Re: Fisher exact test in PROC FREQ

I guess I just used the term marginals the way I'm used to using it.

Marginals is the term I use, and many people use, to mean the row
and column totals. These totals are usually printed in the margins
of the table.

The commonly used chi square test for a 2x2 table is conditional on
the observed marginals totals.

When I use the term: conditional on the total numbers of successes
another way of expressing it might be: given that the number of
successes is restricted to the number that was actually observed.
Both of these expressions indicate that I am referring to conditional
probability.

So a conditional test uses a conditional probability distribution to
evaluate the degree to which the data deviates from the null hypothesis.
But it is necessary to have some background in probability theory
to go into a further discussion that relies on using conditional
probability.

Dean Bross


-----Original Message-----
From: owner-sas-l@listserv.uga.edu [mailtowner-sas-l@listserv.uga.edu]
On Behalf Of RolandRB
Sent: Wednesday, September 10, 2008 1:24 PM
To: sas-l@uga.edu
Subject: Re: Fisher exact test in PROC FREQ

On Sep 10, 4:05 pm, dean.br...@VA.GOV ("Bross, Dean S") wrote:
> I'll try to add a little more insight into this question about
> fixed marginals.

I very much appreciate your efforts.

> The real answer to your question that the number of responders in a
> clinical trial is not a fixed number is to bring in the concept
> of a conditional test.
>
> In other words, when we test the null hypothesis in a 2x2 table,
> we might not choose to answer the question using the interpretation
> of a p value as: What is the probability that you would observe
> data the extreme or more extremely different from the null hypothesis,
> allowing for any possible set of outcomes of the clinical trial.
>
> It generally makes more sense to answer the question as:
> Given that we had this number of responders in the complete
> trial, what is the probability that we would have gotten this
> much or more divergence from the null hypothesis.

So is this a conditional test or an unconditional test if we calculate
based on the number of responders we got?

> In other words, the probabilities are calculated conditional
> on the total number of successes. And this is the same
> calculation that you would do if you assumed the marginals
> were fixed.

Here I am getting confused. When you say "the probabilities are
calculated conditional on the total number of successes" is this for a
conditional test or an unconditional test (or doesn't that question
make sense)?

Does that mean that the population in each treatment was a certain
number and the number of treatment successes was a certain number and
these are the numbers we got and are going to use so these are the
fixed marginals? I can understand "fixed numbers" but why "marginals"?
What does "marginals" mean in this regard?

> So it is actually a conscious decision to use a conditional test
> or an unconditional test.

Are the Chi-square and Fisher exact test conditional or unconditional?
The above has left me a bit confused.

> Now we have to descend deeper and deeper into statistical theory
> to discuss this choice of tests in full, but in this circumstance,
> the number of successes in the entire trial is something called
> an ancillary statistic.
>
> NOW QUOTING:
>
> From Wikipedia, the free encyclopedia
>
> In statistics, an ancillary statistic is a statistic whose probability
> distribution does not depend on which of the probability distributions
> among those being considered is the distribution of the statistical
> population from which the data were taken. This concept was introduced
> by the great statistical geneticist Sir Ronald Fisher.
>
> Unfortunately, it is a hard concept to fully understand.

I'm not sure I have grasped it. The mean of a set of values would be
independent of the distribution of the statistical population (I
assume) but I'm not sure about the distribution of the mean so I guess
this is not an ancillary statistic.

> But it does appear that the best way to make inferences (such as
> making estimates or testing hypotheses) is to evaluate the data
> conditional
> on the ancillary statistic.
>
> Now the 2x2 table has been studied for how many years by
statisticians,
> and the question of whether to use a conditional or unconditional test
> has been asked, and re-asked and re-re-asked. My assessment is that
> we always come back to the conditional tests used by Pearson
> (chi-square)
> and Fisher (Fisher's exact test)

I see you state that these are conditional tests. Conditional on what?

> To me, a very compelling argument that would lead to the Fisher exact
> test is based on the randomization test. This is actually the way
> Fisher
> looked at this type of data.
>
> I'll try to phrase the argument in very few words, but it might not be
> exactly correct.
>
> I'll return to the example data I used before, where there were two
> samples
> which might represent two different treatments. Each sample had 20
> cases.
> In sample 1 there were 6 successes and in sample 2 there were 2
> successes.
>
> The argument relies heavily on the notion that we randomly allocated
the
> 40 subjects to treatments 1 and 2. Now suppose the null hypothesis
that
> both treatments are exactly the same is absolutely true. Whether we
> gave
> a subject treatment one or treatment two would make no difference.
>
> Then in a sense, the results that each subject obtains is simply fixed
> in advance, even if we did not know what their results were going to
be.
> A subject that succeeded would have gotten this success no matter
which
> treatment they were assigned to, and those who failed would have
failed
> no matter which treatment they had been assigned to.
>
> So here is the key idea. The only reason there were more successes in
> sample one is purely due to the randomization placing more of the
> preordained successes in this sample. So to evaluate whether or not
> this randomization is a reasonable explanation for the observed
results
> we look at the probability that you would get this much difference in
> the success rates if the subjects were assigned purely at random. And
> this leads you to the calculation of the Fisher exact test.
>
> Does this explanation help clarify things

I'll have to read it over a few times. I am confused about what are
conditional tests and why. Many thanks for your explanation.

On Sep 10, 7:49*pm, dean.br...@VA.GOV ("Bross, Dean S") wrote:
> I guess I just used the term marginals the way I'm used to using it.
>
> Marginals is the term I use, and many people use, to mean the row
> and column totals. *These totals are usually printed in the margins
> of the table.
>
> The commonly used chi square test for a 2x2 table is conditional on
> the observed marginals totals.
>
> When I use the term: conditional on the total numbers of successes
> another way of expressing it might be: given that the number of
> successes is restricted to the number that was actually observed.
> Both of these expressions indicate that I am referring to conditional
> probability.
>
> So a conditional test uses a conditional probability distribution to
> evaluate the degree to which the data deviates from the null hypothesis.
> But it is necessary to have some background in probability theory
> to go into a further discussion that relies on using conditional
> probability.
>
> Dean Bross

Thanks. I understand that part of it now.