Conditional versus Unconditional Exact Tests for Comparing Two Binomials Cyrus R.Mehta and Pralay Senchaudhuri 4 September 2003 Introduction There are two fundamentally different exact tests for comparing the equality of two binomial probabilities- Fisher's exact test(Fisher,1925),and Barnard's exact test(Barnard,1945).Fisher's exact test(Fisher,1925)is the more popular of the two.In fact,Fisher was bitterly critical of Barnard's proposal for esoteric reasons that we will not go into here.Notwithstanding Fisher's reservations,both tests are available in StatXact. Consider the following example of a vaccine efficacy study(Chan,1998).In a randomized clinical trial of 30 subjects,15 were innoculated with a recombinant DNA influenza vaccine and the 15 were innoculated with a placebo.Twelve of the 15 subjects in the placebo group (80%)eventually became infected with influenza whereas for the vaccine group,only 7 of the 15 subjects (47%)became infected.The data are tabulated as a 2 x 2 table(see Table 1).Suppose me is the probability of infection for the vaccine(or experimental)group and Table 1:Results of Vaccine Efficacy Study Infection Treatment Combined Status Vaccine Placebo Response Yes 7(47%)12(80%) 19 No 8(53%)3(20%) 11 Totals 15 15 30 e is the probability of infection for the placebo (or control)group.What is the exact one-sided p-value for testing the null hypothesis Ho:Te Te. The answer depends on the type of exact test adopted.Fisher's exact test produces an exact p-value of 0.0641. On the other hand Barnard's exact test produces an exact p-value of 0.0341.The smaller,statistically significant, exact p-value produced by Barnard's method is no accident.For 2 x 2 tables,Barnard's test is more powerful than Fisher's,as Barnard noted in his 1945 paper,much to Fisher's chagrin.The purpose of this short article is to explain why Barnard's method is more powerful in this setting and to discuss the limitations of the method. 1Conditional versus Unconditional Exact Tests for Comparing Two Binomials Cyrus R. Mehta and Pralay Senchaudhuri 4 September 2003 Introduction There are two fundamentally different exact tests for comparing the equality of two binomial probabilities – Fisher’s exact test (Fisher, 1925), and Barnard’s exact test (Barnard, 1945). Fisher’s exact test (Fisher, 1925) is the more popular of the two. In fact, Fisher was bitterly critical of Barnard’s proposal for esoteric reasons that we will not go into here. Notwithstanding Fisher’s reservations, both tests are available in StatXact. Consider the following example of a vaccine efficacy study (Chan, 1998). In a randomized clinical trial of 30 subjects, 15 were innoculated with a recombinant DNA influenza vaccine and the 15 were innoculated with a placebo. Twelve of the 15 subjects in the placebo group (80%) eventually became infected with influenza whereas for the vaccine group, only 7 of the 15 subjects (47%) became infected. The data are tabulated as a 2 × 2 table (see Table 1). Suppose πe is the probability of infection for the vaccine (or experimental) group and Table 1: Results of Vaccine Efficacy Study Infection Treatment Combined Status Vaccine Placebo Response Yes 7 (47%) 12 (80%) 19 No 8 (53%) 3 (20%) 11 Totals 15 15 30 πc is the probability of infection for the placebo (or control) group. What is the exact one-sided p-value for testing the null hypothesis H0: πe = πc . The answer depends on the type of exact test adopted. Fisher’s exact test produces an exact p-value of 0.0641. On the other hand Barnard’s exact test produces an exact p-value of 0.0341. The smaller, statistically significant, exact p-value produced by Barnard’s method is no accident. For 2 × 2 tables, Barnard’s test is more powerful than Fisher’s, as Barnard noted in his 1945 paper, much to Fisher’s chagrin. The purpose of this short article is to explain why Barnard’s method is more powerful in this setting and to discuss the limitations of the method. 1