Clasificación y Empaque

Fisher’s exact test is a randomization-based test for 2×2 tables that calculates p-values based on the probability of the observed table under Fisher’s sharp null hy- pothesis that the vaccine has no effect on the number of challenges until infection for any macaque. While Regoes et al. proposed using Fisher’s exact test for RLC studies, this test is not valid for testing (4.2) in that it is not guaranteed to protect against in- flated type I error. To further detail the inappropriateness of the test, we first illustrate its justification for single challenge studies.

4.4.1

Single Challenge Study

Suppose a single challenge study is conducted where four macaques are randomized such that 2 receive vaccine and 2 receive control. The 4₂ = 6 possible treatment assignment combinations are all equally likely, each with probability of 1/6. Assume the observed treatment assignments are Z1 = Z2 = 1 and Z3 = Z4 = 0. Further

assume only one macaque, i= 1 is uninfected (Xobs

1 = 0 and X2obs =X3obs =X4obs = 1).

Under the null where potential outcomes are observed for all macaques, the potential treatment assignments include:

Comb Z1Z2Z3Z4 PiZiXiobs P iZi(1−Xiobs) P i(1−Zi)Xiobs P i(1−Zi)(1−Xiobs) 1 1100 1 1 2 0 2 1010 1 1 2 0 3 1001 1 1 2 0 4 0110 2 0 1 1 5 0101 2 0 1 1 6 0011 2 0 1 1 (4.3)

where combination 1 is observed. The Fisher’s exact p-value forH0 (4.1) versus a one-sided

alternative that vaccine has a protective effect is calculated as the proportion of combinations that result in an outcome distribution at least as extreme as the observed data (in the direction of the alternative). This set includes combinations where the number of infected vaccine macaques is≤1 (combinations 1-3). Therefore the p-value is 3/6 = 0.5.

Since multiple treatment assignment combinations result in the same 2×2 table the calculations for Fisher’s exact test only consider unique tables (e.g., treatment permutation combinations 1-3 all result in the same 2×2 table). As such, the p-value is calculated by summing the probabilities of each unique table that is as or more extreme than the observed table (in the direction of the alternative). The table based on observed data is constructed as: Infected Non-infected Vaccine P iZiXiobs P iZi(1−Xiobs) P iZi Control P i(1−Zi)Xiobs P i(1−Zi)(1−Xiobs) P i(1−Zi) P iXiobs P i(1−Xiobs) n (4.4)

For example (4.3), this table is:

Infected Non-infected

Vaccine 1 1 2

Control 2 0 2

3 1 4

(4.5)

Fisher’s exact test assumes row and column margins are considered fixed. As such, only one other table in addition to (4.5) is possible for example (4.3). This other table switches the cells for vaccine and control macaques such that there is one non-infected vaccine macaque and zero non-infected control macaques. The probability for each table under the null (4.1) is obtained by realizing the first table cell, P

iZiXiobs has a hypergeometric distribution under

therefore, the final p-value is 0.5.

In single challenge studies, the fixed margins assumption holds as the number of macaques assigned each treatment is fixed by design and the number of infected and non-infected macaques are observed, fixed features of the finite population under (4.1).

4.4.2

Repeated Low-Dose Challenge Studies

Expand the example such that Cmax = 2 and all infected macaques are infected on the

first challenge. Thus, (T_iobs, Dobs_i ) = (2,0) for i = 1 and (T_iobs, Dobs_i ) = (1,1) for i= 2,3,4. Regoes et al. propose conducting a Fisher’s exact test of the following table:

Infected Non-infected Vaccine P iZiDiobs P iZi(Tiobs−Diobs) P iZiTiobs Control P i(1−Zi)Dobsi P i(1−Zi)(Tiobs−Dobsi ) P i(1−Zi)Tiobs P iDiobs P i(Tiobs−Dobsi ) P iTiobs (4.6)

For the expanded example, this table is:

Infected Non-infected

Vaccine 1 2 3

Control 2 0 2

3 2 5

(4.7)

To apply Fisher’s exact test, the probabilities of the table given in (4.7) and 2 other tables are calculated. The first table (a) has 1 non-infection event each in the vaccine and control groups and the second table (b) has both non-infection events in the control group. However, it is not possible to observe table (a) as both non-infection events occur within one macaque and therefore the 2 ’non-infected’ events cannot be divided between the vaccine and control rows of the table. In general, Fisher’s exact test in this setting uses the incorrect set of potential tables such that non-zero probabilities of observation are assigned to tables that are

not actually observable when permuting treatment assignments among macaques and zero probabilities are assigned to tables that are observable.

More formally, the assumption of fixed margins required for Fisher’s exact test is not valid in this setting because the number of challenges per treatment group (row margins) are not fixed. This violation is the cause for the inflated type I error. For example, assume a trial withn= 6 macaques where 5 are infected after the first challenge and the remaining macaque remains uninfected. Regardless of treatment assignment, the probability of rejecting the null (4.2) in favor or a two-sided alternative for allα >0.012 is 1 as the Fisher exact test two-sided p-value is always 0.012.