Use an independent samples factorial ANOVA when you have two or more independent variables and one dependent variable, and the independent variables specify which of two or more groups people belong to (for example, one independent variable might be men versus women and, another, people working in sales versus those working in finance). You wish to know if there is a significant difference between the means of the groups across each independent variable. So, if the independent variables were gender and whether people work in production or sales, and the dependent variable was well-being, you might want to know whether there is a significant difference in the well-being of: (a) men and women;
and (b) those who work in production and those who work in sales. You may also wish to know whether there is one or more interaction between the independent variables and the dependent variable. The nature of these interaction effects is explained below.
Example
■ Research has suggested that some managers adopt a task-based approach to
management, focusing on the tasks that need to be achieved, whereas others are more focused on building good relationships with employees. A researcher wishes to examine if there is a relation between whether or not male and female managers are task- or people-focused and the job satisfaction of the employees who work under them. To do so, she examines the job satisfaction of 100 employees. Fifty of them are men and, of this 50, 25 work for task-oriented managers and 25 for people-oriented managers. Another 50 are female, and 22 of them work for task-oriented managers with the remaining 28 working for people-oriented managers. An independent samples factorial ANOVA is used to examine whether the mean level of job
satisfaction is different for people managed by: (a) males versus females; and (b) those with a task-focused manager versus those with a person-focused manager.
Background
With t-tests, and the one-way ANOVA, you are examining whether there is a difference in the means of a dependent variable across two or more levels of a single independent 1111
STATISTICAL METHODS FOR EXAMINING DIFFERENCES
variable. For example, with an independent samples t-test you might examine whether there is a significant difference in mean job commitment across two levels of gender (males and females). However, sometimes you may wish to design research so that you can simultaneously examine whether two or more independent variables affect the depen-dent variable. Perhaps you wish to know not just whether there is a difference in mean job commitment as a function of gender (males versus females), but also as a function of education (degree-educated versus not degree-educated). Data of this type are set out in Table 5.15.
The figures in Table 5.15 indicate the mean job commitment of 20 people meas-ured on a 5-point scale, where the higher the score, the higher the job commitment. So, five degree-educated males have job-commitment scores of 3, 1, 2, 1 and 2 respectively. Obviously, you would be unlikely to have such a small sample of cases in practice, but this is sufficient to illustrate the nature of the data.
How might you analyse these data? Well, one possibility would be to carry out two inde-pendent measures t-tests. First, you could carry out a t-test to examine whether there is a significant difference in the mean job commitment of the 10 males and the 10 females.
Then, you could carry out a second t-test to see if there is a significant difference between the mean job commitment of people who have, and have not, been educated to degree level. Carrying out these t-tests would not be wrong, but there is a danger that something very interesting about the data might be lost if you took this approach.
If you compute the means for the four combinations of job-commitment scores here (i.e.
males degree-educated, males not degree-educated, females degree-educated, females not degree-educated), you obtain the results shown in Table 5.16.
Table 5.16 shows that the mean job-commitment score for males (i.e. 2.8), and the mean for females (2.6), are very similar. Therefore, it is quite possible that you would find no significant difference between the mean job commitment of men and women. Similarly, the mean job-commitment scores of educated employees (2.5) and non degree-educated employees (2.9) are also similar and, again, it is quite possible that you would not find a significant difference between the job commitment of degree-educated and non degree-educated employees. So, carrying out two t-tests, one on level of education, and one on gender, may well tell you that there are no significant differences here, and so you might conclude that gender and level of education are unimportant in relation to job commitment.
However, if this were your conclusion you would probably be making a major over-sight. To understand why, have a look at the graph shown in Figure 5.21. In this graph the
Table 5.15 Job commitment by gender and education level
Degree- Not degree-educated educated
Male 3 3
1 4
2 5
1 5
2 2
Female 3 3
4 1
2 2
3 3
4 1
mean of each possible combination of the independent variables has been plotted.
Furthermore, the two means for degree-educated people have been joined with a line as have the two means for non degree-educated people.
Figure 5.21 shows the mean for degree-educated males, degree-educated females, non degree-educated males and non degree-educated females. This graph clearly shows you something that you would not have known by just carrying out the t-tests. That is, the effect of gender on job commitment depends on whether employees are degree-educated 1111
STATISTICAL METHODS FOR EXAMINING DIFFERENCES
Table 5.16 The mean job commitment of employees by whether or not they are degree-educated
Gender Education Mean N
Males Degree-educated 1.8 5
Not degree-educated 3.8 5
Males overall 2.8 10
Females Degree-educated 3.2 5
Not degree-educated 2.0 5
Females overall 2.6 10
Males and
females overall Degree-educated 2.5 10
Not degree-educated 2.9 10
For all employees 2.7 20
4.0
Figure 5.21 Job commitment: a crossover interaction between gender and education
or not. For degree-educated employees, females show substantially more job commitment than males, whereas for non degree-educated employees, males show markedly more job commitment than females.
What we are actually saying here is that the effect of one independent variable on the dependent variable depends on the other independent variable (in this case, the effect of education on job commitment depends on gender). When the effect of one independent variable on the dependent variable depends on one or more other independent variables, we say that we have an interaction effect.
You can always tell whether you may have an interaction effect by plotting out the means of the various combinations of independent variables on a graph like the one shown in Figure 5.21. In this graph, the two lines connecting the means for males and females cross over each other. For this reason this is referred to as a ‘crossover interaction’. However, not all interactions are crossover interactions. In the graph shown in Figure 5.22 the two lines do not cross each other so there is not a crossover interaction. There is still an interaction though because the lines are not parallel. Whenever you plot the combinations of means in your factorial design and find that a pair of them are not parallel, this indicates an interac-tion in your sample data. Of course, this does not necessarily mean that there is an interaction for the statistical populations you are interested in. Just as a difference in the means of two statistical samples may occur even if there is no difference in the mean of the populations from which they have been drawn (in other words, the null hypothesis is true), so you may find an interaction effect in your sample data even if there is no interaction in the statistical populations from which you have sampled. As a consequence, you need to examine whether any apparent interaction effects are statistically significant.
4.0
3.5
3.0
2.5
2.0
Commitment Education
1.5
Gender
Male Female
Degree-educated Not degree-educated
Figure 5.22 An interaction between gender and education
Factorial analysis of variance is used because, unlike t-tests or the one-way analysis of variance, it allows you to examine interaction effects. In addition, it is also possible to examine main effects which are the effects of one independent variable on the dependent variable when the other independent variable (or variables) is ignored, and simple effects, the effect of an independent variable on the dependent variable at a particular level of another independent variable. There is a possible main effect for every independent vari-able in your research design. So, in the case of the data considered above there is a possible main effect for gender and another possible main effect for education level. To this you add the interaction effects for every possible combination of independent variables, which in this case is simply the combination of gender and education level. If instead of the two inde-pendent variables of gender and education level you added a third, ethnicity (white versus black), you would have three possible main effects (for gender, education level and ethnicity) and four possible interactions (gender with education level, gender with ethnicity, education level with ethnicity, and gender with education level with ethnicity). The inter-action between two independent variables is known as a two-way interinter-action. In the same way, the interaction between three independent variables is a three-way interaction, four independent variables is a four-way interaction and so on.
Factorial ANOVAs are normally referred to by the number of independent variables, and the number of levels of each independent variable. A 2 ⫻2 ANOVA means that there are two independent variables, each with two levels. A 2 ⫻3⫻3⫻5 ANOVA has four independent variables, with the first variable having two levels, the second and third having three levels and the fourth having five levels.
Assumptions and requirements
In ideal circumstances the independent samples factorial ANOVA is used when the samples being compared are drawn from normally distributed populations with the same variance.
However, as explained earlier, ANOVA is a robust statistical technique, and it stands up well to violations of these assumptions most of the time.
The required sample size
The sample size required for an independent samples factorial ANOVA depends primarily on the power required, the significance level adopted and the weakest effect size (i.e. the effect size for the independent variable expected to have the weakest effect size in terms of differences between the two or more means). As for one-way independent measures designs, Cohen (1988) suggests the effect size measure f. If you are in a position to use the results of previous research or a pilot study to estimate the effect size, f, for each inde-pendent variable in your factorial ANOVA, you can do so by using the calculator at www.dewberry-statistics.com (see page 140). When you have established f for each inde-pendent variable, take the smallest one and use it in Table 5.17 to look up the sample size you will require in your study.
1111
STATISTICAL METHODS FOR EXAMINING DIFFERENCES