An online survey was developed as described previously in the methodology chapter (chapter 3) and administered to a sample of respondents who were participants in online communities. The survey text can be found in appendix A. The electronic link to the survey was posted either by the researcher or by a member of the community. Posting of the survey link was accomplished over a period of 8 weeks starting in September 2009 and all surveys were closed to responses in December 2009. Data was collected from users of several online communities in order to achieve a wide array of responses. Potential respondents were emailed a link to the survey and asked if they would both complete it as well as post it on communities in which they took part or visited. A total of 6 links were posted which resulted in 155 responses, out of which 98 were usable, that is, in which the missing data was less than 20%. Excel data files were procured from the Survey
Monkey.com website and cleaned by the researcher to a format suitable for importing into SPSS. Data was screened for outliers and the analysis data functions in SPSS were
subsequently used. Results are detailed in the following sections.
4.2.1.1 Participation Construct Correlation matrices and Cronbach’s alpha.
First, correlation matrices were built for the question group which pertained to
Participation. As there was no presence of hypothesis, a two tailed test was carried out.
Cronbach’s alpha scores were used to assess question group reliability.
The high level of correlation between some factors means that these are probably redundant questions which needed to be excluded. The original survey can be found in the appendices;
the questions correlate to the numbers as seen in the original survey. The questions which were included in the initial analysis in SPSS can be found in the survey in appendix A and were:
Question 5A Only read questions and comments form other members without posting Question 5B Post Questions
Question 5C Post Comments
Question 5D Post Answers to questions from other community members Question 6A Rate the usefulness of other people’s reviews or comments
Question 6A was removed due to low response rate due to inapplicability, as several of the sites on which the survey was posted did not have the capability for users to rate other’s comments. While this feature has become more prevalent, at the time this survey was administered it was still in its infancy.
The researcher followed the guidelines of Fields (2005) in applying the Analyse => Scale Reliability analysis to determine the reliability of the scale. The results showed that through removing item 5A, the Cronbach Alpha score could be increased, and whilst there were three items still measuring the participation scale, this was carried out by the researcher.
Additionally, a factor analysis investigation extracted one component indicating that the three remaining items were in fact measuring the same underlying construct. Table 4.1 details the participation construct findings.
Scale
Cronbach's Alpha Cronbach's Alpha Based on
Standardized Items N of Items
0.884 0.886 3
Table 4.1 Final Participation Item Statistics And Cronbach’s Alpha
It is important to note that this step was carried out by the researcher to ensure a concise and reliable scale could be ascertained when these measures were incorporated into the final survey. The final survey, which was administered to the final sample of news website users, can be found in appendix B and contains all of the original participation items. The items used in the final model analysis in the confirmatory stage of the research can be found in Chapter 5 which includes the confirmatory stage results. Here, Table 5.6 shows the composite reliability of the items used in the final scale, and Table 5.7 demonstrates the T-values and loading of the items resultant from the survey of online news site users.
The purpose of this stage of the research was to develop measures which would capture online “communityness”. The following section continues to describe the process of
developing a scale for the less distinct “Motivation” and “Effect” measures which were proposed by the researcher to be pertinent dimensions in online community participation.
4.2.1.2 Motivation and Effect Constructs
While participation was relatively straightforward as it was conceptually different from the other constructs, the items deemed necessary by the researcher to measure the less distinct concepts of motivation to participate in a community (the preconditions which exist to entice a user to visit or participation in the community) and the effect of participation in the community (or the post visit consequences) were less distinct. In the initial exploratory survey, measurement of motivation and effect were assessed using a multiple item scale.
Following Fields (2005) a correlation matrix was created and examined.
Again, the generally high level of correlation between some items means that there were probably redundant questions which could be eliminated. Due to the high number of items, the Cronbach’s alpha score was also high at .932. Cortina (1993) states that the value of this alpha depends on the number of items on the scale, and therefore it is easier to get a high alpha because of the large number of items. Additionally, Grayson (2004) demonstrated that data sets which have the same alpha can have very different structure; that an alpha may pertain to a data set which comprises several underlying factors. Grayson (2004) demonstrated that an alpha of .8 can be achieved in a scale with one underlying factor, with two moderately correlated factors or with two uncorrelated factors. Cronbach (1951) also recognized this and indicated that if several factors exist then the formula should be applied separately to each factor. Thus the researcher found support for the fact that Cronbach’s alpha values would need to be determined for individual factor item sets.
The next step was therefore to perform a factor analysis on the full set of items measuring the constructs of motivation and effect. Field (2005) writes that the maximum likelihood method and Kaiser’s alpha factoring can be used when we wish to generalize findings to a
larger population and as such alpha factoring was used in this analysis. Since in factor analysis the researcher is interested in finding common underlying dimensions, common variance as opposed to unique variance (variance which is attributed to one measure) or random variance (variance which is attributed to one measure, but not reliably so) is important and was considered. Communality is the measure of the proportion of variance explained by the extracted factors. Literature exists which concludes that solutions derived from principal component analysis differ little from factor analysis techniques (Guadagnoli and Velicer, 1988), but Stevens (1992) holds that for 30 or more variables, and
communalities greater than .7 for all variables, solutions are not likely to be different, but for fewer than 20 variables with low communalities, differences can occur (Field, 2005).
Eigenvalues greater than one were retained, following Kaiser (1960), as the number of variables was less than 30, and the sample size was less than 200, meaning that determining the proper eigenvalue from the inflection point of a curve on a scree plot would not be accurate (Stevens, 1992 in Field, 2005). Field also indicates that the closer the
communalities are to one, the better the factors are at explaining the data and therefore communalities are a good indicator of whether too few factors have been retained.
Of the two types of rotation, orthogonal or oblique, the researcher chose varimax, while even though it is very unlikely that the factors are completely independent of one another (unlikely when measuring psychological attributes), many researchers in the social
sciences, do use orthogonal rotation (e.g. Pura, 2005). Stevens (1992) indicated that a factor loading of .512 should be used as the lowest in sample sizes of around 100, so this is the value which was used in this analysis. Missing values were replaced with the mean.
Field (2005) indicates that before running a factor analysis, a data screening process should be undertaken. If there are variables which do not correlate with any other variable, these should be removed before running factor analysis. In addition Field suggests that if after running a determinant test the determinant of the R-matrix is less than .00001, the data should again be screened to look for variables which correlate very highly (R> .8), and
subsequently finding multicollinearity in the data should raise questions about the questionnaire.
The researcher followed these guidelines in the analysis and removal of items. The resultant factor analysis determined 5 factors accounting for 71.7 % of the variance. The Kaiser-Meyer-Olkin measure of sampling adequacy was very good at .821 (values above .8 are very good: Fields, 2009). Bartlett’s test of sphericity was also significant. Following the principle component analysis, the researcher performed Cronbach’s alpha analysis to retain three items for each construct which showed the highest reliability. These are detailed next.