The main data collection instrument for analysis in the study was the Civic and Political Activities and Attitudes Survey (CPAAS), developed at Tufts University, which gathered data on students’ participation in civic activities as undergraduates, as well as their civic and political attitudes, during the spring of each of their four years at Tufts. A corresponding interview protocol was also developed as a secondary data collection instrument to capture sophomore and senior-year reflections on students’ experiences and involvement and the impact of these on their civic and political attitudes. These follow- up interviews presented students with questions designed to illuminate the factors that motivated students to get involved in particular activities, while also highlighting influential sources of their civic and political attitudes. Appendix A contains the questions included in the interview protocol.
66 The CPAAS was developed after reviewing a number of sources, and was then presented for an expert panel review by Dr. Robert Bringle, Dr. Andrew Furco and Dr. Dwight Giles, known and respected researchers in the areas of civic engagement and service learning. These experts each reviewed the instrument and provided input on the survey instrument to provide further evidence of the entire instrument’s construct validity. The survey was compiled from existing instruments designed to gather information on college students’ civic and political engagement, along with additional items specific to the research question. The items were integrated or adapted from seven validated instruments, as the study draws on past research that has explicated that civic learning outcomes often result from service learning measurements (Bringle, Phillips, & Hudson, 2004; Eyler & Giles, 1999). Two of the seven survey instruments provided questions about involvement in different types of civic and political activities, including community service, voting, and current affairs. The surveys instruments that provided items for the involvement questions on the CPAAS were: 1) the National Youth Survey (CIRCLE, 2002) and 2) the Civic and Political Health of a Nation Survey (Keeter, Zukin, Andolina, & Jenkins, 2002). The remaining five survey instruments provided Likert-type attitudinal items measuring students’ civic and political attitudes across various
dimensions of civic engagement, including their confidence in their ability to make a significant contribution in a community, their commitment to the public good, social justice, and diversity. These survey instruments were: 1) the Baseline Survey of AmeriCorps members (Abt Associates, 2001), 2) the Social Responsibility Inventory (Markus et al., 1993), 3) the Civic Attitude and Skills Questionnaire (Moely, Mercer,
67 Ilustre, Miron, & McFarland, 2002), 4) the Community Service Self-Efficacy Scale (Reeb, Katsuyama, Sammon, & Yoder, 1998), and 5) the Public Service Motivation Scale (Perry, 1996).
Importantly, in past analyses, these final three survey instruments have displayed strong reliability estimates, psychometric properties, and evidence of their respective validity (Bringle et al., 2004; Moely et al., 2002). In particular, each of these survey instruments reported high Cronbach alpha estimates, or coefficient alpha (α), which is the most common measure of internal consistency of items on a scale. This reliability
coefficient expresses the extent to which item responses on a scale are correlated with one another. With regard to the Civic Attitude and Skills Questionnaire, factor analysis was used to define six scales - two of which were used as sources for the CPAAS (Moely et al., 2002). These sub-scales, social justice and diversity attitudes, displayed
considerable reliability (test-retest = .74, .73, α = .70, respectively), with support for their validity being provided by examining correlations to other measures of motivational and racial beliefs (Moely et al., 2002). Through similar analyses, the Community Service Self-Efficacy Scale has shown to be a reliable (test-retest = .62, α > .90) measure of a unidimensional construct as well (Bringle et al., 2004). Likewise, the Public Service Motivation Scale resulted in strong reliability estimates (α = .90), utilized confirmatory factor analysis to substantiate its four sub-scales, and provided support for its convergent and discriminant validity through investigations into relationships to measures of
68 The CPAAS asks a series of questions aimed at examining the extent to which students’ involvement in various activities during their college experience influences their attitudes towards civic engagement. The survey questions focus on both students’ activities as undergraduates, as well as their civic and political attitudes. These
involvement and attitudinal questions are designed to enable a comparative analysis that highlights how students’ choices of activities affect their civic and political attitudes during their undergraduate years. The CPAAS queries students on their involvement in a range of activities in college, including those related to civic activities, political activities, and expression of public voice (Zukin et al., 2006). For each of the 22 activities, students are asked to provide how many hours per year they were involved using a 6-point, Likert- type rating scale: 1 = none, 2 = 10 hours or less, 3 = 11-25 hours, 4 = 26-60 hours, 5 = 61-120 hours, and 6 = more than 120 hours.
The attitudinal section of the CPAAS captures students’ affect towards the importance of and belief in the values of civic engagement. On these 56 questions, students are asked to provide their level of agreement with a variety of statements
expressing various affects towards civic engagement on a 5-point Likert-type rating scale in which: 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree. The directionality of scoring is such that it would be expected for students with high-levels of civic engagement to score highly on these items, whereas it would be expected with students with low-levels of these attributes to score lower. The scale contains thirteen negatively-worded items that are reverse-scored for proper
69 directionality on the scale, such that higher scores reflect stronger positive affect towards civic engagement.