Staff training. The mixed-mode design of NPSAS:08 data collection required the
development of four separate training programs: supervisor training, help desk training, telephone interviewer training, and tracer training. Separate training sessions were held for each of these groups (table 8).
Table 8. NPSAS:08 training sessions for interviewing and tracing staff: 2008 Staff trained
Number of training
sessions1 Time period staff trained Number of
Supervisors 3 January–July 2008 26
Help desk agents 1 February 2008 19
Telephone interviewers 15 March–August 2008 482
Tracing supervisors and tracing specialists 3 March–May 2008 72
1 Number of training sessions indicates the number of training sessions held for unique groups of trainees.
SOURCE: U.S. Department of Education, National Center for Education Statistics, 2007–08 National Postsecondary Student Aid Study (NPSAS:08).
• General training. Before attending project training, telephone interviewers completed 4 hours of comprehensive general training, which outlined call center rules and procedures, sample member refusal-conversion techniques, and an overview of the CATI-CMS. The general training included an overview of survey research, an introduction to survey data collection methods, and responsibilities of telephone interviewers, including
confidentiality requirements and the rights of survey respondents. During their general training, telephone interviewers also learned general interviewing skills, such as
enunciation and maintaining the proper pace. Telephone interviewers completed their general training either in the Raleigh, North Carolina, Call Center with a supervisor, or online with the use of an interactive website (for a list of general training modules, see appendix G).
• Supervisor, help desk, and telephone interviewer training. The interviewer training program was designed to maximize the trainees’ active participation. Materials included a training agenda, an interviewing manual, a list of FAQs, a pronunciation guide, and handouts featuring help text for selected NPSAS questionnaire forms. Interviewer training sessions consisted of lectures, demonstrations, and hands-on practice exercises with the student
instrument and online-assisted coders. Sessions for supervisors, help desk staff, and telephone interviewers included a study overview, an introduction to data collection procedures, a review of the confidentiality requirements, a demonstration interview, an in- depth review of the instrument, a review and discussion of active listening techniques, hands-on practice exercises with the instrument and open-ended coders. Additionally, interviewers were trained to use the CATI-CMS and were given opportunities to practice in CATI and to ask questions. Help desk agents received the same training as telephone interviewers but also received specialized training on the project-specific help desk application and on solutions to common problems. The help desk application was designed to assist agents in documenting problems reported to the study help desk hotline (for a copy of the telephone interviewer training agenda and the training manual’s table of contents, see appendix G).
• Tracer training. New tracers received 12 hours of classroom training and 4 to 8 hours of additional mentoring when they arrived on the production floor. The training included information on RTI’s history and devoted time to the protection of human study subjects. To receive permission to conduct tracing activities, tracers had to pass
certification requirements for protection of study subjects and their information. Training included an introduction to tracing methods, an overview of confidentiality and security matters in the protection of sample members, training in the use of proprietary databases, and instruction on the use of the tracing CMS. Tracers were also trained to use probing to obtain information helpful in locating sample members (for a copy of the tracing training agenda, see appendix G).
• Additional training. In addition to general sessions, help desk agents and select telephone interviewers received a brief training on conducting calls to prompt randomly selected sample members who had not completed the interview by a specified date to do so. Other additional training modules, such as refusal training, were conducted as necessary. Additionally, quality circle meetings were conducted as an extension of the training program and for continual quality improvement. Telephone interviewers and supervisory staff were given the opportunity to ask questions, and as needs were identified, additional training topics were highlighted and addressed in meetings and subsequent posted quality circle notes.
Student website. The student website for NPSAS:08 provided background and general information about the study, the sample, the study sponsor, the study contractor, and confidentiality assurances (figure 7). In addition to the information available on the website, links to other relevant websites (e.g., NCES and RTI) were provided. The student website also provided a way for sampled students to securely log in and complete the interview from the NPSAS home page.
The initial login page provided the link to the web instrument. The login required entry of a specific study username and password, which were provided in the lead letter and subsequent e-mail messages. Sample members could also obtain their study username and password by clicking on the Forgot Password button (to send a request to the project’s e-mail address) or by contacting a help desk agent at the NPSAS toll-free number. The initial login page also provided a way for respondents to submit their telephone number so they could be contacted by an interviewer.
Figure 7. NPSAS:08 student website: 2008
SOURCE: U.S. Department of Education, National Center for Education Statistics, 2007–08 National Postsecondary Student Aid Study (NPSAS:08) website.
The student website also presented a variety of information about the study. Sample members visiting it were provided the following links from the home page:
• About NPSAS linked to information on the study’s mandate and research objectives, with a link provided to NCES reports from previous study cycles.
• Confidentiality linked to information regarding the regulations in place to protect sample members’ confidentiality.
• Endorsements linked to a complete list of all 26 endorsing organizations.
• FAQs linked to questions and answers concerning the NPSAS:08 study.
Web interviews. Sample members were first introduced to the web interview through the lead letter packet. During the first 3 weeks after a sample member’s invitation to participate, he or she was able to complete a web interview or to call the help desk to complete a telephone interview. The begin and end dates for this initial period varied because the sample was selected on a flow basis. A subset of sample members was selected to receive a prompting telephone call approximately 10 days after the beginning of their initial 3-week data collection period. The determination of which sample members were included in the prompting sample was made with the use of paradata
(information that has been collected about the data collection process) to target cases as appropriate (for more details on prompting, see section 3.5.3). The prompting call was to remind sample
members that they had been selected to participate in the NPSAS:08 study and to encourage them to log in to the study website and complete the web interview during the early completion period, while they were still eligible to receive the $30 incentive.
The website was accessible 24 hours a day, 7 days a week, throughout the data collection period, enabling sample members to complete interviews online at their convenience.E-mail prompts periodically were sent to all sample members (for whom a valid e-mail address was available) during the 3-week period to encourage participation and remind them of the address for the study website and the toll-free help desk phone number. Sample members were assured of the confidentiality of their responses and the voluntary nature of the survey, and they were told that they could decline to answer any survey question. The convenience of the web interview was emphasized, especially that it could be completed any time from any location with internet access and that
respondents could discontinue and resume the interview as necessary.
Help desk operations. Anticipating the first student calls after the introductory mailing, the NPSAS:08 help desk opened in conjunction with the first student mailout so as to provide support to web respondents. Help desk staff were available to assist sample members who had questions or problems accessing and completing the web interview. A toll-free help line was established to accept incoming help desk calls. If technical difficulties prevented sample members from completing the web interview, help desk agents, who were also trained to conduct telephone interviews, would encourage sample members to complete a telephone interview rather than attempt the web interview.
The help desk application documented all incoming calls from sample members. In addition to this primary documentation function, it provided the following:
• information required to verify a sample member’s identity;
• login information allowing a sample member to access the web interview;
• systematic documentation of each call; and
• a means for tracking calls that could not be resolved immediately.
The help desk application also provided project staff with reports on the types and frequency of problems experienced by sample members and a means for monitoring the resolution status of all help desk inquiries.
Telephone interviews. After the early response phase expired for each case, the production-interviewing phase began, during which attempts were made to locate and interview study sample members who had not yet completed an interview. Once the interviewee was located, an attempt was made to conduct the full interview by telephone, although the web version was available throughout the data collection period.
All calls were managed by the CATI-CMS, the system used by the call center to track outbound telephone calls and case status. This system included an automated call scheduler that assigned cases to interviewers by case priority and time of day. Case assignment was designed to maximize the likelihood of contacting and interviewing sample members, according to when the sample member was likely to be available. For each case, a call roster prioritized sample member names and telephone numbers for the interviewers. New roster lines were added as dictated by the results of CATI tracing and intensive tracing results.
Some cases required special treatment. To manage those students who initially refused to participate (including those for whom locator sources acted as “gatekeepers”), some interviewers were trained in refusal-conversion techniques. Sample members and locator sources who spoke only Spanish, primarily located in Puerto Rico, were assigned to bilingual CATI interviewers.
An abbreviated interview was developed in part as a tool for Spanish translation and as a tool for nonresponse conversion near the end of the data collection period. Sample members who had been identified as Spanish-speaking were given the opportunity to take the NPSAS:08 student interview by telephone, in Spanish, through the use of the Spanish abbreviated interview. Sample members who had been identified as hard to reach or as refusals were offered the abbreviated interview, which took approximately 10 minutes to complete, as opposed to the approximately 25 minutes for the full interview. Once the sample member was offered the abbreviated interview, it could be completed either by telephone or on the Web. Sample members were offered the same incentive for completing the abbreviated interview.
Use of incentives. Sample members were offered an incentive of $30 to complete a web interview during the early response phase. Sample members were eligible to receive the incentive, regardless of participation mode, provided they completed the interview before their incentive expiration date. No incentive was offered during the production-interviewing phase.
Toward the end of data collection, a different incentive plan was used with particular types of nonrespondents: (1) sample members who initially refused the interview; (2) sample members for whom intensive tracing yielded a good mailing address but no telephone number; and (3) sample members identified as hard to reach (i.e., those with 15 or more call attempts, where contact had been established with the sample member and no “hard” appointment was pending). The incentive offer consisted of a letter from the project director on RTI letterhead or an e-mail tailored to the specific type of nonrespondent. Sample members were promised a $30 check if they completed the interview, regardless of the mode they used to do so. The incentive letters were mailed on a flow basis as respondents met one of the three criteria for designation as a target nonrespondent.
Near the end of data collection, as part of a final effort to convert sample members who still had not responded to the previous incentive offers, all pending cases received a final letter
requesting participation by the end of the data collection period. Postcards and letters reemphasized the importance of the study and offered all remaining nonrespondents a check for $30 if they completed the interview.