Técnicas de medida de estereotipos de la vejez, explícitas e implícitas

To ensure that telephone interviews were accurately administered and that their responses were being accurately coded, telephone interviewers were routinely monitored. The monitors had the ability to observe live interviews (with both visual and audio) without disturbing either the respondent or the interviewer. The monitor would observe up to 20 questions during each interview selected for review. Each question was assessed for two elements of interviewer performance: (1) correct delivery of the question, and (2) accurate keying of the response. To correctly gauge the accuracy and performance of the NPSAS:08 telephone interviewers, monitoring sessions were conducted throughout the data collection period and during all shifts.

Of the 23,042 items monitored, 93 question delivery errors and 31 data entry errors were observed. The overall error rates were low (typically less than 1 percent) and within control limits (figures 17 and 18).27

Figure 17. Question delivery error rates, by week: 2008

The monitoring results were tabulated and posted as weekly and cumulative reports available to project staff. Error rates are typically higher at the beginning of data collection as telephone interviewers become familiar with the NPSAS:08 study and interview. Error rates typically become more stable over the course of the study.

0 2 4 6 8 10 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Error rate Weeks Upper-limit rate Monitored rate Lower-limit rate

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2007–08 National Postsecondary Student Aid Study (NPSAS:08).

27 _{The upper and lower control limits were defined by 3 times the standard error of the proportion of errors to the number of}

Figure 18. Data entry error rates, by week: 2008 0 2 4 6 8 10 12 14 16 18 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Error rate Weeks

Upper Limit Rate Monitored Rate Lower Limit Rate

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2007–08 National Postsecondary Student Aid Study (NPSAS:08).

Quality circle (QC) meetings provided an invaluable opportunity for open communication between call center staff and project staff to address the challenges encountered in data collection, to help ensure data quality, and to provide retrainings as necessary. Meetings were held weekly at the call center, and supervisors selected telephone interviewers to attend on a rotating basis. For interviewing staff unable to attend a given meeting, notes were distributed electronically to the call center supervisory staff, who distributed the notes to their staffs accordingly. Specifically, these meetings addressed elements of the interview and instrument design, addressed interview cooperation tactics, motivated the group in alignment with study goals, and offered exchange of feedback on data collection issues such as the following:

• necessary clarification of interview questions and item responses;

• NPSAS eligibility criteria;

• interviewers’ submission of problem sheets;

• the importance of providing detailed case comments;

• help desk operations;

• Virtual Call Center (VCC) staff issues or concerns;

• methods of gaining cooperation from sample members and associated contacts (e.g., parents); and

• morale boosting and reinforcement of positive interviewing techniques.

Across the QC meetings, a variety of issues were addressed that reinforced specific content from training and contributed to prompt problem solving:

Item administration. Clarification of item responses and interpretation of meanings were addressed. Interviewers discussed items that caused confusion, and project staff provided

explanations and made minor items and help text modifications when necessary. In addition, items that interviewers found difficult to administer were reviewed for further clarification.

Eligibility criteria. Because of the considerable complexity of the eligibility criteria, interviewers were reminded to allow eligibility determination to be made by the programmed instrument.

Problem sheets. Reporting problems when they occur is crucial to quality telephone interviewing. Interviewers were trained to report problems electronically and to provide detail, including the problem that occurred and the specific point in the interview at which it occurred. Review of problem sheets in QC meetings was a critical means by which staff learned to recognize and manage the different problems they would encounter. Issues that were brought to light through problem sheets were also discussed in QC meetings.

Computer-assisted telephone interviewing comments. During the QC meetings, telephone interviewers were reminded to add detailed comments to every record of their actions within the computer-assisted telephone interviewing case management system (CATI-CMS). These comments helped accurately record the event, as well as prepare the next interviewer who would be calling the sample member.

Help desk operations. Help desk staff who attended the QC meetings received specific information about current help desk operations. In addition, help desk agents were able to discuss current issues about which sample members were currently seeking assistance.

Cooperation from sample members. Discussions focused on the difficulty of gaining a sample member’s trust during the initial phases of the call. Refusal-conversion strategies were discussed during QC meetings and adapted as necessary. Interviewers discussed successful approaches for calling refusals and hard-to-reach sample members. They also shared tips for overcoming concerns about individual items and found ways to benefit and learn from each other’s experiences.

4.3.10 Virtual Call Center

In addition to a fully staffed call center located in Raleigh, North Carolina, NPSAS:08 employed a VCC. The VCC provided professionally trained interviewers with the tools to complete their work from home instead of in a traditional centralized call center. Interviewers worked on their own computers and connected remotely to the call center systems that would allow them to conduct interviews. A home visit was conducted for each interviewer to ensure that he or she was

appropriately prepared, with a quiet, secure work area and sufficient computer equipment. VCC interviewers received the same training and preparation for work as interviewers who worked at the call center. Because all VCC interviewers were located in the local area, they were able to attend project training at the Raleigh call center. They did, however, complete their general training online before project training. To accommodate the VCC interviewers for QC meetings, conference calls were established so that staff could participate from their homes. Additionally, QC notes were posted online, eliminating the need to distribute physical copies to interviewers and allowing VCC interviewers to access the notes from a remote location.

To verify the quality of data being collected from staff using VCC technology, project staff regularly reviewed interview data. Overall, no substantial differences in either data quality or

interviewer productivity were found between interviews administered in the call center and those administered in VCC settings. On average, VCC interviewers took slightly longer to complete

interviews than call center interviewers: 29 minutes for call center calls, as opposed to 31 minutes for the VCC, t(778) = 7.64, p < .0001. The rates of missing data for call center completions and VCC completions were also compared. Overall, approximately 4 percent of all items were found to have significantly different rates of missing data for the call center and VCC, indicating that interviewers used proper interviewing and probing techniques.