114. After the key components of your manuals have been internationally verified, and about eight weeks prior to the start of testing, and not much before that, you will need to provide instructions to School Coordinators regarding the collection of student lists. Since it is very important that the student sample represent current enrolments, the lists should not be prepared too far in advance.
115. These instructions appear in the final School Coordinator manual, and you will need to record adaptations to these instructions as part of the process of recording manual adaptations. The individuals who prepare the lists should also be instructed to return the lists to you at least 6 weeks before testing is to begin. This will provide you with adequate time to select the student samples and send the Student Tracking Form (STF) that lists the selected students back to schools at least 2 weeks prior to testing. This schedule may be affected by school vacations and holidays. You should be aware of the impact of these events and modify the schedule accordingly
116. Please note the following: The instructions accompanying the form should convey the following:
• The importance of listing all eligible students, including those who may not be tested due to a disability or limited language experience.
• The fact that any exclusion from the assessment of students who cannot be tested are required to be done AFTER the student sample is selected.
• A definition of PISA-eligible students (in grades 7 or higher born within the population birth date definition that has been agreed with Westat and is recorded on MyPISA).
117. The content of the form should include the following (See the DMM for additional information):
• Student line number, grade, gender, month and year of birth, study program and SEN code. If the population definition begins on day 2 or later in the birth month, then a column for day of birth should be included.
• While it is assumed that the lists of students will contain names, these are not critical to the sampling process as long as the lists contain a unique student identifier. A student identification number assigned by the school, for example, is an acceptable way to identify each student
• There are additional variables that can appear on this form for optional components. If the country is using the UH booklet in general schools, then the list of students needs an extra variable to indicate which SEN students (i.e., those that would otherwise have been excluded) should have the UH booklet. If the country is doing computer based assessments, two other variables for CBA hold the number of available computers, and a comment field about the number of computers.
• If you are selecting an additional grade-based sample and/or other additional population students as a national option (refer to Appendix G) you have to carefully identify on the list of students, those eligible for PISA, and those eligible for grade (or other additional population) sample.
• For this cycle, there is a provision for up to seventeen optional variables on the list of students in KeyQuest. You need to agree with Westat and ACER (via the MAS) on the use of any optional variables. If class sampling is being done, two of these variables need to hold Class name and another for the Class ID that KeyQuest has assigned to each class. If class sampling is not done, the optional variables can be class name, shift, language, or any additional necessary information about students. If used on the list of students, they can be transmitted by KeyQuest into the STF, if desired by the NPM.
118. You will also need to record, as part of the Excel filename for each school’s list of students being imported into KeyQuest, the region (REGION), stratum (STIDSTRT), and school ID (STIDSCH) assigned at the time of school sampling and used on your list of participating schools and agreed as part of your sampling plan. REGION, STIDSTRT and STIDSCH must be used in all parts of your sample tracking and processing
119. For administrative purposes, you may also need to record additional details such as school name, address, telephone number and email address.
120. We recommend that you ask schools to retain a copy of the list in case you need to call the school with questions.
121. It is important that all students with a date of birth within the PISA population birth date definition enrolled in the school be listed. This includes students who may be excluded from testing because of a special educational need or limited experience in the language(s) in which the test is being offered, students who are frequently absent from school, students in all grades 7 and above offered by the school/sampling unit, and students who may be on work placement programmes at the time of testing. It is very important that the lists of students provided by schools are complete lists of all PISA-eligible students since incomplete lists could seriously compromise the quality of the sample.
122. Once a student list is received from a school, please check it to ensure that all age-eligible students and all relevant information about the students has been received using the following as guidelines.
Check for the number of students: The actual total number of students in the list should be compared to the enrolment variable on the SFKQ file.
If the number is similar, then continue with the next check.
If the number is quite different, for example, smaller by a difference of 10, then the school should be contacted, to query them about the student list. The school should be told that a list with more students was expected and ask them to recheck they have included all eligible students (and explain to them again the target population birth dates). If they have left students off the list, an updated file needs to be requested.
If the school confirms the number of age-eligible students in the list is correct, then continue with the additional checks below.
If the number is quite different, for example, greater by 10, then the school should be contacted to query them about the student list. The school should be told that a list with fewer students was expected and ask them to recheck they have included only eligible students (and explain to them again the target population birth dates). If they have extra non-PISA students on the list that should not be there, an updated file needs to be requested.
If the school confirms the number of age-eligible students in the list is correct, then continue with the additional checks below.
Check student details:
Does the student list contain details about: Year or Grade Level
Gender Birth date Etc.
123. If any of this information is missing, or if there are students from only one year level, or only one gender, or students have not been included from the whole target population (for example, missed one month or there are missing levels within gradespan), etc., then send the school a letter, thanking them for sending the student list for PISA, but asking for additional information. Re-attach the list of instructions for preparing the student list, ask them to re- review it, and note that you look forward to soon receiving an updated student list.
7.6.2 Learning the KeyQuest Functions and Operations Required for Student Sampling
and Performing Student Sampling in KeyQuest
124. The details of new KeyQuest functions and operations will be provided in the training materials for the NPM meeting in Singapore in October 2011. NPMs are strongly encouraged to send their data managers for this training, especially those considering national or international options related to sampling such as CBA or an additional grade-based population. The final instructions will be available from the DMM.
125. As NPMs and data managers, it will be extremely important to understand these new KeyQuest functions as the quality of student sampling and the weighting of the student data depends on it.