This quantitative, non-experimental, correlational research study determined whether there is a correlation between how California public middle schools use benchmark exams and their students’ performance on end-of-year standardized tests. As Gall et al., (2007) states, “Correlational research refers to studies in which the purpose is to discover relationships between variables through the use of correlational statistics” (p. 332). “The basic design in correlational research is very simple, involving nothing more than collecting data on two or more variables for each individual in a sample and computing a correlation coefficient” (Gall, Gall & Borg, 2007, p. 335). Correlational statistics are frequently used in test construction and test analysis; therefore, this was the appropriate selection for this study as it identified the effect the predictor variables of benchmark exam utilization, how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and teacher satisfaction with the exams, had on standardized test scores. Specifically looking at results from the California Standards Test and the state accountability measurement tool, the Academic Performance Index (API), the design identified if there was a direct
correlation between how benchmark exams were utilized and the criterion variable of the 2012 API scores.
Research Question This research question for this study is:
benchmark exams be predicted from a linear combination of how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and teacher satisfaction with the exams?
Null Hypothesis The null hypothesis for this study is:
H01: Changes in the API scores of California public middle schools that utilize
benchmark exams cannot be predicted from a linear combination of how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and teacher satisfaction with the exams.
Participants and Setting
The population for this study was California public middle schools that utilized benchmark exams in some capacity. These schools, by definition, are free of charge, state-
funded, co-educational, secular, and offer grades six through eight (Ed-Data, 2014). They were a balanced representation of high-performing (800+), mid-performing (700-799), and low-
performing (below 700) schools as determined by their 2011 API scores. For this study, the sample size was 93 schools. According to Gall et al. (2007), in correlational research, a minimum of 30 participants is desirable. A sample size of 66 is the required minimum for a medium effect size with a statistical power of .7 at the .05 alpha level (Gall et al., 2007, p. 145). Schools were chosen using convenience sampling among all California public middle schools. The researcher sent an email to every California public middle school in the state. Included in the email were the informed consent form and survey. Completion of the survey was entirely voluntary. Schools that agreed to be part of the research study completed the survey and
after having received responses from 100 schools. Noticing that 93 of the 100 responding schools utilized benchmark exams in some capacity, the researcher chose to use those 93 schools as the sample.
As the research study was conducted, the setting did not need to be the campuses of participating schools. Instead, research and analysis took place at home in an online environment, using email, SPSS, and the California Department of Education website.
Instrumentation
The instrument used in this research study was an original online survey created by the researcher and had never been used before. The survey was given the same title as the research study itself and was made available using the Survey Monkey website (see Appendix A). The purpose of this instrument was to first ascertain if participating California public middle schools utilized benchmark exams in some capacity. If the school answered affirmatively to using benchmark exams, the survey then specifically asked for information regarding how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and how satisfied the respondents are with the exams. This instrument was developed to obtain the necessary information to analyze if and to what degree a positive correlation existed between how California public middle schools were utilizing benchmark exams and standardized test scores, specifically their performance on the California Standards Tests.
The survey instrument consisted of eight questions, seven of which were multiple choice. The first question asked the name of the participating school in order to obtain current and
subsequent year API scores. Included in the first question was a disclaimer stating that school identities would be removed upon aggregation of the collected data. Question two asked if the
school utilizes benchmark exams. If the respondent indicated a negative answer to that question, they responded to the next six questions by selecting “NA.” If the respondent indicated the school does utilize benchmark exams in some capacity, they responded to the next six questions with information for the researcher in regards to the predictor variables of the study, such as how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and how satisfied they are with the exams.
The scales of measurement varied from question to question but remained consistent in asking the respondent to select the answer(s) that best describe how the school was currently utilizing benchmark exams. The answer choices for questions two through eight ranged from two possible answers to seven. Scoring was performed by the researcher using the Survey Monkey website and, as there was no specific point value associated with each answer choice, the scoring procedure was an aggregation of data to be entered into the SPSS data sheet. The approximate time to complete the survey was only one minute, as the information required to answer the questions would be common knowledge for the respondent.
Procedures
An email was sent to every public middle school in California asking permission for their participation as outlined in the approved IRB application. IRB approval was granted with an exemption falling under category 46.101 (b)(2,4), which identifies specific situations in which human participants research is exempt from the policy set forth in 45 CFR 46 (see Appendix B). The informed consent form, including a brief explanation of the research study and a link to an electronic survey were included in the email (see Appendix C). Schools willing to participate were asked whether or not they administer benchmark exams, how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the
exams are mandatory, and how satisfied they are with the exams. Responses of participating schools were automatically returned to the researcher using the Survey Monkey program.
An SPPS data spreadsheet was created. The names of all participating schools that utilized benchmark exams in some capacity were listed. The 2011 base API, as obtained from the California Department of Education website, was entered for all schools. Survey response information on predictor variables – how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and teacher satisfaction with the exams – was entered into the SPSS data sheet using numerical codes representing each response. All participating schools’ 2012 growth API scores were then entered into the table for criterion variable analysis, i.e. score change. At this time, the names of the participating schools were deleted.
Data Analysis
Data was analyzed through the use of a linear multiple regression analysis. This research problem, like many in education, involved interrelationships between three or more variables. Multivariate statistics, such as multiple regression, allow researchers to measure and study the degree of relationship among various combination of these variables (Gall et al., 2007, p. 352- 353). Analysis of the data determined whether or not, and to what degree there was a correlation between the predictor variables of how benchmark exams are utilized – specifically how long the benchmark exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and teacher satisfaction with the exams – and the criterion variable of API score change between the 2011 base and 2012 growth API scores of the schools as calculated by the correlation coefficient.
Data screening was performed on each of the variables (API score change, how long the exams have been utilized, how often the exams are administered, how the exams were created, whether or not the exams are mandatory, and teacher satisfaction with the exams) in regard to data inconsistencies. Box and whisker plots were utilized to detect outliers on each of the predictor and criterion variables. Normality was analyzed utilizing the Kolmogorov-Smirnov test. The linear multiple regression analysis was performed to assess the null hypothesis utilizing a .05 alpha level. Correlation coefficients showed a low effect size across all variables.
CHAPTER FOUR: FINDINGS