Insectos de Víctor Rojas

Prior to pilot testing for the WM component task, the task had also been used extensively during thesis research conducted in 2010 and 2011. For that study the WMC reading span task (as per Osaka and Osaka, 1992) and an analytic language ability (ALA) task (Petersen and Al-Haik, 1976; Pimsleur, 1966) were administered to 34 subjects who had varying amounts of experience learning a second language. These tasks (taken together or separately) demonstrated a high positive correlation with success in L2 learning, and demonstrated the task’s validity in predicting proficiency.

The testing of WMC for the present study was conducted after a level of proficiency had already been acquired in L2 Russian. Prior to its use, the task was therefore pilot-tested with participants both before and after an intensive language program. The pilot test was designed to evaluate the reliability of WM in terms of its resilience to improvement as a result of language learning experiences. Testing was conducted with seven participants who were recruited at the intensive Russian Language Institute held in Columbia, South Carolina. Initial testing with these students occurred in the fall, 2011, prior to their beginning a six-month intensive training experience in L2 Russian. The program consisted of four hours of coursework per day, half of which was related to formal instruction and practice of grammatical forms, new lexical items and the learning of proper sentence structure. The other two hours of instruction were

characterized by more implicit and incidental learning activities, including topical or task-based conversation classes and naturalistic interaction with instructors and fellow

105 105

students. Following completion of the six-month program, participants were tested for their working memory span a second time. To prevent any possible long-term memory of the items, for the second trial a different version of the task was developed, with about half (23) of the items remaining the same though not in the same order of appearance, and 25 entirely new sentences added and incorporated into the task. The new items conformed to the type of sentences they replaced in terms of approximate number of syllables, and word length. Both versions of the test contained a total of 48 task items.

The pilot test for WM was successful, and appears to demonstrate no significant improvement in working memory span for any of the participants: of the seven learners tested, two achieved slightly higher scores (by a total of one or two items), three actually showed a slight decrease (two participant scores were a total of one point lower, and one by a total of three points). The remaining two subjects achieved scores that were

identical to initial testing results (See Appendix B).

Proficiency Pilot testing

The test questions (Section A) used to test general oral proficiency in L2 Russian for the present study have been extensively tested and used as an effective tool for diagnosing all major elements of proficiency. These questions represent part of the Oral Proficiency Interview (OPI) that has been used for a number of years as a diagnostic tool for both higher level course placement and final exit testing conducted at the Russian Language Institute (in conjunction with stateside and overseas programs), a year-round intensive language program located at Columbia International University in Columbia, SC. The OPI is based upon the DLPT assessment format used for the past 40 years at the Defense Language Institute. Over 300 participants have participated successfully in this

106 106

testing with the procedure yielding consistent and accurate assessments of learner abilities in reading, writing and speech, and the test has been compared favorably with similar ACTFL and European Framework proficiency tests.

More specific to the goals of the present study in terms of morphological and syntactic accuracy, an additional pilot test was also conducted for the elicitation task. The task questions were piloted with two advanced-level L2 Russian acquirers and three native speakers of Russian, one who is from Ukraine and three from Kazakhstan. Prior to pilot testing some modifications to the test questions were made following an interview with one of the native speakers: the original design utilized photographs that the

participant was to describe following a specific question, although it was found that regardless of the narrowed context of a photo, a learner could always use visual cues to circumlocute around the structure in the response. The modified test demonstrated that the questions used for this task are well constructed to elicit the targeted forms, with results showing that in each narrowly-defined question context, the speaker was forced to utilize the targeted construction in order to properly answer the question. Feedback on the task was also elicited from the native-speaking participants, who confirmed that use of the targeted structures was the only method available to properly answer each item.

Three of the four native-speaking participants responded exactly as predicted on all items used in the task, receiving scores at level 5. Due to researcher error and at least one other factor, one NS participant used a different structure in her attempt to answer one of the sub-questions, although the answer was corrected when further clarification was given. The pilot test was therefore also instrumental in providing the researcher with

107 107

practical guidelines related to the exact manner in which the test should be conducted to ensure the proper elicitation of the target items.