El Humanismo sevillano

Finally, a number of articles have appeared that have examined tools and resources useful for developers of CBLTs. The most general of these is Godwin-Jones ( 2001 ) which explored computerized testing tools, internet applications, test authoring tools, the outlook for CBLTs and useful resources (including testing resources on the internet;

T ABLE 5.3 P apers describing actual CBL Ts and their development

Citation Language Test name Level Purpose of test Skills Test type Papandima-Sophocleous (2008) English (EFL) NEPTON University Placement GRVW IBLT/CBLT/ CALT hybrid Young, Shermis, Brutten & Perkins ( 1996 ) English (ESL) Reading Comp Test University Not clear R CALT Yu & Lowe ( 2007 ) English (EFL) CEOTS University Achievement Oral CBLT Burston & Monville-Burston ( 1995 ) French French CAT University Placement G CALT Laurier ( 1999 ) French French CAPT University Placement GLRVA CALT Dunkel ( 1999b ) Hausa Hausa CAT University Profi ciency L CALT Larson ( 1999 ) Russian R-CARPE University Profi ciency R CALT Larson & Hendricks ( 2009 ) Spanish Part of CLIPS University Diagnostic GW IBLT Zabaleta ( 2007 ) Spanish SPT University Placement GLPRV CBLT Bernstein, Barbier, Rosenfeld & de Jong ( 2004 ) Spanish SST University Profi ciency LS CBLT with automatic scoring Bernhardt, Rivera & Kamil ( 2004 ) Spanish/ German GPT/SPT University Articulation/ Admission GV IBLT Alderson & Huhta ( 2005 ) 14 European DIALANG Mostly university Diagnostic GLRVW IBLT Alderson & Huhta ( 2005 ) 14 European DIALANG Mostly university Diagnostic GLRVWIBLT

related institutions and organizations; existing CBLTs; available online practice tests;

online CBLTs; as well as CBLT development tools and templates).

Polio ( 2001 ) provided a review of an online trial version of Test Pilot , which is a system for authoring and administering tests or surveys online server (currently available at www.clearlearning.com/). Winke and MacGregor ( 2001 ) provided a review of Hot Potatoes (currently available at http://hotpot.uvic.ca/index.php), which is a program that helps teachers create online exercises (or assessment activities) by using templates supplied by the program. Franco, Bratt, Rossier, Gadde, Shriberg, Abrash and Percoda ( 2010 ) briefl y reviewed the literature on speech recognition and then provided good exposure for their product by describing features of Stanford Research Institute (SRI) International’s EduSpeak ® system (www.speechatsri.com/products/eduspeak.shtml) speech recognition and pronunciation scoring software development tool in terms of their strategy for scoring pronunciation at the phrase level or higher. Burstein, Chodorow and Leacock ( 2003 ) also provided excellent exposure for an Educational Testing Service product called CriterionSM. They described CriterionSM as an online essay evaluation system (for general information: www.ets.org/criterion) and evaluated the degree to which CriterionSM was suffi ciently accurate to provide useful feedback (see also Chodorow & Burstein, 2004 ; Chodorow, et al., 2010 ).

Conclusions

As the reference list below will attest, the CBLT literature since Brown ( 1997 ) has been spread out over many journals and books. In addition, it has grown considerably in volume: only 29 of the 96 references in Brown ( 1997 ) were focused directly on CBLT, while 87 of the 89 references in the present chapter are directly related to CBLT.

As mentioned above, Brown ( 1997 ) found that the CBLT literature had covered:

(a) how to use an item bank, (b) how to use new technologies, (c) how to build computer-adaptive tests and (d) how effective computers are in language testing. In the interceding years the literature seems to have moved well beyond the How to . . . topics covered earlier to new topics. First, a number of papers, chapters and books have periodically provided overviews of the CBLT literature. That was not previously common. While CBLT delivery issues like CALT were discussed at the how- to level, today, the discussion of CALT appears to be considerably more sophisticated and in-depth. In addition, while the computer vs traditional testing formats studies continue to be conducted, topics like WBLT, interface architecture, the test takers’

experience with CBLT and CBLT training for teachers are entirely new. As for CBLT content issues, much of the literature appears to now focus on the issues involved in testing the various components of language like vocabulary, speaking or oral skills, writing, listening and/or reading with a particular concern for automatic scoring of spoken and written responses. Moreover, a fairly large number of example CBLTs are actually being developed and described in the literature along with the variety of tools and resources that have increasingly become available for CBLT development.

Brown ( 1997 ) examined the education literature on CBT for clues about the future directions that research on CBLT might take. Since CBT was focusing primarily on computer-adaptive testing (CAT), I predicted that future research in language testing might tend towards addressing ten questions about computer-adaptive language tests (CALTs). To some degree I was correct. However, I was also wrong. I fear that my assumption was that the general educational testing research was always ahead of language testing research and that CBLT research would therefore follow in the footsteps of CBT/CAT research. Instead, what has happened is that, while there was some work on CALT, CBLT also moved into practical and useful directions, perhaps relying on the general CBT literature to sort out the more technical issues of CALT.

However, in all fairness, Brown ( 1997 ) also predicted a number of other areas that CBLT might usefully pursue including (a) testing in intelligent teaching systems, (b) testing on the internet, (c) using speech and handwriting recognition, (d) scoring and analysing speech and writing samples and (e) developing alternative psychometric models that take advantage of the power of computers to analyse more complex information (p. 53). Three out of fi ve of those earlier predictions appear to have come true: (b), (c) and (d) are areas that have burgeoned in the intervening years. The other two, (a) and (e) are areas in which research and development would still benefi t the fi eld.

Clearly, the CBLT literature has pretty much gone its own way which makes sense given that the language part of language testing would naturally lead to a certain expertise and interest in topics like speech recognition and automatic scoring of writing.

In sum, CBLT is not a new area of research, but in recent years, it has clearly been growing and generating a good deal of excitement – perhaps because it allows language testers to respond to changes in the language teaching fi eld, especially in teaching methodologies, using strategies never before available, and to do so much faster, more practically and in ways that are more closely aligned to what and how students are learning in their language classrooms.

References

Alderson, J. C. (1999). Reading constructs and reading assessment. In

M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 49–70). Cambridge: Cambridge University Press.

— (2000a). Technology in testing: The present and the future. System, 28 , 593–603.

— (2000b). Assessing reading . Cambridge: Cambridge University Press.

— (2005). Diagnosing foreign language profi ciency . London: Continuum.

— (2007). Computer-adaptive language testing. In S. Granger (Ed.), Optimizing the role of language in technology-enhanced learning (pp. 1–3). Louvain-la-Neuve, Belgium:

Integrated Digital Language Learning.

— (2009). Test review: Test of English as a Foreign Language: Internet-based Test (TOEFL iBT®). Language Testing, 26 (4), 621–31.

Alderson, J. C., & Huhta, A. (2005). The development of a suite of computer-based diagnostic tests based on the Common European Framework. Language Testing, 22 (3), 301–20.

Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater® V.2. Journal of Technology, Learning, and Assessment, 4 (3), 1–30.

Bernhardt, E. (1999). If reading is reader-based, can there be a computer-adaptive test of reading? In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 1–10). Cambridge: Cambridge University Press.

Bernhardt, E. B., Rivera, R. J., & Kamil, M. L. (2004). The practicality and effi ciency of Web-based placement testing for college-level language programs. Foreign Language Annals, 37 , 356–66.

Bernstein, J., Barbier, I., Rosenfeld, E., & de Jong, J. (2004). Development and validation of an automatic spoken Spanish test. In InSTIL/ICALL 2004 symposium on computer assisted learning, NLP and speech technologies in advanced language learning systems , Venice, Italy, 17–19 June 2004, ISCA Archive.

Blake, R., Wilson, N. L., Cetto, M., & Pardo-Ballester, C. (2008). Measuring oral profi ciency in distance, face-to-face and blended classrooms. Language Learning & Technology, 12 (3), 114–27.

Brown, A., & Iwashita, N. (1996). Language background and item diffi culty: The development of a computer-adaptive test of Japanese. System, 24 (2), 199–206.

Brown, J. D. (1997). Computers in language testing: Present research and some future directions. Language Learning and Technology, 1 (1), 44–59.

— (2009). Computers in language testing: Present research and some future directions. In P.

Hubbard (Ed.), Computer assisted language learning: Critical concepts in linguistics, Volume IV: Present trends and future directions in CALL (pp. 115–37). New York: Routledge.

Burstein, J., Chodorow, M., & Leacock, C. (2003). CriterionSM online essay evaluation: An application for automated evaluation of student essays. Proceedings of the fi fteenth annual conference on innovative applications of Artifi cial Intelligence . Acapulco, Mexico.

Burston, J., & Monville-Burston, M. (1995). Practical design and implementation

considerations of a computer adaptive foreign language test: TIIE Monash/Melbourne French CAT. CALICO Journal, 13 (1), 26–46.

Chalhoub-Deville, M. (Ed.) (1999). Issues in computer-adaptive testing of reading profi ciency. Cambridge: Cambridge University Press.

— (2001). Language testing and technology: Past and future. Language Learning &

Technology, 5 (2), 95–8.

Chalhoub-Deville, M., & Deville, C. (1999). Computer adaptive testing in second language contexts. Annual Review of Applied Linguistics, 19 , 273–99.

Chapelle, C. (1999). From reading theory to testing practice. In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 150–66). Cambridge:

Cambridge University Press.

Chapelle, C. A. (2001). Computer applications in second language acquisition. Cambridge:

Cambridge University Press.

Chapelle, C. A., & Chung, Y.-R. (2010). The promise of NLP and speech processing technologies in language assessment. Language Testing, 27 (3), 301–15.

Chapelle, C. A., & Douglas, D. (2006). Assessing language through computer technology . Cambridge: Cambridge University Press.

Chodorow, M., & Burstein, J. (2004). Beyond essay length: Evaluating e-rater®’s performance on TOEFL® essays. TOEFL Research Reports #73. Princeton, NJ:

Educational Testing Service.

Chodorow, M., Gamon, M., & Tetreault, J. (2010). The utility of article and preposition error correction systems for English language learners: Feedback and assessment.

Language Testing, 27 (3), 419–36.

Choi, I.-C., Kim, K. S., & Boo, J. (2003). Comparability of a paper-based language test and a computer-based language test. Language Testing, 20 (3), 295–320.

Christensen, C. (1997). The innovator’s dilemma: When new technologies cause great fi rms to fail. Boston, MA: Harvard Business School.

Coniam, D. (2006). Evaluating computer-based and paper-based versions of an English-language listening test. ReCALL, 18 (2), 193–211.

Dikli, S. (2006a). An overview of automated scoring of essays. Journal of Technology, Learning, and Assessment, 5 (1).

— (2006b). Automated essay scoring. Turkish Online Journal of Distance Education, 7 (1), article 5 (no page numbers).

Dooey, P. (2008). Language testing and technology: Problems of transition to a new era.

ReCALL, 20 (1), 21–34.

Douglas, D. (2000). Assessing languages for specifi c purposes. Cambridge: Cambridge University Press.

Douglas, D., & Hegelheimer, V. (2007). Assessing language using computer technology.

Annual Review of Applied Linguistics, 27 , 115–32.

Dunkel, P. (1999a). Considerations in developing or using second/foreign language profi ciency computer-adaptive tests. Language Learning & Technology, 2 (2), 77–93.

— (1999b). Research and development of a computer-adaptive test of listening

comprehension in the less-commonly taught language Hausa. In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 91–121).

Cambridge: Cambridge University Press.

Eignor, D. (1999). Selected technical issues in the creation of computer-adaptive tests of second language reading profi ciency. In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 167–81). Cambridge: Cambridge University Press.

Franco, H., Bratt, H., Rossier, R., Gadde, V. R., Shriberg, E., Abrash, V., & Percoda, K. (2010). EduSpeak ^R : A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications. Language Testing, 27 (3), 401–18.

Fulcher, G. (1999). Computerising an English language placement test. English Language Teaching Journal, 53 , 289–99.

— (2003). Interface design in computer-based language testing. Language Testing, 20 (4), 384–408.

Garcia Laborda, J. (2007a). ON THE NET: Introducing standardized EFL/ESL exams.

Language Learning & Technology, 11 (2), 3–9.

— (2007b). From Fulcher to PLEVALEX: Issues in interface design, validity and reliability in internet based language testing. CALL-EJ Online, 9 (1).

— (2009). Interface architecture for testing in foreign language education. Procedia Social and Behavioral Sciences, 1 , 2754–7.

— (2010). Contextual clues in semi-direct interviews for computer assisted language testing. Procedia Social and Behavioral Sciences, 2 , 3592–5.

Garcia Laborda, J., & Litzler, M. F. (2011). Constraints in teacher training for computer assisted language testing implementation. International Education Studies, 4 (2), 13–17.

García Laborda, J., & Magal-Royo, T. (2007). Diseño y validación de la plataforma PLEVALEX como respuesta a los retos de diseño de examines de lenguas para fi nes específi cos. Journal of the European Association of Languages for Specifi c Purposes , 14 , 79–97.

— (2009). Training senior teachers in compulsory computer based language tests.

Procedia Social and Behavioral Sciences, 1 , 141–4.

Garcia Laborda, J., Bakieva, M., Gonzalez-Such, J., & Sevilla Pavon, A. (2010). Item transformation for computer assisted language testing: The adaptation of the Spanish University Entrance Examination. Procedia Social and Behavioral Sciences, 2 , 3586–90.

Garcia Laborda, J., Magal-Royo, T., Macario de Siqueira Rocha, J., & Fernández Álvarez, M. (2010). Ergonomics factors in English as a foreign language testing: The case of PLEVALEX. Computers & Education, 54 , 384–91.

Garrett, N. (1991). Technology in the service of language learning: Trends and issues.

Modern Language Journal, 75, 74–101.

Godwin-Jones, B. (2001). Emerging technologies: Language testing tool and technologies.

Language Learning & Technology, 5 (2), 8–13.

Gorsuch, G. (2004). Test takers’ experiences with computer-administered listening comprehension tests: Interviewing for qualitative explorations of test validity. CALICO Journal, 21 (2), 339–71.

Grabe, W. (1999). Developments in reading research and their implications. In M.

Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 11–47). Cambridge: Cambridge University Press.

Gruba, P., & Corbel, C. (1997). Computer-based testing. In C. Clapham & D. Corson (Eds.), Language testing and assessment (pp. 141–9). Dordrecht: Kluwer Academic.

Jamieson, J. (2005). Trends in computer-based second language assessment. Annual Review of Applied Linguistics, 25 , 228–42.

Jones, L. (2004). Testing L2 vocabulary recognition and recall using pictorial and written test items. Language Learning & Technology, 8 (3), 122–43.

Kenyon, D. M., & Malabonga, V. (2001). Comparing examinee attitudes toward computer-assisted and other oral profi ciency assessments. Language Learning &

Technology, 5 (2), 60–83.

Kenyon, D. M., Malabonga, V., & Carpenter, H. (2001). Response to Norris commentary.

Language Learning & Technology, 5 (2), 106–8.

Kirsch, I., Jamieson, J., Taylor, C., & Eignor, D. (1998). Familiarity among TOEFL examinees. TOEFL Research Report 59 . Educational Testing Service, Princeton, NJ.

Larson, J. W. (1999). Considerations for testing reading profi ciency via computer-adaptive testing. In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 71–90). Cambridge: Cambridge University Press.

— (2000). Testing oral language skills via the computer. CALICO Journal, 18 (1), 53–66.

Larson, J. W., & Hendricks, H. H. (2009). A context-based online diagnostic test of Spanish . CALICO Journal, 26 (2), 309–23.

Laufer, B., & Goldstein, Z. (2004). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54 (3), 399–436.

Laurier, M. (1999). The development of an adaptive test for placement in French. In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 122–35). Cambridge: Cambridge University Press.

Levy, R., Behrens, J. T., & Mislevy, R. J. (2006). Variations in adaptive testing and their online leverage points. In D. D. Williams, S. L. Howell & M. Hricko (Eds.), Online assessment, measurement, and evaluation (pp. 180–202). Hershey, PA: Information Science.

Linacre, J. M. (1999). A measurement approach to computer-adaptive testing of reading comprehension. In M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 182–95). Cambridge: Cambridge University Press.

Liu, M. L., Moore, Z., Graham, L., & Lee, S. (2003). A look at the research on

computer-based technology use in second language learning: A review of the literature from 1990–2000. Journal of Research on Technology in Education, 34 (3), 250–73.

Luecht, R. (1999). The practical utility of Rasch measurement models. In M.

Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp.

196–215). Cambridge: Cambridge University Press.

Macario de Siqueira, J., Martínez-Sáez, A., Sevilla-Pavón, A., & Gimeno-Sanz, A. (2011).

Developing a Web-based system to create, deliver and assess language profi ciency within the PAULEX Universitas Project. Procedia Social and Behavioral Sciences, 15 , 662–66.

McNamara, T. F. (1999). Computer-adaptive testing: A view from outside. In

M. Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp. 136–49). Cambridge: Cambridge University Press.

Meunier, L. E. (1994). Computer adaptive language tests (CALT) offer a great potential for functional testing. Yet, why don’t they? CALICO Journal, 11 (4), 23–40.

— (1995). Human factors in a computer assisted foreign language environment: The effects of gender, personality and keyboard control. CALICO Journal, 13 (2&3), 47–72.

Mislevy, R., Chapelle, C. A., Chung, Y.-R., & Xu, J. (2008). Options for adaptivity in computer-assisted language learning and assessment. In C. A. Chapelle, Y.-R. Chung

& J. Xu (Eds.), Towards adaptive CALL: Natural language processing for diagnostic language assessment (pp. 9–24). Ames, IA: Iowa State University.

Norris, J. M. (2001). Concerns with computerized adaptive oral profi ciency assessment.

Language Learning & Technology, 5 (2), 99–105.

Ockey, G. J. (2009). Developments and challenges in the use of computer-based testing for assessing second language ability. Modern Language Journal, 93 , 836–47.

Papadima-Sophocleous, S. (2008). A Hybrid of a CBT- and a CAT-based New English Placement Test Online (NEPTON). CALICO Journal, 25 (2), 276–304.

Polio, C. (2001). Review of Test Pilot . Language Learning & Technology, 5 (2), 34–7.

Prapphal, K. (2008). Issues and trends in language testing and assessment in Thailand.

Language Testing, 25 , 127–43.

Reed, J. (2000). Assessing vocabulary. Cambridge: Cambridge University Press.

Roever, C. (2001). Web-based language testing. Language Learning & Technology, 5 (2), 84–94.

Sawaki, Y. (2001). Comparability of conventional and computerized tests of reading in a second language. Language Learning & Technology, 5 (2), 38–59.

Taylor, C., Jamieson, J., Eignor, D., & Kirsch, I. (1998). The relationship between computer familiarity and performance on computer-based TOEFL test tasks. TOEFL Research Report 61. Princeton, NJ: Educational Testing Service.

Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defi ning the classroom research agenda. Language Teaching Research, 10 (2), 1–24.

Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University Press.

Winke, P., & MacGregor, D. (2001). Review of Hot Potatoes . Language Learning &

Technology, 5 (2), 28–33.

Young, Y., Shermis, M. D., Brutten, S. R., & Perkins, K. (1996). From conventional to computer-adaptive testing of ESL reading comprehension. System, 24 , 23–40.

Yu, X., & Lowe, J. (2007). Computer assisted testing of spoken English: A study to evaluate the SFLEP college English oral test in China. In F. Khandia (Ed.), 11th CAA international computer assisted assessment conference: Proceedings of the conference on 10th and 11th July 2007 at Loughborough University (pp. 489–502).

Loughborough Leicestershire, UK: Loughborough University.

Zabaleta, F. (2007). Developing a multimedia, computer-based Spanish placement test.

CALICO Journal, 24 (3), 675–92.

Zumbo, B., & MacMillan, P. D. (1999). An overview and some observations on the psychometric models used in computer-adaptive language testing. In M.

Chalhoub-Deville (Ed.), Issues in computer-adaptive testing of reading profi ciency (pp.

216–28). Cambridge: Cambridge University Press.

6

In document 72 Daniela Santonocito (página 32-35)