Start-up Culture
4. The Value of This Approach
Recognize patterns in data sets
ST1.1 Turn in data summary to support/refute government claim
Data summary (see Table 1.7 for possible data summary submissions)
Correct: Player knows to use a large enough sample size and the correct measure.
If Incorrect:
Species other than blackburn: Player likely does not know what a blackburn is
Trait other than body length: Player did not understand what needed to be measured If result does not agree with their answer in Tumbler 1.2: See Quest 1 7a
If sample size is too small: Player does not
understand the importance of a large enough sample Use models and
simulations to make inferences and conclusions
EV3.3 Students use a simulator to see how environmental pressures can affect trials.
Students turn in EvoGlobe and respond to questions.
Data collected includes Globe setting and responses.
Correct: See EvoGlobe settings EV3.3a table.
Potential reasons for Incorrect EvoGlobe:
They don’t understand principle of natural selection.
They don’t understand how to interpret EvoGlobe.
They don’t understand how to use the EvoGlobe.
Figure 2. Example of quest template developed for the Radix Endeavor using the Balanced Design approach.
learning. The two are one. Learning can be defined by two interrelated questions the drive a never-ending cycle of improvement:
• What are you trying to teach?
• How do you know you know it?
As a field, we have largely ignored the second question. The second question is critical, but not just because it helps a teacher or student know where they stand, and what they might need next.
Indeed, those are essential. However it’s critical because when we ask the second question at the beginning of the design process, it changes our foundation. Asking, and answering, that question, sets a content model and conceptual foundation for the design of the strategic learning experience we call learning games. Without such an explicit and defined model, we are making judgments about the game design without a developed evidence model about the actual learning processes involved. As a result, the learning design results from our best judgment rather than from evidence on learning. The Balanced Design framework offers a coherent model that aligns the parts of a given learning construct. We may often start a learning game design with learning goals in mind, but using the Balanced Design framework offers the learning goals and the remaining elements of a complete learning construct as the blueprint for the learning experience at hand.
4.2 Tenet #2: Setting a standard for mapping and documenting learning games
The proposed design framework here offers us something even further and more critically useful to the field: by documenting the learning model undergirding a game, along with other general metadata, we can collectively as a field do a much better job of documenting and sharing knowledge about our work and the actual ‘bones’ of the games we build. Currently, if I asked you “what are the learning goals?” of a given learning game, that answer may or may not be easy to find; and chances are, it will take a fair bit of poking around a website or attached documents to find it. More broadly, the field of education is moving toward standards in data and the representation of data. A prime example is the Learning Resource Metadata Initiative (LRMI) tasked with creating a unified tagging structure for learning content on the web so that it is aligned with schema.org and can better enable search engines to find and display learning content available on the web (see http://www.lrmi.net). Of course, this includes games and learning games are already being tagged and mapped using this structure. However, there are further reasons for the field of learning games to look at taking this practice deeper. Learning games are designed closed- and open-ended experiences. They vary is depth, shape and size. In a quality learning game, the learning is embedded and intertwined in the game experience, and learning content may not be so evident to the user—be they a teacher, student, or parent. By utilizing a shared documentation method, not only will critical information about an existing learning game be mapped out and accessible, it will be done in a way that allows an individual to very easily see what’s happening “under the hood” of a game because they are familiar with the documentation format. Allow the framework would not need to be rigidly followed, if such a framework was generally used to map the contents of a learning game in similar ways it would benefit everyone using, working with, creating and studying learning games.
5.0 CONCLUSIONS: Moving Towards the Next Generation of Learning Games
Although ECD has its roots in formal assessment, and was initially leveraged in the field of learning game design as a means for created assessments that were game-based, we believe this framing of ECD is too narrow and it offers us a toolkit that much more broadly serves the learning design in learning games. A fundamental followed by many in the field is that what makes a quality learning game is when you find the playful game mechanics in the learning content and fuse the two for a quality game—the other end of the spectrum from games where the learning is explicit and then the student is rewarded with game play. Learning game designers may fear that the focus on assessment in games will push their designs towards the latter rather than the former. However, we suggest that in fact, using a modified ECD framework called Balanced Design as presented here, affords you the ability to set a much better foundation for the learning game design, enabling you to avoid this problem and design strategically for game mechanics in the learning content itself.
As we look ahead as to how we may grow as a field, we must be vigilant to focusing growing in ways that better serves the learning in games—and we must work to build bridges between these complex methodologies so that it is accessible to all members of our very diverse, and interdisciplinary design teams.
REFERENCES
[1] Baker, R. S. J. D. & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3-17.
[2] Behrens, J. T., Mislevy, R. J., DiCerbo, K. E. & Levy, R. (2012). Evidence centered design for learning and assessment in the digital world. In M.C. Mayrath, J. Clarke-Midura, D.H. Robinson, G.
Schraw (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 13-53). Charlotte, NC: Information Age Publishing.
[3] Clarke-Midura, J., Mayrath, M., Dede, C. (2012). Thinking outside the bubble: Virtual Performance Assessments for Measuring Inquiry Learning. In M. Mayrath, J. Clarke-Midura, D. Robinson (Eds.) Technology Based Assessment for 21st Century Skills: Theoretical and Practical Implications from Modern Research. New York, NY: Springer-Verlag.
[4] Halverson, R. & Owen, V.E. (in press). Game-Based Assessment: An Integrated Model to Capture Evidence of Learning in Play. International Journal of Learning Technology.
[5] Halverson, R., Owen, V.E., Salmon, A., Kasemodel, C. & Torrisi, B. (2013). ADAGE: An Assessment Data Aggregator for Gaming Environments. Presented at the Games+Learning+Society Conference, June 12-14.
[6] Klopfer, E., Osterweil, S. & Salen, K. (2009). Moving learning games forward: Obstacles, opportunities & openness. An MIT Education Arcade white paper.
[7] Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice, 25(4), 6-20.
[8] Mislevy, R. J., & Riconscente, M. M. (2006). Evidence-centered assessment design: Layers,
structures, and terminology. In S. Downing & T. Haladyna (Eds.), Handbook of test development (pp. 61- 90). Mahwah, NJ: Lawrence Erlbaum Associates.
[9] Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2002). On the roles of task model variables in assessment design. In S. Irvine & P. Kyllonen (Eds.), Item generation for test development (pp. 97-128).
Mahwah, NJ: Lawrence Erlbaum Associates.
[10] Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-67.
[11] Owen, V.E., Halverson, R., & Wills, N. (2012). CyberSTEM: A Game-Based Evidence Model. In A.
Ochsner (Ed.), Proceedings of the 8th Annual Games+Learning+Society Conference. ETC Press:
Pittsburgh, PA.
[12] Pellegrino, J., Chudowsky, N. & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. National Academies Press.
[13] Phillips, V. & Popović, Z. (2012). More than child’s play: Games have potential learning and assessment tools. Kappan, 94(2), 26-30.
[14] Rupp, A.A., Gushta, M., Mislevy, R.J., & Shaffer, D.W. (2010). Evidence-centered Design of Epistemic Games: Measurement Principles for Complex Learning Environments. Journal of Technology, Learning, and Assessment, 8(4).
[15] Shute, V., Hansen, E., & Almond, R. (2008). You can’t fatten a hog by weighing it—or can you?
Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence and Education, 18(4), 289-316.
[16] Shute, V. & Ventura, M. (2013). Stealth Assessment Measuring and Supporting Learning in Video Games. The John D. and Catherine T. MacArthur Foundation Reports on
Digital Media and Learning. Cambridge, MA: MIT Press. Accessible at http://mitpress.mit.edu/books/stealth-assessment
[17] Shute, V. & Torres, R. (2011). Where streams converge: Using evidence-centered design to assess Quest to Learn. In M. Mayrath, J. Clarke-Midura, & D. H. Robinson (Eds.), Technology-based
assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 91- 124). Charlotte, NC: Information Age Publishing.
[18] Squire, K. (2006). From content to context: Videogames as designed experience. Educational researcher, 35(8), 19-29.
[19] Wainess, R., Koenig, A. & Kerr, D. (2011, January). Aligning instruction and assessment with game and simulation design. The National Center for Research on Evaluation, Standards, and Student Testing.
CRESST Report 780.