This dissertation was guided by the vision of one day developing and deploying tutoring systems capable of conversing with students and scaffolding them through any subject imaginable. Access to personalized tutoring at this scale could revolutionize education and greatly aid teachers who are continually faced with increasing class sizes and decreasing budgets. Improvements in tutorial dialogue technologies present a path toward providing every student with an engaging learning experience that enables individual ownership and discovery of knowledge. Progress toward these goals largely hinges on the ability to learn, generalize, and automate more robust and more intelligent dialogue behaviors. The relatively recent convergence of machine learning, natural language processing, and education has presented a new lens in which to explore the linguistic phenomena underlying tutorial dialogue interactions. With such tools at our collective disposal, the momentum and timing are perfect for making intelligent tutoring systems available for every learner. This dissertation is a step towards turning this vision of intelligent tutoring systems into reality.
Bibliography
Agarwal, M. and Mannem, P. Automatic gap-fill question generation from text books auto- matic gap-fill question generation from text books automatic gap-fill questions from text books. In Proceedings of the Sixth Workshop on Innovative Use of NLP for Building Educational Applications. 2011.
Ai, H. and Litman, D. Assessing dialog system user simulation evaluation measures using human judges. In Proceedings of ACL-08: HLT. Columbus, Ohio, USA, 2008.
Alexandersson, J., Buschbeck-Wolf, B., Fujinami, T., Kipp, M., Koch, S., Maier, E., Reithinger, N., Schmitz, B., and Siegel, M. Dialogue acts in verbmobil-2 second edition. Technical report, BMBF, 1998.
Allen, J. F., Miller, B., Ringger, E., and Sikorski, T. A robust system for natural spoken dialogue. In ACL 1996. 1996.
Allen, J. F. and Perrault, C. R. Analyzing intention in utterances. Artificial Intelligence, 15(3):pages 143–178, 1980.
Anderson, J. R. and Reiser, B. The lisp tutor. Byte, 10(4):pages 159–175, 1985.
“Apache OpenNLP Development Community”. Apache OpenNLP Developer Documentation.
Apache, The Apache Software Foundation, 1.5.2-incubating edition, 2010.
Austin, J. L. How to Do Things With Words. Harvard University Press, Cambridge, MA USA, 2nd edition edition, 1962.
Baker, C. F., Fillmore, C. J., and Lowe, J. B. The Berkeley FrameNet Project. In COLING ’98 Proceedings of the 17th International Conference on Computational Linguistics. 1998.
Bangalore, S., Fabbrizio, G., and Stent, A. Learning the structure of task-driven human-human dialogs. IEEE Transactions on Audio, Speech, and Language Processing, 26(7):pages 1249–1259, 2008.
Bangalore, S. and Stent, A. Incremental Parsing Models for Dialogue Task Structure. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009). 2009.
Beck, I. L. and McKeown, M. G. Improving comprehension with Questioning the Author: A Fresh and Expanded View of a Powerful Approach. Scholastic, 2006.
Beck, I. L., McKeown, M. G., Worthy, J., Sandora, C. A., and Kucan, L. Questioning the author: A year-long classroom implementation to engage students with text. The Elementary School Journal, 96(4):pages 387–416, 1996.
Becker, L., Nielsen, R. D., Okoye, I., Sumner, T., and Ward, W. What’s next? Target Concept Identification and Sequencing. In Proceedings of the 3rd Workshop on Question Generation at the 10th International Conference on Intelligent Tutoring Systems, (ITS 2010), pages 35–44. Pittsburgh, Pennsylvania, 2010.
Becker, L., Nielsen, R. D., and Ward, W. What a pilot study says about running a question generation challenge. In Proceedings of the 2nd Workshop on Question Generation. Brighton, England, 2009.
Becker, L., Palmer, M., van Vuuren, S., and Ward, W. Learning to Tutor Like a Tutor: Question Ranking in Context. In Proceedings of the 11th International Conference on Intelligent Tutoring Systems (ITS2012). Chania, Greece, 2012a.
Becker, L., Palmer, M., van Vuuren, S., and Ward, W. Question Ranking and Selection in Tutorial Dialogues. In Proceedings of the 7th Workshop on Building Educational Applications Using NLP. Montreal, Canada, 2012b.
Becker, L., Ward, W., van Vuuren, S., and Palmer, M. DISCUSS: A Dialogue Move Taxonomy Layered Over Semantic Representations. In Proceedings of the 9th International Conference on Computational Semantics (IWCS 2011). Oxford, England, 2011.
Berger, A. L., Pietra, V. J. D., and Pietra, S. A. D. A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, 22(1), 1996.
Bloom, B. S. Taxonomy of educational objectives: The classification of educational goals. Susan Fauer Company, Inc., New York, 1956.
Bloom, B. S. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6):pages 4–16, 1984.
Boyer, K. E. Characterizing the Effectiveness of Tutorial Dialogue with Hidden Markov Models. In Proceedings of the 10th International Conference on Intelligent Tutoring Systems. 2010.
Boyer, K. E., Ha, E., Phillips, R., Wallis, M. D., Vouk, M. A., and Lester, J. C. Dialog act modeling in a complex task-oriented domain. In Proceedings of SIGDIAL 2010, pages 295–305. 2010. Boyer, K. E., Ha, E. Y., Wallis, M., Phillips, R., Vouk, M. A., and Lester, J. C. Discovering Tutorial
Dialogue Strategies with Hidden Markov Models. In Proceedings of AIED ’09, pages 141–148. Brighton, U.K., 2009a.
Boyer, K. E., Lahti, W. J., Phillips, R., Wallis, M. D., Vouk, M. A., and Lester, J. C. An empirically derived question taxonomy for task-oriented tutorial dialogue. In Proceedings of the 2nd Workshop on Question Generation, pages 9–16. Brighton, U.K., 2009b.
Boyer, K. E., Phillips, R., Ha, E. Y., Wallis, M. D., Vouk, M. A., and Lester, J. C. Modeling Dialogue Structure with Adjacency Pair Analysis and Hidden Markov Models. In Proceedings of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies Conference (NAACL HLT) Companion Volume, pages 49–52. 2009c.
125 Bratman, M. E., Israel, D., and Pollack, M. Plans and resource-bounded practical reasoning.
Computational Intelligence, 4:pages 349–355, 1988.
Bub, T. and Schwinn, J. Verbmobil: The evolution of a complex large speech-to-speech transla- tion system. In ICSLP 96. Proceedings., Fourth International Conference on Spoken Language, volume 4, pages 2371–2374. 1996.
Buckley, M. and Wolska, M. A classification of dialogue actions in tutorial dialogue. In Proceedings of COLING 2008, pages 73–80. ACL, 2008. ISBN 978-1-905593-44-6.
Bunt, H. Dynamic interpretation and dialogue theory. The Structure of Multimodal Dialogue, 2000. Bunt, H., Alexandersson, J., Carletta, J., Choe, J.-W., Fang, A. C., Hasida, K., Lee, K., Petukhova, V., Popescu-Belis, A., Romary, L., Soria, C., and Traum, D. Towards an iso standard for dialogue act annotation. In Proceedings of LREC 2010, the Seventh International Conference on Language Resources and Evaluation. Malta, 2010.
Bunt, H. and Romary, L. Towards multimodal content representation. In LREC Workshop on International Standards of Terminology and Language Resources. 2002.
Bunt, H. C. Context and dialogue control. THINK Quarterly, 3(1):pages 19–31, 1994.
Bunt, H. C. Dynamic interpretation and dialogue theory. The Structure of Multimodal Dialogue, 2, 1997.
Bunt, H. C. The DIT++ taxonomy for functional dialogue markup. In Proceedings EDAML 2009. 2009.
Bunt, H. C. and Girard, Y. Designing an open, multidimensional dialogue act taxonomy. In Proceedings of the 9th Workshop on the Semantics and Pragmatics of Dialogue (DIALOR 2005), pages 37–44. 2005.
Cade, W. L., Copeland, J. L., Person, N. K., and D’Mello, S. K. Dialogue modes in expert tutoring. In ITS ’08: Proceedings of the 9th international conference on Intelligent Tutoring Systems, pages 470–479. Springer-Verlag, Berlin, Heidelberg, 2008.
Carletta, J. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):pages 249–254, 1996.
Cawsey, A. J. A belief revision model of repair sequences in dialogue. New directions in intelligent tutoring systems, 1991.
Chi, M., Jordan, P., VanLehn, K., and Hall, M. Reinforcement learning-based feature selection for developing pedagogically effective tutorial dialogue tactics. In Proceedings EDM 2008, pages 258–265. 2008.
Chi, M., Jordan, P. W., VanLehn, K., and Litman, D. J. To elicit or to tell: Does it matter? In Proceedings of AIED 2009, pages 197–204. 2009.
Chi, M., VanLehn, K., and Litman, D. Do micro-level tutorial decisions matter: Applying
reinforcement learning to induce pedagogical tutorial tactics. In V. Aleven, J. Kay, and J. Mostow, editors, Proceedings of the ITS 2010. 2010.
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., and Glaser, R. Self-explanations:how students study and use examples in learning to solve problems. Cognitive Science, 13(2):pages 145–182, 1989.
Chi, M. T. H., De Leeum, N., Chiu, M.-h., and LaVancher, C. Eliciting Self-Explanations Improves Understanding. Cognitive Science, 18, 1994.
Chi, M. T. H. and Roy, M. How adaptive is an expert human tutor? In J. Kay and V. Aleven, editors, Proceedings of ITS 2010, pages 401–412. 2010.
Chi, M. T. H., Siler, S., Jeong, H., Yamauchi, T., and Hausman, R. G. Learning from human tutoring. Cognitive Science, 25:pages 471–533, 2001.
Choi, J. D. and Nicolov, N. K-best, Locally Pruned, Transition-based Dependency Parsing Using Robust Risk Minimization. In Recent Advances in Natural Language Processing, volume V, pages 205–216. John Benjamins, 2009.
Clark, A. and Popescu-Belis, A. Multi-level Dialogue Act Tags. In Proceedings of SIGDIAL 2004, pages 163–170. 2004.
Cohen, P. A., Kulik, J. A., and Kulik, C. L. C. Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19(2):pages 237–248, 1982.
Cohen, P. R. and Perrault, C. R. Elements of a plan-based theory of speech acts. Cognitive Science, 3:pages 177–212, 1979.
Cohen, W. W., Schapire, R. E., and Singer, Y. Learning to Order Things. In Advances in Neural Information Processing Systems 10 (NIPS 1998). 1998.
Cole, R., Vuuren, S. V., Pellom, B., Hacioglu, K., and Ma, J. Perceptive animated interfaces: First steps toward a new paradigm for human-computer interaction. In Proceedings of the IEEE, page 2003. 2003.
Collins, A. Teaching reasoning skills. In S. Chipman, J. Segal, and R. Glaser, editors, Thinking and Learning Skills, volume Vol. 2. Erlbaum, Hillsdale, NJ, 1985.
Collins, A. and Stevens, A. Goals and methods for inquiry teachers. Advances in Instructional Psychology, 2, 1982.
“Computing Research Association”. Grand Challenges in Information Systems (White
Paper). Technical report, Computing Research Association, 2003. Retrieved from
www.cra.org/uploads/documents/resources/rissues.
Core, M. Analyzing and predicting patterns of damsl utterance tags. In AAAI Spring Symposium on Applying Machine Learning to Discourse Processing. Stanford, CA, 1998.
Core, M. G. and Allen, J. F. Coding Dialogs with the DAMSL Annotation Scheme. In AAAI Fall Symposium, pages 28–35. 1997.
Core, M. G., Moore, J. D., and Zinn, C. The role of initiative in tutorial dialogue. In Proceedings of the European Chapter of the Association for Computational Linguistics. 2003.
127 Dligach, D. and Palmer, M. Novel semantic features for verb sense disambiguation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technolgies. 2008.
Engel, R., Kipp, M., and Klesen, M. Predicting Dialogue Acts in VERBMOBIL. Technical report,
Universit¨at des Saarlandes, 1995.
Ferguson, G. and Allen, J. F. TRIPS: An Integrated Intelligent Problem-Solving Assistant. In Proceedings of the 15th National Conference on AI (AAAI-98), pages 26–30. 1998.
Fox, B. A. The Human Tutorial Dialogue Project. Lawrence Erlbaum Associates, Inc., 1993. Geertzen, J. Dialogue act prediction using stochastic context-free grammar induction. In EACL 2009
Workshop on Computational Linguistic Aspects of Grammatical Inference, pages 7–15. 2009. Graesser, A. C., Chipman, P., Haynes, B., and Olney, A. AutoTutor: An intelligent tutoring system
with mixed-initiative dialogue. IEEE Transactions in Education, 48:pages 612–618, 2005. Graesser, A. C., Hu, X., Susarla, S., Harter, D., Person, N. K., Louwerse, M., Olde, B., and the
Tutoring Research Group. Autotutor: An intelligent tutor and conversational tutoring scaffold. In Proceedings for the 10th International Conference of Artificial Intelligence in Education, pages 47–49. San Antonio, TX, 2001.
Graesser, A. C., McNamara, D. S., Louwerse, M., and Cai, Z. Coh-metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36:pages 193–202, 2004.
Graesser, A. C. and Person, N. K. Question asking during tutoring. American Educational Research Journal, 31:pages 104–137, 1994.
Graesser, A. C., Person, N. K., and Magliano, J. P. Collaborative Dialogue Patterns in Naturalistic One-to-One Tutoring. Applied Cognitive Psychology, 9:pages 495–522, 1995.
Heilman, M. and Smith, N. A. Good Question! Statistical Ranking for Question Generation. In Proceedings of NAACL/HLT 2010. 2010a.
Heilman, M. and Smith, N. A. Rating Computer-Generated Questions with Mechanical Turk. In Proceedings of the NAACL/HLT workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk. 2010b.
Hirschberg, J. and Litman, D. Empiracal studies on the disambiguation of cue phrases.
Computational Linguistics, 19(3), 1993.
Jackson, G. T., Person, N. K., and Graesser, A. C. Adaptive Tutorial Dialogue in AutoTutor. In Proceedings of the Workshop on Dialog-based Intelligent Tutoring Systems. 2004.
Jekat, S., Klein, A., Maier, E., Maleck, I., Mast, M., and Quantz, J. J. Dialogue acts in verbmobil. Technical Report 65, BMBF, 1995.
Jeon, M. and Azevedo, R. Automatic analyses of cohesion and coherence in human tutorial dialogues during hypermedia: A comparison among mental model jumpers. In ITS ’08: Proceedings of the 9th international conference on Intelligent Tutoring Systems, pages 690–692. Springer-Verlag, Berlin, Heidelberg, 2008.
Joachims, T. Making large-Scale SVM Learning Practical. In Advances in Kernel Methods - Support Vector Learning. MIT-Press, 1999.
Jordan, P., Hall, B., Ringenberg, M., Cui, Y., and Ros´e, C. P. Tools for authoring a dialogue agent
that participates in learning studies. In Proceedings of AIED 2007, pages 43–50. 2007.
Jurafsky, D., Bates, R., Coccaro, N., Martin, R., Meteer, M., Ries, K., Shriberg, E., Stolcke, A., Taylor, P., and Ess-Dykema, C. V. Automatic detection of discourse structure for speech recogni- tion and understanding. In In Proceedings of the 1997 IEEE Workshop on Speech Recognition and Understanding. 1997.
Jurafsky, D., Elizabeth Shriberg, B. F., and Curl, T. Lexical, Prosodic, and Syntactic Cues for Dialog Acts. In Proceedings of ACL/COLING-98 Workshop on Discourse Relations and Discourse Markers, pages 114–120. 1998.
Katz, S., Allbritton, D., and Connelly, J. Going beyond the problem given: How human tutors use post-solution discussions to suppoert transfer. International Journal of Artificial Intelligence in Education (IJAIED), 2003.
Kendall, M. A New Measure of Rank Correlation. Biometrika, 30(1-2):pages 81–89, 1938.
Kim, J. H., Glass, M., Freedman, R., and Evens, M. W. Learning the Use of Discourse Markers in Tutorial Dialogue Learning the Use of Discourse Markers in Tutorial Dialogue. In Proceedings of the Cognitive Sciences Society. 2000.
Kipper, K., Dang, H. T., and Palmer, M. Class Based Construction of a Verb Lexicon. In Proceedings of the 17th National Conference on Artificial Intelligence AAAI-2000. Austin TX, 2000.
Kl¨uwer, T., Uszkoreit, H., and Xu, F. Using Syntactic and Semantic based Relations for Dialogue
Act Recognition. In Coling 2010: Poster Volume, pages 570–578. 2010.
Lan, K. C., Ho, K. S., Luk, P., Wing, R., and Leong, H. V. Dialogue act recognition using maximum entropy. Journal of the American Society for Information Science and Technology, 59(6):pages 859–874, 2008a. ISSN 1532-2890.
Lan, K. C., Ho, K. S., Pong Luk and, R. W., and Leong, H. V. Dialogue act recognition using maximum entropy. Journal of the American Society for Information Science and Technology, 59(6):pages 859–874, 2008b. ISSN 1532-2882.
Landauer, T., Foltz, P. W., and Laham, D. Introduction to latent semantic analysis. Discourse Processes, 25(259–284), 1998.
Lehnert, W. G. The Process of Question Answering. Erlbaum, Hillsdale, NJ, 1978.
Levenshtein, V. I. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 1965.
Levin, L., Ries, K., Thyme-Gobbel, A., and Lavie, A. Tagging of speech acts and dialogue games in spanish call home. In Proceedings of the Workshop Towards Standards and Tools for Discourse Tagging (ACL’99), pages 42–47. 1999.
Litman, D. and Forbes-Riley, K. Correlations between dialogue acts and learning in spoken tutoring dialogue. Natural Language Engineering, 12(2):pages 161–176, 2006.
129 Litman, D., Moore, J., Dzikovska, M. O., and Farrow, E. Using natural language processing to analyze tutorial dialogue corpora across domains and modalities. In Proceedings 14th International Conference on Artificial Intelligence in Education (AIED). 2009.
Litman, D., Ros´e, C., Forbes-Riley, K., VanLehn, K., Bhembe, D., and Silliman, S. Spoken versus
typed human and computer dialogue tutoring. In Proceedings of Human Language Technology: North American Chapter of the Association for Computational Linguistics HLT:NAACL. 2004. Litman, D. and Silliman, S. ITSPOKE: An Intelligent Tutoring Spoken Dialogue System. In
Companion Proceedings of the Human Language Technology Conference: 4th Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL). 2004. Louwerse, M. M. and Crossley, S. Dialog act classification using n-gram algorithms. In In Proceedings
of the Florida Artificial Intelligence Research Society International Conference (FLAIRS). AAAI Press., 2006a.
Louwerse, M. M. and Crossley, S. A. Dialog Act Classification Using N-Gram Algorithms. In FLAIRS. 2006b.
Makhoul, J., Kubala, F., Schwartz, R., and Weischedel, R. Performance measures for information extraction. In DARPA Broadcast News Workshop. 1999.
Mann, H. B. and Whitney, D. R. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Annals of Mathematical Statistics, 18(1):pages 50–60, 1947.
Mann, W. C. and Matthiesson, C. Rhetorical structure theory and text analysis. Technical report, Information Sciences Institute, California, US, 1989.
Mann, W. C. and Thompson, S. Rhetorical Structure Theory: Description and Construction of Text Structures. In Proceedings of the 3rd Internationa Workshop on Text Generation. 1986. Mann, W. C. and Thompson, S. A. Rhetorical structure theory: Toward a functional theory of text
organization. Text, 8(3):pages 243–281, 1988.
McCallum, A. K. MALLET: A Machine Learning for Language Toolkit.
http://mallet.cs.umass.edu, 2002.
McTear, M. Breakdown and repair in naturally occurring conversation and human-computer dialogue. Social action and artificial intelligence, pages 104–123, 1985.
Murphy, P. K., Wilkinson, I. A. G., Soter, A. O., and Hennessey, M. N. Examining the Effects of Classroom Discussion on Students’ Comprehension of Text: A Meta-Analysis. Journal of Educational Psychology, 101(3):pages 740–764, 2009.
Murray, R. C., VanLehn, K., and Mostow, J. Looking Ahead to Select Tutorial Actions: A Decision-
Theoretic Approach. International Journal of Artificial Intelligence in Education (IJAIED),
14:pages 235–278, 2004.
“National Academy of Engineering”. NAE Grand Challenges for Engineering.
Technical report, National Academy of Engineering, 2008. Retrieved from
Nielsen, R. D. Identifying Key Concepts for Question Generation. In Proceedings of the Workshop on the Question Generation Shared Task and Evaluation Challenge. Arlington, Virginia, 2008. Nielsen, R. D., Buckingham, J., Knoll, G., Marsh, B., and Palen, L. A Taxonomy of Questions for
Question Generation. In Proceedings of the Workshop on the Question Generation Shared Task and Evaluation Challenge. 2008.
Ogren, P. V., Wetzler, P. G., and Bethard, S. ClearTK: A UIMA toolkit for statistical natural language processing. In Towards Enhanced Interoperability for Large HLT Systems: UIMA for NLP workshop at Language Resources and Evaluation Conference (LREC). 2008.
Ohlsson, S., Di Eugenio, B., Chow, B., Fossati, D., Lu, X., and Kershaw, T. C. Beyond the code- and-count analysis of tutoring dialogues. In Proceedings of the 13th International Conference on AI in Education, pages 349–356. 2007.
Olney, A., Louwerse, M., Mathews, E., Marineau, J., Hite-Mitchell, H., and Graesser, A. Utterance Classification in AutoTutor. In Proceedings HLT-NAACL 03 BEA, pages 1–8. 2003.
Palmer, M., Dang, H. T., and Fellbaum, C. Making fine-grained and course grained distinctions both manually and automatically. Journal of Natural Language Engineering, 13(2):pages 137–163, 2006.
Pellom, B. L., Ward, W., and Pradhan, S. S. The cu communicator: an architecture for dialogue systems. In INTERSPEECH, pages 723–726. 2000.