Should we consider Efficiency and Constancy for Adaptation in Intelligent Tutoring Systems?
Presenter: Pedro Manuel Moreno-Marcos
Authors: Pedro Manuel Moreno-Marcos, Dánae Martínez de la Torre, Gabriel González Castro, Pedro J. Muñoz-Merino and Carlos Delgado Kloos
16th International Conference on Intelligent Tutoring Systems 10th June 2020
INDEX
1. INTRODUCTION 2. METHODOLOGY 3. RESULTS
4. CONCLUSIONS AND FUTURE WORK
2
INTRODUCTION
• Adaptation in Intelligent Tutoring Systems (ITSs) – Skill modelling
– Students’ behaviors
• Emotions
• Self-regulated learning
• Focus on two indicators:
– Efficiency – Constancy
3
OBJECTIVES
• O1: Prevalence of efficiency / constancy
• O2: Evolution of efficiency / constancy over time
• O3: Relationship between efficiency / constancy and other variables (e.g., performance)
4
INDEX
1. INTRODUCTION 2. METHODOLOGY 3. RESULTS
4. CONCLUSIONS AND FUTURE WORK
5
EDUCATIONAL CONTEXT
• Data from K-12 students (n = 10,171)
• Interactions with exercises:
– Student identifier – Activity identifier – Timestamp
– Time spent – Grade
6
EFFICIENCY
• Points achieved per unit of time
• : Best grade in exercise
• Time spent in exercise
• Number of attempted exercises
•
7
CONSTANCY
• Inverse of the standard deviation (SD) of daily time spent doing activities
• Time spent in day
• Active days
•
8
INDEX
1. INTRODUCTION 2. METHODOLOGY 3. RESULTS
4. CONCLUSIONS AND FUTURE WORK
9
PREVALENCE
• Ranges for both variables are quite dispersed
10
CLUSTER ANALYSIS: EFFICIENCY
• Six profiles are identified using hierarchical clustering
11
CLUSTER ANALYSIS: CONSTANCY
• Active days and time spent are related to constancy
-1• Students who spend more time → less constant
12
TEMPORAL EVOLUTION
• Little variations of efficiency / constancy over time
13
RELATIONSHIP WITH OTHER VARIABLES
• Analyze the relationship between efficiency / constancy and the following variables:
– active_days – avg_att
– avg_grade – avg_time_ex – naccess
– nactivities – time_spent – total_grade
14
CORRELATION ANALYSIS
• Efficiency /
Constancy are related to:
– avg_time_ex – time_spent
• No relationship with grades
15
RELATIONSHIP WITH TOTAL GRADES
• No strong relationship
between efficiency / constancy and grades
16
REGRESSION ANALYSIS
• Regression models to predict grade:
– Efficiency and constancy: R2 = 0.11
– time_spent and avg_time_ex: R2 = 0.14 – nactivities: R2 = 0.81
• Efficiency and constancy add a different dimension from grades
17
INDEX
1. INTRODUCTION 2. METHODOLOGY 3. RESULTS
4. CONCLUSIONS AND FUTURE WORK
18
CONCLUSIONS
• 1) Big variation of students’ efficiency
• 2) Possibility to cluster students based on efficiency
• 3) Big variation of students’ constancy
• 4) Students who spend more time are less constant
• 5) Possibility to detect students with low engagement and constancy
• 6) More efficient students tend to be more constant
• 7) Efficiency and constancy are not related to grades
• 8) Adaptation based on efficiency and constancy can make sense
19
LIMITATIONS AND FUTURE WORK
• LIMITATIONS
– Different course contexts – Definition of the variables
• FUTURE WORK
– Analysis in courses with known methodology
– Carry out adaptation and interventions based on efficiency/constancy
20
21