INFORMACION GENERAL DEL ÁREA DE ESTUDIO:

The earlier sections in this chapter describe important parts of how CSTutor works. This section aims at providing a more integrated view of how CSTutor works internally, making use of those previously described parts.

Intelligent tutoring starts in CSTutor with the displaying of tasks for the student in a list (see earlier Figure 4.10 and Figure 4.11). The difficulty level of each task is defined by the teacher and is used to order the tasks in the list. Based on the student model, the tasks that are recommended to be completed by the student are displayed with the suffix “<recommended>”. The first recommended task in the list is displayed as the default selection. If desired, the student can change this default selection and select another task from the list.

The student then writes the code in VS for the selected task. When the student presses the button “Is My Code Correct” or “Give Me Help”, the process to check the correctness of the student code is initiated. The main process performed for both buttons is actually the same. The difference is that the process that is performed through the “Is My Code Correct” button supresses all help information, so that it becomes a deliberately-minimal “correct” or “incorrect” feedback.

The process to check the correctness of the student code starts by checking for the existence of a VS project and an open document (see section 5.1.2). If this checking succeeds, the next process is to check the existence of the main components required to build a program, such as the using directive, the namespace, the class,

and the main method (see section 5.1.3). It is also checked whether the student code is syntax error free or not (see section 5.1.1).

When the student code passes all the checks above, the next process performed is writing information to the Prolog knowledge base file. Firstly, the rules and actions are written to this file (These rules and actions are read from the database.) Then the constraints and the goal for the selected task are written to the file. The goal may be composed of several subgoals.

For example, consider a task where the student is asked to display the average of the best four out of five marks that are stored in the array. The total of the best four out five marks can be found by accumulating all five marks and subtracting the result by the lowest mark. This total can then be divided by four to get the average. To achieve this goal, the subgoals for this task are:

• There should be an array with its five elements initialized directly.

• There should be a variable that represents the lowest mark and this variable should be initialized with the value from the first element of the array.

• There should be a variable that represents the total mark and this variable should be initialized with the value from the first element of the array.

• There should be a loop that repeats four times and the repetition counter should start from 1 (to access the second element of the array), end at 4, with increment value 1. This loop is used for two purposes:

o To find the smallest mark in the array o To find the total mark in the array

• The value in the variable that represents the lowest mark should be the same as the smallest value in the array element (when the program completes).

• The value in the variable that represents the total mark should be the same as the total of all values in the array (when the program completes).

• There should be a variable that represents the final mark and its value should be the same as: (total mark – lowest mark) / 4 (when the program completes).

And the constraints for this task are:

• The following variables names must be used:

o studentMark: the array that is used to store five marks. o lowestMark: to store the lowest mark found in the array. o totalMark: to store the total of the marks.

o finalMark: to store the student final mark. o i: to be used as the loop counter.

• The student must use repetition to get the total mark and the smallest mark.

• The loop construct that must be used is “for” statement.

The goal and the subgoals are written to the Prolog knowledge base file as a rule for a predicate named “goal”. The rule that contains the goal and subgoals for this task (written in Prolog format) is:

goal:-

% The first five elements in the array must be initialized hasElement(VarIDArr_0, ArrID, 0),

hasElement(VarIDArr_1, ArrID, 1), hasElement(VarIDArr_2, ArrID, 2), hasElement(VarIDArr_3, ArrID, 3), hasElement(VarIDArr_4, ArrID, 4),

% The 6th element (array index 5) should not be initialized \+hasElement(VarIDArr_0, ArrID, 5).

% the variable that represent the lowest mark should be % initialized with the value from the first

% element of the array, and it should not be assigned inside loop isArrElementOf(VarIDArr_0, ArrID, 0),

hasValue(VarIDArr_0, ValArr_0),

hasInitialValue(VarIDLowestMark, ValArr_0), \+hasInitialValueAt(VarIDLowestMark, ForID),

% The variable that represent the total mark should be % initialized with the value from the first

% element of the array, and it should not be assigned inside loop hasInitialValue(VarIDTotalMark, ValArr_0),

\+hasInitialValueAt(VarIDTotalMark, ForID),

% There should be a for-loop that repeats four times and its % counter should be named i, start from 1, end at 4, incr 1 hasForVariable(ForID, VarIDI),

hasForStartValue(ForID, 1), hasForEndValue(ForID, 4), hasForIncrValue(ForID, 1),

% The lowest mark should have the same value as the array % element if the value in the array element is lower than the % lowest value. This process should be performed inside the loop. hasValue(VarIDLowestMark, ValLowestMark),

isArrElementOf(VarIDArr_i, ArrID, i), hasInitialValue(VarIDArr_i, InitValArr_i), hasValue(ValIDArr_i, ValArr_i),

hasValue(VarIDLowestMark, ValArr_i, [lt(InitValArr_i, ValLowestMark)]), hasVarValueAt(VarIDLowestMark, ForID),

% The total mark should be the same as the initial value of the % total mark plus the value in the array element.

% This process should be performed inside the loop. hasInitialValue(VarIDTotalMark, InitValTotalMark), hasValue(VarIDTotalMark, InitValTotalMark + ValArr_i),

hasVarValueAt(VarIDTotalMark, ForID),

% The final mark should be the same as the calculation form % the following formula: (Total mark – lowest mark) / 4 % and the process should be performed outside the loop. hasValue(VarIDTotalMark, ValTotalMark),

hasValue(VarIDLowestMark, ValLowestMark),

hasValue(VarIDFinalMark, (ValTotalMark-ValLowestMark)/4), \+hasVarValueAt(VarIDFinalMark, ForID),

% The value of the variable final mark should be printed % and it should not be printed inside the loop.

printed(VarIDFinalMark, ValFinalMark), \+printedAt(VarIDFinalMark, ForID).

On the other hand, the constraints for the task are written to the Prolog knowledge base file as a rule for a predicate named “constraint”. The rule that contains the constraints for this task is:

constraint:-

% Constraints regarding the variable names exist(ArrID, studentMark),

exist(VarIDLowestMark, lowestMark), exist(VarIDTotalMark, totalMark), % The repetition should use “for” loop % with variable named “i” as the loop counter forExist(ForID),

exist(VarIDI, i),

hasForVariable(ForID, VarIDI)

The last items added to the Prolog knowledge base file are the predicates extracted from the student’s code as well as the actions activated. For example, let’s say that the student writes the following code in his/her attempt to answer to the task given above. The predicates created and the actions activated from this code can be

seen in Appendix E: Predicates Created and Actions Activated from a Particular Student Program.

static void Main(string[] args) {

double[] studentMarks = { 10, 20, 30, 40 }; double lowestMark = 0;

double totalMark = studentMarks[0];

for (int i = 1; i < studentMarks.Count(); i++) {

if (studentMarks[i] > lowestMark) lowestMark = studentMarks[i];

totalMark = totalMark + studentMarks[i]; }

double final = (totalMark - lowestMark) /4;

Console.WriteLine("The student final mark is: {0}", final); }

There are five errors in the code above. One is a constraint error and the other four are logical errors. The constraint error comes from the use of the variable name “final” – instead of “finalMark” – as the name of the variable that is used to store the final mark of the student. The four logical errors in the code above are:

• The array is not initialized with five values:

The student code: double[] studentMarks = { 10, 20, 30, 40 };

The correct code: double[] studentMarks = { 10, 20, 30, 40, 50 };

• The variable “lowestMark” is not initialized with the first element of the array:

The student code: double lowestMark = 0;

• The loop counter does not end with four. The use of “<” in the “i < studentMarks.Count()” will create a predicate that states that the loop counter ends with one number less than the value in the “studentMarks.Count()”. Because the student only initializes four elements in the array then the statement “i < studentMarks.Count()” will yield three as the end value of the loop counter. This error will disappear automatically when the student initialize five elements in the array.

• The condition operator that is used to change the value in the lowest mark is incorrect.

The student code: if (studentMarks[i] > lowestMark)

The correct code: if (studentMarks[i] < lowestMark)

When all the above components have been written to the Prolog knowledge base file, it is loaded back into memory. This allows CSTutor to run the Prolog inference engine – to check if the goal is satisfied by the existing predicates from the student’s code or not (see section 5.3). This can be performed in CSTutor by calling the predicate named “goal” above.

If the goal is satisfied, then CSTutor uses the Prolog inference engine to check whether there are any constraints that are not satisfied by the student code. Similar process to the goal checking can be performed to do the constraint checking.

However, if the goal is not satisfied, more detailed inferencing is performed to find the subgoal that cannot be satisfied. This can be done by excluding some parts

of the subgoals and perform the inferencing process again. If the inferencing is success then we know that the student code cannot satisfy the excluded subgoal.

For the student code in the example above, the predicates generated from the student code as well as the activated actions (see Appendix E) cannot satisfy the following subgoals:

• hasElement(VarIDArr_4, ArrID, 4)

When this subgoal cannot be satisfied, it shows that there is no code to initialize array element at index 4 (the fifth array element).

• hasInitialValue(VarIDLowestMark, ValArr_0)

When this subgoal cannot be satisfied, it shows that there is no code to initialize the variable that represents the lowest mark with the value of the first element of the array.

• hasForEndValue(ForID, 4)

When this subgoal cannot be satisfied, it shows that the end value of the loop counter is not four.

• hasValue(VarIDLowestMark, ValArr_i, [lt(InitValArr_i, ValLowestMark)])

When this subgoal cannot be satisfied, it shows that the value in the variable lowest mark is not assigned with the value of the array element at index i, with the correct condition.

The help, that is related to the subgoal or constraint that cannot be satisfied, is then displayed to help the student. More information about the help is described in section 4.4. Some examples of the first level of help, for the corresponding logical errors above, are:

• The array should be initialized with exactly five arbitrary numbers (no more and no less than five numbers).

• The variable "lowestMark" should be initialized with the mark from the array element at index 0.

• The conditional part of your "for" statement should be modified so the counter variable can have the values: 1, 2, 3, 4.

• There should be an “if" and assignment statement inside the loop that enables variable "lowestMark" to have the same value as the smallest value in the array element.

When the student code satisfies the goal and the constraints, the next process is to check if the student code is written efficiently or not. It is possible that the student may use some extra lines of code that are not necessary to solve the task. Therefore this next process is to find any unnecessary items in the student code (see section 4.4.5 and the last part of section 5.3)

The student code is regarded as being correct when the code is capable of solving the required task and any unnecessary lines of code have been removed. When this happens, the probabilities in the Bayesian Network student model are updated based on the new evidence (see section 5.4).

Based on those updated value in Bayesian Network, CSTutor updates the list of the recommended tasks for the student. The list of these tasks is then displayed and the process is restarted again.

5.7

Summary

This chapter discussed the implementation of CSTutor. In the first section, the chapter discussed how CSTutor can be integrated into Visual Studio (VS). It then continued with a discussion of how to parse the student code entered into VS, inferencing about the code, and using the results to update the model of the student knowledge.

6

EVALUATION

The first section in this chapter gives a description of how the evaluation was performed, for both the first and second releases of CSTutor. Because we had a sufficient number of respondents for the second release only, this chapter reports the results from that second release.

Section 6.2 evaluates the effectiveness of CSTutor across all respondents. The degrees of correlation between each of three variables (i.e. the length of time spent in CSTutor, the number of help requests, and the number of tasks completed) with the student increase in score are evaluated. In section 6.3, I refined the evaluation by dividing the respondents into two categories: the respondents who had any background in programming; and the respondents who had no background in programming. This was done to see if either category got more benefit from CSTutor.

Section 6.4 and its subsections describe the evaluation that was performed to see if CSTutor attracts student interest in learning to program. There are three issues that were used to measure if CSTutor can do this, that is its ease-of-use, its ability to help the student learn to program, and its learning approach attractiveness. The final section, 6.5, describes student recommendations on possible extensions to CSTutor.