ADVANCES IN TECHNOLOGY-BASED EDUCATION: TOWARDS A KNOWLEDGE
BASED SOCIETY
Proceedings of the II International Conference on Multimedia and Information & Communication Technologies in Education
m-ICTE2003
www.formatex.org/micte2003/micte2003.htm Badajoz, Spain, December 3-6
th2003
Edited by A.Méndez-Vilas J.A.Mesa González
J.Mesa González
ISBN Volume I, (Pages 1-658): 84-96212-10-6 ISBN Volume II, (Pages 659-1335): 84-96212-11-4 ISBN Volume III, (Pages 1336-2026): 84-96212-12-2
Published by:
JUNTA DE EXTREMADURA,
Consejería de Educación, Ciencia y Tecnología (Badajoz, Spain), 2003
Printed in Spain
CONTINUOUS ASSESSMENT IN ONLINE-TEACHING: THE CASE OF AN OBJECT-ORIENTED PROGRAMMING COURSE
MIGUEL-ÁNGEL SICILIA, ESTEVE MARINÉ, JUAN-MANUEL DODERO, ELENA RODRÍGUEZ, Universitat Oberta de Catalunya, SPAIN.
E-mail: {, msicilia, emarine, jmdodero, mrodriguezgo, }@uoc.edu
Continuous assessment approaches provide a number of benefits, including the availability of more accurate data reaching the tutor early enough to modify instruction. In the context of virtual universities, continuous assessment becomes a key tool to reduce student dropout and improve the student-centered learning process. In this paper, a concrete experience in designing, implementing and evaluating a continuous assessment program in the context of a CS1 course is described. Both quantitative and qualitative data about four semesters are presented, along with the rationale for the evolution of the assessment’s design, and a discussion of the most important lessons learned.
KEYWORDS: continuous assessment, virtual universities.
1 Introduction and Context
The Universitat Oberta de Catalunya (UOC)1 is a young university created by initiative of the Generalitat Government of Catalonia, aimed at offering high qualitity distance teaching through the application of an innovative pedagogical system and the use of interactive multimedia technologies. The UOC began its activity in 1995 (Computer Science and Multimedia studies began in 1997), complementing the Catalan university system, putting higher education within the reach of social sectors which prefer to take advantage of distance education, due to professional activity, place of residence, age or other personal reasons. The UOC expanded its initial offering to regulated studies in Spanish in September 2000 thanks to an alliance with the Planeta Publishing Group, giving rise to the Planeta-UOC initiative. Computer Science and Multimedia studies in Spanish began in 2001, offering two three-year degrees: “Technical Engineering in Computer Management”
(TECM) and “Technical Engineering in Computer Systems” (TECS).
The UOC centers its educational system on the following concepts: learning materials, the Virtual Campus, and continuous assessment. Learning materials include hardcopy and multimedia materials, as well as software in the case of Computer Science and Multimedia studies. The Virtual Campus allows overcoming time and geographical barriers, facilitating communication among all members of the university community through the use of the Internet. Thanks to the Virtual Campus, from an academic perspective, students receive personalized attention, work jointly with their peers and tutors , and have access to other university services (for example, the Virtual Library). Finally, continuous assessment is the pedagogical model fostered by the UOC (or better, a combination of continuous assessment and evaluation according to [5]). It allows students to face with knowledge in a particular way, forcing them to solve problems, to search for more information and to work in a cooperative way. Through continuous assessment and with the help of their tutors, students formatively evaluate their learning. It also plays a central role in the student’s formal evaluation process. It should be noted also that continuous assessment is provided as an alternative, so students that cannot keep up with it or do not pass it still have the option of a conventional final evaluation exam. In some cases, the assessments obtained through continuous assessment are confirmed by a ‘validation test’, to guarantee the final learning outcome. Tutors are the experts of the subjects where students are enrolled. Their main functions include to guide and encourage, and to evaluate the progress made by students in the learning process of a specific subject. Typically, an academic coordinator guides the activity of a group of tutors sharing the same subject. In addition to the just described elements, Computer Science and Multimedia studies consider an additional evaluation element:
“Practice” activities that include the development of software artifacts. When a subject considers such activity, it becomes a mandatory requirement to pass the subject. The student population is mainly comprised by professionals aged 25 or above with part-time dedication to the studies, and they are grouped in “virtual classrooms” of 70 or below with a tutor assigned.
In this paper we report and discuss the results of a concrete subject of the UOC Computer Science and Multimedia studies titled Programming Foundations II (PF II), which is a half-year 7.5 credits compulsory subject both for TECM and TECS (essentially, the CS1 PL6 core topic according to ACM/IEEE curriculum guidelines [1]). PF II focuses on the comprehension of the central concepts of the Object-Oriented (OO) paradigm and the acquisition of the required knowledge and abilities to develop OO programs. Currently, the
1 http://www.uoc.edu/web/eng/index.html
pedagogical design of PF II follows a practical approach that includes modeling problems – using the UML notation, see [2] – and its later implementation in the Java programming language [3]. PF II is the continuation of Programming Foundations I where the basic concepts of structured programming are introduced (so that an imperative-first approach is taken according to [1]). PF II has its continuation in other subjects including Data Structures, Software Development Techniques and Software Engineering.
The initial continuous assessment design of PF II has been revised along the time, and relatively stable learning outcomes have been observed, using a sequencing approach that parallels modeling and programming practice with an increasing level of complexity. Here we discuss the strengths and weaknesses of that scheme in the light of historical data. The rest of this paper is structured as follows. Section 2 describes the original motivation of the continuous assessment design in PF II. Section 3 provides quantitative and qualitative data about the first four semesters of the application of such design. Finally, concluding remarks are provided in Section 4.
2 Motivation and Rationale for Continuous Assessment
The UOC pedagogical model has been implemented in PF II as follows. Learning materials include hardcopy materials (OO foundations and Java manuals) and software material (Java development environment and the Visio 2002 Professional UML-drawing tool). The interaction is organized around two kinds of virtual classrooms: theory and laboratory, with their corresponding tutors. The theory classroom objective is to guide the study of both theoretical and practical aspects of the contents, complementing learning materials, while the laboratory classroom is intended to solve software installation problems and to guide programming abilities’
learning. Continuous assessment includes several differentiated activities (called in the context of the university Continuous Evaluation Test, CET). In addition, a mandatory “Practice” activity (consisting on consists of two parts: object modeling and implementation) is developed in groups of two students. Table 1 summarizes the objectives and sequencing of current CETs and Practice activities (an initial CET called CET-0 is used to motivate students to engage in continuous assessment and to advance software material installation, but it does not impact in the final evaluation). It also provides the approximate weight of each activity along with its approximate deadline (in a frame time of about sixteen weeks of activity). Obviously, the assessment activities design is guided by the learning objectives of the subject, which have been presented in the previous section.
It is important to note that the two first semesters, CET 1 and CET 2 were merged into a unique CET, CET 1.
Therefore, PF II continuous assessment included only three CETs.
Table 1. Description of Assessment Activities
Activity Importance Schedule Main objectives Typical Activities CET-1 10-20% 6th week Essential syntactical aspects of
Java and objects.
Development of small Java programs (similar to those in PF I), short practical questions and exercises involving error correction.
CET-2 10-15% 7th week Essential OO modeling concepts (classes and associations)
Small modeling cases.
CET-3 60% 11th week OO programming abilities. Implementation of cases proposed in CET-2 (or similar).
CET-4 10% 15th week Advanced object concepts and
its Java implementation. Concrete and short design problems (about inheritance, interfaces exceptions, class services, etc), with an emphasis on justifying solutions.
Practice-1 20% 9th week OO modeling concepts for a relatively complex situation.
Justified static OO model for the case study to be implemented in Practice- 2 (or a superset of it).
Practice-2 80% 14th week OO programming abilities. Implementation related to the model of Practice-1.
As showed in Table 1, programming and OO notions are introduced progressively and simultaneously.
This enables an early acquisition of basic programming and unit-testing abilities (CET-1) along with an initial contact with modeling (CET-2) that are expanded in subsequent activities. The limited credit allotted to these two activities make possible some compensation with a second stage that comprises the work on CET-3 (programming) and Practice-1 (modeling). The last stage comprises work on advanced elements (CET-4), and on a relatively complex programming assignment (Practice-2), intended to produce a situated learning scenario [7] in which the Lab’s tutor scaffolds students through several phases of work breakdown. The sequencing of
the contents and classroom interactions (practices, suggested readings, discussions, etc.) are sequenced to guide the students to the concepts and techniques required for the activities in Table 1 at a constant pace. Previous experiences had shown the effectiveness of introducing partial continuous assessment techniques for teaching OO programming [4] in non-virtual settings, pointing to the importance of instructor-student interaction. In our virtual setting, this interaction takes place after assigning the grades for the given CET, as student-to-student or tutor-initiated discussions around a proposed solution for the activity. The quick turnaround of grading enhances the post-activity interactions, resulting in improved opportunities for discussion, just as reported in [6].
3 Analysis and Evolution of the Continuous Assessment Approach
The overall outcomes of the continuous assessment approach are provided in Figure 1. It should be noted that
‘participation’ refers to students that follow the activities till the end of the semester. The increase in continuous assessment participation rates in the last semester may be due to slight changes in the difficulty and organization of activities, in the light of student feedback.
0,0%
5,0%
10,0%
15,0%
20,0%
25,0%
30,0%
35,0%
40,0%
45,0%
2001-1 2001-2 2002-1 2002-2 Semester
Participation rate Fail rate/Participation Fail rate/Registration
Figure 1. Overall outcomes of the first four semesters of PF II
Figure 1 depicts the low failure rates for students engaged in continuous evaluation (5-10%), and also the relatively high participation rates in continuous assessment. These facts contrast with the (overall) low rates of success in the conventional final exam, due to the fact that less than 5% of students engage in this option. This clearly indicates a preference of students for continuous assessment, consistent with studies conducted by the UOC at a global level that indicate a very positive satisfaction with such methodology.
Examining in detail abandon points in the schedule described in the precious section, it deserves a mention the fact that a very small proportion of students that have passed CET-3 fail or abandon Practice-2. This indicates that CET-3 clearly helps students engaged in continuous assessment to master the final assignment (that is mandatory for all the students to pass the subject). In general terms, participation rates decrease from an initial engagement of around a 50-60% to a 30-40% in CET-3 (or CET-2 in the case of the first two semesters).
Figure 2 and 3 show these tendencies along with fail rates for each of the CETs. The first three semesters are similar to that respect, but the last semester experienced lower failure rates in the first two CETs, possibly due to changes in their level of difficulty.
0%
10%
20%
30%
40%
50%
60%
70%
PEC 1 PEC 2 PEC 3
Continuing Assessment Test
Participation rate
2001-1 Participation rate 2001-2 Participation rate 2001-1 Fail
rate/Participation 2001-2 Fail rate/participation
Figure 2. Participation and fail rates of PF II semesters 2001-1 and 2001-2
0%
10%
20%
30%
40%
50%
60%
70%
PEC 1 PEC 2 PEC 3 PEC 4
Continuing Assessment Test
Participation rate
2002-1 Participation rate 2002-2 Participation rate 2002-1 Fail
rate/Participation 2002-2 Fail rate/Participation
Figure 3. Participation and fail rates of PF II semesters 2002-1 and 2002-2
The continuous assessment scheme just described has proven useful to obtain stable learning outcomes for the sub-population of students that early engage in virtual classroom activities and are able of increasing the effort spent in the activities at the second stage of the semester. Student feedback point to the difficulty of maintaining a sustained level of effort in the activities due to the overload of assignments from different subjects, but this problem is systemic to distance education, especially whenever the student population is mainly comprised by professionals that course the degree as a second, part-time activity.
4 Conclusions
A continuous assessment design for distance learning of OO fundamental modeling and programming concepts has been described. The method has resulted in stable success outcomes for students that early engage in assessment activities. Minor changes in the level of difficulty and orientation of the activities have resulted in small changes in the overall outcome.
5 References
1. Joint IEEE-ACM Task Force on Computing Curricula, Computing Curricula 2001 Computer Science, Final Report, December 15, 2001.
2. Arnold, K., and Gosling, J. The Java Programming Language, 2nd ed. Addison-Wesley, Reading, MA, USA, 1998.
3. Booch, G., Rumbaugh, J. and Jacobson, I. The Unified Modeling Language User Guide. Addison- Wesley, Reading, MA, USA, 1999.
4. Duke, R., Salzman, E., Burmeister, J., Poon, J. and Murray, L. Teaching programming to beginners - choosing the language is just the first step. In Proceedings of the Australasian conference on Computing education, ACM Press (2000), pp. 79–86.
5. Parker, P.E., Fleming, P.D., Beyerlein, S., Apple, D.K. and Krumsieg, K. Differentiating assessment from evaluation as a continuous improvement tool. In Proceedings of the Frontiers in Education Conference, IEEE Press (2001).
6. Carswell, L. The “Virtual University”: toward an Internet paradigm? ACM SIGCSE Bulletin 30(3) (1998) pp. 46–50.
7. Lave, J., & Wenger, E. Situated Learning: Legitimate Peripheral Participation. Cambridge, UK:
Cambridge University Press (1990).