G LADYS G ORGA 1 , C ECILIA S ANZ 1 , C RISTINA M ADOZ

1 Instituto de Investigación en Informática LIDI – School of Computer Science – National University of La Plata. 50 y 120, La Plata, Buenos Aires, Argentina

{ggorga, csanz, cmadoz}@lidi.info.unlp.edu.ar

Abstract. A proposal of a model to assess ICT-mediated educational processes is presented. The education quality background is analyzed, particularly in relation to distance education. The model proposed is described, and then, its strengths and weaknesses are discussed, based on its application to some educational experiences. Finally, the results are analyzed and some conclusions are presented.

Keywords: quality, education, ICTs, assessment model.

1. Introduction

The evolution of our society is currently supported, among other aspects, by the development of new Information and Communication Technologies, driven by advances in computer science and telematics, that affect every aspect of life.

Higher Education Institutions (IES) are not exempt from this situation, and they must take advantage of these resources to their maximum potential, so that they can generate innovative proposals that help them achieve their goals.

In this sense, IES will have to set clear quality standards and criteria to achieve their goals taking into account, mainly, the critical role they play in the current knowledge society.

Nowadays, there is a vast rather than intense proliferation and production of distance education programs, but there is no associated reflexive or deep assessment process. To do this, criteria such as filters should be conceived and established to check if these educational proposals are reliable and offer quality education [1]

To determine the set of quality criteria to be used by the IES, either for on- site or distance programs, the specific contexts in which these activities take place should be analyzed, also including, but not limited to, their components, critical issues, and the actors and their characteristics, needs and demands. All these aspects related to the goals sought will determine the roads that the educational project will follow within a quality and continued improvement context.

2. Educational Quality Background

There is no consensus regarding the concept of quality, particularly in higher education. Many authors state that the concept of quality is relative. However, the current trend is to consider that the University should analyze its quality in relation to its nature and role, its purposes and goals. Thus, quality includes the relevance of university goals and the extent to which these are achieved [2]

García Aretio states that “…the various power or thought groups can consider that quality has been achieved based on the starting premise and the intended goals.” On the other hand, he mentions that a large number of the various proposals and models are rooted on the European Quality Model (TQM), since they focus their interest mainly on customer satisfaction, which is based on continued improvement, process measurement and maximum attention to processes, team work, and individual responsibilities [3].

However, there are other authors, such as Barberá et al [4], who state that “…the first mistake is copying, almost to the T, the business quality models and apply them to education; neither form nor substance can be transferred to educational environments as it is being done.”

As mentioned earlier, there are different opinions on how to consider quality and its assessment.

In the case of ICT-mediated educational processes, there is also a diversity of aspects to consider when defining quality criteria and how to achieve it.

In [5], several renowned university members with experience in distance education programs analyze various strategic components that affect quality. For instance, at Monash University in Australia, digital educational materials are considered to be a key aspect, as well as the appropriate training of both teachers and students to be able to use these materials. At the University of Texas, the issue of quality is considered to be based on three main cornerstones – the quality of educational materials, presentation, and student attention; whereas at the US Open University, quality is viewed from the following perspectives: content and design of educational materials, teacher- student communication and with the institution in general, and the use of appropriate technology to support these processes.

In [6], continuous assessment processes are considered as a tool to ensure education quality. For ICT-based learning to be successful, its organization, program consistency including teacher and student training, course design, the appropriate incorporation of emerging technologies, and an effective assessment of the educational process have to be considered.

García Aretio [7] proposes a model that he describes as being inclusive and that allows developing and monitoring total quality, and that is linked to the context, goals, inputs, processes, results, and improvements. The criteria to be considered in this model are: Functionality (consistency between goals and educational results in relation to needs and contextual reality), Efficacy or effectiveness (consistency between educational goals and the results obtained), Efficiency (consistency between inputs, processes, means and achievements or educational results), Availability (consistency between the

XVIIARGENTINE CONGRESS OF COMPUTER SCIENCE 110

goals and objectives proposed by the institution and human, material and economic resources that are available to start the process), Information

(consistency between the results obtained and the improvement proposals that are offered in the corresponding report), Innovation (consistency between the specific catalogue of improvements required – developing strengths and correcting weaknesses – to achieve goals and the decision to innovate and review goals, inputs and processes).

In the following section, the proposal designed by the authors of this paper is presented. The goal of this model is assessing the quality of a distance educational system, program or process, and it incorporates some of the elements included on the previous list. This model can also be adapted to hybrid educational processes, where on-site meetings are combined with distance work and learning strategies.

3. ECALEAD - Educational Model to Assess the Quality of

Distance Processes

ECALEAD is a quality assessment model for distance educational systems or hybrid models. It is based on a layered assessment that starts with a first layer that analyzes general aspects, and ends with a third layer that applies certain measures to analyze the criteria and indicators defined in the two previous layers to assess a specific context. As layer depth increases, they become more specific in relation to aspects typical of ICT-mediated education. The model can be adapted based on the specific purpose of the assessment, context, and the specific needs in each case.

To better explain each of the layers, some of the components that are part of an educational system have been summarized (Figure 1). Thus, the educational institution is inserted in a certain socio-cultural, political and economical context that has certain demands and needs. There are indicators that the IES have into account to learn what the context requires or the direction in which they are evolving.

Fig. 1. System components to be taken into account for the quality assessment

Even though students are part of this context we refer to, they have been assigned a central role as component in the system, since they have specific interests in relation to the institution, they are part of it, and expect certain actions or results from the IES. They can provide information about their needs and expectations. Similarly, the IES have an effect on students (based on their goals/objectives). Students experience the institution and can give their opinion about it. External Assessment Agents are another essential component in the system; they gather information about the institution and give their specific feedback to achieve evolution in educational quality.

In order to operate, the institution defines its objectives/goals and the processes to achieve them, it has various types of resources (which are organized and distributed based on certain strategies), it pursues results that are assessed by means of internal and/or external process and, ultimately, it determines the aspects that can be improved. We understand that, if the institution achieves functionality, availability, efficacy, efficiency, information and innovation (based on Aretio's definitions), it has progressed significantly towards ensuring the quality of the system.

In the following paragraphs, each of the layers in the model proposed is described.

Layer 1: In this layer, the general criteria to be considered in the model