As Miao points out in the chapter above, self assessment has by far the highest priority in the literature surveyed. This is line with the high priority given in this final reporting period to the completion of Task Force 7, responsible for integrating QTI tests into the Personal Competence Manager (situated in task 6.1) This involved the adaptation and extension of the WP6 infrastructure for QTI so that it could provide competence self-assessment tests within the Personal Competence Manager.
In terms of the analysis in the published chapter above we note that self assessment is a relatively simple form of assessment, and that it is often used as a preparation for other types of assessment, for example 360 degrees. It was therefore sensible to use similarly simple methods to support self assessment, making use of the additional flexibility which they offer. This was the logic of implementing self assessment as portlets within the Personal Competence Manager.
4.2.1 Take QTI test portlets
Users of the Personal Competence Manager have to provide information about themselves and the competences which they have achieved before they can be given guidance by the system about their best path towards achieving their competence development goals. They can do this by indicating their level of competence on a scale. In some cases learners are well aware of their competence levels, or may have taken prior courses in the system or have qualifications which clarify this, but in others they may well overestimate or underestimate this substantially. Consequently there is a need for a simple self-test functionality which can assist them in assessing their level of competence. The QTI editor in ReCourse could have been used to create such tests, but this would have required the use of a desktop application, to create and upload the tests, and then a publishing process on the server. A simpler solution was required, and so portlets were created with which teachers and others could create simple self-assessment tests to be made available to members of the community who sought to achieve a particular competence.
In this way simple assessments can be provided within the Personal Competence Manager, in the same way as simple learning activities can be provided within the system. These can be authored simply and conveniently as required. which could create and deliver simple tests from within the Personal Competence Manager, ensuring that the user can make an informed judgment about the competence levels which they should ascribe to themselves. Using the Learning Design Toolkit more sophisticated assessments can be developed using ReCourse, uploaded to the PCM, provisioned for groups of learners, and made available in Learning Paths.
This work was carried out within work package 6, but the of authoring and testing functionality is an integral part of the Personal Competence Manager, and is delivered as part of that system in WP3.
Extensive work was required to ready the NewAPIS runtime system for use with the PCM. This involved decoupling NewAPIS from the LD runtime systems with which it had previously been bundled. A “miniserver” was developed which can list and serve xml files stored in a folder, publish tests, and can handle multiple connections
and distinguishes users by their IP. All the user management and the database was shared with Liferay. An adaptation of all the tablenames and sql syntax was done in order to make the package compatible with mysql. An interface was designed in order to handle the output of tests, and the input interface was also modified to deliver the XMLs of the QTI questionnaires.
In related work a CSS was created for NewAPIS so that the look and feel of the output of newAPIS could closely match that of the Liferay based PCM. Accordingly the render functions of newAPIS package were cleaned out of colours and layout information. Other changes were made to the distribution of content on the screen, and new tables and subsections added for distributing graphical information. 4.3 Assessment templates
In the chapter included above in section 4.1, the characteristics of four forms of assessment which are currently receiving substantial interest were systematically analysed from the perspective of process orchestration technologies. This was done by taking the first ten articles found in both databases, most of which provided examples of assessment types. These served as the input for the descriptions of the four forms of assessment.
This was done in order to
a) confirm that the Learning Design Toolkit would be able to express these forms of assessment
b) in order to prepare for exemplifying them with example UOLs c) identify specific modelling problems and propose solutions.
This work informed the development of templates for the Learning Design Toolkit, and the following templates have been prepared:
360 degree Peer assessment Formative assessment Summative assessment
Adaptive learning based on QTI test Learning Outcomes negotiation.
More details about these templates, and others included in the ReCourse Learning Design Editor, are available in Appendix
The templates are available for download from DSpace at
http://hdl.handle.net/1820/2283
Additionally an example was produced which models a real-world performance assessment, in which three assessors use the same assessment form to evaluate three performances (playing violin) of a candidate. The final score is a weighted mean of all evaluations11.
11