In formative assessment—conducted on a continuous, ongoing basis throughout the course—the focus is on evaluating the learners’ progress and development and providing targeted feedback and suggestions for improvement. It can also reveal important information to the trainer about how effective the training has been thus far, either collectively or individually, and where adjustments may be needed. For example:
A pre-course assessment is conducted before training begins, providing a baseline measure of learners’ existing knowledge, skills and attitudes, and is used to develop an individualized learning plan.
A trainer conducts a skills assessment (using a checklist) while a learner practices
active management of third stage of labor with an anatomic model; based on the checklist, the trainer coaches the learner through steps where he is having difficulty.
In observing learners role play HIV counseling, the trainer notes that the majority are missing several of the same critical points; she decides to add some case studies to the next morning’s schedule that will reinforce these key points, eliminating another activity that she feels is not as important for this particular group.
Key Features of Formative Assessment
Here are some key features of formative assessment, which is essential for learners to develop competency throughout the course.
Incorporates of range of formal and informal tools, such as role plays, case studies, games, quizzes, skills checklists, written assessments, skills demonstrations,
discussions and many more. Almost anything that happens in the classroom can be used as a tool for formative assessment—an opportunity for learning, assessing progress and providing feedback.
Can be unstructured and flexible. Again, tools designed especially for formative
assessment can be modified and new approaches developed in the field. These tools can be used at any time throughout the course.
Is non-threatening. They may be scored, but they are not “graded” in the sense that they have a direct impact on whether a learner advances. Learners can score their own work, even, and are often encouraged to ask questions about the content.
Involves direct and immediate feedback. Whether asking group or individual questions, doing group exercises, games or reviewing homework, direct and immediate feedback should be provided.
Can provide structured information on learners’ understanding of a certain topic, perhaps through a quiz or homework assignment. Trainers can use such information to evaluate mastery of content to date and revise training accordingly.
Facilitates learning—helping learners learn by reinforcing important information, giving the trainer an idea of learner progress so that she/he can focus on learning activities and practice that will directly address learner needs. Skills practice and coaching sessions are a great example of how to use formative assessment to help learners learn.
Formative Assessment in Practice
As a learner progresses through the course, the trainer performs formative assessment in the classroom and the clinical setting. Again, this type of assessment helps learners develop the desired competencies—primarily through feedback—and also prepares them for summative assessments, those critical “decision points” during the course when knowledge, skills and attitudes are formally evaluated. (This section provides detailed information on issues or practices unique to formative assessment, while the details of how to conduct many of the activities mentioned are covered in Chapters 5 and 6.) In the Classroom3
Building knowledge
To help learners acquire new knowledge, the trainer presents information using a variety of techniques to help them retain and understand it. Through the use of tools/methods— such as oral quizzes, written tests and exercises (see Box, next page), case studies, games and questioning techniques—and by providing learners ample feedback, the trainer is
able to highlight key points, reinforce correct information and correct misinformation— as well as to assess learners’ level of knowledge. By the end of this phase in training, learners will take a validated objective written examination (summative assessment) (page 7-14), which will enable the trainer to determine whether they have in fact acquired the knowledge necessary to move on to the skills portion of the course.
Using Written Tests in Formative Assessment*
Subjective written tests (e.g., short-answer questions, essay questions, written assignments), which are often used for formative assessment, may be more difficult to score than
objective tests, such as the validated objective written examination used in summative assessment. Use of an answer key (perhaps outlining main points a “correct” answer should cover) is recommended, and may be provided, for ease of scoring. Blind testing/scoring is also recommended to eliminate any biases.
Either way, however, because the purpose of formative assessment is to provide feedback to learners to help them improve their performance, it is not necessary to assign a numerical score to the assessment. The results of formative assessments may be reported to learners on a scale such as poor, fair, good or excellent. The crucial aspect of formative
assessment is to explain to the learners why they got questions wrong or received a given rating, and how they can improve the results when reassessed on the same topic in the future.
Following are some ways to help learners learn from knowledge assessments:
z Ensure that they understand which course learning objective corresponds with any incorrect responses, so that they know where to focus their energies.
z Instruct learners to review the materials related to the questions they missed.
z Give learners an opportunity to ask you questions about any test items on which they scored poorly or that they did not understand.
z Discuss answers as a group (protecting anonymity), asking learners the reasons why different answers are correct or incorrect.
(Additional guidance for conducting written tests is provided on page 7-15.)
Note: If many learners had trouble with the same questions, either the teaching methods or materials may not have adequately addressed the corresponding learning objective(s), or the questions (in formative assessment tools only) may need to be rewritten. Adapting the teaching methods/materials to better address the problem areas might also be considered. These issues are further discussed in “Using Assessment to Evaluate and Guide Training” (page 7-19).
* Written tests and exercises are highlighted here but may in fact be used in any phase during the course.
Building skills in a simulated setting
In this phase of the course, the trainer conducts clinical demonstrations to provide learners with a clear picture of the skills to be learned. The trainer is able to assess and build the learner’s level of competence in practicing these skills through the use of two key methods/tools—direct observation and structured feedback reports—which are further discussed below. By the end of this phase in training, learners’ performance will be formally evaluated according to the validated skills checklist (summative assessment), which will enable the trainer to determine whether they have the level of skills competence needed to practice their skills in with actual clients in a clinical setting. (Remember: Actual competency can only be achieved with actual patients in a clinical setting.)
Direct observation: This is the most valid way to assess learners’ skills in both formative and summative assessment and can be conducted by the trainer or the learner’s peers (and, later, by clinical staff). The main components of direct observation as a key formative assessment tool are:
Observation: As a learner practices his/her newly acquired skills with validated skills checklists (Exhibit 7-3) and any other tools and equipment needed, the observer— follows along with the same checklist—noting which steps are performed correctly and which are performed incorrectly or missed.
Asking questions: As learners proceed through a given task or procedure, the observer
asks questions about what they are doing, why they are doing it, etc. This probing allows the trainer to assess fully the learner’s ability to integrate knowledge, skills and attitudes.
Providing feedback: When steps are missed or performed incorrectly, the observer coaches and provides positive, constructive feedback to the learner to aid him/her in getting back on track.
Additional guidance for conducting skills assessments using checklists is provided on page 7-17.
With the appropriate guides, direct observation can also be used to assess learners’ demonstrated attitudes, as well as communication and clinical decision-making skills— through a variety of simulations (e.g., taking a history, diagnosing illnesses based on patient information, even clinical decision-making).
Tip: In many training situations, it is difficult to observe each individual learner, particularly when time is limited and the ratio of learners to trainers is high. Nevertheless, several techniques can be used to overcome the obstacles to direct observation. For example, assessments can be “staggered” by dividing learners into small groups and sending them for practice and assessment at different times throughout the session or course. Learners can also be videotaped so that the trainer and the learner can review together following the practice session.
Exhibit 7-3. More about Using Validated Skills Checklists
Although checklists focus only on the essential steps or tasks involved in a specific
competency, they contain sufficient detail to permit (1) the learner to understand exactly what is involved in specific skill or activity and (2) the clinical trainer to effectively and objectively evaluate and record the learner’s overall performance. Using checklists in competency- based clinical training:
• Ensures that learners have mastered the clinical skills and activities, first with models or in role play, and then with clients
• Ensures that all learners will have their skills measured according to the same standard • Forms the basis for follow-up observations and evaluations
The checklist is first used formatively, to develop learner competency. Following along with the checklist, the trainer and/or peers will observe the learner’s performance on models— providing coaching and feedback as needed. After learners demonstrate competency on models or in role play, they can practice their skills with actual clients under supervision, and the checklist is once again used to assess their performance.
When clinical practice is completed, the checklist—together with the clinical trainer’s review of the learner’s case logs, skill portfolio and any medical records—becomes a tool for summative assessment; it provides objective documentation of the learner’s level of
performance. Furthermore, it serves as one part of the process of attesting that the learner is qualified to provide the clinical service (e.g., male circumcision, postpartum family planning, diagnosis and management of pre-eclampsia/eclampsia). Like other tools used in summative assessment (e.g., the post-course knowledge assessment) the competency-based checklists used in skills development are developed and validated by a group of subject matter experts. As such, they should not be modified.
Sample 7-1 contains a checklist for providing post-test PMTCT counseling for a woman with