Anexo A ( Clase 1)

Again, the four main criteria reviewers use to judge the quality of the design to this point involve goal, learner, context, and assessment considerations. In evaluating the congruence among the various design elements, the following steps were taken:

1. Organize and present the materials.

2. Judge the congruence among the materials and the instructional goal.

3. Judge the appropriateness of the materials for the target learners.

4. Judge the congruence of the performance objectives and test items with the performance and learning contexts.

5. Judge the clarity of the materials.

Organization Table 7.6 contains a partial design evaluation chart for the in-structional goal on leading group discussions. The first column contains selected

Performance Objective for Main Step

Prescription for Frequency Count Observation Instrument (used by evaluator during simula-tions and actual meetings)

6. During simulated meetings com-prised of new leaders and held in the department’s conference rooms, manage cooperative group interac-tion. Discussion members should participate freely, volunteer ideas, cooperate fully with leader and other members.

During actual problem-solving meetings held at a designated site on campus or in the community, lead group discussions aimed at solving problems currently existing.

The following categories will be used relative to main step 6: Manage cooperative group interaction in both the learning and performance contexts.

A. Engendering actions demonstrated Frequency 1. ________________________________ _________

2. ________________________________ _________

(etc.)

B. Defusing actions demonstrated Frequency 1. ________________________________ _________

2. ________________________________ _________

(etc.)

C. Stress-alleviating actions demonstrated Frequency 1. ________________________________ _________

2. ________________________________ _________

(etc.)

D. Rating of overall quality of cooperative group interaction (circle one)

Mild 1 2 3 4 5 Excellent

Continued

t a b l e

7 . 5

subordinate skills for step 6, “Manage cooperative group interaction,” the second column contains the performance objectives for the selected skills, and the third column includes matching test items for each of the objectives. Only a few of the skills, objectives, and test items are needed to illustrate the analy-sis process. A thorough analyanaly-sis would include all the skills, objectives, and items developed to this point. In addition to the chart, a copy of the goal analysis ( Figure 4.8 ) , learner characteristics ( Table 5.4 ) , performance site characteristics ( Table 5.5 ) , and learning site characteristics ( Table 5.6 ) , should be available for the reviewers. They could also benefit from the summary of assessment criteria included in the rubric at the end of this chapter. With the materials organized, the second step of judging the congruence between the materials and the goal can be undertaken.

Congruence For this analysis reviewers will need the goal analysis ( Figure 4.8 ) and the subordinate skills in Table 7.6 . Notice that the subordinate skills illustrated Performance use as a group leader to encourage and stifle cooperative discussion during your meetings. What direct actions might you take as the leader to encourage member participation and coopera-tion? (Create response lines for ten responses.) leader actions most likely to encourage and a minus (−) before those actions likely to stifle cooperative group interaction.

___ 1. Introduces all members who attend the meeting.

___ 2. Emphasizes status differences among group members.

___ 3. Glances around the group welcomingly.

___ 4. Names a particular group mem-ber to start discussions.

___ 5. Comments positively after each person as commented (etc.)

As you observe (name) manage the meeting, what actions did he/she take to engender or encourage cooperative group behavior?

Actions to Engender

Cooperation Frequency

1. _______________ _________

2. _______________ _________

3. _______________ _________

(etc.)

A Section of a Design Evaluation Chart for the Instructional Goal “Lead Group Discussions Aimed at Solving Problems,” Step 6, “Manage Cooperative Group Interaction”

t a b l e

7 . 6

Case Study: Group Leadership Training 165

are worded the same as those in the goal analysis except for the addition of the term stifling that has been added to subordinate skill 6.4. Reviewers and designers would need to discuss this addition to determine whether it was appropriate and intended or an inadvertent addition that should be corrected.

Using Table 7.6 only, reviewers should compare the subordinate skills in col-umn one with the performance objectives in colcol-umn two. The skills do appear to be exactly the same except for skill 6.4, where the action term is classify, whereas the objective specifies indicate. There is no meaningful difference between these two terms in this instance.

Finally, reviewers should compare the congruence between the performance objectives in column two and the test items in column three. Examine the items for their congruence with the conditions, behavior, and criteria separately rather than holistically. Reviewers followed the same process the designer used in the previous section, and they judged the items and objectives to be congruent across conditions, behavior, and criteria.

Contexts The third step is to judge the congruence of the performance and learn-ing contexts with the goal, objectives, and test items in the design evaluation table.

Because reviewers previously judged the goal, subordinate skills, and performance objectives for congruence and decided they were consistent, they can simply con-centrate on the performance objectives ( Table 7.6 ) and the description of contexts ( Tables 5.5 and 5.6 ) . Following their review, they believed that leaders could transfer the skills described in the materials to their campus and community meetings. The meetings will be held on campus or in selected community locations. Little addi-tional resources (e.g., costs, resources, time, personnel, facilities) would be needed, because meetings are likely to occur related to learners’ jobs or volunteer activities.

Learners The congruence among the goal, skills, objectives, and test items has already been established; thus, reviewers must review the feasibility of the mate-rials ( Figure 4.8 and Table 7.6 ) for the learner ( Table 5.4 ) . Reviewers with greater familiarity with target learners concluded that attaining the instructional goal,

“Lead group discussions aimed at solving problems,” appeared to be feasible for masters level students in leadership. They also believed that students would most likely succeed in managing cooperative meetings given instruction. They thought the performance objectives and test items were reasonable and should assist the leaders in developing skills that would be usable not only on campus, but also on their jobs. In addition, they judged the materials to be free of bias (i.e., gender, cultural, or racial).

Materials Clarity Finally reviewers considered the clarity of the materials; they needed the goal analysis ( Figure 4.8 ) and the design evaluation chart ( Table 7.6 ) to complete their work. They perceived the structure of the goal analysis to be logical, with the six main steps describing the chronology of events in managing a meeting. The verbal information tables (e.g., Table 4.1 ) were judged to be neces-sary to illuminate the designer’s intended meaning of the linked subordinate skills.

They thought that neither the learners nor the instructors would know what to do if these tables were eliminated from the materials. They also considered the content to be appropriate in scope for student leaders, and thought leaders could succeed given instruction in performing the tasks. Reviewers could interpret the performance objectives and test items, and thought these materials were clear. In addition, they judged the test items to be consistent with the assessment criteria stated in the rubric.

Readers interested in school curriculum examples should review the subordi-nate skills, performance objectives, and corresponding assessments for the writing composition case study in Appendix E .

The following rubric contains a summary of criteria to use in developing and evaluating criterion-referenced assessments. The first section contains criteria appropriate regardless of the assessment format, and the sec-ond section describes criteria appropriate for alternative assessments including product development, live performances, and attitudes.

Designer note: If an element is not relevant for your plan, mark NA in the No column.

No Some Yes A. All Assessment Formats (Objective and Alternative)

1. Goal-Centered Criteria Are the items, directions, and rubrics congruent with the components of the terminal and performance objectives including:

___ ___ ___ a. Conditions?

___ ___ ___ b. Behavior?

___ ___ ___ c. Content?

___ ___ ___ d. Criteria?

2. Learner-Centered Criteria Are the items and directions congruent with target learners’:

___ ___ ___ a. Vocabulary, language level?

___ ___ ___ b. Developmental level (e.g., complexity, abstractness, guidance)?

___ ___ ___ c. Background, experience, environment?

___ ___ ___ d. Experience with testing format and equipment?

___ ___ ___ e. Motivation and interest?

___ ___ ___ f. Cultural, racial, gender needs (lack bias)?

3. Context-Centered Criteria for Assessments Is/are:

___ ___ ___ a. Items and directions authentic for the contexts?

___ ___ ___ b. Items and directions feasible for the contexts?

___ ___ ___ c. Required equipment/tools available?

___ ___ ___ d. Adequate time available for administration, scoring, analysis?

___ ___ ___ e. Adequate personnel available for administration?