Complicaciones

Perhaps one reason Middle States found inconsistency in SLO assessment was because of the lack of documentation of the process and, therefore, the challenge in communicating a unified vision internally and externally. Publishing an Assessment Guide that documents a step-by-step process was a pivotal step to improvement. As I continue to use Heifetz et al.’s (2009) experimental mindset, he cautions against using it during a state of emergency so presenting the Assessment Guide as a “solution” (rather than an experiment) had to be explained internally. The Assessment Committee emphasized that, much like the assessment process itself, the guide would undergo a yearly review and be updated as necessary. The message that the guide was subject to revision signaled that it would also be rooted in continuous improvement even though at the point of its initial publication it was more like a solution.

The PDSA cycle was the guiding tool in the development of the Assessment Guide framing the continuous improvement of Small School’s process and procedures (see Appendix B). The process laid out in the Assessment Guide was an initial attempt to add more structure to the existing process, which was not previously well documented. The first step in the development of the Assessment Guide relied on benchmarking and best practices data from other schools as well as

the consultation from assessment experts in the field. This was the planning phase of this PDSA cycle and also developed a strategy for the assessment process and a means to communicate it.

The Assessment Guide documents the process, informs the development of curriculum maps, SLOs, assessment plans, and other assessment-related activities. Throughout the year notes were made about how the guide could be improved based on how people used the guide, questions people asked, or suggestions people made (do). One year after its initial publication, the annual review and update included the addition of an assessment coordinator job description, a revision to the assessment process including a more user-friendly timeline, and modification to the charge of the assessment committee to be more specific (study and act). Step 4 – communicating the vision – was repeated when faculty were notified that a revised Assessment Guide had been published to the institution’s portal (Kotter, 2012). Several of these modifications were made after I was executing a PDSA cycle throughout the first year of the Assessment Committee. The lessons that I learned about the inefficiency of the committee (study) drove action to include clearer expectations documented in the second iteration of the Assessment Guide (act).

3.2.2.1 Aligning the System

Another important part of systematizing the approach to assessment at Small School was developing a Closing the Loop Day. This is an end-of-the-year day dedicated to student learning outcome assessment attended by faculty in order to give them the opportunity to communicate and collaborate about what students are learning. This event is a documented item in the Assessment Guide. I viewed the first annual Closing the Loop Day in 2019 as the culmination of the year’s events and a springboard to sustainability of our new culture of assessment. I developed a brief PowerPoint presentation that set the stage for the day by retrospectively looking at the educational policies committee 2017-18 general education report to inform the planning for the 2019-20

general education assessment cycle. After giving a summary of key recommendations for assessment from the educational policies committee report, I presented some brief information about the Association of American Colleges and Universities (AAC&U) VALUE (2009) rubrics in order to practice assessment. Faculty first performed an assessment of a senior capstone academic poster individually (all faculty were looking at the same poster), next faculty discussed their ratings in small groups of no more than 8 per table, and finally faculty participated in a large group discussion. This exercise was set up very much in the spirit of a score norming session to ensure that faculty had the same understanding of definitions and were using the VALUE rubric (critical thinking in this case) in the same way.

Besides building faculty’s connection and ownership of general education, another important first step accomplished in 2018 was the creation of a general education curriculum map. As Suskie (2015) describes “curriculum alignment is ensuring that your course, program, or general education curriculum is designed to give students enough opportunity to achieve your key learning outcomes” (p. 135). This was the college’s initial attempt in getting faculty to think about the connections and alignment of their courses with general education SLOs. It also exposed significant gaps in the assumptions that faculty have made about the requirements. This activity seemed to jumpstart thinking among faculty about how alignment in the curriculum is key to success. Closing the Loop Day capitalized on this momentum.

Assessment expert Peggy Maki (2010) states “establishing formal institutional times for collective discussion of, reflection on, and interpretation of results leading to actions to improve student learning” (p. 284) will foster a collective responsibility within the faculty and contribute to a learning organization. The Closing the Loop Day provided such a structure for discussion,

reflection, and interpretation of results. It also took advantage of collaboration among faculty within and across disciplines. She continues,

campuses can develop another channel of communication by bringing together members of a campus community to share and interpret institution- and program-level assessment results in formal campus wide forums designed to (1) learn about institutional practices, (2) explore new practices, (3) learn about assessment results, and (4) build institutional learning based on those results (p. 289).

Rallis and Lawrence (2017) also endorse collaboration and inquiry in assessment. They say, “The overriding question asked in inquiry processes should not be: ‘Is it valid?’ but rather ‘Is this a useful way of understanding, and potentially acting upon, a given problem?’ The users hold the key — they determine truth or value” (p. 26). Smith and Gordon (2018) agree noting that many schools purposefully connect faculty and other assessment practitioners with novice faculty assessors to build capacity and encourage idea sharing cross campus. The notion of validity and adequate sampling was a topic discussed during Closing the Loop Day. It gave me an opportunity to push faculty’s thinking away from statistical significance (a default for no action) and think about student learning in the spirit of continuous improvement.

While Closing the Loop Day was a first attempt at an institutional assessment event, it has potential to become an engrained part of Small School’s assessment culture especially for the general education program – currently undergoing a revision. Academic administration is committed to this event as it serves to formalize and systematize the assessment approach to general education, which Papadimitriou (2018) suggests is a positive step in becoming an organizational routine. The observational data, informal conversations, group discussion, and the data from the evaluation form helped to measure the effectiveness and inform fine-tuning for next

year’s event. The data also provide rich evidence that Small School is working to improve its assessment of the general education program and informs the revision process.