De la escolarización

Introduction

Given a lack of research regarding professional development for in-service teachers on computer science and computational thinking, this study sought to provide foundational understanding around possible approaches to professional development. Based on existing literature, one area that showed promise for additional research was the potential impact that the interface modality of instructional resources may have on the confidence and competence of the teachers involved (Berland & Lee, 2011; Duncan et al., 2017; Fletcher & Lu, 2009). In some studies, analog or tangible interfaces were seen to increase both competence and confidence as well as increase engagement within a playful environment (Horn et al., 2009; Schneider et al., 2011). Other studies suggested that despite evidence for analog preferences in children and adults, digital interfaces may still play an important role depending on the instructional objectives (Horn et al., 2012; Strawhacker & Bers, 2015).

Based on gaps in existing literature, this study investigated potential differences in elementary teachers’ confidence and competence depending on the use of either analog or digital instructional resources during professional development. Specifically, two

research questions were investigated in a qualitative study.

1. Is the confidence of elementary teachers in an initial computer science professional development interaction different depending on the use of an analog or digital teaching tool?

2. Is the competence of elementary teachers in an initial computer science professional development interaction different depending on the use of an analog or digital teaching tool?

Research Design

This study was conducted as a qualitative, single-case experimental design using a non-concurrent, multiple-baseline approach with comparisons across subjects. Single- case experimental design is recognized in the literature as an experimental approach appropriate for use with smaller population sizes when studying larger groups is not feasible (Hitchcock, Kratochwill, & Chezan, 2015; Smith, 2012). In this instance, a lack of prior research made calculating an effect size and a requisite sample size for a

traditional experimental design difficult. Single-case designs can still be used to establish a causal relationship and allow some generalization (Hitchcock et al., 2015).

As a full experimental design, single-case studies include the expected elements of control and randomization. In a multiple-baseline study, control is provided through the repeated measurements of a single subject comparing baseline data and intervention data (Barlow, Nock, & Hersen, 2009). Each subject in the experiment acts as her or his own control through comparison of the baseline to the intervention phase with all factors other than the independent variable being held constant (Kennedy, 2005; Kratochwill et al., 2013). Replication of results with respect to treatment effects is sought between subjects to increase external validity in the results of the study (Barlow et al., 2009).

In this study, subjects drawn from a single-case of in-service elementary teachers were compared with respect to response to different interface modalities used within

analog or digital – was the independent variable assessed. Instruments described in the following section were used to assess the subjects’ confidence about and competence with various aspects of computer science as encountered in a computational thinking instructional approach to the topic. The control for each subject was established during a baseline phase followed by a randomly timed intervention with additional measurements in a intervention phase. The start of the baseline phases were non-concurrent to

accommodate the schedules of the subjects and the researcher. Given the highly personal nature of the study around professional development, however, the lack of concurrency was considered to have little impact.

Subjects were selected from multiple schools to reduce the threats to internal validity from teacher interactions. The study was designed to include six subjects though two subjects withdrew leaving an actual n of four. Subjects were randomly assigned to the two experimental groups, analog and digital, resulting in three subjects for each as planned and two each as implemented. One subject from each group withdrew after randomized assignment but before the actual study began. Within each group, subjects were randomly selected for the order in which they received the intervention. Subjects in the study underwent a total of eight assessments made up of three to five each for

baseline and treatment phases depending on the randomized implementation timing. This randomization was recommended by Kratochwill and Levin (2010) to increase internal validity and to allow the use of additional statistical tests.

The intervention introduced teachers to basic concepts of computational thinking including commands given to robots, basic control flow statements, and functions. The role of functions as a method of bundling, naming, and then enacting a group of repeated

or commonly used commands was highlighted in the professional development. For the digital group, the intervention used the Scratch Jr. application on iPad Mini tablets. The analog group used the Robot Turtles board game. Both resources are designed for use by primary grade students in kindergarten through second grade. Additionally, each tool had a similar approach to defining and calling functions using a specific command; the function frog card in Robot Turtles and the envelope/message block in Scratch Jr. The professional development provided by the researcher covered the basic concept of a function as a repeated pattern expressed as an algorithm as well as the specific method of defining and calling a function in the resource being used.

Research Context

This study was conducted at rural elementary schools within the service area of the Genesee Valley Educational Partnership in Western New York. The Partnership serves 22 school districts across four counties with a total student population of 22,339. There are 26 public elementary schools in the region with a total of 632.5 teachers and 11,521 students in kindergarten through sixth grade (GVEP). The districts in the region tend to fall below average district wealth for New York State. The mean combined wealth ratio for the region is 0.59 meaning that these districts are about 60% as wealthy as the New York State average (nydatabases.com).

None of the elementary schools had established a formal computer science program when this study began. Six of the schools had been involved with STEM activities through a New York State Learning Technologies Grant run by the Genesee Valley School Library System. As Director of the School Library System, the researcher

avoided those six schools. The current state grant established the researcher as a credible and trusted provider of technology professional development in the region amongst principals and superintendents.

There are future opportunities for a computationally literate population in the Genesee Valley region. The region falls between the greater metro areas of Buffalo and Rochester. Both cities are investing heavily in scientific and technical industries

including the integrated photonics hub in Rochester. Genesee County also has an

emerging advanced manufacturing project that will provide hundreds of jobs for middle- skills workers who are computationally savvy (Spector & Sharp, 2015). To educate the workforce to meet future needs, the schools in the Genesee Valley region need to emphasize computational thinking as the foundation of computer science and other technology understandings. This study compared the efficacy of different approaches to professional development for in-service teachers to reach that goal.

Research Participants

The case being investigated for this proposed study was elementary teachers with limited computer science experience who were ideally teaching computational thinking for the first time. Given the younger age alignment for resources that were used, the sampling frame was limited to kindergarten and first grade teachers. The actual subjects were purposively selected from responses to a call for participation shared by principals from the region. The selection excluded those who had prior professional development or instructional experience in computer science or programming. This was intended to isolate the impact of the intervention and amplify any changes between the baseline and

the intervention measurements. Additionally, subjects were selected from a single gender to remove potential gender impacts.

Instruments Used in Data Collection

There were two primary instruments used within this study. Teacher confidence was measured using an adapted version of the Computer Programming Self-Efficacy Scale initially developed by Ramalingam and Wiedenbeck (1998) and modified by Kong (2017) for elementary student programmers. Teachers’ competence was measured using the Computational Thinking Test designed by Román-González (2015) that had

undergone content and criterion validation (Román-González, 2015; Román-González et al., 2017). Both of these instruments were adapted for this study to better address the focus on teachers. Stewart, Thrasher, Goldberg, & Shea (2012), writing in the context of health research, noted that adaptation of an existing measurement is an acceptable practice in order to meet the specific self-reporting needs when investigating a smaller, distinct population.

Computer Programming Self-Efficacy Scale for Confidence. The Computer

Programming Self-Efficacy Scale (CPSES) was initially developed by Ramalingam and Wiedenbeck (1998) to assess college students learning the C++ programming language. The CPSES has undergone numerous adaptations for use in studies of different

populations and different programming languages. The adapted version used in this study was based upon modification by Kong (2017) for elementary student programmers. Additional adaptations by Kukul, Gökçearslan, and Günbatar (2017) for high school programmers and by Tsai, Wang, and Hsu (2018) for middle school programmers were

made according to the definitions used by Stewart et al. (2012) as compared to the CPSES adaptation validated by Kong. For example, the contextual focus of statements from Kong’s scale such as “I can code with. . .” were adapted to address an instructional context as “I can teach the use of. . .” (p. 99). Additional language adjustments were made to clarify terminology as introduced in the professional development for this study while maintaining the same underlying concepts of the CPSES version validated by Kong (2017).

The adapted Elementary Teacher Computer Programing Self-Efficacy Scale (see Appendix A) included 15-items that asked subjects to rate their level of confidence with statements about teaching computer programming. Each statement was rated on an 11- point scale ranging from 0 (Not at all Confident) to 10 (Highly Confident) based on the recommendations of Bandura (2006). Instructions were provided for the subjects asking them to rate their confidence about their ability to complete the instructional objective at the point in time of the measurement (Bandura, 2006).

Computational Thinking Test for Competence. The adapted Computational

Thinking Test (see Appendix B) originally designed by Román-González (2015) included 28, multiple-choice questions covering a variety of computational thinking concepts. Given the strength of the validation studies for the current instrument, no questions were added or removed. However, permission was received (see Appendix C) to adapt the graphical representation of the questions to reflect the interfaces of both Robot Turtles and Scratch Jr. This moderate, content adaptation meant that teachers were assessed in the same graphical interface environment as they learned and taught (Stewart et al., 2012).

Challenges for adapting the Computational Thinking Test. Prior to the

adaptations, additional consideration was given to the potential impact the incongruity of movement commands between the original graphical representation of the Computational Thinking Test and the adapted version using graphics from Robot Turtles and Scratch Jr. could have on the assessment. In the Scratch Jr. programming language, the kitten or other sprites are moved around the screen using absolute directional commands. This means that the command arrow ➡ means move to the right, not move forward. Even if the kitten is facing to the left, the ➡ command will move the sprite to the right one unit making it look like the kitten is moving backwards. This is different from other

programming situations, such as the Robot Turtles board game, where movement commands are relative. In Robot Turtles, the forward movement command is always interpreted as movement of one unit in the direction that the turtle is facing. Therefore, movement is relative to the turtle’s point of view.

A search did not reveal any definite statement as to why the Scratch Jr. team decided to use absolute movement as opposed to relative movement commands as are found in the Scratch language. However, there are clear indications that this was a decision based on a desire to implement a coordinate grid system upon which movement would happen. “The grid was designed to help children understand the rules of

measurement for each programming block. It addresses the countable unit of

measurement for linear movement. For example, a character programed to ‘Move Right 10’ glides 10 grid cells rather than 10 pixels or an arbitrary unit” (Bers, 2017, pp. 122- 123). The designers intended the movement to reflect movement students would see on a

interpretation is reinforced by the intentionality of having the script execute horizontally as opposed to vertically as would happen with text code. “This choice reinforces print- awareness and English literacy skills” (Bers, 2017, p. 119).

In terms of potential impact on this study, the issue was that the intended

instrument for measuring competence used a mixture of absolute and relative movement commands. Of concern was that the absolute and relative questions were incongruous with the adapted graphics. Within the original instrument, the questions with absolute movement were graphically presented in a style more similar to the cards from Robot Turtles with arrows of movement while the relative movement commands were written using Scratch coding blocks. This was incongruous with what the subjects might expect from their experiences using the resources for the study with Robot Turtles using relative and Scratch Jr. using absolute movement. Research by Bruner and Postman (1949) on recognition in the case of incongruous stimuli suggested that the differences between absolute and relative movement commands should be small. While this study found a significant difference (t = 3.76, p <.01) in the time it took to recognize a playing card where colors of suits were reversed, the difficulty of a subject to identify an incongruous card dropped rapidly after repeated interactions with miscolored cards. This suggested that if subjects in this proposed study are informed before interacting with the

measurement, and then are presented with example questions that introduced and reinforced the incongruous use of relative and absolute movement, the impact should be minimized.

A second potential impact was identified stemming from the incongruity of notation styles. The difference in symbol notation and representation of movement

between the proposed instrument and what the subjects in this proposed study learned in their respective interactions with Robot Turtles and Scratch Jr. was investigated. In writing about learning math, Bruner and Kenney (1965) noted that an understanding of the abstract, foundational concept was more important than the concrete representation of a mathematical interaction. Bruner and Kenney (1965) observed that the students “had not only understood the abstractions they had learned, but also had a store of concrete images that served to exemplify the abstractions. When they searched for a way to deal with new problems, the task was usually carried out not simply by abstract means but also by ‘matching up’ images” (Bruner & Kenney, 1965, pp. 56-57). This suggested that as long as the subjects in this proposed study had a chance to experience the multiple concrete images of movement commands as seen and explained in the example questions of the instrument, they should be able to match these images up with their abstracted understanding of different possible movement types. While this learning would need to be self-directed during the baseline assessment interactions, additional explanation could be provided during the intervention. During the professional development intervention, it was reinforced that the move type seen by the subject in the resource used – relative for those in the Robot Turtles group, and absolute for those in the Scratch Jr. group – was not the only type.

Based on this review, the potential impacts of the incongruity in iconographic representation and movement command style were judged to be of limited consequence to the study. As such, the adaptations were made to the graphical representations of the test questions and answer choices to use images from Robot Turtles and Scratch Jr.

Following the recommendations of Stewart et al. (2012), efforts were taken to maintain all other aspects of the questions whenever possible.

Procedures

The first step in the implementation of this study was to seek and obtain approval for the research from the St. John Fisher Institutional Review Board. Following approval of the experimental design, subjects were sought using a call for participants sent through principals as defined above. After selection, the researcher completed a randomization process as recommended by Kratochwill and Levin (2010) to determine placement within either the analog or digital group. A second randomization process was used to create an order of intervention for subjects within each group. Randomization was completed using the list randomizer from Random.org. The last step in the initial phase of the study was to meet with the subjects to inform them of the full parameters of the study and to seek consent. The multiple-baseline approach required additional explanation for subjects. During the consent meeting, subjects were reassured that during the baseline phase they may not be able to answer the questions on the instrument for assessing competence in computer science. It was explained that this was an expected part of the research design and that the subjects should not independently seek out information on the topic.

Ethical considerations. In terms of ethical considerations, subjects were notified

that pseudonyms would be used for data collection, analysis, and reporting. They were given an opportunity to select a pseudonym at the time of informed consent. The pseudonym key was stored as an encrypted note in an industry standard password vault on the researcher’s phone protected with two-factor authentication until completion of the dissertation process and then was securely deleted. The fully anonymized data was stored

in an encrypted, non-synchronized, Dropbox folder in an account protected with two- factor authentication. It was also explained that the research findings would be

disseminated in this dissertation and would be shared at conferences and in other possible publications. The findings were shared with the subjects at the conclusion of the research study.

Study procedure. This study was designed for implementation during an 8-day

period. The 8 days for the study were scheduled non-currently to accommodate the calendars for the subjects and the researcher. In the initial meeting, following receipt of consent, the researcher presented the subjects with examples of the two instruments. At the start of the 8-day study period, a link to access the online instruments was sent to subjects via a daily email. Subjects completed a specified number of baseline assessments as indicated by the randomization process. On the 4th _{to 6}th _{days of the intervention, again}

depending on randomized order, the researcher met with each subject to provide professional development on teaching with either the analog or digital resource. Two sessions ranging from 45 minutes to an hour took place on succeeding days. During and after the intervention, subjects continued to take the daily assessments. The subjects each completed a total of eight instruments throughout the study.

The first professional development (see Appendix D) focused on the use of directional commands within the analog or digital resource as a way to move the character around the play area. Basic conditional statements based on If/Then