Cuarta categoría: Tendencias de la evaluación de aprendizajes

The reliability of manual tracking using the grid sketching method was assessed during Australian football matches (Veale et al., 2007). This research used one observer, therefore only intra-observer analysis was performed. Two observations of three separate matches displayed high reliability (r = 0.98) (Veale et al., 2007). Despite minimal variation from the observer, the validity of this method was not determined. In addition, the use of correlation analysis between observations may not be a true measure of reliability.

2.1.1.2 Validity

Validation of a manual tracking system in soccer was carried out by comparing the observers’ grid sketchings to a pre-determined criterion measure of distance (Reilly & Thomas, 1976). Correlations within each intensity zone (walking, jogging, cruising, sprinting, and backwards movement) were high (r = 0.91 to 0.97) (Reilly & Thomas, 1976). Despite these positive outcomes, this method only allows researchers to observe one athlete at a time, with the post-hoc analysis being time consuming.

2.1.2 Video tracking systems

The manual analysis concept was developed further with the addition of video systems intending to enhance the quality of the information (Dawson et al., 2004b, a; Dobson & Keogh, 2007). This method allowed the observer to record movement and analyse the information later with greater accuracy. Movement patterns and intensities could be recorded with greater precision by stopping and starting the video. The methodology for video time-motion analysis includes single camera analysis following one player, and multi-camera protocols designed to capture the entire playing area. Using multiple cameras allowed the observer to track more than one player. Despite this, the process is labour intensive and requires consistent filming and analysis techniques (Dobson & Keogh, 2007). In addition, video tracking does not facilitate real-time analysis, minimising the influence of the information during actual matches (Mohr et al., 2003; Deutsch et al., 2007). Various video analysis techniques exist, some simply recording the time and frequency of efforts (McInnes et al., 1995), whereas others have attempted to estimate the distance covered by an individual player (Deutsch et al., 1998; Dawson et al., 2004b, a). Commonly used measures include total movement distance, distance and time in various intensity zones (Spencer et al., 2002; Dawson et al., 2004b, a; Duthie et al., 2005; Burgess et al., 2006; Deutsch et al., 2007), length, time and frequency of efforts (Bangsbo et al., 1991; Duthie et al., 2005; Deutsch et al., 2007), and work to rest ratios including time between maximal efforts (Dawson et al., 2004b, a; Spencer et al., 2004).

2.1.2.1 Reliability

The reliability of video tracking has been established across multiple team sports. In most cases researchers determine their own reliability statistic as part of the studies methodology. Intra and inter-observer reliability assessments are common practice as

human testers are required to collect and analyse data. In one study, video-based TMA was utilised to track the movement patterns of Australian football players in matches (Dawson et al., 2004b). Only one observer was used to view and analyse the video footage, therefore only intra-observer reliability was calculated. The frequency and total time of movements and match related activities were assessed for reliability. Pearson’s correlation coefficient (r) was used to display relationships between trial, and the coefficient of variation (CV%), which is the technical error of measurement expressed as a percentage, was used to display the magnitude of variation (Dawson et al., 2004b, a). The variables incorporated into the reliability testing included movement intensities zones (standing, walking, jogging, fast running, and sprinting), changes of direction whilst sprinting, and various game based activities such as kicking, handballing and marking. No significant differences from trial one to two were found, whilst relationships were strong (r = >0.96). The CV% which represents the magnitude of difference was <4% for frequency, and between 7 and 11% for total time of the various movements and activities (Appleby & Dawson, 2002; Dawson et al., 2004b, a). The error associated with this data collection method is high in relative terms as an underestimation of up to 11% could have an impact on the interpretation of the information.

Rugby union and hockey research displayed similar intra-observer reliability statistics for total time and frequency of movements in various intensity zones (Duthie et al., 2003b; Spencer et al., 2004). Rugby analysis displayed moderate reliability for total time and frequency of low intensity movements such as walking (CV= 8.7 & 7.2%), jogging (CV= 5.8 & 4.3%) and striding (CV= 7.4 & 6.9%) whereas sprinting (CV=

10.9 & 10.3%) was harder to replicate. Interestingly, the recognition of stationary or resting (CV= 11.1 & 13.6%) periods by observers showed a larger CV%, this may

have a significant effect on the time and frequency of work bouts during a match (Duthie et al., 2003b). Similarly, hockey analysis from matches showed that stationary or resting (CV= 9.4 & 9.8%) periods were more difficult to repeatedly observe compared to low intensity movement (Walk: CV= 5.9 & 5.7%; Jogging: CV=

5.4 & 8.8%). Sprinting (CV= 8.1 & 7.3%) still showed a substantial error, but not to the extent of the previous study (Spencer et al., 2004). Rugby and hockey fields are standardised across all competitions with multiple line markings at pre-determined intervals. With references points for the observers to utilise, it could have been assumed that reliability would be higher than an oval playing field, however, this was not the case.

Basketball time-motion analysis research also determined intra-observer reliability as part of the testing protocol (McInnes et al., 1995). The magnitude of difference between trials was determined using the CV%. Similarly, the percentage of time in lower intensity activity (Walk/Jog: CV= 4.1 to 4.9%) was easier to replicate compared to running and sprinting (CV= 8.8 and 5.6%, respectively) (McInnes et al., 1995). The reliability of timing individual efforts decreased as intensity increased. These efforts are often shorter in duration and difficult to time accurately. Frequency analysis showed high reliability, aside from the running category, which can be difficult to isolate from jogging and sprinting as the action often appears similar. Basketball specific movements such as lateral shuffling were slightly less reliable than common locomotor categories, as various techniques are used and distances are often shorter and difficult to categorise (McInnes et al., 1995). The variation in movement type between sports, along with different field dimensions and game styles emphasises the need for specific reliability assessments of each sport. (Duthie et al., 2003b).

Records of inter-observer reliability are less frequent as most investigations use one observer, possibly to eliminate the inter-observer variation. One investigation, during rugby union activity displayed an inter-observer reliability of CV= <8.7% for frequency of efforts, total time, and the mean duration of efforts in each intensity zone from one observer to another (Deutsch et al., 2007). Soccer research displayed higher inter-observer reliability for measuring the frequency of efforts, and mean duration in various intensity zones (CV= <4%) (Bangsbo et al., 1991). Unfortunately these investigations did not provide statistics for each individual intensity zone.

2.1.2.2 Validity

Validation studies of video time-motion analysis are limited, as a criterion measure of locomotor activity in matches was difficult to obtain. Researchers from one study developed a video of rugby match play where the distances travelled by an individual was known (Deutsch et al., 1998). The observers predicted distance was compared to the known distance to calculate the validity of this specific time-motion analysis method (Deutsch et al., 1998). The error associated with this method was CV=

<4.94% (r = 0.73 to 0.93). The main source of error was associated with alternative movements such as lateral and backward running (Deutsch et al., 1998). Alternative methods such as the frequency and time length of efforts were not validated.

2.1.3 Computer-based tracking systems

The introduction of computer-based tracking systems enabled researchers to analyse data more efficiently post match. These systems usually required a mouse or mouse- pen to trace individual movements using a scaled down model of the playing pitch on a computer (Burgess et al., 2006; Edgecomb & Norton, 2006). Additionally, computerised key activation systems, and software packages that detect movement based on video recordings from matches also exist (Abdelkrim et al., 2007; Duffield

& Drinkwater, 2008). Although the information gained from these systems is similar to video tracking, the data can be entered into a computer system for automated analysis and considerably less manual labour.

2.1.3.1 Reliability

The reliability of various CBT systems has been assessed in isolated research (Edgecomb & Norton, 2006), and as part of the methodology in a number of studies (Burgess et al., 2006; Abdelkrim et al., 2007). One time-motion analysis study of soccer players assessed the intra- and inter-observer reliability of the computer tracing method (Trak Performance Software [SportsTec Pty Ltd., Sydney]) (Burgess et al., 2006). Variations of CV= 4.6% for the same observer were calculated over 5 halves of match play on two separate occasions. The inter-observer reliability was calculated with Pearson’s correlation (r = 0.98), instead of the previously mentioned CV%. Both these measure were assessed only for total distance covered, intensity zones and frequencies were not tested (Burgess et al., 2006). Soon after, an isolated study designed to establish the reliability of the same computer tracking method was conducted (Edgecomb & Norton, 2006). Australian football players were tracked over multiple courses of known measurement, and during actual matches (Edgecomb & Norton, 2006). A sample of four observers displayed intra-observer reliability of CV=

<3.3% and 4.7%, for the known courses and in actual matches, respectively (Edgecomb & Norton, 2006). Inter-observer reliability was assessed by tracking the same player by two separate observers, again using a known course and in actual matches. These results indicated that a difference of CV= 6.1% between observers in analysing matches, and CV= 4.4% for the known course (Edgecomb & Norton, 2006). Another system designed to track basketball players (PC foot 4.0) used a computer- based program to analyse video recording of the entire playing area (Abdelkrim et al.,

2007). This system differed from the other methods in that the computer used statistical modelling to determine the change of position of each player from frame to frame (0.04 s). It was stated in the methodology that intra- and inter-observer reliability would be determined; however results were not expressed separately. The sprinting (CV≤ 3.6%) and sports specific high intensity movements such as shuffling (CV≤ 3.9%) displayed the highest error (Abdelkrim et al., 2007). The lower intensity activity of walking (CV ≤ 2.9%), jogging (CV ≤ 2.6%), and low-specific movements (CV ≤ 2.9%) displayed slightly less error. These results align with video TMA reliability, suggesting that as the speed of movement increases so does the measurement error (Duthie et al., 2003; Spencer et al., 2004). The measures assessed for reliability were the frequency of efforts at various intensities, average time of the efforts, and the percentage of live time, which is the time spent by the player whilst the clock was running. The live time measure was the most difficult to replicate with the error being between CV= 2.7 to 3.8%.

2.1.3.2 Validity

As part of the previously described reliability study (Section 2.1.3.1), validity was also determined by comparing the predicted distance of four observers, to the known distance of multiple course measured out using a calibrated trundle wheel (Edgecomb & Norton, 2006). Over 176 trials, observers overestimated distance by CV= 5.8%. As the distance of the known courses increased the error decreased (Edgecomb & Norton, 2006) (Figure 2.1). It was concluded that overestimations may result from slight sideways movements of the mouse or mouse pen. This is more likely to occur at lower velocities.

Figure 2.1. Tracking distances for CBT vs. actual distances are shown. In panel (A) there was a significant correlation between distances measured using both systems. Panel (B) illustrates that there was a systematic error as a function of actual distance (slope is different from zero, p < 0.0001). That is, the mean relative error decreased across actual distances. Also, the absolute error is greater over shorter distances when compared to those over longer distances. The relative error in panel (B) was calculated as ((CBT distance − trundle wheel distance)/trundle wheel distance) × 100. Reproduced from (Edgercomb & Norton, 2006).