Instrumento de Análisis: Prueba de Estándares

Elliot (2000) delineates four main reasons for the slow uptake in the use of dynamic assessment that are relevant to consider in South Africa, even though he operates mainly within the domain of educational psychology. Firstly, DA does not consist of a single set of procedures that can be acquired easily with little training, but rather the range of models, techniques, methods, and purposes is vast and potentially confusing to the practitioner. Secondly, dynamic approaches are often time consuming and makes for a slow turnover of assessments. Thirdly, few opportunities for training exists ̶ combined with limited available expertise. Finally, these approaches, by tradition, are more qualitative in nature, and subsequently do not fit well with Western models of professional thinking, which are “proudly empiricist” and view change “through differences in amount rather than differences in kind” (Sutton, as cited in Elliot, 2000, p. 716).

This has led to opinions that dynamic assessment should perhaps not be seen in the same light as traditional assessments and that psychometric properties should be relegated to secondary considerations. Grigorenko and Sternberg (as cited by Elliot, 2000) note that, unlike researchers whose focus tends to be the measurement of change, teachers and applied psychologists are generally more interested in promoting change; thus, for them assessment is most effective when it provides clinical insights that can inform subsequent intervention. As such, Elliot argues that rather than endeavouring to develop superior tools to undertake traditional functions (classification, selection, and prediction), dynamic assessment researchers should highlight the requirement for theoretical, methodological, and professional paradigm shifts and justify this change by emphasising the particular contribution dynamic approaches can make to educational

intervention. The judgements about modifiability/potential will assist practitioners in deciding on the appropriateness of special education placement, predicting future educational success, and gaining an understanding of the nature of the individual’s specific strengths and weaknesses and should thus assist in identifying potentially effective interventions. This should particularly assist in the assignment of children to special education – although there are poignant and complex philosophical and practical considerations associated with the application of potential to placement in education, which are beyond the scope of this thesis (Elliot, 2000).

Due to the socio-economic and educational disadvantages many South Africans experienced in the past, and currently, there is, however, a need for assessment that focusses on potential, rather than current ability (Murphy & Maree, 2006), for the purposes of utilisation in the organisational environment, particularly in the context of a multi-stage selection model.

This means that adequate measurement and construct clarity is needed to the extent where the practice is psychometrically and legally justifiable. One does notice, however, that most studies involving the validation of a learning potential model for predicting learning performance, for the most part, have happened outside of the business or work environment – and rather particularly in the educational field, as mentioned earlier.

Many dynamic batteries do not adequately address issues such as reliability and validity, however, any model that seeks inclusion in the psychometric model, needs to be scientifically sound and should certainly provide benefits in terms of their use over and above what is available from traditional measures. Dynamic assessment lends itself more easily to methodological faults and is vulnerable to exploitation for unscientific purposes, more so than traditional tests, due to its precarious handling of psychometric issues. Construct fuzziness, procedural spuriousness, instructional aloofness, instrument inadequacy, and labour intensiveness, are all issues associated with learning potential assessment, that certainly do not inspire confidence when one considers the needs of practice (Jitendra & Kameenui, as cited by Murphy & Maree, 2006). Issues of reliability and validity are not always seen as being part of the paradigm from which dynamic assessment springs. Sternberg (as cited by Murphy & Maree), a strong critic of the dynamic movement, states that the psychometric soundness of DA tools need to be demonstrated.

Dynamic assessment in South Africa is regarded as a process involving a pre-test and post-test design, with some learning or mediation involved in between – that is not strictly defined. The straightforward difference between pre- and post-test scores are seen to be an indication of ‘potential’ (Murphy & Maree, 2006). These difference scores (or gain scores), merely obtained by subtracting pre-test and post-test scores is, however, not always an effective means of assessing change in the subject and cannot be interpreted directly as indicating potential. Recommendations on solving this have not always been forthcoming, however. When taken in its purest form, dynamic assessment emphasises extensive individual mediation and remediation and does not

emphasise quantitative values. Even though the pre- and post-test scores may have a large degree of reliability, this in no way ensures the high reliability of the difference score. The cumulative effect of error measurement most assuredly influences the difference score. Early studies also merely subtracted pre-test from post-test, though one has to consider the appropriateness of this strategy. In recognition of psychometric problems that result from the use of gain-scores, that is, focusing on the difference between pre- and post-test measures, it is the post-test performance that is generally considered to be more statistically useful (Guthke & Wingenfield, as cited in Elliot, 2003).

As Embertson (as cited by Fernández-Ballesteros & Calero, 2000) states, the reliability of the gain-score depends on, and will be affected by, the reliability of the two scores on which it is based. To solve this problem, Schottke, Bartram, and Wiedl (as cited by Fernández-Ballesteros & Calero) developed a mathematical algorithm, based on linear regression that allows the establishment of a significant gain criterion. This algorithm takes into account the error of estimation of the gain-score from which the classification of the learners into gainers, non-gainers, and high scorers starts out.

Difference scores can be used to differentiate between a disorder and a difference. When pre- test to post-test scores show a significant increase, it shows the ability to learn. A significant gain score is an indication that learning skills related to the task were not present due to some background factor. Small gain scores present two possibilities: The first option is that the pre-test score was not significantly low, and the post-test score is similar to that of the pre-test, indicating that this test-taker did not need to be assessed in this area. However, if the score is significantly lower than average on the pre-test and does not show a significant gain, then a learning disorder may exist that prevented learning (Gutierrez-Clellen & Pena, as cited by Elliot, 2003).

Gain-scores may thus prove problematic when used for comparative purposes. The purpose of dynamic assessment is to determine the extent of learning that can take place in the ZPD during a specifically designed session. Dynamic assessment has been motivated by the inadequacy of conventional tests to provide information about individual differences of learning ability and specific learning processes, which can be large for individuals who are highly comparable otherwise. This is especially magnified with people who suffer mental retardation or those who are culturally different. While Vygotsky included both the initial level of functioning and the ZPD in explaining his theory, the difference score has often (incorrectly) been referred to as that which indicates potential. Because much of the early research in DA involved low-ability examinees with similar (low) initial levels of performance, the initial focus was only on the ZPD or difference score obtained. This can only be done in special cases where the initial levels of performance are equal. However, in all other cases, when one needs to interpret the results of individuals where there are differences in the initial level of the performance and quite likely also differences in the ZPD,

pre-test, and post-test results must be included in the interpretation because, for all such cases, the use of the ZPD (difference) scores without reference to the level at which they occur provides incomplete information. If one were to only consider the difference score as learning potential, then a person with a high (near maximum) initial score (due to being highly intelligent and adaptive, and having an extremely high level of learning potential), will show little to no improvement from pre-test to post-test. His/her difference score will thus be smaller than that of a person who is a low performing individual, due to this person likely being able to produce a bigger gain in scores, despite actually having lower real learning potential. To state this case more plainly, a high-performing academic, who would be able to master a large array of new experiences, would produce both a high pre-test and post-test score due to their high development level. This person would thus show a low gain score, whereas, in comparison, a person with a poor development background and low education would perhaps, by achieving a low pre-test score, and an only slightly higher post-test score, due to learning more from the novel teaching element of the DA than the academic, be able to produce a bigger difference-score. In this example, if the difference score was recorded as being outright learning potential, one might mistakenly conclude that a genius academic will be outperformed in a learning task than a person who is far less capable. Vygotsky’s proposed use of both the actual developmental level and the ZPD is thus essential to achieve logical and practically useful interpretations. The focus is on the ability to learn, thus current accomplishment in learning should be taken into consideration (Elliot, 2003).

In South Africa, DA is usually aimed at disadvantaged students and is seen to be culture and bias free, but, as mentioned, takes very long to administer and is very costly to implement. In South Africa, DA is often implemented merely by making use of conventional testing instruments, implemented in a dynamic manner. Sometimes it is a form of assessment that comprises only mediation and individual attention that is wholly qualitative and thus not suitable for larger samples. In practice, it is supposed to be a form of assessment which involves two aspects, namely mediation and assessment, which emphasises previous deficiencies in learning opportunities, allowing all learners to participate on the same level, as far as is possible. The bulk of South African research in this field places emphasis on the assessment aspect, often not considering mediation or remediation at all, or only paying scant attention to it. Murphy and Maree (2006) cite various studies that deem DA unsuitable for the South African context now, due to time constraints, costs, and the less robust reliability and validity indices when compared to more static assessment procedures.

Test batteries in South Africa containing the word ‘potential’ in their titles, apparently convey the meanings inherent in the terms “ability” and “aptitude”, which represent different aspects of intellectual functioning and measurement. These terms are often used interchangeably and thus need to be clarified and understood by practitioners, for the sake of the future of dynamic

assessment in South Africa. There is also the perception that dynamic assessment refers to disadvantaged learners only, although this is certainly the main field of the application thereof, as mentioned above. It can also be applied to gifted learners in disadvantaging circumstances and to average and above-average learners who are struggling with some aspect of cognitive functioning. Those learners from cultures that are not considered mainstream or who have received inadequate mediated learning in the past can also benefit from dynamic assessment measures (Murphy & Maree, 2006).

Murphy and Maree (2006) looks at research in South Africa in response to dynamic assessment. An analysis of various theses, dissertations, and published research (29 studies in total) shows that 21 (or 72 percent) of them have a positive view of dynamic assessment, in that it is able to predict performance to the same degree or better than conventional tests. Six of the studies found partial support and cautiously encouraging results for dynamic assessment in South Africa. These studies include findings such as the types of learners who may or may not benefit from dynamic assessment, as well as evidence suggesting that exposure to training within a testing situation allows a better pre- to post- test transfer score. Certain sub-tests of the LPAD do correlate with certain school subjects. In terms of culture fair testing, there have been results showing that dynamic assessment batteries should be used as initial tools before static measures or in combination with them. Some of the negative findings include that mediation as a strategy may indeed not prove effective in modifying performance. The degree to which a student’s performance is modifiable may also not predict or constitute academic success. Also, in some instances static measures exceeded the predictive capabilities of the dynamic measures. Of the 36 tertiary institutions surveyed in the study, 27 percent used dynamic assessment either in the past or at that point in time There are also other negative results, especially with the Learning Performance Assessment Device (LPAD), in contexts of gifted populations; and also, when attempting to predict academic success at a multicultural tertiary institution using the Ability, Processing of Information and Learning Battery (APIL-B) (discussed later) (Murphy & Maree).

The research is also marred by methodological issues as far as mediation techniques are concerned. Inadequate sample size is also frequently listed as a problem, resulting in poor statistical defensibility of results. It is perhaps too much to expect that researchers should develop and norm criterion tests as part of their studies, however making use of normed criterion variables on samples that are not the normed population skews the results and findings when learning potential tests are compared to these criterion measures (Murphy & Maree, 2006).

According to Elliot (2000) findings suggest that dynamic assessment has failed to produce consistently superior predictive power compared to traditional intelligence measures (Grigorenko & Sternberg, as cited by Elliot). It is however important to realise that traditional intelligence tests do not measure the same construct as learning potential tests, although there is a significant

degree of overlap. Predications based on dynamic measures require an appropriately contingent intervention if they are to be realised (Feuerstein; Haywood, as cited by Shayer & Beasley). Thus, the focus is on the potential gains for the individual with short-term instructional inputs, and what degree of assistance is required – thus, the present readiness to respond to intervention, and not a long-term prediction of performance. Lidz (as cited by Elliot, 2003) mentions, as a word of caution however, that those dynamic approaches that appear to best fit scientific requirements often appear to have less utility for diagnosis and intervention

In South Africa, the individual administration of learning potential tests is, although it is the most desirable method, not a viable option. The benefits of standardisation, as always, include that a group of learners can be assessed at any one time, and issues of psychometric importance such as reliability and validity can be monitored and measured more easily – thus ensuring equity of test administration (Murphy & Maree, 2006).

There are some locally developed and normed learning potential assessment devices resulting from a shift away from internationally developed tests in South Africa. The majority of studies in South Africa have secondary or tertiary education as their field of application and the main emphasis was placed on the prediction of academic success. While the field of application for dynamic assessment has traditionally remained in the arena of education, this is not the exclusive domain of the approach. It is versatile enough to warrant further research in other contexts in South Africa. Dynamic approaches should integrate important psychometric aspects into its own framework, while simultaneously espousing positive change as its primary goal. One way of doing this is to standardise dynamic assessment tools while remaining cognisant of the fact that norms do not play as large a role in dynamic assessment as they do in conventional assessments (De Beer, 2006; Taylor, 1994). Many researchers claim that there is a large pool of data available on the topic, although the research does seem to be rather fragmented. Viewed collectively however, dynamic assessment research results support the case for dynamic assessment to be implemented and used on a wider scale. Collaborative research is however needed.