11. ANEXOS
11.4. Anexo IV: Transcripción de las entrevistas
A number of variations on Kappa have been developed since Monserud and Leemans (1992) first applied this method for the accuracy assessment of spatial simulation models. Despite the existence of alternative map‐comparison measures (Turner et al., 1989; Couto, 2003), Kappa and its varieties have since become the predominant measure to compare categorical maps. A full discussion of all methods for accuracy assessment goes beyond the scope of this paper;
instead Kappa, Fuzzy Kappa, Kappa Simulation and Fuzzy Kappa simulation, all used in the case study application, are discussed in the context of assessing the results of land‐use models. All four have the same basic structure – the observed agreement corrected by the expected agreement – but the definition of the expected agreement differs. Similarly all four have the same range of possible values – between ‐1 and 1, with 0 indicating an agreement as can be expected by consider the amount of change in the simulation period. For that reason the absolute values of these measures have no intrinsic meaning in the context of land‐use modelling. Land‐use models applied to areas and/or periods with little change will generally yield higher scores than when applied to areas and/or periods with many land‐use changes, regardless of the accuracy of these changes. indicates that a model has some predictive accuracy. Kappa and Kappa Simulation only consider crisp land‐use classes and crisp locations. This means they cannot account for near‐hits in terms of classes that are somewhat similar but not entirely, or in terms of nearby locations with the same land‐use type or transition. As near hits can be valuable result for a land‐use modeller, it is justified to also account for them in model assessment methods. Due to fuzziness in location, only Fuzzy Kappa and FKS are explicitly spatial, while Kappa and Kappa Simulation can also be applied to other, non‐spatial, categorical datasets.
The combination of accounting for the amount of change and near‐hits makes FKS arguably the most appropriate method out of the four Kappa variations
discussed for the assessment of the predictive accuracy of land‐use models.
However, this advantage comes at the cost of subjectivity, introduced by the modeller in the values in the similarity matrix.
However, the use of Kappa to assess the predictive accuracy of land‐use models is not undisputed: Pontius and Millones (2011) criticize Kappa statistics for several reasons; however, they do not provide an appropriate alternative, as their suggested approach does not assess the predictive accuracy of land‐use models, but instead provides insights in the types of error made. We see two possible directions to improve accuracy assessment methods: this paper presents an improved model for the expected agreement, which solves part of their critique; an alternative approach is the application of reference models such as presented in Hagen‐Zanker and Lajoie (2008). Some modellers estimate
Kappa Crisp A stochastic model of random allocation of land use classes
Fuzzy Kappa Fuzzy A stochastic model of random allocation of land use classes
Kappa Simulation Crisp A stochastic model of random allocation of class transitions relative to the initial map land‐use changes, corrected for the expected agreement and using a fuzzy interpretation of land‐use transitions. This algorithm combines properties of Fuzzy Kappa (Hagen‐Zanker, 2009) and Kappa Simulation (Van Vliet et al., 2011) in a single map comparison method. FKS has several important advantages over other map comparison methods available to assess the predictive accuracy of
land use models: It allows to differentiate between land‐use changes and land‐
use persistence because it is based on land‐use transitions rather than land‐use classes; it differentiates between near‐hits and complete misses because it uses a fuzzy interpretation of land‐use transitions; and no benchmark is needed in the assessment of land‐use models because there is an appropriate reference model implicit to this method. Moreover, by adjusting the similarity matrix FKS can be tailored to assess specific types of land‐use changes, such as the simulation of urban growth or deforestation, by changing the similarity matrix. The assessment of a specific type of land‐use change can be very useful in relation to the aim of a particular modelling study, such as studying urban growth or deforestation.
Due to its properties, FKS is very suitable for the assessment of the results of land‐use models. This was shown by applying this new method to assess land‐
use maps generated by different land‐use models. Results show that FKS, like Kappa Simulation, can differentiate between similarity due to persistence and similarity due to correct changes, which is of crucial importance in land‐use modelling. Moreover, a comparison between scores for FKS and Fuzzy Kappa show that this method is indeed capable to distinguish between near‐hits and complete misses, which is very relevant for interpreting and communicating the results of land‐use models. It should be noted that a comprehensive assessment of the results of land‐use models includes an assessment of several map properties (Hagen‐Zanker and Martens, 2008). Hence FKS is very suitable to assess the predictive accuracy but should be complemented with methods that assess the process accuracy (Brown et al., 2005).