This section focuses mainly on similarity of form and structure as it relates to perception, cognition. Similarity as a process has many potential applications and it is important to various context and to a diverse range of disciplines. Similarity is often considered as a fundamental mechanism that enables humans as well as computers to grouping things together, learning, making comparison, and develop categories. As Quine said “there is nothing more basic to thought and language than our sense of similarity”, (Ibid: 1969: p. 116). The notion of similarity appears fundamental to theories of perception, learning, and judgement, but it has also been argued that it has weaknesses. However many researchers and scholars have argued that similarity can only suffice to explain a limited number of mental phenomena. According to some similarity is (i) too vague and unconstrained to provide a complete explanation of how we form categories or make comparisons, (ii) not elaborate enough to account for all categories, and (iii) is highly context depended (Goldstone, 1994). It has been argued that more or less everything might be seen in a sense similar to everything else in some respect. Obviously that possess a serious challenge to similarity (ibid: 127).
When Hume introduced the term resemblance for discussing similarity he recognized that similarity as a process is of great importance in making comparison between objects and concepts, but he also noticed that not all properties have equal weight in assessing similarity. He argued that when a quality becomes very common amongst many objects or concepts these properties lose their significance in establishing links between two or more objects. According to Hume these properties are given less weight as a criteria in a similarity judgement, as we are inclined to nominate less attention to these features because the possibilities of choice become immense. Based on these it could be said that having common properties might not be adequate to explain similarity. Another approach to consider resemblance/similarity is the concept of likeness, were a number of objects are comparatively tested for similarity against each other. For example (a) might be considered to be more like object (b) rather than (c), (Gamboa, 2007)Another alternative to property based resemblance is that resemblances occur on higher-level attributes this involves abstract relations or causal-structural. In abstract resemblance we have emergent or contextual
57
Figure 29. Methods used to assess dissimilarity and the advantages and disadvantages of each method (Kriegeskorte & Mur, 2012).
property that establishes similarity or might be seen as resembling. For example empirical research findings show that subject judged a raccoon and a snake to be more similar when the word pet was presented above the two representation than when no context was provided, (Barsalou, 1999).
Goodman who did theoretical work in analysing similarity/resemblance suggested that similarity or likeness between two units such as X and Y cannot be established until a third contextual/psychological property Z defines in which respect X and Y are compared, (Goodman,
58
1972). Consequently, when a person is asked two make a similarity judgement between two units without defining Z property guessing what Z property/ies will be used in the judgement is hard to predict. Goodman suggested that determining what criterion is psychologically important when making a similarity judgements is a hard task (Goldstone, 1994). It is therefore reasonable to say that similarity does not only depend on structural and objective features but it also has a psychological dimension to it.
Figure 30 illustrates nicely the problems deciding which attribute will be psychological important in a similarity judgement. Similar problems arise in computational feature matching. For example when using criteria for sound selection/ identification based on similarity or dissimilarity features. In the example of the figure below we could distinguish between two features that would suggest different similarity classification, one based on the outline or the second based on texture. Determining which on will be psychologically important it is a hard task as the following experiment suggests. Students were asked to classify the images in Figure 30 in two group, were each group consisted of two images. One third of the students grouped A-B, C- D the one third A-D, B-C and the other one third did not answer the question, (Jehan, 2005). These demonstrates some of the difficulties that might be faced and the decisions that have to be made, when it is required to classify objects or other entities based on set of criteria.
The selection criteria based on which a subset is selected is in a dialectic relationship with interest and intentions of the subject who makes the similarity judgement. Interests and intentions that define selection criteria might vary depending on context. As for example Barsalou findings show that subjects judged a raccoon and a snake to be more similar when the word pet was presented above the two representation than when no context was provided, (L. W. Barsalou, 1999), or similarly with the example we have seen earlier were a cat and dog are considered more similar if we introduce a bird in the comparison that if we just compare the two.
Figure 30. Simple example of the feature selection problem when making a similarity judgment in the visual domain (Jehan, 2005).
59
3.7 Conclusions
This chapter reviews literature that investigates the cognitive mechanism that mediate cross- modal binding and congruency effects. As it was discussed in this chapter, at first glance, sensory modalities might appear to be of such disparate nature. Nevertheless sensory modalities appear to converge and overlap in a number of ways. Understanding better the equivalences between the sensory modalities can deepen our knowledge of how we perceive and experience information in the environment. Research findings have shown that for a successful multimodal binding to occur, the relationship between two modal objects and their attributes must be plausible in terms of: (i) of time (i.e. synchrony), (ii) space (i.e. collocation), and (iii) prior experience of similar events and phenomena. The underlying principles that mediate cross-modal binding and congruency effects, particularly beyond spatiotemporal integration are poorly understood. The need for systematic empirical work to shed light on the multimodal nature of perception is evident.
Furthermore, as I have argued, different methods are better suited for studying cross- modal correspondence at different hierarchical levels of sensory information processing. Speeded methods are well suited for studying structural similarity and statistical correspondence, while unspeeded methods are more suitable for studying cross-modal feature correspondence at either the statistical or the semantic level. Hence, speeded and unspeeded methods are complimentary. Moreover, in applied context such as in human computer interaction and the design of cross- modal mappings both methods can be applied to assess the effectiveness of the mapping between the users’ sensorimotor input and the system output.
60