Competencia

In tasks that require figure-ground segmentation, such as object discrimination, recognition and navigation, the contours that separate objects from their background may not always be best defined by changes in luminance. However other dimensions of colour such as hue and saturation provide additional ways to segregate an image, helping to identify objects as well as their context (Goffaux et al., 2005; Rousselet, Joubert & Fabre-Thorpe, 2005; Torralba, 2009). Rivest and Cavanagh (1996) found that contours as defined by luminance, colour, motion and visual texture were equally weighted and integrated when establishing discrete objects. However they noted that “discontinuities in luminance created by shadows are not reliably linked to object contours, whereas

continuities in other attributes (e.g. colour, motion and texture) are much more reliably linked to object contours.” Despite visual perception using a conjunction of multiple forms of information to define object boundaries, this is not possible with the most commonly researched SSDs.

The use of luminance alone in SSDs means that luminance-boundaries need to also be object-boundaries. This results in experimental stimuli for greyscale SSDs such as the ‘vOICe’ or ‘Brainport’ using single colour high contrast objects and backgrounds under consistent lighting conditions for object recognition and navigation (Auvray, Hanneton & O'Regan, 2007; Chebat, Schneider, Kupers & Ptito, 2011; Segond et al., 2005 – see fig. 4.1). The use of greyscale SSDs in more natural environments have large limitations, in which an objects’ signal varies depending on camera positioning, light positioning, light intensity, object characteristics (e.g. dark or bumpy) alongside potentially complex background information not relevant to the user (Brown, Macpherson & Ward, 2011; Capalbo & Glenney, 2009).

Figure 4.1. Left side shows object discrimination stimuli used in Auvray et al., 2007; Right side shows navigation stimuli used in Chebat et al., 2011. Both experiments use high contrast black and white stimuli with consistent lighting conditions, aspects that are less assured in complex natural environments.

In contrast to greyscale approaches which feature one boundary for edges (dark-light), colour features two additional boundaries, namely saturation (grey-colourful) and hue (red, yellow, green, blue). These three boundaries can be utilised in any combination to extract an object from its background. In computer-vision studies, a combination of luminance and hue information has been found to be reliable for computer navigation and object recognition even in shadowed environments (Crisman & Thorpe, 1993; Orwell, Remagnino & Jones, 2001). Computerised object segmentation

under variable lighting can be aided through using hue as a more stable object marker, this can also be used to reduce the influence of shadows similar to colour constancy mechanisms (Salvador, Cavallaro & Ebrahimi, 2004). Similarly in human perception, Gur and Akri (1992) found that luminance and hue processing was enhanced by the integration between these in the same stimuli (see also Syrkin & Gur, 1997). As such, the conjunction between multiple forms of colour information may be beneficial over those same cues in isolation.

Previous studies on colour in sensory substitution have used a wide variety of tasks that incorporate colour information, from simplistic abstract representations of colour for simple shapes or flags on a computer (Ancuti, Ancuti & Bekaert, 2009; Burch, 2012). It could be argued that understanding abstract colours through sound is more a measure of how well participants can represent distinct sounds with new labels. Interestingly this point has been well illustrated previously where colour information has even been presented without a visual context or application, where participants were able to obtain an understanding of a specific 'colour space' through how perceptually distinct two given points are from one another in this space even when the participants did not know it was colour that was being represented (Kahol et al., 2006). By comparison, some tasks have involved live feeds of artificially produced highly saturated colours in a real environment, such as through matching differently coloured socks (Bologna et al., 2008), navigating along painted lines outside (Bologna et al., 2010) or recognising coloured doors inside (Meers & Ward, 2004). Finally some devices have been used on naturally occuring colours such as through identifying a variety of fruit indoors (Capalbo & Glenney, 2009), the use of natural colours has the advantage of both being a part of the evolutionary basis from which colour vision originally developed (Jacobs, 2009) and introduces meaningful complexity in terms of shades of colour that reflect real environments. Another additional factor to consider is that the colours of objects vary according to their environmental lighting conditions so a 'red apple' will be a variety of shades of red as you move it across a variety of lighting conditions. This element of colour comprehension by colour SSD users has to date not been addressed.

There have been previous attempts to compare the use of colour and greyscale in sensory substitution; however these have come with methodological problems. Ancuti, Ancuti and Bekaert (2009) compared the colour based ColEnViSon and greyscale vOICe SSDs on discriminating the patterns of flags using four participants. Both SSDs performed similarly for questions of orientation and complexity; however the ColEnViSon had a marked improvement for identifying the number of colours present and recognition of specific flags. This is to be expected since similarly luminant

colours would be indistinguishable on the vOICe, capping its performance. Besides the amount of information, the style of presentation to the user also differs, for instance, pitch denotes vertical location in the vOICe and luminance in the ColEnViSon. This makes the influence of different translations difficult to gauge. Similarly, Capalbo and Glenney (2009) compared their colour SSD, the Kromophone, which converts a single point of colour into sound with the vOICe for light localisation and fruit recognition tasks. Their preliminary findings using three to six participants found that the Kromophone was superior for localisation and recognition as well as more resistant to changes in environmental illumination. However the Kromophone and vOICe are also difficult to compare because they convert different spatial dimensions and use different translations for luminance information making it difficult to conclude whether differences in information content or translation underpin these differences. The present experiment seeks to address these concerns by using the same SSD to convert either luminance or colour information, as well as present this information using a variety of representations. From this it becomes possible to disentangle the influence of information content from information presentation to the user. The next section examines the variety of colour-spaces relevant to this experiment as well as potential ways this can be turned into sound.