The virtual assistants in Clique describe the information space of available programs, tasks, subtasks, and their content. Clique places these virtual assistants around the user, separating them spatially and acoustically to maximize segregation and selection when they speak or play sounds concurrently. Table 5.1 states the names, responsibilities, and vocal properties of each assistant. Figure 5.1 depicts their spatial relationships to the user.
Names Responsibilities Vocal Properties
Content Reports content to the user (e.g., text in an email, the names of files, appoint- ments on a calendar). Indicates spe- cial properties, or states, of content re- ported (e.g., misspelled words, empha- sized text). Part of the primary view.
American male voice, 100% speech volume, 60% sound volume, 0° azimuth
Summary Reports orientation information to the user (e.g., index of the current email out of the total number of emails, time spent reading a document out of the to- tal time it would take to read the doc- ument). Indicates the type, or role, of the current task (e.g., browsing a list, editing text). Part of the preview.
American female voice, 86% speech volume, 60% sound volume, -10° azimuth
Narrator Confirms user actions by echoing im- portant input (e.g., characters typed when editing and searching). Part of the primary view.
British female voice, 78% speech volume, 60% sound volume, 10°azimuth
Related Summarizes changes in tasks and sub- tasks related to the active subtask when either changes (e.g., number of messages available after selecting a new mailbox, Web browser download com- pletion notice while browsing a Web page). Part of the preview and periph- eral view.
Deep American male voice, 70% speech vol- ume, 55% sound vol- ume, -60°azimuth
Unrelated Reports summaries of tasks and sub- tasks unrelated to the active subtask when the state of the unrelated compo- nent changes (e.g., Web browser down- load complete notice while writing an email). Part of the peripheral view.
Hoarse American male voice, 70% speech vol- ume, 55% sound vol- ume, 60° azimuth
Context Reports navigation among tasks when a task starts, completes, cancels, or re- sumes. Part of the overview.
No speech, 55% sound volume, not spatial- ized (i.e., inside the user’s head).
Table 5.1: Virtual assistant names, their responsibilities, and vocal properties distin- guishing their speech streams. Volume percentages are given to indicate relative levels, but their unit intervals are sound hardware dependent.
Earcons
The assistants use earcons to represent actions taken by the user or the system. The action earcons use timbre, rhythm, pitch, consonance, and duration to convey infor- mation about events. The instrumental timbre of the earcon indicates the role of the task in which the action occurred. The rhythm names the type of the action itself. The sequence of pitches (i.e., melody) in the earcon distinguishes actions within a class of related actions. The consonance or dissonance of the earcon, according to Western Music tonality, conveys the success or failure, the positive or negative consequence, of an action. The density of notes in the earcon represents the complexity and importance of the action. The duration of all earcons is greater than 200 ms to ensure fusion of the sound stream, but not much longer than a second to keep the display responsive. The tempo of all action earcons is rapid, nearly 200 beats per minute. Chapter 2 categorizes this set of action sounds as a familial, or hierarchical, earcon grammar.
Table 5.2 names the actions indicated by earcons in Clique, shows the earcon melody on a treble clef staff, and describes their use, acoustic properties, and relationships.
Action Staff Description
Task started
A new task loaded and is ready for userinteraction. Timbre represents the role of the task. Rhythm is the same as task completed, cancelled, and resumed. Task completed
The user completed an existing tasksuccessfully. Timbre represents the role of the task. Rhythm is the same as task started, cancelled, and resumed.
Task cancelled
The user cancelled an existing task without completing it. Timbre repre- sents the role of the task. Rhythm is the same as task started, completed, and resumed.
Action Staff Description
Task resumed
The user resumed an existing task bymaking it the active task. Timbre rep- resents the role of the task. Rhythm is the same as task started, completed, and cancelled.
Wrapped
The user jumped from the last item ina collection to the first or vice versa. The collection can be a set of subtasks or a set of content items. Timbre in- dicates the role of the task containing a set of subtasks. Timbre is always a flute when wrapping within a subtask. DeletedThe user deleted an item in a collectionsubtask. Timbre is a flute. Moved to next
line
The user browsed to the next line in an editable text document. Timbre is a flute. Rhythm is the same as moved to previous line.
Moved to previ-
ous line
The user browsed to the previous line in an editable text document. Timbre is a flute. Rhythm is the same as moved to next line. Joined
The user joined two words or lines bydeleting a space in an editable text doc- ument. Timbre is a flute. Rhythm is the same as splitting two lines or words. Harmony plays when joining lines, but not words. Rhythm is the same as split. SplitThe user split two words or lines by in-serting a space in an editable text doc- ument. Timbre is a flute. Rhythm is the same as joining two lines. Har- mony plays when splitting lines, but not words. Rhythm is the same as joined.
Refused
The system refused to take an action in its present state. Timbre is a baritone saxophone.
Action Staff Description
Warned
The system warned the user about the consequences of taking an action. Tim- bre is a baritone saxophone.
Table 5.2: Clique action earcons.
The assistants also use earcons to represent stateful properties of subtasks or their content. The state earcons are all percussive to distinguish them from action earcons. Beyond this simple classification, the state earcons share no other relationship with one another. According to Chapter 2, this state earcon grammar is representational.
Table 5.3 names the states indicated by earcons in Clique and describes its use, acoustic properties, and relationships to other earcons.
State Properties
Missing A subtask is currently unavailable. Timbre is a melodic drum.
Misspelled A word is misspelled. Timbre is a tambourine.
Navigable An item in a collection contains additional items. Tim- bre is a drum.
Beginning The reading position is at the beginning of a body of text. Timbre is a timpani drum.
End The reading position is at the end of a body of text. Timbre is a timpani drum.
Recording The system is recording or no longer recording speech output. Timbre is guitar fret noise.
Searching The system is accepting or no longer accepting user in- put for a keyword search. Timbre is a sandblock.
Auditory Icons
The assistants use auditory icons to identify the roles of subtasks and content in Clique. The role of a subtask informs the user of what interactions it supports. Therefore, the mapping between the auditory icons used and the subtasks represented hints at the possible actions in the current context (e.g., commands for browsing a list). Similarly, the role of a piece of content identifies its purpose in a body of content. In this case, the mapping distinguishes a segment of content from other content (e.g., a heading in a document).
Because many computer interactions cannot be directly related to the sounds of ev- eryday, physical events, Clique uses symbolic and metaphorical auditory icons as defined in Chapter 2. The duration of each auditory icon is at least 200 ms to ensure fusion of the sound stream, but less than one second to keep up with fast paced interaction.
Table 5.4 names the subtask roles represented by auditory icons in Clique and de- scribes the sounds themselves. The definition of each role and their differences is left for Section 5.3.3 of this chapter.
Role Sound
List Fast rippling through a deck of cards Tree Wind blowing tree leaves and branches Text Papers shuffling
Table Plates and silverware clattering Hyperlink Zapping
Heading text Marching footsteps Editable text Typing on a keyboard
Table 5.4: Clique role auditory icons.