Diseño de la excursión

The study was structured as a role playing scenario involving a single participant and me in each session. I asked the participant to assume the role of a personal assistant to the chief executive officer (CEO) of the imaginary XYZ corporation. I explained that the four responsibilities of the CEO’s assistant were the following:

1. Completing the tasks assigned by the CEO via email.

2. Helping fellow XYZ employees schedule meetings with the CEO. 3. Answering emails from people outside the company.

4. Running errands by car or on foot for the CEO.

I stated that, in the past, the assistant had found it difficult to meet the first three requirements while carrying out the fourth. The assistant simply could not walk or drive safely while using the laptop, and had to either take frequent breaks to work on the laptop or save all the computer work for when he or she returned to the office. The CEO was not particularly happy with either solution.

I went on to describe the CEO’s latest attempt at overcoming this problem. The CEO had recently purchased a new software package for the assistant’s laptop. Once installed, the package would allow the assistant work with any program on the laptop by conversing with it as if it were a real, intelligent person. I stated that I would be playing the part of this new application throughout the scenario.

I explained that the CEO had high hopes for the new software. The CEO believed it would allow the assistant to continue running errands while answering email, scheduling appointments, and looking up information on the Web. To test this theory, the CEO was now asking the assistant to perform an initial, thirty minute evaluation of the new software before taking it on the road. The participant was told he or she would be performing this evaluation in the role of the assistant.

To summarize, I reiterated that I would be acting as an audio-only intermediary between the participant and a typical desktop computer. The participant would not be allowed to see the computer screen or touch its mouse and keyboard. Instead, the par- ticipant would be expected to speak to me to give commands and ask questions, which I would carry out on the computer or answer appropriately. I stated that the participant’s goal was to see how many tasks assigned by email he or she could successfully complete during the thirty minute trial.

Rules

After introducing the scenario, I read aloud the rules to which I would adhere while playing the part of the auditory display:

ˆ I would perform user commands exactly as specified, if possible.

ˆ I would remember information, if specified (e.g. “Skip all emails from [email protected]”, “Remember that URL for later”).

ˆ I would speak in order to respond to user questions and to report feedback from an application.

ˆ I would never interrupt the user but will always allow the user to interrupt me. ˆ I would notify the user of important events during breaks in the conversation. ˆ I would ask the user to clarify vague commands (e.g. “Handle all my email for

me”).

ˆ I would guide the user out of intractable problems.

ˆ I would not look at the user in order to prevent communication via back-channels (e.g. hand gestures, facial expressions).

To further clarify the rules and give the participant an idea of how the interaction might proceed, I next played three pre-recorded audio examples. The audio segments featured a female user conversing with a male auditory display in the following manner: 1. The user asks me to open email program, start a new message, states the recipient, and asked what other information is needed to complete the email. The auditory display reports the other fields that need to be filled in before the message can be sent.

2. The user asks me to open a Web browser, interrupts the display as it starts to read the first page that loads, and tells the display to announce only the title of the page and the number of links when a page first loads. The display responds by providing the request page name and number of links, then announces that three new messages just arrived in the email program. The user asks to hear the subjects of the new messages.

sender and subject. The user asks for the body of the message, and, upon hearing that it contains a link, asks the display to activate it immediately.

Tasks

The tasks to be completed by the participant were initiated by email messages received by the participant throughout the study. At the start of the scenario, ten such emails were located in the participant’s inbox awaiting processing. Up to four new emails arrived later during the session at predetermined times or in response to user actions. Table 3.2 gives an overview of the emails sorted by their default ordering in the inbox.

Not every email was reviewed or received by every participant. Some participants ran out of time before reading all of the emails. Other participants never satisfied the pre-conditions for later messages.

Discussion

At the completion of the scenario, I asked each participant the following series of ques- tions about the experience. Additional questions were asked as appropriate to help clarify some responses and delve deeper into topics of interest to the study.

ˆ Do you have any comments or questions?

ˆ What task was the easiest to complete? The hardest?

ˆ How well do you think you did in completing the assigned tasks?

ˆ What questions did you ask repeatedly? What commands did you give repeatedly? ˆ What feedback from the auditory display did you find useful? Useless?

ˆ How did this experience compare with the way you normally use a computer? ˆ What did you like the most about the interaction? Like the least?

# Request Possible solution 1 CEO requests 1/10/05 be set aside

in his calendar for a site visit, asks that all current appointments be rescheduled

Visit calendar, block off day for site visit, email two people to reschedule their meetings

4 XYZ employee Leo Marcos asks for a copy of an email the assistant sent to him a week earlier about the new XYZ product

Find message in sent mailbox, for- ward to requester

5 CEO asks that the proposal docu- ment attached to an email from Bob Brown be forwarded to him when it is received

6 CEO informs the assistant of the location drive, folder, and zip file of the new company catalog (H drive, documents folder, catalog zip), states which file in the zip can be sent to people outside the com- pany (catalog for customers) 8 Non-XYZ employee asks for a copy

of the company catalog

Attach file found in drive, folder, zip archive given in #6

9 XYZ employee Amy Georges asks to set up a meeting for 1/10/05 at 11 AM, asks for a suggested meeting time as close to her preferred time as possible but no earlier if her pre- ferred time is not available

Visit calendar, review appointments on 1/11/05, suggest 1 PM

11 XYZ employee Bob Brown says he is unable to send the proposal via email, provides a link to his Web site, username, and password for the assistant to download it

Follow link, find proposal link on page, enter username and password, report link broken to Bob

12 XYZ employee Leo Marcos says thanks for the forwarded copy 13 CEO asks for a direct link to the

video library Web page on the UNC- CS department Web site

Visit UNC homepage, visit local search page, search for “video li- brary,” visit first search result link, copy URL, email URL to CEO 14 XYZ employee Amy Georges ac-

cepts the suggested time slot or re- iterates her desire for a suggestion

Table 3.2: User tasks in the ideal auditory display study assigned by email. Email numbers missing from the table were spam messages that could be ignored or deleted. Receipt of the final four emails was dependent on completion of previous tasks.

ˆ Do you ever listen to news or music while performing other tasks? If so, what tasks?

ˆ Would you passively listen to your email or Web pages while performing those tasks? Would you actively browse them?