Mediciones Cualitativas

Before  using  the  application  in  the  hospital  with  patients,  it  is  important  to  have   as  few  usability  and  design  errors  as  possible.  The  experiment  described  in   section  7.5  can  only  be  executed  once;  changing  the  application  during  the   testing  on  patients  could  alter  the  results  and  is  therefore  not  possible.  Time   wise,  it  is  also  sensible  to  prevent  design  errors  to  creep  in  to  the  

implementation,  since  fixing  this  after  the  implementation  has  been  completed   would  take  more  time.  The  designs  presented  in  the  previous  section  are   therefore  tested  on  usability  issues.    

This  usability  testing  is  also  in  line  with  the  iteration  and  prototyping  paradigm   suggested  by  Dix,  Finlay,  Abowd,  &  Beale  (2004,  p.  220).  This  checking  of  

(possibly  paper-­‐based)  prototypes  “whether  they  are  acceptable  and  where   there  is  room  for  improvement”  can  go  on  until  there  are  “no  more  problems  

that  can  economically  be  fixed”.  For  this  project  there  is  only  time  for  one   iteration  of  usability  testing.  

A  problem  with  summative  usability  studies  is  that  the  system  is  judged  based   on  predefined  performance  goals.  Setting  these  goals  beforehand  is  difficult  and   achieving  the  goals  may  not  necessarily  mean  that  the  final  system  has  high   usability.  Usability  metrics  “rely  on  measurements  of  very  specific  user  actions  in   very  specific  situations”.  In  the  early  stages  of  design,  the  designer  does  not   know  yet  which  actions  and  situations  there  will  be  (Dix,  Finlay,  Abowd,  &  Beale,   2004,  p.  241).  This  usability  testing  will  therefore  be  formative,  and  focus  more   on  qualitative  information  (e.g.  users’  ideas  about  improvements)  than  

measuring  e.g.  efficiency,  effectiveness  and  satisfaction  levels.    

The  usability  testing  is  done  in  a  Wizard-­‐of-­‐Oz  simulation  where  the  designs  of   section  5.2  are  used  as  a  paper-­‐prototype  to  simulate  the  application.  

Participants  of  the  usability  study  should  generally  be  as  close  as  possible  to  the   target  group  of  the  application.  However,  it’s  not  possible  to  test  the  design  on   patients  of  the  hospital,  and  therefore  healthy  people  are  used.  Since  the   persuasion  elements  target  hospital  patients  with  their  specific  behaviour,  

persuasion  is  not  part  of  this  usability  testing.  The  target  group  of  the  application   is  very  broad,  so  there  are  no  specific  conditions  for  the  testing  participants  only   that  it  is  generalized  and  non-­‐specific.  The  well-­‐known  article  of  Nielsen  and   Landauer  (1993)  shows  that  after  five  users  the  number  of  usability  problems   found  decreases  considerable.  They  also  estimate  that  the  optimal  number  of   users  for  user  testing  in  a  small  project  would  be  seven.  Although  it  has  to  be   mentioned  that  this  static  number  of  users  is  controversial  (Lewis,  2001),  and   some  more  difficult  approaches  have  been  proposed  (Woolrych  &  Cockton,   2001).  

First,  all  participants  have  to  sign  a  consent  form,  which  states  that  they  give   informed  consent  to  the  usability  testing.  The  consent  form  also  serves  as  a   standardized  way  of  explaining  the  usability  test.  Therefore  it  will  give  some   background  information  about  the  research,  where  and  how  the  application  is   going  to  be  used  and  explains  the  procedure.  The  consent  form  is  based  on  the   template  from  the  User  Experience  Group  of  Indiana  University  (2005)  and  can   be  found  in  appendix  A.1.  

Second,  the  participants  in  the  usability  testing  do  a  walkthrough,  which  means   that  they  go  through  the  (simulated)  system  by  doing  some  tasks.  These  tasks   should  of  course  not  be  too  specific  (e.g.  press  the  green  button)  because  that   would  contradict  the  goal  of  the  testing,  which  is  finding  unclear  or  problematic   design  issues.  The  users’  actions  are  video  recorded  and  users  are  asked  to  think   aloud.  Thinking  aloud  has  the  advantage  of  getting  insight  into  the  reasons  why   users  do  certain  actions,  and  can  be  the  basis  for  questions  after  the  tasks  (Dix,   Finlay,  Abowd,  &  Beale,  2004,  p.  343-­‐347).    

There  are  three  tasks  the  participants  have  to  do.  The  time  it  takes  the  user  to   complete  this  task  is  measured.  If  the  user  cannot  succeed,  a  hint  will  be  given,   but  this  is  mentioned  in  the  results.  The  three  tasks  are:  

• 15  minutes  before  lunch  you  receive  a  notification  (figure  5.1);  open  it  

• After  lunch  you  have  to  fill  in  what  you  ate  and  drank  (figure  5.3).  Select  

something,  answer  the  questions  (figure  5.4)  and  go  to  the  nutritional   overview  (figure  5.6).  

• Go  to  the  social  section  (figure  5.7)  and  visit  Anna’s  profile  (figure  5.8).  

Third,  to  obtain  some  general  information  and  impressions  of  the  user,  a  small   questionnaire  will  be  held.  Usability  questions  have  a  five-­‐point  scale  and  two   questions  are  open-­‐ended.  The  questionnaire  is  based  on  the  Standardized   Computer  System  Usability  Questionnaire  at  IBM  (Lewis,  1995).  This  survey  is   also  recommended  by  Lazar,  Feng,  &  Hochheiser  (2010)  as  one  of  the  

recommended  tools  for  usability  testing  and  evaluation.  However,  seven   questions  (about  completion  of  work,  productiveness,  error  messages  and   documentation)  not  related  to  the  application  under  observation  were  removed.   The  questionnaire  also  includes  questions  about  the  age,  gender,  experience  with   tablets  and  hospital  visits  of  the  user.  The  complete  questionnaire  can  be  found   in  appendix  A.2.  

As  has  been  mentioned,  the  focus  of  this  usability  testing  will  be  on  qualitative   information.  The  fourth  and  last  part  of  the  usability  testing  is  therefore  a  semi-­‐ structured  interview  with  questions  related  to  the  observation.  This  interview  is   done  after  the  questionnaire,  to  prevent  biased  responses  (Brooke,  1996).  The   thinking  aloud  and  other  remarks  made  by  the  participants  –  in  combination   with  this  interview  –  will  be  used  to  help  clarifying  reasons  why  users  behave  or   think  a  certain  way  (Dix,  Finlay,  Abowd,  &  Beale,  2004,  p.  348).