Remates Combinatoriales

Considering  that  the  trend  of  including  peer-­‐feedback  strategies  in  MOOCs  will   continue  to  increase  (O’Toole,  2013;  Pilli  &  Admiraal,  2017)  and  that  little  is  actually   known  about  current  practices,  the  need  to  explore  this  field  further  becomes  more   evident.  Taking  into  account  that  MOOC  participants  are  diverse  and  that  their   participation  in  the  courses  is  voluntary,  the  present  study  aimed  at  inquiring  into   the  nature  of  peer-­‐feedback  in  the  MOOC  “Managing  the  Arts:  Marketing  for  

Cultural  Organizations”.  The  method  of  content  analysis  and  a  regression  model  with   a  Poisson  distribution  were  employed  to  explore  and  describe  three  different  

aspects  of  feedback  in  the  course:  the  quantity  and  quality  of  feedback  provided   throughout  the  course;  the  feedback  provided  by  students  and  supporters;  and  the   feedback  provided  by  females  and  males  in  their  roles  as  students  and  supporters.  

To  this  end,  three  specific  research  questions  were  formulated:  

1.  How  does  feedback  evolve  over  time  in  terms  of  quantity  and  quality?  

The  quantity  of  feedback  provided  in  the  MOOC  was  inconstant  but  displayed  a   positive  trend.  The  quantity  of  badges  awarded  or  the  types  of  badges  awarded  do   not  offer  a  reliable  association  with  the  quality  of  feedback  provided,  especially  if   participants  rating  comments  were  not  the  ones  for  which  these  were  intended.  In   fact,  and  in  agreement  with  Prins  et  al.  (2006)  and  Read  et  al.  (2005),  the  value  of   feedback  appears  to  lie  in  the  perception  of  the  person  receiving  it  or  reading  it,  and   the  moment  in  learning  in  which  feedback  was  received  or  read.  This  perception   might  be  intensified  by  individual  characteristics  of  the  learners,  which  as  expected  

are  directly  associated  with  their  sociocultural  context  and  their  previous  collected   experiences.  According  to  the  findings  of  this  study,  the  types  of  feedback  perceived   as  quality  feedback  include  a  combination  of  types  within  the  dimensions  of  

feedback  introduced  by  Narciss  and  Huth  (2004),  and  Nicol  and  Macfarlane-­‐Dick   (2006):  cognitive,  metacognitive  and  motivational.  The  evolution  of  the  expected   types  of  feedback  required  for  the  suggested  types  of  tasks  was  constant,  whereas   the  other  types  displayed  alternating  shifts.  

2.  How  does  the  nature  of  feedback  differ  between  students  and  supporters?  

Differences  were  encountered  in  the  use  of  five  types  of  feedback  contained  in  the   cognitive  and  motivational  dimensions  of  feedback,  whereby  supporters  overtook   students.  These  differences  are  believed  to  be  strongly  associated  with  the  

contextual  situations  of  the  learning  phases  of  the  course  and  to  how  each  type  of   role  experienced  them  (in  relation  to  individual  aims  for  the  course,  possibilities  and   limitations).  Despite  the  differences,  both  types  of  participants  taking  part  in  this   voluntary  activity  shared  a  specific  commonality,  related  to  a  certain  level  of   confidence  (Falchikov,  2005),  thus,  allowing  them  to  engage  in  this  cognitively   demanding  activity.  Furthermore,  the  level  of  confidence  appears  to  combine   linguistic  and  academic  aspects,  as  well  as  specific  knowledge  on  the  topic  treated.  

3.  How  does  the  nature  of  feedback  differ  between  female  and  male  participants  in   their  respective  roles  as  students  or  supporters?  

Despite  the  fact  that  no  differences  could  be  established  for  the  use  of  peer-­‐

they  performed.  The  differences  encountered  challenge  findings  from  other  studies   in  which  gender  stereotypes  are  reinforced.  This  suggests  that  the  characteristics  of   the  learning  context,  the  dynamics  developed  therein,  as  well  as  the  empowering  or   supressing  situations  experienced  by  the  participants,  to  name  a  few,  greatly  impact   participants’  behaviour  during  learning  and  participation.  Moreover,  the  differences   encountered  recognise  the  individuality  in  each  participant  and  acknowledge  that   sociocultural  factors,  personal  skilfulness  and  the  situational  context  may  have   played  a  decisive  role  regarding  the  nature  of  feedback  provided.  

Despite  the  fact  that  the  number  of  participants  who  took  part  in  the  peer-­‐ assessment  activity  was  significantly  small  compared  to  the  number  of  enrolled   participants,  it  can  be  suggested  that  their  diversity  did  not  represent  a  risk  to  the   purpose  and  value  of  the  very  last  step  of  the  cycle  of  the  evaluation  process:   assessment,  which  was  expressed  as  a  concern  in  the  introduction  of  this  study.   Their  participation  was  key  in  understanding  the  nature  of  feedback  in  such  a  course   and  in  confirming  that  manifold  variables  underpin  the  intricacies  of  peer-­‐feedback.   Although  these  variables  can  be  researched  independently,  it  appears  impractical  to   address  many  of  them  in  one  study,  as  well  as  study  their  correlations.  Limitations  of   various  types  emerge,  as  presented  previously.  This  case  did  not  present  pedagogical   differences  between  the  learning  phases,  and  if  there  would  have  been  any,  these   would  have  only  represented  important  but  not  exclusive  indicators  for  the  nature  of   peer-­‐feedback  provided.  It  is  to  consider  that  the  characteristics  of  the  learning   setting  (participants’  diversity  and  voluntary  participation)  combined  with  the   individual  lived  experiences  are  equally  important,  increasing  the  complexity  of  the  

case.  A  number  of  contextual  and  individual  factors  may  play  a  great  role  in  the   nature  of  peer-­‐feedback  identified  in  the  courses.  These  factors  are  not  assumed  to   be  present  simultaneously,  but  to  emerge  depending  on  situations  experienced  at  an   individual  and  collective  level.  Examples  of  factors  are:  time,  mood,  engagement,   access  in  its  different  dimensions,  internal  and  external  incentives  and  the  works  to   be  assessed.  It  is  believed  that  because  every  MOOC  happens  in  a  unique  way,   different  variables  of  interest  will  arise  based  on  those  different  situational  contexts.   This  case  study  provides  a  unique  view  of  peer-­‐feedback  in  a  less  conventional  type   of  MOOC  that  is  team-­‐based,  offers  two  different  modes  of  participation,  and  where   different  supporting  actors  are  present.  In  contrast  to  other  MOOCs  reported  in  the   literature,  this  course  assigns  an  equal  value  to  all  three  steps  of  the  learning  cycle:   teach,  learn  and  assess,  and  the  way  this  is  accomplished  is  reflected  in  the  different   elements  integrated  in  its  pedagogical  design.  This  study  pioneers  the  exploration  of   the  field  of  peer-­‐feedback  in  MOOCs,  and  sheds  light  on  some  of  the  hitherto  

unexplored  variables  that  are:  feedback  quantity  and  quality,  and  feedback  in   relation  to  participant’s  role  and  gender.  

It  is  evident  that  although  a  specific  set  of  assessment  criteria  was  provided  in  the   analysed  course,  participants  found  their  own  ways  to  express  the  feedback  that   they  felt  was  important,  or  that  was  more  accessible  for  them  in  terms  of  language,   among  others.  This  is  reflected  in  the  emergence  of  20  types  of  feedback.  

Consequently,  it  appears  that  assessment  criteria  should  be  clearly  presented  as  a   suggestion  and  not  as  an  imposition.  The  latter  may  have  a  negative  impact  in  the   quantity  and  also  quality  of  feedback  provided,  which  can  lead  to  disappointment  for  

those  providing  feedback  but  also  to  those  receiving  it.  In  the  course  analysed,  these   criteria  were  fixed  items  that  were  in  an  automatically  generated  template.  

At  the  same  time,  mechanisms  for  motivating  learners  in  taking  part  in  this  

assessment  strategy  need  to  be  further  researched,  to  be  able  to  offer  and  support  a   lively  dynamic  in  a  learning  setting  that  interacts  with  and  depends  on  its  

community.  

The  inclusion  of  peer-­‐feedback  practices  in  such  context  is  undoubtedly  challenging,   as  the  degree  of  expectations  vary  along  with  the  topic  of  the  course,  participants’   diversity  and  own  lived  experiences.  Thus,  future  research  could  analyse  peer-­‐ feedback  in  relation  to  the  general  topic  of  the  course  and  a  common  background  of   the  participants,  as  this  can  evidently  have  a  direct  impact  on  the  types  of  feedback   that  are  used  and  that  seem  to  fulfil  the  expectations  of  those  particular  groups.  This   understanding  would  help  instructional  designers  to  focus  on  encouraging  those   types  of  feedback  that  should  be  provided,  thus  supporting  what  Kop  and  Fournier   (2010)  suggest:  

In  order  to  develop  empowering  learning  environments  that  foster  active   learning,  designers  and  developers  of  such  environments  first  need  to   understand  the  factors  that  influence  people’s  attitudes,  intentions  and   behaviours.  They  must  also  understand  the  prerequisites  for  people  to  thrive   in  such  environments  in  order  to  create  favourable  components  and  

conditions.  (p.  5)