Durante el período de desembolsos

Mobile Mathematics Learning System

The fourth case study was conducted in the CCD MobiLe research project and aimed at

exploring the motivational aspects of South African learners in mobile mathematics learning context. Although the motivational aspects are not in the scope of this thesis, the study provided an interesting case for defining UX goals based on user research and literature.

Furthermore, experiences from usage data logging during the study inspired the formulation

of the third research question RQ3, further expanding the scope of the thesis. The study was

carried out between June 2014 – April 2015 and is reported in P2.

Participants. Participants were Microsoft Math service users who had registered to the web

service. 53 responses (22 female, 42%) were used in the analysis. Respondents’ age was between 14 and 42 years (M = 18.3, SD = 3.87), with one respondent younger than 16 years and three older than 19 years. 45 (85%) were in a primary school or college, 5 (9%) were higher education students, one in working life, and 2 (4%) in other life situation. 68% rated math in general to be

easy or very easy, suggesting that the sample was slightly biased towards learners skilled in math. (P2)

Procedure and methods. A two-part questionnaire was integrated into the service platform

and advertised in the news section of the service. Respondents were instructed to answer both questionnaires in order, after which they could participate in a lucky draw of five free airtime coupons for mobile data. In addition to respondents’ background, the questions included topics such as user experience, motivation, behavioral intentions, and use context (see P2 for question details). The survey was open for one month in December 2014. Log data from the service usage that were used in the analysis included the number of completed separate mathematical quizzes by each participant. (P2)

Data analysis. Responses to qualitative questions were initially coded and categorized by one

researcher and iterated with another researcher, after which the second round of coding was conducted, resulting in the final categorization of the responses. Quantitative data were analyzed with SPSS software, using appropriate non-parametric tests. Log data were analyzed with SPSS for making sub-group analysis and inspecting the actual usage of the service. Initial UX goals were derived from the analysis results and literature by one of the researchers and then iterated together with another researcher to form a set of three proposed UX goals. (P2)

3.2.5 Case Study V: Exploring Perceptions Towards Usage Data Logging

and Supporting Visual Data Analytics Tool Development in

Industrial Manufacturing Automation Context

The fifth case study was carried out during a collaborative research project with a manufacturing automation supplier company (UXUS project). The aim was to support the

utilization of usage data logging in R&D related activities by developing a visual data analytics tool in close collaboration with the practitioners from the company. This case study

was included to the thesis because it further explored the theme of usage data logging from Study

IV and offered an interesting case of long-term UX evaluation of a product in development, i.e.

the data analytics tool. During the long-term evaluation period of the analytics tool between May – November 2015, repeated interviews and questionnaires were carried out to understand what kind of perceptions the practitioners had towards usage data logging, how practitioners’ current work practices related to data logging and what kind of needs they had that usage data logging could support. Tool evaluations with the practitioners were carried out to provide feedback for the developers of the usage data analytics tool. The results summarize the perceptions towards usage data logging in manufacturing automation context and provide guidelines to support the development of visual data analytics tools for logged usage data in manufacturing automation context. (P5 & P6)

Participants. Six employees from the company participated the long-term study, but one

developer left the study midway. The respondents represented roles with tasks such as 1) management of research and innovation development, 2) technical customer support, 3) product management, and 4) software development. Respondent details are presented in P5.

Procedure and methods. The study process was inspired by the Multi-dimensional In-depth

Long-term Case Study (MILC) approach as described by Shneiderman and Plaisant (2006). Over the six-month study period, feedback was collected from the updated versions of the developed tool with several methods, including 1) user observations with interviews and 2) online surveys with the practitioners from the company, and 3) heuristic evaluations with external evaluators. Log data were collected to follow the analytics tool usage over its development period. User observations with interviews provided mainly qualitative data regarding the usability of the tool and practitioners’ perceptions towards usage data logging, while also supporting the understanding of practitioners’ current work tasks and needs related to data logging. Questionnaires provided quantitative data concerning specific factors related to the tool’s user experience over time. Two researchers planned and conducted the data collection during the first two iterations of the tool, after which one researcher carried out the data collection during the following three iterations. (P5 & P6)

Data analysis. Qualitative data were analyzed with conventional content analysis (Hsieh and

Shannon, 2005) by coding comments regarding different perceptions towards usage data logging into representative categories. The first set of the guidelines to support usage data logging tool development was created by one of the researchers and then discussed and iterated with the whole research team. (P5 & P6)