This section discusses how the main research contributions of this thesis relate to the product development life cycle and can support product development activities. In Figure 9, the main research contributions are anchored to the steps in UX and the product development life cycle model by Roto et al. (2014) presented in Figure 5. In the adapted product life cycle model in Figure 9, one can see that the evaluation data gathered after product launch, as suggested by the findings in P3, can be used to update the current product (in case of, e.g., software products) or
to inform the design process of the next product version. The research contributions of this thesis mainly relate to investigation (steps 1, 2, 4) and evaluation activities (steps 6 and 10) during the product development life cycle as illustrated in Figure 9.
First, this research provided examples of UX goals, a notation for verbally describing UX goals, a description of characteristics for a good UX goal, and instructions for defining and evaluating UX goals. All these can be utilized in the very first stages of UX goal definition (steps 1–4). These steps include various investigative activities when the development team aims at understanding the current tasks (e.g., familiarization with the context by interviewing users), and
7. Implement
Steps in Product
Development Life Cycle
1. Gain insights 2. Set UX goals 3. Design implications 4. UX targets Refine UX goals 5. Concept design 8. Go to market
9. After sales support
10. Collect feedback
Updating the current or developing the
next product
Research Contributions
6. Evaluate
Notation for describing UX goals
Characteristics for a good UX goal
Instructions for defining andevaluating UX goals
Experience Goal Elicitation Process modelUX Goals
Create
Feedback
Sources for Insight and Inspiration Empathy: Inspiration from Designer's Empathic
Understanding of Users' World
Vision: Inspiration from Investigating the
Deep Reasons for Product Existence and Envisioning Renewal
Theory: Deriving Experience Goals from
Scientific Understanding of Human Beings
Technology: Experience Goals Identified
Based on Possibilities and Challenges of a New Technology
Brand: Experience Goals Derived from
Company and Brand Image
Tentative Experience Goals Prioritizing
Prioritized Experience Goals Choosing Chosen Target Experience Goals
Defining Experience Goals Approaches for Processing Information (examples) Brainstorming Bodystorming Co-design Participatory design
Means (e.g., artifacts) to communicate experience goals (examples) Sketches Mood boards Personas Scenarios User journeys Presentations Reports Stakeholders
Designers, Researchers, Users, Clients, Experts in the field, Developers, Management, etc.
Perceived usefulness of long-term UX evaluation results
Benefits and challenges of specific methods in long-term UXevaluation studies
Perceived benefits of usage data logging for product development purposes
A set of questions to inspire discussions regarding the feasibility of usage data logging
Guidelines for developing and evaluating visual data analytics tools for logged usage dataUsage data logging Long-term UX evaluations
Research Contributions and Their Relation to
the Product Development Life Cycle
Figure 9. The relation of the research contributions to the steps in the product development life cycle. The Product Development Life Cycle on the right from Roto et al. 2014, adapted with permission (see Figure 6 for original).
defining UX goals and other requirements for the developed system (Roto et al. 2017). The Experience Goal Elicitation Process (Figure 7) illustrates the process for setting UX goals. It expands the model by Roto et al. (2014) by including examples of means (e.g., artifacts) for communicating UX goals with the development team and relevant stakeholders, whose feedback can further contribute to the UX goal definition process. UX goals should be operationalized as measurable UX targets to assess how well they have been achieved during the later phases of the development life cycle. These evaluations can happen during the iterative design and implementation phases (steps 4-7), or when collecting feedback from the product on the market (step 10).
When the measurement of UX goals, i.e., UX targets, is considered, methods for their evaluation should also be decided. Evaluation activities include the evaluation of the tool in relation to UX targets and evaluation of the feasibility of initial UX goals (Roto et al. 2017). In our studies, evaluation activities included evaluations made during the development process (step 6) and when collecting feedback after product launch (step 10). This research provided empirical findings regarding the usefulness of long-term UX evaluation results and the identified benefits and challenges of different long-term UX evaluation methods. These findings can inform UX designers and researchers during investigative activities when they consider which methods to use for measuring UX targets (step 4), or later on during the actual evaluation activities (steps 6 and 10). Evaluating the fulfillment of UX goals might require a high-fidelity prototype or even a finalized product that is used over an extended period. In these cases, long-term UX evaluation methods may prove to be beneficial. Finally, the developed DrawUX survey tool can be used in evaluation activities during the product development (step 6) or for evaluating products already on the market (step 10).
If feasible, the possibility of usage data logging should be considered in the early phases of product development. However, sometimes usage data logging is implemented to a product already on the market, such as in Study V. The set of questions for companies considering the utilization of usage data logging (P5, p. 433) might prove useful in these early phases. If a decision is made to utilize usage data logging, e.g., based on the expected benefits of it (see Table 10), and there is a need to start a development of new or integration process of some available visual data analytics tool, the proposed guidelines (P6) can provide support for this process. It is advisable to develop analytics tools at the same time when the logging capabilities of the developed system are implemented, so that the data wrangling activities (i.e., mapping and transferring the “raw” data to suitable form for an analytics tool) and visualizations to the data analytics tool can be designed and discussed at the same time. If such a tool for analyzing logged usage data is implemented, it can be utilized in evaluation activities during or after product development (steps 6 and 10) to further inform product development activities, such as continuous UI development (P5).
Tools and methods for long-term UX evaluations, such as those utilized in this thesis, are only some of the many methods available for evaluating how well UX targets are achieved. To some extent, usage data logging could also provide information to assess if certain UX targets,
especially pragmatic ones, are reached (step 4). However, considering the subjective nature of user experience, other methods, such as surveys measuring specific aspects of UX, or qualitative approaches, such as user interviews, might be more suitable for assessing UX targets.
The conducted studies in this thesis did not relate to all steps in the product development life cycle, but included products already on the market (e.g., digital sports equipment, pruning shears, flexible manufacturing systems) or in development (visual data analytics tool). However, these findings can provide guidance especially for UX designers, product developers, and product managers working in academia or industry, when a) investigating the feasibility of UX goals,
long-term UX evaluations, or usage data logging to support their product development activities; b) conducting long-term UX evaluations of their products; or c) participating in development of visual data analytics tools for logged usage data.