Subdirección de Telecomunicaciones y Servicios 1 Área de Proyectos y Redes

Ideation is one of the universal constants in sketching in relation to research, and forms part of almost every (if not all) disciplines. Ideation in this context relates to the quick generation and brainstorming of many different visual ideas and is utilised at the beginning of a project phase, although it can also be used as part of a user-centred process. Examples from the field include: sketching used to increase divergent thinking via an iterative process [73]; digitisation via the creation of an electronic “back-of-the-napkin” to encourage creativity [92]; used as a ideation method to generate ideas for future interfaces by participants in user studies (see Chapter 6); or to examine how novice designers utilise differing ideation methodologies with their basic sketches [191].

3.7.2

Input

Direct input is a use of sketching that is novel to HCI and computing, in contrast to traditional, freehand sketching where the resulting image is a visual output only. The sketch-as-input relies upon complex computational processes to recognise lines, shapes, distances and stroke-widths — as well as intended meaning. Thesketch-as-input is an essential part of sketch-based interfaces [6] and is made possible by software allowing forsketch recognition. In this context, sketches can be input in several ways — directly by stylus, from a scanned or photographed image of a freehand sketch, from a pre-existing digital source, or by gesture inside programs. Examples from the literature include direct freehand input of maths symbols and diagrams (MathPad2[169]); network sketches or UML (Tahuti system [100]); using 2D sketches to convert into 3D representations (Teddy [117]); and there are even sketch recognition languages (Laddersketch recognition language [99]).

3.7.3

Output

The visual output of freehand sketching is an easy concept to grasp, but the way in which sketching output can be generated within HCI is enhanced in comparison to art and design processes. The sketch as output can relate not only to an image on paper or screen, but sketches can now generated by Neural Networks or other programs utilising a variety of imagery, and rendered as if they were a freehand representation for both generic scenes [348] and portraits of people from photographic input [29]. The output can also be coupled with

direct sketch input to create refined versions of freehand sketches as part of the beautification process via line correction (PortraitSketch[352]), or used as a workable interface design (SILK [167]).

3.7.4

Tool

Tool-based sketching is an intermediary concept, where sketching is utilised as a means to an end. This relates to the sketch as having a purpose beyond simple input or output — rather, it provides a service in a specific context. In this category, sketching can be used in education research, where the sketch has been gamified and made into a way of improving draughtsmanship [347]. The sketch can also be a tool for increasing interactivity in paper prototyping [95], to direct movement in animation (Motion Doodles [308]), or control enhanced [170] or sketched visualisations [22]. Thesketch-as-tool is also part of sketch-based image search/retrieval [52].

3.7.5

Iteration

Sketching is perfectly suited to the iterative design process, where a product or image has been decided upon, but requires refinement or further ideation. Iterative sketches in HCI relate to the incremental development of visual ideas, much as they would in design disciplines, but are used in relation to the prototyping process, on paper or on screen. Similar to ideation, the iterative sketching process is a visual record of ideas [13], but also the development of those ideas [271]. In HCI research iteration has been used to examine ways of exploring and analysing ideas from researchers in tangible interfaces [243], or as a way of providing an animated record of sketched iterations that can be played back to examine evolution [267].

3.7.6

Evidence

Sketched visuals, including iterations can be a form of record for processes and findings in a highly accessible manner. This kind of usage relates to the published findings and documentation relating to research projects in HCI, and can be purely process related, or show the results from computational output (for example). Though not universally utilised at present, Pictorials containing sketches are becoming more popular for documenting processes within HCI [88], and at conferences or other live events, sketch-noting and scribing are being embraced as adding value to both attendance and the legacy of events [335, 195]. Sketches are also used as a (private) documentary of meetings and collaboration, both in

research and industry [337, 336], and in public as a form of expression on social media (UbiSketch[40]).

3.7.7

Elaboration

Sketching can be conducted on existing items to add value and to aid understanding via a process of annotation. Annotative processes within HCI take on new meaning out-with making notes or doodles on a text when they become interactive, transferable or can be transmitted across the world in real-time. Elaboration in this context has been examined from both the interface development side [9] and in a practical manner to assist medical professionals in tracking illness and making decisions about surgery [135]. Interactive whiteboards allow exploration of visuals [171], whereas in collaborative work, sketched annotation can aid decision-making [55]. Finally, in tangible interfaces, sketched visuals can add meaning to non-planar surfaces [197].

3.7.8

Dialogue

Given the universal nature of the visual sketch, it is therefore a logical extension of the concept that it be used as a form of dialogue. Sketching on interactive tables enhances the collaborative experience [98, 119], and can also be used as a way of communicating concepts remotely via digital pen/paper capture, where textual language cannot adequately demonstrate meaning (PaperSketch[341]).Sketching user experiences[23] has long been used as a visual method to open a dialogue between researcher and user, and put ideas into believable contexts, and this has been extended into co-creating comic-strips in cyber-security [179] or creating complex visual icon libraries to allow novices to engage in sketching dialogue [178].