Adrian Second et al. [107] crowdsource a user study to measure3Dobject viewpoint good- ness, in which participants are shown two different viewpoint images of an ‘object’ and are asked: “Which of these two views do you prefer?”. The results of the large user study are then used to optimise the parameters of a computational model for viewpoint goodness. The learned model is used to predict people’s preferred views for different varieties of objects. Au- thors in [20] propose the concept of ‘schelling points’, which are ‘salient’ feature points on3D surface having several fundamental applications in computer graphics. In their study, they ask users to select points on the surface of an object mesh that they think will also be selected by other users. The collected data from humans is analysed using local and global shape proper- ties such as symmetry and curvature. The same properties are further used to predict where ‘schelling’ points on the surface of an input mesh will be.
Gingold et al [46] introduce human micro-tasks to solve perceptual problems in graphics, for example in the problem of augmenting an image with high-level semantic information such as symmetry can be aided by human input (Figure 2.2.1). They emphasise on the idea that hu- mans are good at visual tasks such as tagging an image, while computers are good at numerical computations. Thus, in their approach they define an algorithm that uses ‘human processors’ for small visual tasks along with digital processors.
Nghiem et al. [88] demonstrate a system to exploit workers at crowdsourcing platforms to build semantic links between a product’s textual description and it’s corresponding3Dvi- sualisation. On a web-page based interface, participants read textual product description and locate product features on the rendered3Dmodel of the same product. For example, in the textual description of a digital camera, participants may read ‘shutter button to capture photo’
Figure 2.2.1: The process envisioned in [46] to involve human computation (HC) to solve problems in computer graphics and vision. The figure depicts a human who is us- ing an interactive application to solve a perceptual problem such as “create depth lay- ers”. The application code invokes a Human Computation (HC) algorithm to utilise hu- man processors (HP). Specifically, The HC algorithm takes advantage of crowd of human workers to solve perceptual tasks and give results back to the main application.
Figure 2.2.2: Crowdsourcing interface used in [61] to learn tactile mesh saliency. In (a) authors show, as part of instructions before attempting the task (or ‘HIT’ in Mechanical Turk terms), two examples of images with correct answers. The participants are asked to ‘imagine the virtual shape as if it were a real-world object, and to choose which point is more salient (i.e. grasp to pick up, press, or touch for statue) compared to the other or that they have the same saliency’. In (b), authors show two examples of real questions.
in the product description and locate the same on the rendered3Dmodel of the camera. This linkage is useful in enhancing online3Dproduct browsing experience for customers.
Lau et al. [61] use Amazon Mechanical Turk crowdsourcing platform to collect mesh saliency data to measure tactile mesh saliency. They ask humans to compare between pairs of vertices of a mesh and decide which vertex is more salient (Figure 2.2.2). In another work [60], they learn a model of perceived softness of virtual3Dobjects. Similar to previous work, they collect crowdsourced data where humans rank their perception of the softness of vertex pairs on virtual3Dmodels.
Authors in [112] use Amazon Mechnical Turk to collect ratings of geometric human bodies with respect to 30 body attributes, such as curvy, fit, heavyset, round apple etc. Using the
collected data they learn a linear function relating these ratings to3Dhuman shape parameters. Specifically, they learn a mapping between a linguistic body space and a geometric body space. An important finding of their work is that humans share an understanding of the3Dmeaning of shape attributes used in their work.
The work presented in [41] builds a data-driven model of style similarity for 2D clip art with crowdsourced data. In order to collect large amount of data on human style preferences, authors use crowdsourcing platform to show each participant questions having three pieces of clip art A, B, and C, and ask: “Is A more similar to B or to C?” (Figure 2.2.3 a) The collected data is used in a linear ‘metric learning’ method to develop a style distance measure between two given pieces of clip-art. The distance metric is build by computing an over-complete set of features encoding ‘colour’, ‘texture’, ‘strokes’, and ‘shading’.
Jun-Yan Zhu et al. [139], use popular crowdsourcing platform Amazon Mechanical Turk to collect pairwise comparisons (e.g., “Is expression A more attractive than B?”) to score the at- tractiveness of facial expressions to train a model to automatically predict facial attractiveness of different expressions of a person. They note that collecting data for such studies in pairwise comparisons is a common approach since it is much harder for people to provide an absolute score. In their approach, a novel active learning scheme is also described to help both cus- tomise the learned model to the user’s data and select the user’s top expressions across a range of seriousness levels.
Liu et al. [67] learn a measure of style compatibility for furniture models using a combi- nation of crowdsourcing and machine learning. In each task they pair one furniture item (say chair) with six other different pieces of furniture from another category (say tables), and ask participants to select two pairs that are stylistically similar. In this way, using one task they are able to gather more style similarity data. Lun et al. [74] use similar setup as in [41], except the participants are given two additional options: “can not tell-both B and C”, “can not tell- neither B nor C”.
Sean Bell and Kavita Bala [9], learn an embedding for visual search in interior design. Specifically, they learn a distance metric between an object in-situ (i.e., a chair in a drawing room image) and independent product image of that object (i.e., product image with white background). They use a unique crowdsourced pipeline to collect a large number of pairings between scene images and the individual product images. For data collection, participants are asked to draw bounding boxes around a product appearing in a scene with other objects. With this data, they design a deep convolutional neural network to learn an a distance function.
Koyama et al. [58], present a method to incorporate crowdsourced human computations for a traditional design optimisation problem dealing with parameter tweaking in graphics de- sign. An example of such problems is when a designer has to spend a lot of time in color en- hancement of photographs because parameters such as ‘brightness’ and ‘contrast’ need a care- ful tweaking to obtain pleasing results. In this method, the participants are asked to perform a
Figure 2.2.3: Examples of tasks presented to participants for collecting style similarity data for clip-art (a) and3D shapes (b) in [41] and [74], respectively. In (a), left image shows an example in which style of source clip-art ’A’ is matched with target clip-art ’C’, and right triplet shows a real style matching task. In (b), there buildings ’A’, ’B’, and ’C’ are shown where participants are asked to click on either ’B’ or ’C’ based on which they think matches more in style with ’A’.
sequence of single-slider manipulation micro-tasks to adjust the parameters. Data collected in this manner from crow is used in a novel technique extending Bayesian optimisation to allow many manipulation tasks using a single slider.