Resultados de la simulación, en el lado anódico de la batería

In this chapter, the use of coding for reliable classification using unreliable crowd workers has been proposed. Using different crowdsourcing models, it has been shown that coding-based methods in crowdsourcing can more efficiently use human cognitive energy over traditional majority-based methods. Since minimum Hamming distance decoding is equivalent to MAP decoding in this setting, the anonymity of unreliable crowd workers is not a problem. We have shown that better crowds yield better performance. In other words, crowds which have higher reliabilities on an average perform better than the crowds with lower reliabilities. Although dependent observations have been considered in typical sensor networks, human decision makers in a crowdsourcing sys- tem give rise to multiple levels of dependencies. This work provides a mathematical framework to analyze the effect of such dependencies in crowdsourcing systems and provides some insights into the design aspects. By considering a model with peer-dependent reward scheme among crowd workers, it has been shown that pairing among workers can improve performance. Note that this aspect of rewards is a distinctive feature of crowdsourcing systems. It has also been shown that diversity in the system is desirable since the performance degrades when the worker observations are dependent.

large, such as fine-grained image classification for building encyclopedias like Visipedia2_{. In such}

applications, one might need to classify among more than 161 breeds of dogs or 10000 species of birds. Designing easy-to-answer binary questions using the proposed scheme will greatly simplify the workers’ tasks.

CHAPTER

8

HUMAN-MACHINE

INFERENCE

NETWORKS

(HUMAINS)

8.1

Introduction

In the previous chapters, systems with only sensors or only humans were studied. In this chapter, a framework for human-machine inference networks (HuMaINs) is presented. Note that the previous chapters focused on specific inference problems. An inference problem can also be referred to as problem-solving process, where the problem to be solved is that of inferring about a phenomenon. For example, an object classification task answers the question: “Which class does a given object belong to?". In other words, it solves the problem of classifying the given object into various possible classes.

In traditional economics, cognitive psychology, and artificial intelligence (AI) literature, the problem-solving process is described in terms of searching a problem space, which consists of various states of the problem, starting with the initial state and ending at the goal state [4]. Each path from the initial state represents a possible strategy which can be used. There could be multiple paths between the initial and the goal state which are the solutions to the problem. The focus here is on the case where there is a single path between the initial and the goal state. The problem-

solving process is to identify this solution path among the multiple paths emanating from the initial state. Other paths lead to other goal states. Continuing with the example of object classification problem, the initial state is the unclassified object and the final state is the classified object. Each path from the initial state is a potential class for the given object, and the solution path is the true class. Identifying the solution path corresponds to the correct classification of the object. In this chapter, a problem-solving framework is considered and the benefits associated with human- machine collaboration to solve problems in an efficient manner are emphasized.

The first step for such a search is to determine the set of available strategies, i.e., the strategy space. For the object classification problem, this refers to identifying the set of possible classes that the object may belong to. The second step is to evaluate the strategies to determine the best strategy as the solution. For the object classification problem, this refers to observing the character- istics of the object and determining the true class by evaluating the possible classes. In traditional economic theory, a rational decision maker is assumed to have the knowledge of the set of pos-

sible alternatives1_{, has the capability to evaluate the consequences of each alternative, and has a}

utility function which he/she tries to maximize to determine the optimal strategy [93]. However, it has long been debated that humans are not rational but are bounded rational agents. Under the bounded rationality framework [93, 127], decision makers are cognitively limited and have limited time, limited information, and limited resources. The set of alternatives is not completely known

a priorinor are the decision makers perfectly aware of the consequences of choosing a particular

alternative. Therefore, the decision maker might not always determine the best strategy for solving the problem.

On the other hand, machines are rational in the sense that they have stronger/larger memory for storing alternatives and have the computational capability to more accurately evaluate the con- sequences of a particular alternative. Therefore, a machine can aid a human in fast and accurate problem-solving. The goal of this chapter is to develop a framework for human-machine collabo- ration for problem-solving and illustrate its benefits. There are two basic ways in which a machine

can aid a human: in gaining knowledge about the set of alternatives, and in accurately evaluating the consequences of a chosen alternative. For example, in medical diagnosis, the human doctors might only look at a subset of possible diagnoses based on the symptoms, due to the cognitive lim- itations of humans. On the other hand, a machine with a database consisting of a much larger set of diseases can provide this human doctor with more exhaustive set of diagnoses by evaluating the symptoms in a timely manner. Such a machine can support the doctor in recognizing some of the rare diseases and also provide corresponding recommendations which could have been overlooked

by the doctor. This is partly the idea behind IBM’s Watson, M. D.2 _{Another example is the task}

of pattern recognition. While humans are good at identifying new patterns, machines are good at searching for specific patterns. Therefore, in a pattern recognition task, humans can provide new patterns which the machines can search for.

As stated by Lubart in [88], when the machine supports the human in determining the set of alternatives, the machine is acting as a coach to the human to discover new alternatives. When the machine is helping the human by evaluating the effects of a particular alternative, the machine is acting as a colleague to the human in solving a problem. We discuss these two problems in this chapter, with more focus on the first one, to illustrate the benefits of using human machine collaboration for problem-solving.