De la Rúa, Fernando, TEORIA GENERAL DEL PROCESO Editorial Depalma.

munities

Given the interdisciplinary approach of this dissertation, the academic con- tributions span several domains. Specifically, the studies and the models combined reference major concepts from cognitive science, game theory, human factors, human- computer interaction, and machine learning. As a result, we organized the implica- tions and contributions of this research into three major areas: artificial intelligence, team cognition, and collective intelligence.

10.3.1

Artificial intelligence

As discussed in Ch 1, most artificial intelligence research right now is being conducted with the aim of replacing as opposed to augmenting human effort. This dissertation shows several ways of keeping human effort in the loop by validating the need to study human-machine teamwork as AI becomes more sophisticated.

Specifically, as reinforcement learning evolves, RL agents should be studied in a human-machine team setting as well in order to better understand the complex dynamics of these autonomous agents interacting with humans in the real world. Furthermore, the AI community should not assume that agents will be naturally willing to cooperate with humans, or that cooperative behaviors in any one scenario necessarily translates to generalized willingness to cooperate in many other contexts.

To that end, our experimental setups (especially in Study 1 and Study 2) establish a robust methodology to examine human-machine teamwork by testing co- operative dynamics and coordination in multi-agent systems. Leveraging game theory in this way is useful precisely because deviations from Nash Equilibria highlight the distinctive dynamics governing cooperation and coordination in multi-agent systems, which in this context can serve as useful proxies for human-machine teamwork. This approach is very valuable for AI-safety researchers because it enables the precise and quantitative analysis of the cooperative dynamics between humans and AIs.

Overall, this dissertation extends prior findings of the value of game theory to study the behavior of multi-agent systems by showing how game theory can inform the design of experimental setups to identify the different ways humans and AIs can cooperate and coordinate under different incentive structures. Specifically, our methodology provides a strong and reliable basis for future researchers to test and make inferences about the ways in which different RL models, different beliefs about whether the other player is an AI or a human, as well as different game theory models with different Nash Equilibria influence the cooperative dynamics of human-machine teams.

10.3.2

Team cognition

This dissertation strongly emphasises the need to approach human-machine teamwork not from just the human or only the machine perspective, but from both. Human-machine teamwork is a complex phenomenon that, just as the studies show, varies wildly in different contexts and at different scales. Thus, this dissertation shows that human-machine teamwork can only be understood through an interdisciplinary perspective that includes both cognitive science as well as computer science.

Specifically, our results go contrary to the expectations traditionally set by the teamwork literature. For instance, Study 1, 2 and 3 invalidated the anticipated need for advanced natural language processing to effectively study human-machine teamwork, thereby showing that human-machine teamwork can emerge despite lack of communication [Bates and Weischedel, 2006]. Specifically, this dissertation showed how RL and our methodology for developing agents obviate the need for natural lan- guage processing because human-machine teamwork between humans and RL agents stems not from communication, as is the case with humans, but from cooperation and coordination that is much more akin to the game theoretical dynamics in multi-agent systems.

Besides communication, however, human-machine teams should make us re- consider what we know about situational awareness and its sources. Fan et al’s (2010) speculated that situational awareness in human-machine teams would come with higher cognitive load; our data does not support this claim, as situational aware- ness did not result in fatigue and lower levels of team satisfaction as measured by the survey. Similarly, our data on situational awareness as measured by NeoCITIES does not highlight the coordination deficiencies resulting from a lack of direct communica- tion in human-machine teams as suggested by Demir et al (2017); it shows precisely the opposite insofar as the human-machine teams displayed higher levels of situational awareness than human-only teams where communication was more frequent. Indeed, the superior performance exhibited by the machine-only teams in NeoCITIES shows that sequencing and synchronicity improve alongside the RL agent’s skill-level in the simulation.

More importantly, this dissertation underscores the difference between au- tomation and autonomy as outlined in Chapter 2. The prior literature makes several assumptions with regards to the alleged pivotal role predictability and directability

play in human-machine teamwork [Klien et al., 2004, Christoffersen and Woods, 2002]. Such assumptions tend to be predicated upon an automation-centric view of AI and human-machine teamwork because it presupposes that the best way to achieve human- machine teamwork is by designing agents entirely around human dynamics and be- haviors. Instead, this dissertation framed the human-machine team as a human- autonomy problem, thereby sidestepping many of these design assumptions in order to incorporate cutting-edge AI architecture without limiting researchers to any par- ticular model. The subsequent results ended up aligning with McNeese et al’s (2019) theory that humans and machines can retain interdependence and yet still function effectively as long as they share a common goal.

10.3.3

Collective intelligence

This dissertation strongly advocates for the design of prediction markets with human factors principles. Prediction markets are collective intelligence mechanisms, and thus only a stronger foundation in cognitive science can inform the design of better prediction markets by identifying the underlying mechanisms behind collective intelligence. However, this dissertation also shows that researchers ought to also approach prediction markets from a multi-agent teamwork perspective in order to discover better ways to accommodate the introduction of artificial intelligence so that humans and machines can work together and thus produce higher levels of collective intelligence.

Specifically, our results suggest that we have only scratched the surface of the potential designs for trading bots in prediction markets. Specifically, the prior lit- erature has assumed that it should be up to the human participants to place bets to realign prices with historical base rates [Atanasov et al., 2016]. For example, a

human-only prediction market expects that whenever the prices of “yes” and “no” shares don’t add up to 1, then human traders would buy and sell those shares un- til the prices converge towards the right values. This assumption however stands in sharp contrast with the marginal trader hypothesis, for the capital accumulated by essentially arbitraging temporary inconsistencies between the prices of “yes” and “no” shares would give those traders more resources to influence future prices despite the lack of better information. It thus makes sense to think have bots perform that func- tion, thereby removing noise from a participant’s behavior as they therefore become limited in making directional bets as opposed to speculative trades.

Many other possibilities exist that can inspire future research, such as mean- reversion bots who buy and sell shares as they deviate from a moving average. More specifically however, the design of trading bots aimed at enhancing the collective intelligence of a prediction market should be informed by the biases the literature has identified in prediction markets. Our design was very much informed by Dudik et al’s (2017) in classifying the sources and impacts of different biases. Since sampling error arises from the noisiness of the local knowledge of the participants, we included the randomized trading bots so that the direction of the variation would not be biased in any particular direction. Since market-maker bias arises from players overshooting or undershooting their estimates based on how they are rewarded by functions used to facilitate trading, we relied upon the volatility emerging from the bots trading to incentive initial trading, which would not exhibit any specific pattern detectable and exploitable by the human participants. Lastly, since converge error arises from the market price constantly fluctuating, we included the other type of trading bot whose entire objective is to stabilize prices by removing excess volatility from the system.