Infrastructure projects with delegation - an agent-based model to simulate the interaction history

(1)

Infrastructure projects with delegation:

an agent-based model to simulate the

interaction history

Author: DavidP´aez

Supervisor: Mauricio S´anchez

A thesis submitted in fulfilment of the requirements for the degree of Master of Science

in the

Department of Civil and Environmental Engineering

(2)

I, David P´aez, declare that this thesis titled, ’Infrastructure projects with delegation: an agent-based model to simulate the interaction history’ and the work presented in it are my own. I confirm that:

⌅ This work was done wholly or mainly while in candidature for a research degree at this University.

⌅ Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated.

⌅ Where I have consulted the published work of others, this is always clearly at-tributed.

⌅ Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work.

⌅ I have acknowledged all main sources of help.

⌅ Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed myself.

Signed:

Date:

(3)

Abstract

Faculty of Engineering

Department of Civil and Environmental Engineering

Master of Science

Infrastructure projects with delegation: an agent-based model to simulate

the interaction history

by David P´aez

This work is a innovative approach to the problem of delegation in the procurement of infrastructure projects (e.g., public-private partnerships). A hybrid agent-based model was developed to simulate the interaction between the stakeholders, the natural envi-ronment and the physical infrastructure system.

(4)

The articles listed below are related to the research work presented in this document and were submitted for publication.

• Paez-Perez, David & Sanchez-Silva, Mauricio (2014). Modeling the principal-agent interaction in the operation of infrastructure systems. 4th International Symposium on Life-Cycle Civil Engineering, IALCCE 2014.

• Paez-Perez, David & Sanchez-Silva, Mauricio (2015). A dynamic principal-agent framework for modeling the performance of infrastructure. European Journal of Operational Research. (under review)

• Paez-Perez, David & Sanchez-Silva, Mauricio (2015). Principal-agent dynamic interaction in the context of the life-cycle operation of infrastructure systems. Second International Conference on Performance-based and Lifecycle Structural Engineering, PLSE 2015. (under review)

(5)

Declaration of Authorship i

Abstract ii

Publications iii

Contents iv

List of Figures vi

List of Tables vii

Notation viii

1 Introduction 1

1.1 Introduction. . . 1

1.2 Principal agent problem formulation . . . 5

1.2.1 Basic formulation. . . 5

1.2.2 Limitations of the PA model . . . 7

Aggregation problem . . . 7

Assumption on knowledge of output . . . 8

1.3 Continuous sequential model . . . 9

1.3.1 Problem overview . . . 9

1.3.2 Dynamic interaction . . . 12

1.4 System dynamics . . . 13

2 Model 18 2.1 Mathematical formulation of the hybrid model . . . 18

2.1.1 Hybrid system dynamics. . . 18

2.1.2 Actions . . . 19

2.1.3 Functions . . . 23

Demand . . . 23

Revenue rate . . . 23

Environmental forces . . . 23

Infrastructure response . . . 23

Maintenance cost . . . 24

Utility functions . . . 24

2.1.4 Parameters . . . 24

2.1.5 Strategy sets . . . 25

(6)

2.1.6 State space and transitions . . . 26

2.1.7 Game execution . . . 27

2.2 Implementing the game . . . 27

2.2.1 Entities . . . 28

2.2.2 Simulation . . . 30

2.3 Numerical experiments . . . 31

2.3.1 Construction and operation of a highway segment . . . 31

Experiment 1 . . . 33

2.3.2 Analysis of experiments . . . 36

3 Discussion 38 3.1 Discussion . . . 38

3.1.1 Validation . . . 38

Theory. . . 38

Model . . . 39

Program . . . 39

Operational . . . 39

Empirical . . . 39

3.1.2 Future work. . . 40

Simultaneous adverse selection and moral hazard . . . 40

Data and validation of the descriptive model . . . 40

Exploration of strategy space and parameter space . . . 41

Risk-sharing scheme . . . 41

Spatial awareness . . . 41

Optimization . . . 41

3.2 Conclusions . . . 42

A Algorithms 44

(7)

1.1 Interaction diagram . . . 3

1.2 Principal-agent problem with moral hazard . . . 6

1.3 Types of deterioration . . . 9

1.4 Unit of sequential game . . . 11

1.5 Game tree and performance infrastructure history . . . 11

1.6 Stock-flow diagram of dynamic principal-agent game . . . 14

2.1 Performance in hybrid time . . . 19

2.2 Game state chart . . . 20

2.3 Simplified class diagram . . . 29

2.4 Use case of simulation model . . . 31

2.5 Shock response function . . . 33

2.6 Experiments 1, 2, and 4 . . . 35

2.7 Experiment 5 . . . 36

2.8 Experiment 6 . . . 36

3.1 Use case of optimization model . . . 42

(8)

1.1 Available actions to players . . . 10

2.1 Spaces and action sets . . . 22

2.2 Parameters in the model . . . 25

2.3 Figures from numerical experiments . . . 34

(9)

Symbol Description

Section1.2

q Output ✓ Noise introduced by Nature

w Wage transfered to the agent uA Agent’s utility

e Agent’s e↵ort uP Principal’s utility

Section1.3

V Performance of infrastructure rf Revenue rate function

tm Contract duration k⇤ Performance threshold

h Payment schedule sL Penalty fee strategy

Section2.1

bA Agent’s monetary balance ⌅ Shock action set

bP Principal’s monetary balance i Information known toi-th player

Set of states of the game df Demand function

Current state of the game fc Continuous environmental force

A1 Agent’s action set rc Continuous response function

A2 Principal’s action set rd Discrete response function

A3 Nature’s action set Maintenance cost function

O Null action set ai An action of thei-th player

⇤ Voluntary maintenance action set si A strategy of thei-th player

M Mandatory maintenance action set a An action profile

⌥ Performance space of infrastructure s A strategy profile

⌥! Maintenance space Si Strategy set of thei-th player

C Contract o↵er action set S Strategy space of the game

I Inspection action set Problem parameters

L Penalty fee action set _G Game realization function

(10)

Introduction

1.1 Introduction

Since infrastructure systems are conceived to serve basic necessities of society, public institutions are responsible for their creation and persistence. However, the processes that entail its development are so complex that sometimes public institutions are not prepared to manage them efficiently. This situation has paved the way for procurement methods where these complex tasks are delegated to specialized private third parties who are able to inject private capital investment and deal with complex technical aspects of design, construction and maintenance. Today, one of the most widely used category of this kind of delegation is the Public-Private Partnership (PPP) [1–4]

A PPP is defined as a medium to long term arrangement where the public sector (e.g., a government agency) delegates some services or works to the private sector (e.g., a private firm), having agreed on objectives and conditions for the delivery. For clarity and consistency with other literature on the subject, we will refer to the government agency as theprincipal and treat it with female gender. The private firm will be referred to as theagent, with male gender. The services or works delegated to the agent are often either the enhancement of existing infrastructure or the design and construction of new infrastructure. Once this is completed, the public works are transfered temporarily to the agent —usually for a period ranging between 10 and 30 years—in which he assumes the responsibility of maintaining (i.e., performing maintenance works or updates to counteract deterioration) and operating the infrastructure (i.e., carrying out all the logistics necessary to provide the intended service) while receiving the rent produced by its operation. The agent also agrees to share risks with the principal. Those risks are related to design and construction costs, market demand, service and maintenance costs. It is common that in order to make the project economically attractive to the

(11)

agent, the principal must provide subsidy: a payment schedule transfered to the agent during the infrastructure’s operation. In order to ensure that the mentioned ‘objectives and conditions’ of the arrangement are fulfilled, the principal will keep track of certain performance indicators of the infrastructure by executing inspections. At the end of this contracted period, the government takes back control of the infrastructure system.

The central thesis that we want to convey is that the history and the success —or failure—of an infrastructure project featuring delegation (such as a PPP) results from the interplay of all the following aspects:

1. the economic game between the principal and the agent,

2. the regulatory framework and contractual design that constraint their interaction,

3. the behavior of the physical infrastructure system and

4. the natural environment in which the infrastructure is embedded.

Although in principle, this thesis might seem self-evident, it is not exercised in practice; a systematic framework that integrates these aspects does not exist. We want to propose a model for such framework with the aim of showing the mechanisms by which these four aspects influence the result of the interaction. Additionally, we present suggestions that could transform the model into a decision support system for government agencies.

Even though the term partnership suggests that principal and agent are united by a legal partnership and have intrinsic motivation for cooperating to achieve the greater good, this is not the case. Their relationship is a contractual one. As Yescombe points out [1, p.3], ‘partnership’ in this context is mostly a political slogan. In fact, the use of a PPP procurement method implies that the following circumstances will appear:

• Information asymmetry: Two circumstances contribute to this. First, the princi-pal seeks for an agent who knows how to accomplish something that she herself cannot accomplish. This condition always holds between the principal and any competent agent, even before any contractual arrangement is made. Second, once their contractual relationship has started, the agent’s actions are generally unob-servable to the principal. The agent is aware of his own level of e↵ort (i.e., how diligent his maintenance interventions have been) and output level (i.e., a perfor-mance indicator that is the result of his activities as measurable quantities of the infrastructure attributes). He can also predict the infrastructure behavior much better than the principal. On the other hand, the principal does not know how much e↵ort the agent has employed in maintenance interventions and he can only

(12)

estimate the performance of the infrastructure by actively inspecting it via discrete samples.

• Conflicting goals: Although it is possible that some specific set of circumstances could induce the principal and agent to cooperate —that is, if any strategy change that benefited one party would automatically benefit the other—without the need of an enforcing mechanism, it seldom occurs in practice. Instead, we observe the following. The principal wants to maximize her monetary balance and some perfor-mance measure of the infrastructure. The agent on the other hand simply wants to maximize his monetary balance. Their goals clash because a greater e↵ort from the agent —while increasing the infrastructure performance, thus increasing the prin-cipal’s satisfaction—will decrease his own monetary gain. On the other hand, the increase of the payment schedule —while increasing the agent’s satisfaction—will decrease the principal’s monetary balance.

• Stochasticity: the physical infrastructure system is fundamentally a stochastic system. Thus, from the point of view of a player (principal or agent), the merit of an action to be deployed at the present time instant is uncertain.

The act of delegation when these three features exist creates a moral hazard. The regulatory framework and the contractual design of the interaction is the main leverage point that the principal can use to control the moral hazard problem. On the whole, the delegation relationship is summarized in Figure 1.1

From the literature dedicated to contracts in infrastructure projects we could identify two approaches. The first approach [5, 6] is deductive and quantitative. It is based on economics and game theory. It uses closed form representations for the idealized

Infrastructure System Principal Agent Nature Performs inspections Performs operations Influences behaviour Influences behaviour Returns utility Displays information Pays wage Contract Bound by Imposes penalties Modify Through concensus Bound by Through concensus

(13)

interaction of fully rational economic agents. It deals with information constraints, risk preferences, utility functions and optimization problems. The second approach [1,7] is inductive, and mostly qualitative. It is focused on the interface of finance, regulation and institutions. It is often sustained by experience, the extrapolation from past events and guided by subjective opinion.

The first approach is fundamentally prescriptive. It is able to produce precise quantita-tive specifications at the cost of reducing the complexity by imposing overly simplified assumptions on the problem so that it becomes mathematically tractable. This approach addresses the first component of our thesis. The second approach is fundamentally de-scriptive. It makes reference to the minute details that involve the formation and persis-tence of a PPP. However, while being empirical and close to concrete examples, it often lacks the ability to produce a rigorous prescription of contract design. This approach addresses the second component of our thesis.

Neither the first nor the second approach address the third and fourth components of our thesis. They do not model the problem as a dynamic and path dependent interac-tion and they both overlook the fact that the physical system deteriorates over time, which would in turn elicit reactive actions from players. In summary, economic and management research on this topic has not studied their principles in the context of a physical reality that influences players. Fortunately, research on deterioration models for many kinds of civil infrastructure assets does exist (e.g., [8–11]). These models ef-fectively make the connection between: the properties of physical objects that compose the infrastructure system, the operations exerted on them, the pressures and demands coming from their environment and the resultant change in physical condition measured with some performance index.

Thus, it seems feasible to create a solution that is able to encompass all four compo-nents of our thesis: a framework focused on designing contracts based on a reliable, reproducible quantitative model that acknowledges the intricate details of real economic and operational interactions and the inevitable deterioration of a physical infrastructure system under environmental pressures.

In this work, we will develop an agent-based simulation model capable of tracing an interaction history between the principal, the agent, the natural environment and its e↵ect on the infrastructure system. From such interaction we will calculate the utility for each player, which will rate the goodness of the delegation relationship that emerges out of certain player’s strategies and problem parameters. The formulation of such framework is challenging, since it encompasses several disciplines. We will progressively introduce the necessary theoretical background and use it to build the definition of our model.

(14)

This document is organized in the following structure. In Section 1.2, we present the traditional principal-agent framework and highlight its limitations. In Section 1.3 we propose an alternative interaction game that is the basis of the simulation model. In Section1.4a conceptual model of the interaction process is presented as a dynamic sys-tem, making explicit the dependence relationships. Then, the formulation of the model is described in detail in Section 2.1. The broad characteristics of the implementation in the form of a hybrid agent-based model are explained in Section 2.2. In Section 2.3

we present a set of numerical experiments and further analyses that highlight relevant aspects of the model. In Section 3.2we discuss the validation of the model and provide suggestions for future work. We conclude in Section 3.2 by stressing the advantages of this approach and highlighting the importance of unifying methods of diverse disciplines in order to design and manage socio-technical systems.

1.2 Principal agent problem formulation

This section is focused on theGame theory perspective, from which we will address the Principal-Agent problem. For an introduction on the key concepts of Game Theory, see Leyton-Brown and Shoham [12], Rasmusen [13] and Fudenberg and Tirole [14].

1.2.1 Basic formulation

A principal-agent (PA) problem is one in which an uninformed player (the principal) delegates a task to an informed player (the agent) in exchange for a wage. La↵ont and Martimort [15] present a detailed study of principal-agent models. PA models in general are presented in two versions: adverse selection and moral hazard. We will focus on the moral hazard problem. Dutta and Radner [16] present a review of several variations of the principal-agent problem with moral hazard. Specifically related to concessions in public works, a theoretical investigation of Build-Operate-Transfer (BOT) contracts is presented by Auriol [5].

In the principal-agent problem with moral hazard (see Rasmusen [13, Ch. 7.2]), the principal wants the agent to carry out some task, quantifiable by the variableq(output). The output represents the quality of the work performed by the agent or the produced quantity of some good. The principal o↵ers a contract to the agent that specifies a wage w(q): a payment function that is contingent upon the output. The agent decides whether to accept or reject the o↵er. If the agent accepts, he exerts an e↵ort e that increases the agent’s level of production (@q/@e >0) but the principal cannot observe the agent’s e↵ort. The output producedq(e,✓) not only depends on e↵ort but on a random

(15)

variable ✓ 2 R which is privately chosen by “Nature” according to some probability function. Nature chooses ✓ after the agent exerts his e↵ort, and neither the principal nor the agent can observe✓. Finally, the contract is executed: the output q is observed by the principal and the wage w is transfered. The graphical representation of the principal-agent problem with moral hazard is shown in Figure1.2.

It is assumed that the contract is binding, meaning that the principal can credibly commit to pay the agent according to the function w(_·) at the end of the game. The execution of the contract produces the utilities or payo↵s for the players. Thus if the the principal’s and agent’s utility areuP anduArespectively, the principal’s utilityuP(q, w) increases with output (@uP/@q >0) and decreases with wage transfered (@uP/@w <0). On the other hand, the agent’s utilityuA(e, w) increases with wage received (@uA/@w > 0) and decreases with e↵ort exerted (@uA/@e <0). On the whole, the principal’s problem can be defined as:

max

w(·) E[uP(q(˜e,✓), w(q(˜e,✓)))] (1.1)

subject to

˜

e= arg max

e E[uA(e, w(q(e,✓)))] (1.2) ¯

uE[uA(˜e, w(q(˜e,✓)))] (1.3)

The restriction shown in equation 1.2 is called incentive compatibility constraint. It ensures that the agent voluntarily selects his e↵ort for a given contract. Also, the

(16)

expression 1.3, called the participation constraint, which ensures that the agent prefers the contract to alternative activities that would provide him with a reservation utility ¯

u.

The issue of moral hazard appears because the output is random; i.e.,q is the combina-tion of the agent’s e↵ort e and the realization of ‘pure luck’ ✓ (determined by random exogenous circumstances), therefore the production level is only a noisy signal of the agent’s e↵ort. Because of the stochasticity of the output, it is impossible to directly condition the agent’s rewards to the e↵ort he has chosen.

In spite of this difficulty, the principal would like to design a wage function w(·) that maximizes her expected utility while acknowledging that the agent will also maximize his own. The PA problem is a bi-level optimization problem [17][18] because one of the restrictions (the incentive compatibility constraint, where the agent maximizes his own utility) appears defined as a lower-level optimization problem within the upper-level optimization problem of maximizing the principal’s utility.

Analyses of the principal-agent problem in economic literature mostly address canonical static versions such as the one previously described. Other specialized approaches that recognize the problem as a dynamic interaction are less common and fairly recent. Some examples are the applications of performance-based incentives in a dynamical principal-agent model by Plambeck and Zenios [19], the dynamic moral hazard problem with infinitely repeated actions described by Bolton and Dewatripont [20] and the continuous-time models by Sannikov [21] and Cvitanic and Zhang [22].

1.2.2 Limitations of the PA model

In general, PA models with moral hazard account for the asymmetric information be-tween the principal and the agent —created when the agent performs a hidden ac-tion—and give insight on how the contract (also called compensation scheme or payment schedule) should be set in order to maximize the principal’s utility. They assume that the output is observable to the principal, so that the compensation can be contingent on it. PA models have two shortcomings at representing the problem of infrastruc-ture procurement; one related to aggregation and the other with the assumption on the knowledge of output.

Aggregation problem One of our goals is to model the e↵ects of the players strate-gies on the dynamic behavior of the infrastructure system. For this reason, it is necessary for us to describe the actions of players as sequential events arranged along a time dimen-sion. Most PA models, however, do not take this approach, but rather, use aggregate

(17)

variables of e↵orteand outputq, while denoting their relationship as a functional form

q(e). Certainly, this simplification is useful for some circumstances and there can be a correspondence between prescriptions of PA models and management problems in firms [23]. However, is it not possible to fully represent certain systems [24] using aggrega-tion. In our context, the aggregation of discrete inter-temporal actions of players into final variables cannot be used to study the infrastructure’s dynamics in which we are interested; we need to model the PA game as a dynamic interaction.

Assumption on knowledge of output The second inconvenience of the PA models is their assumption that the output is automatically known to the principal just after the e↵ort is exerted. In our problem, the principal is by default ignorant of the state of the infrastructure system (i.e., the equivalent of the output in the PA models); she must actively monitor this output by performing costly inspections to obtain at best a good estimate. Only then she can use a compensation scheme contingent upon the output estimate. Thus, the costly monitoring of the performance must be an essential part of the game.

Considering these two shortcomings, we propose (Section 1.3) a dynamic model where the compensation scheme could be arranged in such a way that a flat wage is adjusted by penalty fees every time a violation of the performance threshold k⇤ is discovered, with the hope of discouraging shirking. If we assume that monitoring is realized by a series of discrete and instantaneous inspections, a sequential inspection game arises. Inspection problems in game theory literature have been modeled as sequential games between an inspector and an inspectee, whose actions occur at discrete time stages [25–

28]. Although the idea of the conflicting inspector-inspectee relationship can be used to describe our problem, we will use di↵erent modeling assumptions in order to capture its complexity.

Notice that the limitations of the PA model presented in the last subsection point out that: (1) the players actions and problem variables for the whole of the interaction cannot be meaningfully represented by single real values (e.g., e↵ort e and output q); and that (2) the inspection is a requisite for the principal to estimate the state of the infrastructure However, the basic idea of the PA problem as a bi-level optimization program remains valid. Therefore, it is only the two specific limitations that must be adapted to better represent our problem. On the basis of these shortcomings, the following continuous sequential model is proposed.

(18)

1.3 Continuous sequential model

1.3.1 Problem overview

The continuous sequential game we will describe in this section consists of the dynamic interaction between Principal, Agent and Nature and its e↵ect on the performance of an infrastructure system.

The objective of the principal (e.g., government agency) is to maximize her utility (i.e., maximizing the expected performance while minimizing inspection costs) and her ac-tions consist on performing inspecac-tions and imposing penalty fees, so as to induce the agent to behave according to her interests. The agent (e.g., private contractor) is re-sponsible for the operation, which is translated into performing voluntary maintenance on the infrastructure and performing mandatory maintenance if the principal detects that the performance level is below a specified threshold. Note that inspections and vol-untary maintenance interventions are proactive actions that do not require triggers to occur. Finally, Nature encompasses all physical, natural phenomena that a↵ect the sys-tem but that are not in control of the agent nor the principal. We introduce the ‘nature’ player—a common recourse in game theory—as a participant who does not have pref-erences and who chooses actions randomly according to some probability distribution instead of strategically. This allows us to model the uncertainty and randomness of the environment into the model. Nature exerts continuous and discrete perturbations to the infrastructure system, which cause progressive or instantaneous degradation [9,29,30]. Most physical systems exhibit a combination of the two mechanisms. These two mech-anisms are shown in Figure 1.3.

In our model all maintenance costs (voluntary maintenance or mandatory maintenance) must be paid by the agent, whether they are a result of the agent’s negligence for dealing with progressive degradation or the result of stochastic shocks from the natural

(19)

environment. For the sake of simplicity, our model does not implement a risk-sharing scheme, since this alone presents several challenges. See further comments in Section

3.2. The actions of the three players (Principal, Agent and Nature) are summarized in Table1.1.

In the sequential continuous game problem, at the beginning of the game (i.e., t0) the

principal selects the parameters of the contract: its durationtm, the payment scheduleh that the principal’s promises to pay to the agent, the revenue rate functionrf which is the income rate that the agent receives as a function of the demand of the infrastructure, the performance thresholdk⇤, and the penalty policysL. This initial move by the principal completely defines the contract. Then, the possibilities of proactive actions by each player and the reactive actions that become available as a result are shown in Figure

1.4. This is the fundamental building block of the game. It defines the schedule of actions occurring instantaneously at any time t. What may occur in the game at time

t+dt can be constructed simply by connecting the terminal node of the path that was just realized with the root node of another copy of a tree unit. In Figure 1.5 there are two sample history paths of the game. Path 1 starts out with the principal selecting a contract where the performance threshold is k⇤ = 60 while in Path 2, he selects a contract with k⇤ = 40. Each path is a complete interaction history; it is a possibility of how the game can unfold and what the outcomes can be, e.g., (u1_A, u1_P) or (u2_A, u2_P). One can think of each game history as a spaced concatenation of the realized path of many tree blocks presented in Figure1.4.

Table 1.1: Available actions available to each player and their respective decision variables

Player Action Decision variables

Agent Voluntary maintenance

Time

Performance goal Mandatory maintenance Performance goal

Principal

Contract o↵er

Contract duration Payment schedule Revenue function Performance threshold

Inspection Time

Selection penalty fee Monetary value

Nature Shock Time

(20)

P

N A

Selection of penalty fee

Mandatory maintenance

Inspection Shock _maintenanceVoluntary

P

A Root node

Terminal node

Possible path Example of realized path Continuous action space

Figure 1.4: Unit of sequential game

Figure 1.5: Representation of two out of infinite possible paths of the sequential game. Top: game tree with players’ actions marked as P, A and N for principal, agent and nature, respectively. Bottom: the performance history of an infrastructure system

(21)

1.3.2 Dynamic interaction

In this sequential game, the ordering of proactive actions is not predetermined and the timing of moves is free for players to decide. All decisions made by the players is based on the following informational settings:

• The value of the performance threshold and the penalty policy is public informa-tion.

• The agent knows the stochastic nature of the progressive structural degradation process.

• The agent is aware of inspections once they occur (ex post).

Given these conditions, each player uses a strategy to play the game. A strategy is an algorithm that tells a player which actions to perform contingent upon the perceived current state of the world and recalled information. Strategies are immutable through-out the realization of a game. This does not imply a loss of generality, because any strategy composed of a combination of other strategies—even guided by some control algorithm—is itself precisely defined and thus immutable. Players are able to keep his-toric records and have unlimited recall of observed information and their own executed actions. Whether this ability is used or not depends on the strategy that a player is deploying. The principal’s utility depends on his monetary balance and some measure of the cumulative perceived performance of the infrastructure. The agent’s utility depends on his monetary balance alone. Players dynamically perceive their utility as the game evolves.

The method of backward induction can be used to find a subgame perfect Nash equilibria for finite sequential games with perfect information. However, the proposed game does not meet these conditions since it is an infinite game of imperfect information. It is infinite for two reasons. The first is the potentially uncountably infinite number of nodes derived from the continuity of the time dimension. The second is that this game has at least one continuum action set, thus from a decision point of such action, uncountably infinite paths branch o↵. Because the conditions mentioned are not met, the algorithm of backward induction is not well defined [14, p. 91] and is therefore not applicable to our problem.

So far, we have described the time-dependent nature of the problem, freedom in ordering of moves, continuum action spaces, and proactive and reactive actions. What is less clear is how the di↵erent components of the problem influence each another. A system

(22)

dynamics model, which we address in the following section, will clarify this issue by describing these dependence relationships.

1.4 System dynamics

We need a way to define more precisely the relationship between all the di↵erent parts of the problem which we have only addressed qualitatively. Also, we need to define the mechanism by which the system (principal, agent, nature, and the infrastructure system) evolves continuously when no player is performing actions (i.e., the evolution between every pair of consecutive actions in the branches of the game tree shown in Figure 1.5). In order to resolve this need, we will frame the concepts of game theory presented so far, within the context of system dynamics.

System Dynamics(SD) [31,32] is a mathematical methodology that seeks to simulate the behavior of a complex system by identifying its parts and the connections between those parts in the form of relationships of dependence. A system dynamics model describes complex system in terms of stocks, flows and feedback loops. A stock represents the accumulation of a substance and a flow represents the rate of its movement between stocks. The behavior of flows can be dependent on the value of various stocks and other flows. Therefore, a dynamic system is described mathematically as a set of state variables and a system of di↵erential equations that govern their evolution throughout time. When an element (stock or flow) can influence itself through a closed loop of dependency relationships, the group of elements involved is called a feedback loop. Stock and flow diagrams are useful to make the dependency between parts of a system explicit and comprehensible.

We make use of this idea in Figure 1.6, in order to represent the complex relationship occurring in the infrastructure life-cycle performance problem we are modeling. The stocks in this model, depicted as tanks, are: (1) the performance of the infrastructure; (2) the agent’s monetary balance; and (3) the principal’s monetary balance.

The performance of the infrastructure is increased by flows of continuous and discrete maintenance works and is decreased by flows of progressive (continuous) and shock-based (instantaneous) degradation.

Nature exerts its influence in the form of progressive and instantaneous environmental forces. These forces are received by the infrastructure system and are translated into progressive and instantaneous degradations through a response function. Such response function also depends on the current level of performance and the demand of users which could potentially have a damaging or repairing e↵ect.

(23)

Figure 1.6: Conceptual stock-flow diagram of dynamic PA game. Each element is identified with the color of the entity that owns it or controls it.

(24)

The agent’s balance is increased by the payment schedule promised by the principal and the revenue caused by the operation of the infrastructure. It is decreased by the main-tenance costs and penalty fees. The revenue rate depends on the users’ demand, which in turn depends on the infrastructure’s performance. Maintenance costs are determined through the actions that the voluntary maintenance and the mandatory maintenance strategies implement. Penalty fees are determined by the penalty strategy.

On the other hand, the principal’s balance is decreased by the payment schedule agreed in the contract and by the cost of inspections. The values of the payment schedule are transfered to the agent. When the inspection strategy dictates an inspection action, its cost is instantaneously subtracted from the principal’s balance and a record of the current performance is observed and stored by the principal. When an inspection is executed, a level of compliance is calculated based on the perceived performance from an inspection and the performance threshold.

A cloud represents a source or sink of a flow. They mark the boundary of our system. We do not keep track of anything that is beyond the clouds. For instance, while the monetary value coming out of the agent’s balance through the maintenance cost flow may actually be received by a subcontractor who is hired to perform a specific reparation, it is a process that is outside the scope of our analysis, therefore we decide to ignore it.

Small circles in Figure1.6represent calculated elements or simple parameters. Rhombi represent strategies. They receive some input and produce signals to other components with information about dictated actions. Since the information input of strategies de-pend completely upon their particular internal structure, no dede-pendency is drawn as a definitive relationship. Instead, the possibility of such dependencies is drawn with dotted lines. Although the existence of these relationships seem intuitive, they do not imply necessary dependence but rather the possibility to information access if needed. As an illustration of this, suppose the agent chose a strategy for both mandatory and voluntary maintenance that at fixed time intervals would increase the performance by a fixed amount. The execution of such strategies would not require the agent to know the performance previous to the intervention. In contrast, if the voluntary maintenance strategy dictated that each intervention would bring the infrastructure to a specific performance, then the knowledge of its previous value would indeed be required.

When the suggested dependencies with dotted lines are considered, interesting feedback loops appear. Take for example the voluntary maintenance strategy. It sends an in-formation signal of a maintenance action to the maintenance flow, which increases the infrastructure’s performance stock. In turn the infrastructure’s performance a↵ects the next action produced by the voluntary maintenance strategy. This feedback loop is very simple and its e↵ect may be instantaneous, but others run across more components and

(25)

may exhibit delays. For instance, the action produced by the voluntary maintenance strategy controls the maintenance flow, which increases the infrastructure’s performance. The value of performance may be observed and registered by the principal, who will use it to estimate the level of compliance based on a chosen performance threshold. Such compliance is an input of the penalty strategy, which sends a signal to the penalty fee flow, which in turn a↵ects the agent’s balance. Finally, a change in the agent’s balance will a↵ect the next action dictated by the voluntary maintenance strategy itself, thus completing the loop.

In the dynamic model proposed in this document, the natural forces have been explicitly related to other components of the system and such forces are in fact actions. In doing so, we are implying that Nature can be considered as a third player in the game.

In summary, the dynamic system model proposed here is a more coherent integration of the parts that compose our PA problem for an infrastructure life-cycle by including the following aspects:

• Original PA problem: the existence of two players (principal and agent) with information constraints (asymmetry included), conflicting goals and a promised wage or payment schedule agreed in a contract between the two parties.

• Natural environment: a fictitious player called Nature whose actions are uncertain for the principal and the agent.

• Infrastructure system: the continuous and discrete dynamics of an infrastructure system upon which the principal, agent and nature operate.

• Inspection game: the necessary costly inspections that the principal uses to learn information about the infrastructure and estimate agent’s actions. The definition of agent’s legal and illegal actions according to a specified minimum performance threshold. Also the inclusion of threats in the contract in the form of penalty fees to be imposed on the agent if a violation of the minimum threshold is detected during an inspection.

• Players’ actions: the definition of specific actions by which the players interact. For the principal, they are the selection of the contract (e.g., payment schedule, performance threshold), the execution of inspections and the imposition of penalty fees. For the agent, they are the execution of a voluntary maintenance and the execution of a mandatory maintenance. For nature, they are the imposition of discrete shocks and continuous deteriorating forces.

• Exogenous parameters: the parameters that should be chosen by the modeler to match a particular instance of the problem, such as the response function of the

(26)

infrastructure system, the character of the users’ demand and the revenue earned by the agent as a result of the operation of the infrastructure system.

This section has presented a perspective of the problem in terms of a dynamic system. In the next section we put forward a mathematical description of this dynamic system.

(27)

Model

2.1 Mathematical formulation of the hybrid model

With the stock-flow diagram shown in Figure 1.6 we were able to describe graphically how the di↵erent parts of the problem relate to each other. The current section is devoted to present a formal description of such system.

2.1.1 Hybrid system dynamics

The combination of continuous and discrete behavior in a system is denoted with the term hybrid [33]. Most aspects of the theory of hybrid system dynamics are applied to physical systems (often mechanical or electrical), however it is general enough to be applied to other phenomena as well [33]. A hybrid system can move throughout its state space both in a continuous and instantaneous manner. The continuous evolution of the system is given by a di↵erential equation (or set of di↵erential equations) called a flow map. On the other hand, the instantaneous evolution is described by a recurrence relation, called a jump map. Furthermore, there are certain conditions that determine whether the system flows or jumps at each particular instant.

It is characteristic of our problem that some aspects are better described as continuous and smooth and others as discrete and sudden. For example, the progressive deterio-ration of a physical system can be modeled as a continuous process, while the action of a player is better represented as a discrete event that causes instantaneous change. We argue that the framework of hybrid systems is a useful and natural analogy that encompasses the perspective of game theory (interaction of players through execution of instantaneous actions) and system dynamics (the smooth evolution of variables described by di↵erential equations).

(28)

Hybrid models are parametrized by a set E ⇢{R 0⇥N}, where the vector (t, j)2E,

defines the time tand the order of discrete jumps j in the system state. Note that the hybrid time domain allows the possibility of more than one jump occurring at the same value of continuous ordinary timetwhile capturing the order of its occurrence. Also, it allows to unambiguously refer to the state of the system that exists just before or after an instantaneous event.

0 1 2 0

1

2

Figure 2.1: Representation of performanceV parametrized in hybrid time (t, j).

As an illustrative example, let us suppose that the state of an infrastructure system is defined by the scalar variable x = V 2 X, denoting its performance. When the performance of the infrastructure is plotted (see upper right plot in Figure 2.1), the time intervals where the performance value is continuous and smooth are separated by sudden jumps that occur as a result of shocks and maintenance actions. The regions where the continuous evolution occurs are governed by a di↵erential equation. On the other hand, sudden jumps are governed by a recurrence relation, which in the context of this problem is an abstraction for the collective action taken by the players when they are confronted with some state of the world. The time hybrid time domain is depicted in the lower right plot of Figure 2.1. Let us propose that the game between Principal, Agent and Nature is a hybrid system, where its state is defined by a vectorx₂X, where

X is the system state space. This notion will be expanded further in Section2.1.7.

2.1.2 Actions

In the proposed model there are three players that interact with each other permanently: Agent, Principal and Nature, which will be denoted as Player 1, 2 and 3 respectively. By including Nature as a third player, the game has three proactive actions: inspection (principal), shock- - instantaneous degradation- (nature; e.g., earthquake) and volun-tary maintenance (agent). Also, between actions, the system may or may not degrade

(29)

continuously. There are two reactive actions that occur when a violation is detected: mandatory maintenance and the selection of an enforced penalty fee.

The game has four states defined by the set ={0,1,2,3}whose elements represent:

• = 0: the initial state, where the principal o↵ers the contract to the agent and he accepts it

• = 1: all players must select a proactive action

• = 2: a detection has occurred and the principal must select a penalty fee

• = 3: the penalty has been charged and the agent must perform a mandatory maintenance

The chart in Figure 2.2 shows the possible states of the game, the actions available to each actor and the events that trigger the transition between states.

Principal Agent

1. Unconstrained Detection

* Null action * Inspection

* Penalty fee * Null action

* Vol. maint

* Mand. maint Mand. maint

execution Entry point

Nature

* Null action * Shock

2. Violation detected

3. Penalty applied Penalty

charge

1

2

3

* Null action * Null action

0

0. Initial * Null action * Contract oﬀer * Null action

Figure 2.2: State chart of the game. Each state shows the correspondent available actions for players.

We refer to the space of all possible instances of a particular action as an action set. Let us now formally describe the actions shown in Figure 2.2as action sets for each player as a function of time and the state of the game. The variable ₂ indicates the current state of the game.

(30)

A(₁t,j) = 8 > > > > > > > < > > > > > > > :

O = 0

{O_[⇤(t,j)_} = 1

O = 2

M(t,j) _{= 3}

(2.1)

where O is the null action set, ⇤(t,j) is the voluntary maintenance action set and

M(t,j) _{is the mandatory maintenance action set. A unique maintenance action is}

defined by a vector (vi, vf), denoting the performance before and after its execu-tion. These vectors constitute the voluntary maintenance action set⇤(t,j) and the mandatory maintenance action set M(t,j).

• Principal’s action set: the complete action set for the principal (player 2) is

A(₂t,j) =

8 > > > > > > > < > > > > > > > :

C = 0

{O[I} = 1

L = 2

O = 3

(2.2)

whereC is the contract o↵er action set,_I is the inspection action set andLis the penalty fee action set. The inspection action setI contains only an element which represents the execution of an inspection. The values contained in the penalty fee action set L represent the possible monetary penalty that the principal imposes on the agent after a violation is detected.

A contract o↵er is a vector (tm, h, rf, k⇤, sL)2C, where tm2R 0 is the contract

duration, h = _{ht : t 2 R 0} is the payment schedule, rf : R 0 7! R 0 is the

revenue rate function (see Equation2.5),k⇤2⌥is the performance threshold and

sL is the penalty fee strategy to which the principal commits to use in case of detections.

• Nature’s action set:

A(₃t,j)=

8 > > > > > > > < > > > > > > > :

O = 0

{O[⌅} = 1

O = 2

O = 3

(31)

where ⌅ is the shock action set. The values contained in the shock action set ⌅ represent the magnitude of instantaneous environmental force. Shocks may be used to model earthquakes, floods, fires or other catastrophic events that can be considered as sudden events.

The details of other sets that make up A(₁t,j), A₂(t,j) and A(₃t,j) are formally defined in Table 2.1. The null action set is included to take into account that this game allows only one proactive action to be executed at a time, therefore whenever a player executes an action, the others must necessarily be forced to choose ˜o for the time instant. Also, all three players may simultaneously choose to do nothing at a time t, in which case, the continuous environmental force will cause the infrastructure to deteriorate. With the inclusion of a null action available to the players in the unconstrained state = 1, it can be asserted that they all make a choice of action continuously.

Table 2.1: Spaces and action sets

Spaces

⌥=_{v₂_R_|vmin vvmax} Performance space of infrastructure system

⌥! ={(vi, vf)2⌥2 | vi< vf} Maintenance space

Action sets

O=_{o˜_} Null action set

C Contract o↵er action set

⇤(t,j)={(vi, vf)2⌥! |vi =V(t,j)} Voluntary maintenance action set

M(t,j)={(vi, vf)2⌥!|vi =V(t,j)^vf k⇤} Mandatory maintenance action set

I={◆} Inspection action set

L_⇢R 0 Penalty fee action set

⌅⇢R 0 Shock action set

The information setting is central to the definition of a game. The information accessible to thei-th player at time (t, j) is described by the variable (_it,j). In particular, the vari-ables i will provide the considerations of information accessibility described in Section

1.3where the sequential inspection game was defined. We will assume they provide the

i-th agent with information previously recorded by him as well as information signals that are currently perceived .

This section defined the action sets that enclose the possible decisions that players can take. This was done by expressing them as mathematical sets, thus a particular action is a single element within a set. As a way to present this in a

(32)

2.1.3 Functions

The continuous evolution of our dynamic model is driven by di↵erential equations. As seen in Figure 1.6, the rate of change of the stocks are a↵ected by auxiliary functions connecting various components of the system. Most of them correspond to parameters of the model. Here, we present a description of each function.

Demand The usage level of the infrastructure per time unit is expressed in the model as a user demand function

d=df(V, t)2R 0 (2.4)

that may represent the number of concurrent users in the case of a public transportation system or the specific stress exerted on the infrastructure; for instance, the number of Equivalent Standard Axial Loads (ESAL) per time unit in the case of a pavement structure or the power demand in Megawatts of an electrical distribution network.

Revenue rate The revenue function

r =rf(d)2R 0 (2.5)

is the continuous income stream per time unit that the agent receives as a result of the operation of the infrastructure system. In the case where users of the infrastructure directly pay the agent for the service provided (e.g., a toll road) the revenue rate is determined by the instantaneous demand from users.

Environmental forces The environment naturally imposes disturbances that can directly change the state of the infrastructure. The term environmental forces is used to refer to such disturbances. The continuous environmental force

˜

f =fc(t)2R 0 (2.6)

may represent eroding factors like rain and wind in geotechnical structures or sea waves in coastal structures. However, its nature does not necessarily imply mechanical stress. It can also be used to model the presence of chemical corrosives like chloride that dete-riorates reinforced concrete structures or changes in the water table that may a↵ect the reliability of a foundation.

Infrastructure response An infrastructure system is assumed to have the property of a known response function to environmental forces (both continuous and discrete)

(33)

and the demand level (i.e., usage level). The response function can be separated into a continuous response function

c =rc( ˜f , d, V, t)2R 0 (2.7)

that determines the rate of progressive deterioration and a discrete response function

s=rd( ˆf , V, t)2R 0 (2.8)

that produces the shock-based deterioration, where ˆf ₂⌅.

Maintenance cost The maintenance cost function

:⌥! 7!R>0 (2.9)

maps a maintenance intervention to its respective cost. This function describes the level of efficiency of the agent’s operations.

Additionally, we define the general form of the players utility functions:

Utility functions The agent’s utility is given by the function

uA=u1(bA)2R (2.10)

wherebA is the agent’s monetary balance and @u1/@bA>0. The principal’s utility is

uP =u2(bP,v˜)2R (2.11)

where bP is the principal’s monetary balance, ˜v is a measure (e.g., the mean value) derived from the performance observations, @u2/@bP >0 and @u2/@v >˜ 0.

2.1.4 Parameters

The parameters of the model are summarized in Table 2.2. These are elements that are associated with a particular instance of the problem. Five of them are values: vmin and

vmax are the limits of the performance space,v0 is the performance of the infrastructure

at the beginning of the game,b0_Ais the initial agent’s balance which is normally composed of the initial payment received from the principal (if any) minus the initial investment or construction cost, andc◆ is the cost that the principal incurs every time she inspects the

(34)

infrastructure. Seven other parameters are functions, which were defined in the previous subsection.

As we mentioned before, nature does not have preferences and does not act strategically. Therefore, the parameters3is simply an algorithm that dictates actions from the action

space A3 (see Equation 2.3). The (strategy) algorithm is chosen by the modeler to

re-semble environmental pressures from a particular problem instance. The next subsection specifies what a strategy is and how it relates to actions.

Table 2.2: Parameters in the model

Related entity Symbol Parameter

Problem df Demand function

Infrastructure

vmin Null performance

vmax Maximum performance

v0 Initial performance

rc Continuous response function

rd Discrete response function

Nature fc Continuous environmental force

s3 Nature’s strategy (shocks)

Agent Maintenance cost function

b0_A Agent’s initial balance

uA Agent’s utility function

Principal c◆ Cost single inspection

uP Principal’s utility function

2.1.5 Strategy sets

At every time instant, eachi-th player performs an actionai2Ai(t, j) dictated by some strategysi. A strategy for thei-th player is the relation

ai =si( (it,j), )2A

(t,j)

i (2.12)

This notation implies that the strategysiproduces actions according to the action space that is available to player iat time (t, j) as a function of the state of the game . An agent’s strategy s1 encapsulates a voluntary maintenance strategy and a mandatory

maintenance strategy. Similarly, a principal’s strategy s2 encapsulates the inspection

strategy and the penalty fee strategy. In the case of nature, s3 simply contains the

shock strategy. The realization of one game has a unique combination of strategies, also called strategy profile denoted as s = (s1, s2, s3). If we define an information vector

(35)

a=s( ) (2.13)

transforms an information vector into an action profilea= (a1, a2, a3) which is the joint

selection of actions at a particular time in the game. The strategy set Si of the i-th player contains all his available strategies. In order to play the game, the i-th player selects a specific strategy out of his strategy set si 2 Si. All possible combinations of players strategies are included in the strategy space

S={S1⇥S2⇥S3} (2.14)

A strategy profile is therefore a point within the strategy space, so that s2S.

2.1.6 State space and transitions

If X is the state space of the game, then it is defined as X =_{X1⇥X2⇥. . .⇥Xn},

where n is the number of variables that compose the state of the game and Xn is the state space of the n-th variable. If x 2 X is a particular state, in our problem it is composed of at least the variables

x= (V, bA, bP, , . . .) (2.15)

thus, the state space of our problem is X ={⌥⇥R⇥R⇥ ⇥. . .}. Ellipses are used because there is a large number of variables that we could track in the model which are present in any computational implementation of the problem. However, for the scope of this work, we are only interested in those explicitly written in Equation 2.15.

It is possible now to interpret each path in the game tree in Figure 1.5 as a possible trajectory of the state vectorxwithin the state spaceX, whose motion was dictated by the aggregation of the players’ strategies and the naturally occurring phenomena of the physical infrastructure system.

The process where the players select and perform their actions, depicted in Figure 1.4

and Figure2.2 are the equivalent of a jump map

x(t,j+1) =⌧(x(t,j), (t,j)) (2.16)

(36)

dV

dt =rf(df(V, t)) (2.17)

dbA

dt =rc(fc(t), df(V, t), V, t) (2.18)

which both depend onV andt.

A complete summary of the notation of the mathematical formulation presented in this section is shown in 7

2.1.7 Game execution

The exact process by which the hybrid system evolves —both discretely (jump map) and continuously (flow map)—is presented in the Algorithm 1 in A. All variables, strategy and action spaces, functions and parameters are included in a detailed process that describes the actual execution of a game realization.

2.2 Implementing the game

As the formulation for the game progressively grows in complexity, it becomes more difficult to fit its structure into the basic models provided by game theory. Then, in order to recreate the game as formulated in the previous section without incurring in further simplifications, we developed an hybrid simulation model that combines System Dynamics (SD) and Agent-Based Modeling (ABM).

Agent-based modeling is a simulation method composed of agents that interact within a given environment [34]. Each agent can be autonomous and adaptive. Their aggregate interaction dictates the evolution of the whole system, which often exhibits complexity out of very basic rules of individual behavior. For this reason, ABMs are useful to model the properties of complex systems [35]. Agent-based modeling is also related to the fields of discrete and continuous dynamical systems, multi-agent systems [36] and game theory [37]. ABM has also influenced the social sciences [38]. The advent of powerful computers combined with the ABM paradigm has provided a framework to better understand complex emergent phenomena in social systems composed of individuals (i.e, agents) by simulating the evolution of their interactions [39].

Due to its generality, agent-based models have begun to be applied to engineered sys-tems. The remarkable advantage of ABMs is that besides simulating socio-economic

(37)

interactions, they could simultaneously emulate a realistic description of physical inter-actions between agents. The addition of the latter feature is not yet widely used, and we have found only in Sanford Bernhardt and McNeil [40] an instance with the mentioned feature being specifically targeted at modeling the life-cycle of civil infrastructure with the perspective of a socio-technical system.

The proposed model combines system dynamics with agent-based modeling; an overview on the design of hybrid AB-SD simulation models can be found in Swinerd [41]. The selection of a hybrid SD-AB simulation corresponds nicely with the properties of hybrid system dynamics presented in section 2.1 to formally describe the game. They both share the ability to represent continuous and discrete processes.

In the present work the basic principal-agent game is extended and modeled as a sequen-tial game between autonomous players and their environment. All moves are computed by a strategy that each player selects at the beginning of the game. The simulation is a time-dependent game that in its finished state produces an interaction history from which the aggregate utility for each player can be calculated. The realization of a game is therefore a transformation of the players strategies and the problem parameters into the utilities

(uA, uP) =G(sA, sP, ) (2.19)

where sA 2 S1 , sP 2 S2. Strategies for inspection, maintenance and penalty policy

selection can be introduced in the strategy set of a player (i.e., Si) and be selected to evaluate G. The result of such game will show the emergent e↵ect of these strategies throughout the life-cycle of the infrastructure.

2.2.1 Entities

The agent-oriented computational paradigm [42] o↵ers valuable guidance on the design of agent-based simulation models. The hybrid simulation model was implemented in MAT-LAB as an object-oriented program where the problem is represented by objects with attributes, methods and associations [43]. The entire model as presented in this docu-ment is available at the public repository https://github.com/davpaez/contract-design. We will not explain the details of the implementation, since many configurations of the actual computer program could replicate the model described in Section2.1. Instead, we present only a brief overview of the structure of the object-oriented program. The class diagram in Figure2.3 shows the most important components and their relationships.

(38)

Realization

Principal Agent Nature

Infrastructure Contract

Strategy GameEvaluation

DecisionRule

+ principal + agent + nature

+ infrastructure + contract

+ contractStrategy + inspectionStrategy

+ mandMaintStrategy + volMaintStrategy

+ penaltyStrategy

1

+ realizations 1..*

+ decisionRuleArray

+ shockStrategy

1 1 1 1 1

1 1

1

1...*

Figure 2.3: Simplified class diagram of the ABM, features the most important asso-ciation relationships. Inheritance relationships are not shown.

• The players Principal, Agent and Nature are classes. Each of them has an asso-ciation to one or more strategy classes. It is by using the strategy objects that players compute their actions.

• The specific strategy objects for each player are assigned at the beginning of the game. The Principal has an inspection strategy object. The agent has a mandatory maintenance strategy object and a voluntary maintenance strategy object. The player’s also have associations to classes that store information about their known events, observed variables and monetary flows (not shown in class diagram).

• Nature contains a shock strategy; i.e., a strategy that defines the occurrence of sudden events that may decrease the system state. This class is the only associated with an infrastructure object.

• Infrastructure defines attributes such as response functions and a record of the performance history.

• Contract contains the penalty fee strategy and other information about the game considered as common knowledge.

(39)

• The Realization class contains the interaction history of a complete and indepen-dent realization of the game. A realization object is composed of a contract, a principal, an agent, an infrastructure and a nature object.

• TheGameEvaluation class defines all the input data necessary to run simulations of a particular game setting. It contains an array of one or more Realization objects, each of which is executed according to the same game settings.

The classes Principal, Agent, Nature and Contract have strategies that produce the action profile at every time instant. Strategies are composed of objects called decision rules. The use of a decision rule is to compute and return one or more decision variables of an action. Thus, a strategy responsible for producing an action with m decision variables may contain up to m decision rules that collectively compute the value of all decision variables. Decision rules are therefore the building blocks of the player’s behavior, strategies are collections of decision rules that compute the decision variables required to define an action.

2.2.2 Simulation

When the system is within the flow set, the model uses a numeric method for solving ordinary di↵erential equations. This behavior in the model is typical of a SD simulation. When the system is within the jump set, the model behaves as an ABM simulation. In contrast with the mathematical formulation in the definition of the hybrid system, instead of continuously asking players to produce an action profile, the Realization class arranges an iterative process where it asks players to submit the next action they wished to perform, while ignoring null actions. Only the earliest action is allowed to be executed. If the current time of the system is less than the time of the allowed action, the system evolves according to the flow map until such time is reached. Then the action is executed and players are asked to submit their next action once again. In this manner, the process is repeated until the duration of the contract is reached. This is in practice identical to the Algorithm 1.

Since the model proposed aims to represents granular characteristics, it also needs a considerable number of parameters. These parameters are related to the specific problem instance that the modeler wants to simulate. They broadly refer to properties of the problem, the infrastructure system, the contract, nature, the principal and the agent. The input data requirements were summarized in Table 2.2.

Having provided the input data requirements, what remain to be specified in order to run a game realization are the player’s strategies themselves. By making use of

(40)

the decision rule/strategy hierarchy described earlier, the modeler can devise various strategies and assign them to players. By doing so, the modeler forms a strategy profile which combined with the parameters are used to evaluate the game G(s, ). The use of the simulation model —with a government agency in the role of user/modeler—is summarized in Figure2.4. Controlled parameters are those that the government agency (i.e., principal) can voluntarily select, such as the kind of infrastructure system and the meaning and measure of the performance level (e.g., vmin and vmax). Uncontrolled parameters are a condition of the real-world system, such as the distribution of shocks exerted by natures3, or the demand functiondf.

2.3 Numerical experiments

We have covered in Section2.1 the theoretical structure of the problem and in Section

2.2the arrangement of its implementation as a hybrid AB-SD model. This section will present numerical experiments for a specific problem instance. We will characterize the following problem by specifying the parameters listed in Table 2.2 and the strategies that each player deploys.

2.3.1 Construction and operation of a highway segment

Consider a PPP whose goal is the construction and maintenance of a segment of some interurban highway. The principal is a government agency who is charge of the local transportation network. The agent is a private firm or consortium of firms with access to credit and the capacity to provide the service required. The infrastructure system is a road segment built with a flexible pavement structure.

The performance of a road is defined as its ability to serve traffic. Examples of existing road performance measures are the PSI (Present Serviceability Index, a ride quality

(41)

rating) and the IRI (International Roughness Index, a roughness estimation based on a measured longitudinal road profile)[44]. These measures are very useful in practice for road management because they are related to vehicle operation costs [45]. The infrastructure performance space is ⌥ = [0,100]. We assume the traffic flow is mostly composed of small vehicles which cause imperceptible damage to the pavement structure; thus, the response function is independent of the infrastructure usage. On the other hand, environmental conditions do influence the response function. User’s demand come in the form of vehicle traffic moving along the road segment, measured as vehicles per year. The demand function depends on the performance level

df(V) =

✓

V vmin

vmax vmin

◆10

↵ (2.20)

where ↵ = 2.8_⇥107 vehicles per year is the demand at the maximum performance. Suppose that the presence of water in the pavement structure contribute to its deterio-ration. Then, the environmental force function would be the instantaneous precipitation, measured as millimeters per year

fc(t) = 612·sin(4⇡t) + 1032 (2.21)

This function resembles a bi-modal rainfall pattern with dry periods peaking on January and August (420 mm/year) and rainy periods peaking on April and October (1644 mm/year).

For those experiments with shock environmental force enabled, it is implemented by a shock strategy that produces shocks at exponentially distributed time intervals with = 1.5 with a force that is distributed log-normal with mean 7 years andCOV = 0.07. The shock environmental force is given in cm/m: centimeters of vertical displacement in one meter of longitudinal distance along the road. Such displacements can be the result of seismic activity.

The shock response function is defined by the surface shown in Figure2.5.

The initial construction cost invested by the agent is cc = 875. We will make the assumption that the agent is risk neutral and that his utility is exactly equal to the balance of the monetary values he perceives. We can define such function as

uA= (h cc) ⇢ µ (2.22)

where h is the payment schedule, cc is the agent’s initial investment (i.e., construction cost), ⇢represents the penalties imposed by the principal andµ the maintenance costs. The model assumes cc to be independent of the players’ strategies. The components