Probabilistic learning strategies applied to agreement negotiation for meeting scheduling

(1)

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

T H E S I S

Master of Science in Intelligent Systems

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey

By

Gisela Medina Ramirez

December 2008

(2)

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

THESIS

Master of Science in Intelligent Systems

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey

By

Gisela Medina Ramirez

December 2008

(3)

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey

Graduate Division in Mechatronics and Information Technology

Thesis committee members recommend the present thesis of Gisela Medina Ramirez to be accepted in partial fulfillment of the requirements for the degree of Master of

Science in:

Intelligent Systems

Thesis Committee:

Leonardo Garrido L., Ph.D.

Thesis advisor

Ramon Brena P., Ph.D.

Examiner

Hugo Terashima M., Ph.D.

Examiner

Joaquin Acevedo Mascarua, Ph.D.

Chairman of the Graduate Programs Engineer School

December 2008

(4)

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

By

Gisela Medina Ramirez

THESIS

Submitted to the Graduate Program in Mechatronics and Information Technology in partial fulfillment of the requirements for the degree of Master of Science in

Intelligent Systems

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey Campus Monterrey

Monterrey, N.L. December 2008

(5)

To my new baby whom in the last nine months has changed my life. To my mother for her inconditional support in every decision I have taken in my life. To my father for his love and comprehension, for let me take all the decisions in my life. To my

boyfriend for his patience, love, friendship and effort to travel to Monterrey.

(6)

Acknowledgements

I want to thank to the Instituto Tecnologico y de Estudios Superiores de Monterrey to give me credit facilities for my master degree. Furthermore I appreciate to the Center for Intelligent Systems for the academic support and the facilities to use the computer equipment during the years of my postgraduate course. Studies of this nature would be impossible without access to primary texts, journals and works of reference and criticism, for these I must thank the ITESM Campus Monterrey for the library facilities.

I wanted to thank my family for everything that have done for me during my residence in Monterrey, I always received comprehension and affection no matter the distance.

My greatest debt is to PhD. Leonardo Garrido Luna, his careful reading of my original manuscript and his detailed, perceptive comments contributed in no mean way to the shape and clarity of the finished thesis. It was at his suggestion that I compiled the first appendix and the index, and I have indicated my debt to his insight in the interpretation of the texts in numerous places in the notes.

Also I appreciate the help of the commite members, PhD. Hugo Terashima and PhD. Ramon Brena for the review of my thesis including their valuable comments which contribute to the final version of my thesis.

Gisela Medina Ramirez

Instituto Tecnol´ogico y de Estudios Superiores de Monterrey December 2008

(7)

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

Gisela Medina Ramirez, M.C.

Instituto Tecnol´ogico y de Estudios Superiores de Monterrey, 2008

Thesis advisor: Ph.D. Leonardo Garrido Luna

This document presents the thesis required to get the degree of Master in Science in Intelligent Systems. One of the problems more commonly found in Artificial Intel- ligence is related to negotiation where all participants involved in the decision process have to reach an agreement in a way that minimize costs for each participant; this problem is known as Distributed Agreement Problem. Our specific problem is focused on Multiagent Meeting Scheduling where several variables are considered for selecting the best schedule which fulfills the requirements and preferences of each participant.

Recently there is a necessity to find optimal solutions to problems of this type to satisfy the demands on time, quality and productivity into a competitive society. The research work described in this thesis is mainly focused on probabilistic multiagent learning for the selection of the best strategy applied in specific scenarios. We are using the approach of Crawford ([Crawford 05b]) about Playbooks to select the best negotiation strategy taken for an agent that acts as organizer or invitee in a meeting scheduling negotiation. In this approach the agent who uses the playbook is the only learning agent in the environment, the others agents are using static strategies. In our research work we implement several negotiation strategies and use the playbook to select the best one; our contribution is to use new and adapted strategies using several heuristics to make the agents more cooperative or competitive. This research work is focus on the development of modified algorithms applied to our multiagent meeting scheduling, in specific modified negotiation strategies helped by some learning algorithms. We put all these together and did a set of experiments to see the complexity, usefulness and performance of the algorithms in several scenarios.

(8)

List of Tables

3.1 Conditions for calculating multiplier according to utility function value 76 3.2 Interaction matrix useful to calculate all interactions between agents

who participate together in others meeting teams. Specially this matrix contains all teams from which each pair of participants interacts. This matrix is an example with 4 agents only. . . 77 3.3 Interaction matrix with averaged strategies weights calculated in each

cell, that is, the weights apported by each pair of participants when they participate together in other meeting teams. AW is represented by the vector of weights. This matrix is an example with 4 agents only. . . . 77 3.4 This represents a vector where each entry is related to the strategy

weights calculated for each participant. This average weights summa- rize the experience of every participant in other meeting teams. P W corresponds to weights by participant. This vector is an example with 4 agents only. . . 78 4.1 Profiles of agents involved in experimentation for Learning Other Cal-

endars . . . 87 4.2 Profiles of meetings run simultaneously in simulation for Learning Other

Calendars . . . 88 4.3 Profiles of strategies run in each simulation for Learning Other Cal-

endars. This strategies apply only to invitees. Organizer has a fixed strategy Of f er(2, 7, 2) . . . 88 4.4 Probability distribution for assignment of participants during the gener-

ation of random meetings . . . 95 4.5 Table with settings for experiment one . . . 96 4.6 Comparison of Strategies. The total of experiments are 21168 consider-

ing the combination of all parameters. The value 3024 is for experiments where the organizer is using a static strategy (to compare the behavior of his tactic with all others. The value 756 is given when the densities are fixed for both organizer and invitees and the strategy of the organizer is fixed. At the end all these experiments are created and analyzed.) . . 103

(12)

4.7 Comparison of Strategies. Profiles of preferences used for the experiment that compares all strategies. The problem generator builds the profile of all the meetings and participants used in the simulation. These preferences profiles are distributed like that to have a variety of agents into the negotiation and to be able to do correct comparisons. . . 103 4.8 Comparison of Strategies. These are the profiles of densities for the

experiment. See that no more than the half of the density is allowed, this is because we try to test the strategies in scenarios where the participants could be in fully or partial availability. The strategies do not allow rescheduling so if the calendar is almost full there are no chances to schedule a meeting. . . 103 4.9 Comparison of Strategies. The meetings are generated with 1, 3 or 6

invitees. These are good parameters to test the strategies and we see that the maximum number of invitees are 6, so the performance of strategies depends on the strategies themselves. It would be easy to schedule a meeting with only one invitee but it is more difficult to schedule it when more invitees are involved. . . 104 4.10 Comparison of Strategies. We chose a serie of strategies for OfferNKB,

Time and Behaviour with different parameters to find the strategies that work fine in specific situations . . . 104 4.11 Experiment 1. Use the time tactic to generate the experience data.

This table contains all the parameters needed to run the experiment for predictive tactics. . . 113 4.12 Table with settings for experiment one in playbook for single agent . . 118 4.13 Table with teams for experiment one in playbook for single agent . . . 118 4.14 Table with preferences for experiment one in playbook for single agent 118 4.15 Playbook. There are 7 strategies in the playbook where the learner

is selecting his best option at each point of choice into the Playbook Learning algorithm . . . 119

(13)

List of Figures

1.1 Effort vs. Time spent in Meeting Scheduling Domain . . . 1 1.2 Problem context for meeting scheduling process where several agents are

acting behalf of their users . . . 5 2.1 Examples of Concession Curves for the Polynomial Time-dependent Fam-

ily of Tactics . . . 19 2.2 Function schema to eval external features like preferences using eval

function . . . 35 3.1 Diagram for describing OfferNKB negotiation strategy through meeting

time proposal messages. . . 47 3.2 Analysis of scenario that shows functionality of OfferNKB strategy . . 51 3.3 Time Tactic using increasing exponential function . . . 54 3.4 Time Tactic using increasing polynomial function . . . 55 3.5 Time Tactic using increasing function varying β parameter with 0.2, 1

and 10 values . . . 55 3.6 Behaviour Tactic using δ = 2 for organizer. Two invites are using time

tactics. The red curve (upper) represents the first invitee using time strategy with beta 10. The blue curve (lower) represents the second invitee using time strategy with beta 0.3. And finally the green curve (average curve between these two invitees) represents the behavior strategy of the organizer . . . 59 3.7 Solution model for learning to negotiate using Crawford approach ex-

tended to meeting groups . . . 73 4.1 Calendar inferred is Inv1. We notice in this result that calendar density

is near to 0.5, the real density of this invitee. There were more than the half of well inferred slots but the number of bad inferred slots was medium high. We deduce that the strategy used in each simulation influenced this result as the other ones. . . 90

(14)

4.2 Calendar inferred is Inv2. We notice in this result that calendar density is up to 0.0, the real density of this invitee. Almost all slots were well inferred and there were not bad inferred slots. From the well deduced slots, more than a half have a probability between 0.7 and 0.9. If we compare this result with the other agents we found that this density is less than all others. . . 91 4.3 Calendar inferred is Inv3. We notice in this result that calendar density

is not near to 0.1, the real density of this invitee. There is more than a half of well inferred availability but the number of bad inferred in also high. From the well deduced slots, more than a half have a probability between 0.7 and 0.9. Although this is a favorable result the calendar density is not well precise. Compared with the others agent’s density this inferred calendar is set in the second place, that is correct because the first calendar with less density is from Inv2. . . 92 4.4 Calendar inferred is Inv4. We notice in this result that calendar density

is not so near to 0.2, the real density of this invitee. There were more than the half of well inferred slots but the number of bad inferred slots was medium high. As the results from other agents the number of bad inferred is considerablely high, but there are more well inferred than bad inferred. . . 93 4.5 Here is showing real and learned preferences for all invitees. Although

there are a small number of preferences learned correctly, the quantity is balanced by the measures done for each preference that is not learned correctly. . . 94 4.6 Shows the average iterations as result of the OfferNKB negotiator exe-

cution obtained after 10 runs . . . 97 4.7 Shows total of messages sent into an OfferNKB negotiator run 10 times 98 4.8 Shows the computational cost in time for the execution of OfferNKB

negotiator after 10 runs . . . 98 4.9 Shows percentage of meetings scheduled and canceled among the execu-

tion of OfferNKB negotiator after 10 runs. The number is low because of the preferences and the amount of slots exchanged . . . 99 4.10 Shows the percentage of final density in calendar of all participants.

The number is low because of the testing scenario and behavior of the OfferNKB negotiator after 10 runs . . . 99 4.11 The first chart shows the average iterations. The second one shows the

total of messages sent. The third shows the computational cost in time for the execution of the strategies. The total of runs are 10. . . 101

(15)

4.12 The first chart shows the percentage of meetings scheduled using the specific strategies. The second shows the percentage of final density in the calendar of all participants. This runs 10 times. . . 102 4.13 Comparing Strategies. Measure of utility using maximum number of

invitees. This figure shows the utility values for organizer and invitee using this configuration. The number of invitees involved in a meeting is an important factor which affects the overall performance of the negotiations. . . 110 4.14 Comparing Strategies. Measure of utility using strategies. This figure

shows the utility values for organizer and invitee using this configuration.

The strategy of the invitee involved in a meeting is an important factor which affects the overall performance of the negotiations. . . 111 4.15 Comparing Strategies. Measure of utility using preferences of invitees

with preference for organizer fixed to 0. This figure shows the utility values for organizer and invitee using this configuration. The preferences of the invitee involved in a meeting is an important factor which affects the overall performance of the negotiations. . . 112 4.16 Learned vs. Real preferences for each invitee: I0, I1, I2, I3, I4 and I5.

This preferences were learned from the results of the experiments where the organizer is using the strategy Of f erN KB(3, 5, 2) and the total of meetings generated were 108 where all the invitees are taking different strategies. The meetings also have different number of invitees. . . 115 4.17 It shows the first try using Predictive vs. Other strategies. The first chart

shows the utility for organizer and invitee. The second is averaging both to quality the performance of the Predictive. . . 117 4.18 Shows team weights adaptation for team 1 when does not consider past

experiences. Adaptation until 50 points of choice, averaged after 30 runs using 4 strategies of OfferNKB negotiator where the number of slots offered are 3 and 10 and number of rounds to do an evaluation is 5 an 10.120 4.19 Shows team weights adaptation for team 2 when does not consider past

experiences. Adaptation until 50 points of choice, averaged after 30 runs using 4 strategies of OfferNKB negotiator where the number of slots offered are 3 and 10 and number of rounds to do an evaluation is 5 an 10.121 4.20 Shows team weights adaptation for team 3 when does not consider past

experiences. Adaptation until 50 points of choice, averaged after 30 runs using 4 strategies of OfferNKB negotiator where the number of slots offered are 3 and 10 and number of rounds to do an evaluation is 5 an 10.122

(16)

4.21 Shows team weights adaptation for team 4 when does not consider past experiences. Adaptation until 50 points of choice, averaged after 30 runs using 4 strategies of OfferNKB negotiator where the number of slots offered are 3 and 10 and number of rounds to do an evaluation is 5 an 10.123 4.22 Shows team weights adaptation for team 1 when does not consider past

experiences. Adaptation until 50 points of choice, averaged after 30 runs using different strategies. . . 123 4.23 Shows team weights adaptation for team 2 when does not consider past

experiences. Adaptation until 50 points of choice, averaged after 30 runs using different strategies. . . 126 4.26 Comparison of performance between different strategies and the Play-

book learner related to the best utility . . . 127 4.27 Shows team weights adaptation for team 1 when past experiences are

considered. Adaptation until 50 points of choice, averaged after 30 runs using 4 strategies of OfferNKB negotiator where the number of slots offered are 3 and 10 and number of rounds to do an evaluation is 5 an 10.128 4.28 Shows team weights adaptation for team 2 when past experiences are

considered. Adaptation until 50 points of choice, averaged after 30 runs using 4 strategies of OfferNKB negotiator where the number of slots offered are 3 and 10 and number of rounds to do an evaluation is 5 an 10.131 5.1 Results obtained in mean of (a) CPU time; (b) number or percentage

of scheduled meetings and (c) number of exchanged messages using the Hassine´s work (in the left side) and our thesis work (in the right side). 134 5.2 Comparison of playbooks used by a) Crawford and b) our approach . . 136

(17)

5.3 Comparison of learning other calendar’s algorithm results using Leffert

article (left side) and our implementation (right side). . . 138

5.4 Jean Oh’s results for learning calendar preferences. . . 139

5.5 Our results for learning other calendar preferences. . . 140

A.1 The class model for the simulator . . . 147

(18)

Chapter 1 Introduction

Nowadays many organizations use technology for improving performance of com- munications such as email systems and shared resources which allow us to send and receive information in an automatic form, but there are still many tasks to improve which make easy the interchange of information between people. Some artificial intelligence techniques based on intelligent agents could improve such systems in a way that agents have knowledge of interests, priorities and preferences of people to which they represent for executing actions in an automatic way. Meeting scheduling is one of the topics which could take advantage of these techniques for being a kind of tasks, tedious and time consuming.

Figure 1.1: Effort vs. Time spent in Meeting Scheduling Domain

The figure 1.1 shows the time spent in scheduling meetings using a simple negotiation protocol. The x axis shows the effort represented by the number of meetings

(19)

that several teams are trying to schedule concurrently. The y axis represents the total of milliseconds spent to schedule them. The graph shows that the time spent increases according to the number of meetings; this is a normal behavior in meeting scheduling scenarios because the complexity of the agreement in negotiations. In the graph the maximum time spent is almost 1667 minutes (28 hours) to schedule 160 meetings concurrently. In this thesis we propose to implement several negotiation strategies to schedule meetings in less effort time but with the same complexity given by the total of meetings scheduled concurrently.

The main topic of this thesis is learning in distributed multiagent environment where each agent is able to learn to select the best strategy or action considering some variables which take part in the problem. In research work of Bowling and Veloso [Bowling 02b] they proposed a learning strategy using Game Theory and Markov Decision Process (MDP) called stochastic games. It is a good approach since they considered reinforcement learning and searching of best solution to satisfy properties of rationality and convergence. A research work also interesting is the proposed by Crawford and Veloso [Crawford 05b] focused on scheduling which use learning to select best strategies through approximation of Playbooks where each agent has an individual book of strategies and in a given situation it is able to select the best between them.

Applying specific strategies and algorithms of Multiagent Learning in problem solving in distributed multiagent environments is a challenge very interesting to inves- tigate and for which many valuable and relevant research work have emerged. Many problems we face in home, work, industry, city and school are types of problems where many entities are interacting to reach an agreement favorable to all participants.

1.1 Motivation

A society is a set of members which can have similiar or different interests given circumstances and situations in which they take part where their relationship is determined by protocols which define politics and rules in cooperation schemas, competition and negotiation. The study is focused in societies where the number of members is considerably large and the preferences in many situations are different. For example, in student societies where there are hundreds of students in all campus, each of one defines a set of school and personal preferences. When it is time to take a decision in which many people is considered then conflicts of interests and preferences emerge and they can avoid reaching an agreement in less time as possible. Because of this, there are many algorithms which have been focused to solve negotiation problems and coordination in a distributed multiagent environment.

Meeting scheduling is one of the more interesting topics nowadays because its concurrent and simultaneous nature. Also the work realized by others is focused on

(20)

a specific feature of meeting scheduling such as dynamic environments, learning, user modeling, user preferences and so on. In our work, there is a collection of bibliography and information taken from more recent research work. Although the meeting scheduling is solved from different perspectives, it has been an important problem which has not been solved completely because the level of difficulty to study every part of the problem.

There is a set of well-documented strategies to solve the meeting scheduling problem and use them in harmony to reach a favorable result. These strategies are related to learning during the negotiation protocol ([Crawford 05a], [Ho 03]), learning user preferences ([Oh 05a], [Stuart E. Middleton 04], [Melinda T. Gervasio 05]) and modeling user by experience ([Ben Geisler 01]). To apply correctly these strategies is necessary to interact many times with the system, to have enough information for deciding how and where to apply them. That is because many techniques fail to solve meeting scheduling problem and some research work is done for dealing with it.

1.2 Problem definition

Our research is focused on the category of Multiagent Meeting Scheduling (MMS) where the goal is to coordinate the calendars of each participant for scheduling a meeting in a time favorable to all participants. Preferences and interests of users can be public or private. Public means preferences are known by all participants. Private means only agent knows his calendar preferences. In the real world such preferences can change over time because of the interaction the agents have with their environment.

Normally, the meeting scheduling process is a tedious and long task which con- sumes time because there are many meetings which are being negotiated in a simultaneous way. A personal agent should act in favor of his user and negotiate considering his private personal preferences. The goal in this case is to keep the privacy of information during negotiation process. This makes agents to be autonomous and independent of his user. At the end of meeting negotiation process all participants agree with the final decision. Besides it is considered that preferences are changing over time and depend of circumstances and environment in which they interact. As a result of this, their strategies also change to allow competition and cooperation and because of this the problem becomes very dynamic. As in real world, persons change their preferences and strategies (take decisions) when they know their past experiences and that is because they act in a different way when scheduling a meeting under several circumstances. Knowing the environment where they act means to know also those involved in the process. User modeling is a feature considered to define the real context of the problem.

Figure 1.2 shows in a conceptual way the problem of meeting scheduling into a school society. There are two groups, (MAT and MIT) with two participants each

(21)

one and private preferences. Suppose a third participant proposes one meeting A for at hM onday, 8 : 00i and other participant proposes other meeting B for at hM oday, 8 : 30i and both of them have a duration of one hour. The organizer sends invitations to all participants and these check their calendar availability and personal preferences to confirm, refuse or propose another day and hour. The organizer receives all offers and generates a new one. The process continues until an agreement is reached or when they are cancelled.

Formally a meeting schedule consists of a group of meetings for a group of persons (as described by [Sen 91]). Given a set of n meetings and k attendees, a scheduling problem is represented as S = (A, M ) where A = 1, 2, . . . , k is the set of attendees and M = m₁, m₂, . . . , m_n is the set of meetings to be scheduled. A time slot is represented as a date, hour pair hD, Hi. A set of contiguous time slots is called a time interval. A meeting named i is represented by a tuple:

m_i = (A_i, h_i, l_i, w_i, S_i, a_i, d_i, f_i, T_i) where

A_i ⊆ A is a set of attendees of the meeting; h_i ∈ A_i is an attendee who will host the meeting (organizer); l_i is the required length of the meeting in hours; 0 < w_i < 1 is the weight or priority assigned to the meeting; S_i contains a set of possible starting times on the calendar for the meeting. If |S_i| = 1 the meeting is said to be constrained (the exact interval to be used for meeting is pre-specified); if S_iincludes all time slots on the host calendar (assuming that it was empty) from which a meeting of length li can be started, then the meeting is said to be unconstrained; otherwise, the meeting is semi constrained; a_i is the time at which h_i becomes aware of the need to schedule m_i; d_i is the deadline by which the host h_i needs to schedule the meeting m_i; f_i is the time at which the final decision is made about whether m_i can be scheduled and, if so, when it will take place; T_i is the time interval for which the meeting m_i is finally scheduled and is represented by an ordered set (hDi, Hii), hDi, Hi+ 1i , . . . , hDi, Hi+ li− 1i, (here Di

gives the date and H_i gives the starting hour for which meeting m_i is scheduled) if the meeting could be scheduled, and by empty otherwise. Additional considerations include cancellation and rescheduling of meetings, scheduling rooms for meetings, scheduling meeting with subset of specified attendees, priorities for meetings, hierarchy of each agent (or level of importance) and duration of meetings.

Particularly in this thesis we use some values for these parameters. li is set to 50 time slots (hours), w_i is set to 1 because all the meetings are considered with the same high priority, S_i is set according to the negotiation strategy (it refers to the total of slots to negotiate between each iteration in the negotiation process), d_i is set to 50 that corresponds to the total of iterations allowed in the negotiation process. In order to facilitate the negotiation protocol and the application of the negotiation strategies,

(22)

Figure 1.2: Problem context for meeting scheduling process where several agents are acting behalf of their users

(23)

the development of this thesis is not considering rescheduling and cancellation of meetings. Rescheduling and cancellation are two additional process that affect the results of the different negotiation strategies we suggest and we prefer to implement a simple negotiation protocol without these two features to facilitate the result analysis. The problem is delimited to schedule only meetings based on the definition explained previ- ously, all meetings have the same priority, all invitees have the same level of importance and all meetings have a duration of 1 hour. The importance of these considerations is to facilitate the implementation of the negotiation protocol and the result analysis of the negotiation strategies. The user preferences are private which allow keeping con- fidentiality in information. Normally as a result of interaction between participants, knowledge of each one is gained, and it starts to model the users. This is also a desirable feature because is closer to reality where each person learns to interact with the others.

Many research work has been done related to meeting scheduling into the artificial intelligence field whose results have been good and the contribution of knowledge allows the generation of new topics of study. Many people study the scheduling problem inside the frame of Distributed Artificial Intelligence as the work of Sen and Durfee [Sen 91] which introduce heuristics. Given the distributed nature of the problem, the better approach is the paradigm of multiagent systems in which many work has been realized. One of them is the work of Modi and Veloso [Modi 05] which use the paradigm of Private and Incremental Multiagent Agreement Problem (piMAP) for the solution of meeting scheduling finding new strategies of decision (bumping strategies)which allow to the agents the arrival of new meetings and rescheduling of others to protect the privacy of the user’s information. The work of Miyashita [Miyashita 96] is another example of modeling negotiation strategies through the uses of cases, a supervised technique for learning in negotiation. The work of Enric Plaza ([Plaza 02]) refers that cooperation and learning are two ways in which an agent can improve its performance. He analyzes the multiagent learning as a framework to analyze the tradeoff between cooperation and learning in multiagent systems. All these research is done around negotiation and learning. Our meeting scheduling problem is analyzed using several negotiation algorithms where cooperation, competitiveness and learning are their principal components.

1.3 Hypothesis

The learning algorithms and the concept of playbooks in multiagent environments applied to the meeting scheduling problem along with the use of negotiation strategies, ensure the convergence of the good solution, increasing the utility value during the negotiation process between several agents which act of behalf of their users. Good solution means to reduce the negotiation time, to have more scheduled meetings and

(24)

to get a high global utility, all these compared with fixed strategies without learning phase.

This thesis finds to validate the following hypothesis.

1. The agents act in an effective way (based on their preferences) increasing the global utility into a negotiation protocol using the learning strategy based on playbooks

2. The global utility can be increased during the negotiation process without sacri- ficing the convergence of the optimal solution, using negotiation strategies with concessions

3. The learning of the other agent’s preferences allows the execution of an effective negotiation

4. The use of different negotiation strategies in the playbooks ensures the convergence of the best and the worst strategy

5. The use of predictive strategies increases the utility of the agents in the negotiation in comparison with other strategies

6. When the organizer concedes more than the invitees allows to the negotiation finishes with high utility for all the participants

7. The number of scheduled meetings increases when there are agents conceding most of the time, but the global utility decreases

8. The convergence of the best and worst solution using playbooks with experience is faster than using playbooks without experience

1.4 Objectives

The main objective of this thesis is to use a combination of strategies of probabilistic reinforcement learning and techniques of negotiations used in multiagent distributed environment with the approach of Playbooks where each agent associates a set of particular strategies and learns to recognize when use each of one in his environment.

The particular objectives to realize in this thesis of research are the following.

• Development of the meeting scheduling multiagent framework or testbed used to test and analyze the negotiation strategies described and implemented in this thesis.

(25)

• Development of negotiation strategies based on a communication protocol where the organizer starts the negotiation with one or more initial proposals. Invi- tees replies with either counter proposals or only an accept/refuse anwser. The organizer mediates the negotiation being more cooperative than all the invitees.

• Development of several negotiation strategies based on heuristics and learning phase applied to meeting scheduling problems using the meeting scheduling multiagent framework as the testbed. Each negotiation strategy must implement specific features of the meeting scheduling problem domain.

• The implementation of the negotiation strategies must take into account concession functions to allow participants behave more or less cooperative.

• Get the total of scheduled meetings using several negotiation strategies, each using different heuristics and learning phase.

• Development of the playbook that supports several negotiation strategies in order to select the best strategy applied to a specific meeting scheduling problem in- stance. The base for this implementation is taken from the article [Crawford 05b]

that was modified to allow new negotiation strategies and meeting teams.

• Get the calendar information (calendar density) of all the participants using learning algorithms. This information must be known by the learning agent previous to the process of learning the calendar preferences.

• Get the user preferences (calendar preferences) of all the participants using learning algorithms. This information must be known by the learning agent previous to the negotiation and any negotiation strategy could use it when it requires.

• Development of a predictive negotiation strategy which uses additional information of each participant such as calendar density and calendar preferences.

1.5 Thesis organization

The organization of this thesis is as follows. The current chapter introduces the problem of multiagent meeting scheduling and presents how several approached have used to solve the problem. The chapter 2 introduces the fundamentals of this thesis such as techniques and research works where topics of learning techniques are presented.

This chapter is essential to understand the work of the thesis because it has detailed each technique. Chapter 3 presents the solution model which use a combination of several techniques described in chapter 2. It details all features and describes the design of each algorithm used. Chapter 4 presents the experimentation methodology used to design

(26)

all testing scenarios and the assumptions considered before analyze the results. Also it includes analysis of results detailed for each testing scenario and compares them.

Chapter 5 shows the work related to this thesis where several techniques are used and an analysis of them are described. Chapter 6 describes the future work that can be done after the termination of this thesis and final conclusions. Also it includes a section of our contributions in this thesis.

(27)

Chapter 2 Fundamentals

In artificial intelligence, learning is a topic of great interest for the design of an intelligent and autonomous agent that allows making his decisions in a rational way.

The doctoral thesis of Bowling [Bowling 03a] focus to the topic of Multiagent Learning applied to Stochastic Games using game theory and reinforcement learning in cooperative and competitive environments where agents have limitations for acting in a rational way. Other work of Bowling [Bowling 04] related to his doctoral thesis defines several evaluation criteria for algorithms of multiagent learning in environments totally dynamic where recognize two main attributes for being evaluated: convergence and de- ception. Show that the learning algorithm GIGA-WoLF satisfies these two attributes in real worlds with limitations. To measure the impact of limitations of agents during the learning process, specially in the computation of Nash equilibrium, the work of Bowling and Veloso [Bowling 02a] makes a detailed study. For negotiating in a strategic way, agents need to learn to select the best negotiation strategy with other agents, and for that the work of Crawford and Veloso [Crawford 05b] introduces the concept of playbook which defines a set of techniques used during the selection process.

Another of the topics of importance in artificial intelligence is the coordination of a set of autonomous agents for the generation of an effective multiagent system.

The work of Bowling and McCracken [Bowling 05] refers to a simultaneous learning in a non cooperative environment using multiple robots and although the study is applied to a soccer game scenario with autonomous robots it can be generalized to other applications as meeting scheduling. The proposed architecture uses the concept of playbooks as a general strategy of planning where each team defines a set of strategies or plays related with each other agent in the scenario and these are adapted while they interact with their environment. In the same non cooperative environment of the soccer game, the authors Bowling and Veloso [Bowling 03b] introduce a new simultaneous learning algorithm, GraWolf, which combines the learning techniques of policies based on gradients with the variable rate of learning WoLF, Win or Learn Fast.

There are a great variety of techniques and methods used to solve the problems related to meeting scheduling, such as the negotiation strategies (Game Theory), Multi- agent Learning and Playbooks. For ensuring the convergence to a more rational solution

(28)

the agent can recognize or anticipate the actions of their opponents, and for that several techniques related to user modeling have been studied through the observation of their actions. In the other hand, the multiagent learning is also used for doing intelligent agents to act in a rational way in specific situations given the knowledge gained in previous interactions.

2.1 Multiagent Meeting Scheduling as a Negotia- tion Problem

In previous chapter we defined the meeting scheduling problem. To start this section we rewrite this definition: it is the act of finding an available time between all participants in a meeting in the least number of iterations, allowing that each participant acts in a cooperative or competitive way to protect his preferences and interests.

Meeting scheduling into a multiagent distributed environment is a hard task because several meetings are done at the same time.

According to [Sen 91], meeting scheduling is an example of a resource allocation problem where the principal resource under consideration is people’s time. Resource allocation in a distributed, dynamically changing environment is difficult due to the distribution of information needed for decision making, the dynamic nature of the system (which may lead to changing/conflicting goals), the limited bandwidth of the communication channel and communication delays between parts of the system. Centralizing control in a single resource allocator suffers from serious drawbacks including a lack of robustness (if the allocator fails, the entire system collapses) and its communication and computation demands on a single bottleneck process. To overcome the limitations in centralized resource allocation, we instead distribute decision making among the processes controlling the separate resources. Distributed resource allocation, therefore, involves the cooperative solution of resource allocation problems among a network of decision makers, and falls into the subclass of DAI known as Cooperative Distributed Problem Solving (CDPS).

Individual agents in a Distributed Meeting Scheduling (DMS) system have only partial knowledge of system-wide goals because they do not know all the meetings that are currently being scheduled or have already been scheduled by other agents. Hence they must exchange relevant information to build local schedules that fit into glob- ally consistent schedule. To enable information exchange, the agents need a common communication protocol for negotiating over meeting times. Moreover, because users must be able to understand and accept how the agents interact, the protocol must be well-defined and straightforward while still providing sufficient flexibility. For negotiation it is desirable a negotiation protocol in which agents can exchange information. In that protocol, each meeting has a particular agent who is responsible for it, called the

(29)

host or organizer. The host contacts other attendees of the meeting (who are called invitees) to announce the meeting and collects bids (availability information). This process could be repeated several times as they search through different parts of their respective calendars. In between, other meetings could be undergoing scheduling; in general, an agent can simultaneously be involved in scheduling any number of meetings, acting as a host for some and an invitee for others.

How well this protocol performs in efficiently converging on good schedules is strongly impacted by heuristic strategies about what information to exchange and how to model tentatively scheduled meetings. Strategies for communication must balance demands for privacy (which lead to exchanging less information) with demands for quickly converging on meeting times (which can be sped up by exchanging more information). Strategies for modeling tentatively scheduled meetings can range from block- ing off tentative times for a meeting unless and until the arrangements fall through, to ignoring tentative commitments about a meeting when scheduling other meetings.

Negotiation protocol follows several heuristic strategies such as described below.

• Announcement strategies. Determine how a meeting is announced, and usually involve proposing some number of possible times. The slots of time can be ranked with a number or a label according to a criteria such as preference or a specific negotiation rule.

• Bidding strategies. Determine what information an invitee sends back based on an announcement. The alternatives are to rank slots proposed by organizer or making a contra proposal or both.

• Commitment strategies. Determine to block off a slot of time during all negotiation process or only at the end when a full agreement on a meeting time is reached.

2.2 Negotiation

The negotiation definition applied to meeting scheduling problems, as described in the previous section, is composed of a communication protocol, user preferences (related to time preferences) and the negotiation protocol, all these described in this section.

2.2.1 Comunication Protocol

Negotiation strategies are based on a specific protocol which allows communication between all participants, distinguishing between organizer and invitees. Every organizer must start a meeting negotiation through a meeting proposal which is sent

(30)

to each participant. Each meeting proposal is represented by a message which contains specific information of the meeting to be scheduled. Main features implemented into this protocol are listed below.

1. Meeting time proposals. An organizer is responsible for sending a meeting negotiation message to all invitees. When the message reach an invitee, this is responsible to send a response to the organizer according to his time preferences for each time proposal.

2. Initially it is not allowed neither to cancel a previous scheduled meeting nor reschedule it.

3. Cancellation as a mean of an undo operation when a negotiation for a meeting is not reached. That is, if organizer found a slot of time suitable for all participants and sent them a confirmation but one or more invitees have their slot of time not available at that specific moment then a cancellation message is sent to undo the operation. The same occurs when is the organizer whose slot of time is not available.

4. There is not pending slots. When a slot is a candidate for schedule the meeting, that is all invitees have voted for it with high values, it is not marked as pending and this allow other concurrent negotiations take it into account for future proposals.

5. Confirmation messages. There are two types of confirmation, one is when a slot of time is found and negotiation is finished successfully, and the second is when negotiation has not reached after a maximum total of interaction.

2.2.2 Preferences

One variable into the meeting scheduling is given by calendar preferences of each participant. The participant could have either a public or private calendar. When the participant shows all free time slots to all participants then the calendar is public.

When he hides his calendar to all the participants then the calendar is private. Calen- dar preferences is given by the set of time slots the participant prefers before others.

One participant could prefer to schedule meetings in the mornings and others in the afternoon; others participants could prefer to schedule on Mondays instead of Fridays (in any hour). It could happen the partipants prefer to have all theirs meetings one after another in the same day and have the rest of the days free. The negotiation protocols used to implement a simple preference model based on time slot selection (no matter the day of the week).

(31)

2.2.3 Simple Negotiation Protocol

There are some research work related to meeting negotiation protocols which have been used into testbed environments to test any new algorithms. There are two types of calendars: open and close. The open calendar are public calendars, i.e., all others agents know about it and it is easy to agree in one free slot in a reduced number of iterations.

Although this is a simple strategy the disadvantage is the lost of privacy, not ideal in real scenarios. Now the more common calendar used in testbeds close to real scenarios is private which not deal with lost of privacy because each agent does not have to show all his calendar at beginning. Agents using one of these two types of calendars act as competitive and cooperative: cooperative because to formulate new proposals agents have to analyze their own past proposals and from the others; competitive because they also negotiate within their own preferences. Determine how competitive or cooperative is an agent depends of the variations of the negotiation protocol which we will describe in this chapter.

Open calendar behavior is described as follows (detailed in algorithms 2.1 and 2.2): organizer knows all invitees calendars and without any other prior knowledge he is able to make a proposal and send it to the invitees. They have to accept or refuse the proposal according to their preferences and availability. Organizer collects all responses and if any is refused then he continues making more proposals, but if all are accepted then the negotiation is finished with a scheduled time agreement. The negotiation could finish when all slots have been proposed or maximum number of rounds has been reached.

Private (close) calendar behavior is described as follows (detailed in algorithms 2.3 and 2.4): organizer does not know all other calendars, so he makes a proposal according to his own preferences and availability: the proposal size is n, it means, n slots of time are proposed at a time. When invitees receive proposal their make their responses according also to their own preferences and availability. Organizer then collects all responses and makes an intersection to find a common proposal (all invitees must accept that slot). If all agree then organizer sends the confirmed slot. If not then the negotiation continues until it reaches a maximum number of rounds or all slots have been proposed.

2.2.4 Other Negotiation Protocols

There are several articles related to negotiation protocols which we are going to mention in this paragraph. First, [de los Angeles Constantino Gonzalez 95] designed a distributed meeting scheduling protocol using KQML protocol. She presented a negotiation protocol for meeting scheduling based on an intelligent personal multiagent system. In the protocol each user had an agent who knows his commitments and

(32)

Algorithm 2.1 Negotiation using Open Calendars: Organizer Behaviour Require: All calendars are visible, one slot in proposal

Ensure: Protocol finishes at specific number of rounds

while max rounds not reached and all slots have not been proposed do inspects invitees calendars

finds best choice according to his preferences and availability of own and others calendars

if best choice found then sends proposal to invitees

Asynchronous communication: Here waits for all responses collects all responses

if all invitees accept proposal then

sends confirmation to invitees with agreement negotiation finishes with agreement

end if else

organizer sends a termination signal to invitees negotiation finishes with no agreement

end if end while

Algorithm 2.2 Negotiation using Open Calendars: Invitee Behaviour Require: Receives organizer proposals

while receive organizer proposals do receive proposal

evaluate proposal according to preferences and availability responses with accept or refuse

end while

Algorithm 2.3 Negotiation using Private Calendars: Organizer Behaviour Require: Organizer’s calendar, n slots in proposal

Ensure: Protocol finishes at specific number of rounds

while max rounds not reached and all slots have not been proposed do makes a not-yet-proposed proposal with n slots available and prefered sends proposal to invitees

Asynchronous communication: Here waits for all responses collects all responses

if all invitees agree in one slot then

sends confirmation to invitees with agreement negotiation finishes with agreement

end if end while

(33)

Algorithm 2.4 Negotiation using Private Calendars: Invitee Behaviour Require: Receives organizer proposals

while receive organizer proposals do receive proposal

evaluate proposal according to preferences and availability

responses with accept or refuse in every slot in proposal from size n end while

preferences. She proposed an entity-relationship model to design the meeting scheduling problem.

The article [Debenham 07] is an example of another negotiation protocol development based on three different types of agent. Two negotiation agents, each rep- resenting an individual, which develop consecutive offers, supported by information, while requesting information from its opponent. A mediator agent, with experience of prior negotiations, suggests how the negotiation may develop. A failed negotiation is a missed opportunity. An observer agent analyses failures looking for new opportunities.

The integration of negotiation theory and data mining enables the curious negotiator to discover and exploit negotiation opportunities. This type of negotiation protocol is useful in topics like ecommerce where a high number of success is required.

The article [Shintani 00] is an example of negotiation protocol that uses persuation applied to the meeting scheduling problem. In the meeting scheduler, an agent negotiates with other agents about making a public schedule by referring user’s private schedules and preferences. They proposed the persuation method called multiple negotiations based on the multi attribute utility theory. In the multiple negotiations, each agent has an opportunity for persuading the others by conducting all patterns of negotiation. In order to reach the consensus, agents try to persuade the other agents (it should be conducted concurrently to facilitate multiple negotiations). The disadvantage of this negotiation protocol is the concurrency and therefore the performance.

Given the nature of the meeting scheduling problem this protocol design is not well applied.

2.3 Negotiation Algorithms

Multiagent meeting scheduling is defined as a particular subset of the general negotiation problem, so the study of negotiation frameworks is useful to design a specific solution for this type of problems. This section summarizes the background for the development of this thesis using some articles as reference. The solution model for the thesis, as describes in tne next chapter, uses a combination of negotiation algorithms and ideas taken from these articles; therefore it is interesting to describe them in this

(34)

section, emphasizing their most important features. It is important to point out that we modified these algorithms to be applied to the meeting scheduling problem domain.

In the next chapter we focus on these important modifications.

2.3.1 Negotiation Algorithm Based on Functions

This section describes a negotiation framework based on functions which design the behavior of a competitive and cooperative agent. These functions are formulated and explained in [Hou 01]. It is important to mention that these functions are not implemented for meeting scheduling problems and the challenge of this thesis is to modify them to be applied for this domain. One of the goals of this thesis is to implement several negotiation strategies and compare them to analyze the specific performance in different situations. Some of these strategies are implemented using this design. If the original framework works for numerical domains (i.e. buyer seller domain) then it has to work for the meeting scheduling domain; this is our hypothesis and we will demonstrate it.

To get better individual or social outcomes, the software agents require appropriate tactics. A tactic is the decision policy for choosing actions in different situations.

Because negotiation is an interactive process, the outcome is not only determined by an agents own tactic but it is also influenced by the other agents choices. This charac- teristic makes it difficult to find an optimal tactic.

Game theory treats the negotiation as a kind of game and negotiating agents as the players in a game. Game theory provides formal concepts to analyse the strategic interaction among agents in negotiation. However, game theory has two fundamen- tal assumptions: common knowledge and perfect computational rationality. In the first assumption, all information about the possible strategies, the outcome with each configuration of strategies, etc., are common knowledge known to each agent. Perfect computational rationality assumes a negotiation agent has unbounded computational power. With this power, agents can actually find the optimal strategies at the beginning of the game. These assumptions do not necessarily hold for real-world negotiation and therefore make it difficult to apply game theory in practice.

Since an agent has limited computational power and incomplete knowledge about other agents, it is necessary for an agent to produce offers based on their own criteria, such as, time limits or resource availability. Using this approach were proposed a negotiation model and three families of negotiation tactics, namely: time-dependent, resource-dependent and behaviour-dependent tactics.

Negotiation model

In this negotiation model, two parties negotiate on an issue. The two parties adopt two conflicting roles. The negotiation is a process of two parties making alternate

(35)

offers. Let x^t_s→b is the offer proposed by the seller s to the buyer b for a negotiation issue at time t. x^t_s→b is delimited by [min, max], the range of all possible offers by s.

The negotiation is to determine a value x (x ∈ [min^s, max^s] ∩ [min^b, max^b]) which is mutually acceptable to s and b. max^b is actually the reservation value of b, that is, any value larger than max^b won’t be accepted by b. min^sis the reservation value of s, that is, any value smaller than min^s won´t be acceptable to s. Each agent a has a scoring (or utility) function Vâ: Dâ → [0, 1] that assigns a score to value x in Dâ. The scoring functions are either monotonically increasing for the seller or decreasing for the buyer.

A negotiation is an alternating-offer process terminated by accept or withdraw.

Agent a’s response at t_n to agent b’s offer x^t_b→aⁿ⁻¹ sent at time t_n−1 is defined as:

response^a(tⁿ, x^t_b→aⁿ⁻¹) =







withdraw(a, b) if tⁿ> t^a_max

accept(a, b, x^t_b→aⁿ⁻¹) if V^a(x^t_b→aⁿ⁻¹) ≥ V^a(x^t_a→bⁿ ) of f er(a, b, x^t_a→bⁿ ) otherwise

(2.1)

where x^t_a→bⁿ is the counter offer a offers to b when offer x^t_b→aⁿ⁻¹ is not accepted by a.

t^a_max is as deadline by which a must have completed the negotiation.

Offers are generated by functions called tactics. A tactic generates a value for a single negotiation issue based on a single criterion such as the time available, the resource remaining, or the opponents behaviour.

Time-dependent family of tactics

In this family of tactics time is the predominant factor. All of the tactics in this family prescribe that an agent a concedes to their reservation value by their t^a_max.

What differentiates them is the shape of their concession curves (figure 2.1). The offer of agent a to agent b at time t < t^a_max is modelled by a function α^a which is dependent on time:

x^t_a→b=

minâ+ αâ(t)(maxâ− minâ) if Vâ decreasing

minâ+ (1 − αâ(t))(maxâ− minâ) if Vâ increasing (2.2) where 0 ≤ αâ(t) ≤ 1, and can be in one of the following two forms:

1. Polynomial:

αâ(t) = kâ+ (1 − kâ) min(t, tâ_max) tâ_max

1/β

(2.3) 2. Exponential:

(36)

Figure 2.1: Examples of Concession Curves for the Polynomial Time-dependent Family of Tactics

α^a(t) = e⁽¹⁻

min(t,tamax) tamax )^βlnk^a

(2.4) where kâ (∈ [0, 1]) determines the initial offer at t = 0 in [minâ, maxâ]

Both forms are parameterised by a value β ∈ R⁺ that determines the rate at which an agent approaches their reservation value. The expressions above represent an infinite number of possible tactics, one for each value of β. There are three types of tactics in this family: Boulware (β 1) where an agent does not start conceding until the deadline is nearly up. Conceder (β 1) where an agent will start giving ground fairly quickly. And, Linear (β = 1) where an agent concedes the same amount in each round of the negotiation.

Resource-dependent family of tactics

These tactics generate offers depending on how a particular resource is being consumed. They become progressively more conciliatory as the quantity of resource diminishes.

()αâ(t) = kâ+ (1 − kâ)e^−resourceâ^(t) (2.5)

(37)

where resource^a(t) = _|X^µt^a

a↔b|, and µ^a is the time agent a considers reasonable to spend on negotiation, and |X_a↔b^t | is the number of messages exchanged in the negotiation, i.e. the communication cost.

Behaviour-dependent family of tactics

These tactics base their actions on their opponent’s behaviour.

1. Relative Tit-For-Tat: Agent a reproduce, in percentage terms, the behaviour that their opponent b exhibited in the previous δ ≥ 1 rounds.

x^t_a→bⁿ⁺¹ = min(max( x^t_b→a^n−2δ

x^t_b→a^n−2δ+2x^t_a→bⁿ⁻¹, min^a), max^a) (2.6) 2. Random Absolute Tit-For-Tat: Same as Relative Tit-For-Tat, except that the

behaviour is imitated in absolute terms.

x^t_a→bⁿ⁺¹ = min(max(x^t_a→bⁿ⁻¹+ (x^t_b→a^n−2δ − x^t_b→a^n−2δ+2) + (−1)^sR(M ), minâ), maxâ) (2.7) where R(M ) is a function that generates a random value in the interval [0, M ], and s = 0 if Vâ is decreasing and s = 1 if Vâ is increasing.

3. Average Tit-For-Tat: Uses the average of the percentage change in a window of γ ≥ 1 of the opponent’s history.

x^t_a→bⁿ⁺¹ = min(max(x^t_b→a^n−2γ

x^t_b→aⁿ x^t_a→bⁿ⁻¹, min^a), max^a) (2.8) In order to obtain a better outcome in negotiation, an agent needs to find out some information about their opponent. However, in competitive bilateral negotiation, due to the diversity and complexity of negotiation in practice, a negotiator has little information about their opponent. Under this uncertainty, although an agent can choose a tactic with best performance on average, such as resource tactic or Linear time-dependent tactic, both tactics can not guarantee an agent the best deal. Since there is no optimal tactic for all negotiation environments, an agent has to choose the most effective tactic when facing agents with different tactics. Typically, an agent does not declare their tactic. In fact, the only available information in most cases is the previous offers from the other negotiating agent.

Solution model

The proposed approach uses nonlinear regression to learn an opponents tactic.

Using the results obtained an agent can:

(38)

• discriminate which of the three families of tactics their opponent has adopted (i.e.

time-dependent, resource-dependent or behaviour-dependent tactic)

• predict the opponents offers

• identify the opponents tactic type in the early stages of negotiation (e.g. Boul- ware, Conceder or Linear in the time-dependent family)

• estimate the opponents reservation value and deadline using the identified tactic type

The information about the opponents tactic type can be used to guide the predicting agents choice of tactic. Furthermore, by identifying the opponents deadline and reservation value, the predicting agent can either avoid breakdown or terminate unprofitable negotiations. See [Hou 01] to details about solution model proposed by Hou using nonlinear methods.

2.3.2 Negotiation Algorithm Based on Prediction

This section describes a new negotiation algorithm based on prediction. It uses the negotiation framework based on functions described in the previous section. This algorithm was proposed in [Jakub Brzostowski 06] by Brzostowski. We use it in our thesis for multiagent meeting scheduling problems. For this purpose, the algorithm was modified to manage non numerical data (described in solution model chapter).

We rewrite the most important features of the algorithm in this section to understand the phases where the modification is done. The prediction that the algorithm makes is based on the knowledge that the opponent agent is using any negotiation strategy based on functions, so the suggested model considers the time and behavior variations during the negotiation process.

Typically, there are two main factors that influence the negotiation agent behaviour: time and imitation. The agents behaviour depends to some extent on time and it depends to some extent on imitation. In order to be able to assess the degrees of this dependency appropriate criteria will be introduced in a form of functions mapping the sequences of previous actions of both parties in the current encounter into interval [0, 1]. These two criteria should state to what extent our partner responds to its time constraint and to what extent does it respond to our actions.

Time-dependency

The time-dependant tactic may be modelled by the use of polynomial or exponential function. The whole family of polynomial or exponential functions has a constant sign of all derivatives and this feature is treated as a necessary condition for

Probabilistic learning strategies applied to agreement negotiation for meeting scheduling

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

Master of Science in Intelligent Systems

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey

Gisela Medina Ramirez

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

Master of Science in Intelligent Systems

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey

Gisela Medina Ramirez

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey

Graduate Division in Mechatronics and Information Technology

Thesis Committee:

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

Gisela Medina Ramirez

Instituto Tecnol´ ogico y de Estudios Superiores de Monterrey Campus Monterrey

Acknowledgements

Gisela Medina Ramirez

Probabilistic learning strategies applied to agreement negotiation for meeting

scheduling

Contents

List of Tables

List of Figures

Chapter 1

Introduction

1.1 Motivation

1.2 Problem definition

1.3 Hypothesis

1.4 Objectives

1.5 Thesis organization

Chapter 2

Fundamentals

2.1 Multiagent Meeting Scheduling as a Negotia- tion Problem

2.2 Negotiation

2.2.1 Comunication Protocol

2.2.2 Preferences

2.2.3 Simple Negotiation Protocol

2.2.4 Other Negotiation Protocols

2.3 Negotiation Algorithms

2.3.1 Negotiation Algorithm Based on Functions

2.3.2 Negotiation Algorithm Based on Prediction