An important issue associated with planning for networked multi-agent teams is that plan- ning algorithms rely on underlying system models, which are often subject to uncertainty. This uncertainty can result from many sources including: inaccurate modeling due to simpli- fications, assumptions, and/or parameter errors; fundamentally nondeterministic processes (e.g. sensor readings, stochastic dynamics); and dynamic local information changes. Differ- ences between planner models and actual system dynamics cause degradations in mission performance. Furthermore, the impact of these discrepancies on the overall quality of the plan is typically hard to quantify in advance due to nonlinear effects, coupling between tasks and agents, and interdependencies between system constraints. For example, longer- than-expected task service times not only affect the scores received for those tasks, but also impact the arrival times of subsequent tasks in an agent’s path (see Figure 2-3). These types of propagation effects can be catastrophic in certain environments (e.g. time-critical missions), however, if uncertainty models of planning parameters are available they can be leveraged to create robust plans that explicitly hedge against the inherent uncertainty to improve mission performance.
For example, consider the problem definition introduced in Eq. (2.6) but where the planning parameters θ are random variables. Assume that a model of the uncertainty is available, where θ ∈ Θ and is distributed according to the joint probability density function (PDF), f (θ). Stochastic planning algorithms can use the information provided in f (θ) to create plans that explicitly account for the variability and coupling of the uncertain
(a) Proposed Schedule (b) Actual Schedule
Figure 2-3: Illustration of task coupling given uncertain task durations. Figure (a) shows the agent’s proposed schedule. Figure (b) shows the agent’s actual achieved task durations, where the longer-than-expected duration of the first task impacted the arrival times for the second and third tasks as well. In time-critical missions, this would result in lower-than- expected task scores for all three impacted tasks.
for uncertainty in the planning formulation. Perhaps the most common approach is to maximize the expected mission performance [28], where the objective function from Eq. (2.6) becomes, max x,τ Eθ Na X i=1 Nt X j=1 cij(τij(pi(xi)), θ) xij (2.7)
Note that optimizing Eq. (2.7) is not the same as deterministically planning using the
mean values of uncertain planning parameters ¯θ = Eθ{θ}, which can often lead to poor
planning performance since the problem formulation fails to capture the nontrivial coupling of uncertainty in scores, dynamics and constraints. This is especially problematic when scores are coupled, and can lead to biased predictions that drastically misrepresent the actual expected performance.
While optimizing Eq. (2.7) provides a plan that maximizes the expected performance of the system, an actual single run execution of this best expected plan is still subject to the uncertainty in the environment, and may result in a relatively poor plan (worse than expected) with some non-zero probability. If the current mission tolerance to failure is very low, a more conservative planning objective involves maximizing the worst-case scenario (sometimes referred to as robust in the literature [33]),
max x,τ minθ Na X i=1 Nt X j=1 cij(τij(pi(xi)), θ) xij (2.8)
Optimizing Eq. (2.8) guarantees that the plan execution will result in a score no worse than that predicted by the algorithm, however, in general, execution of the robust plan typically leads to scores higher than the worst case, and for missions of general interest, maximizing this worst-case lower bound is usually far too conservative.
Several robust optimization and stochastic optimization methods, that find the best so- lution within some predefined risk threshold, have been developed to mitigate this issue of conservatism [33, 157]. One such approach involves optimizing a risk-adjusted expected per- formance, where a risk function R(cij) biases the original cost function cijtowards more con-
servative solutions to account for the acceptable level of risk. Another approach is to bound
the domain of the uncertainty set θ to be within certain ranges, θ ∈ [Θmin, Θmax] ⊂ Θ
(e.g. ellipsoids for specified confidence intervals [24]), or to take on a set of discrete repre- sentative values θ ∈ [θ1, . . . , θk] ⊂ Θ (e.g. by constructing representative uncertainty sets
[32, 56]), thus limiting the support of the uncertain parameters. Classical robust convex op- timization techniques can then be used to solve the resulting approximate problem [25, 31]. Although these approaches provide methods to control the conservatism or risk associated with the planning solution, there are a few issues which makes their practical implementa-
tion difficult. Firstly, it is not usually obvious how to design the risk functions R(cij) or
the bounded uncertainty sets for θ, and the selection of these is typically problem specific, time consuming and ad-hoc. A more serious issue, however, is that when a global risk threshold is available, the metric of interest is the cumulative risk of the total solution, not the individual parameter or task risks, and it is difficult to quantify how these individual parameter bounds will affect the risk of the global solution. Nonlinearities in the cost func- tions, complex variable coupling and interdependencies, and discrete optimization effects, often affect the solution in unpredictable ways, and it is therefore hard to ensure that the total mission outcome is within the desired risk threshold by simply bounding the support of the random variables individually.
An alternative approach that guarantees that the global mission performance will be within a certain risk threshold is the chance-constrained formulation [45, 66, 157],
max x,τ y (2.9) s.t. Pθ Na X i=1 Nt X j=1 cij(τij(pi(xi)), θ) xij > y ≥ 1 −
The goal of Eq. (2.9) is to maximize the worst-case score within the allowable risk threshold specified by (can be interpreted as guaranteeing that solution will be at least as good as y with probability greater than or equal to 1 − ). When = 0, the score is guaranteed to be at least y with probability one (absolute worst-case), and the chance-constrained formulation reduces to the robust formulation of Eq. (2.8). When = 0.5, the chance- constrained formulation is equivalent to optimizing the median performance of the system (similar to optimizing the mean for symmetric distributions). Thus setting the risk value appropriately provides control over the conservatism of the solution within a consistent framework. The main drawback of the chance-constrained formulation is that it makes the optimization difficult to solve (analytically and computationally), especially given the extensive coupling between agents and tasks (double sum over distributions). Previous work has mainly considered linear or quadratic optimization with continuous variables [45, 66, 157], where, under special circumstances, optimal solutions and bounds can be found analytically. However, the task allocation formulation presented in Eq. (2.6) is a mixed- integer program, and these techniques cannot be easily extended to discrete optimization, especially given nonlinear and heterogeneous score functions with coupled distributions. Furthermore, these solution strategies are centralized and cannot be trivially extended to distributed environments. This thesis addresses these issues by proposing robust distributed planning strategies that can incorporate the stochastic metrics described in this section to improve the performance of multi-agent networked teams operating in uncertain and dynamic environments.