Revisiting Classical Planning

(1)

Introduction to Automated Planning

Revisiting Classical Planning

(2)

PART I

Introduction to Automated Planning

(3)

4. A drawing or diagram made to scale showing the structure or arrangement of something.

5. In perspective rendering, one of several imaginary planes

perpendicular to the line of vision between the viewer and the object being depicted.

6. A program or policy stipulating a service or benefit: a pension plan.

Synonyms: blueprint, design, project, scheme, strategy

plan n.

1. A scheme, program, or method worked out beforehand for the accomplishment of an objective: a plan of attack.

2. A proposed or tentative project or course of action: had no plans for the evening.

3. A systematic arrangement of elements or important parts; a configuration or outline: a seating plan; the plan of a story.

(4)

plan n.

1. A scheme, program, or

method worked out beforehand for the

accomplishment of an objective:

a plan of attack.

(5)

Generating Plans of Action

z Origin of Automated planning: computer programs to aid human planners

Project management (consumer software)

Automatic schedule generation

» various OR and AI techniques

z For some problems, we would like generate plans (or pieces of plans) automatically

Much more difficult

Automated-planning research is starting to pay off

z Here are some examples …

(6)

z Autonomous planning, scheduling, control

NASA: Jet Propulsion Lab and Ames Research Center

z Remote Agent

Experiment (RAX)

Deep Space 1

z Mars Exploration Rover (MER)

Space Exploration

(7)

z Sheet-metal bending machines - Amada Corporation

Software to plan the sequence of bends

[Gupta and Bourne, J. Manufacturing Sci. and Engr., 1999]

Manufacturing

(8)

z Bridge Baron - Great Game Products

1997 world champion of computer bridge

[Smith, Nau, and Throop, AI Magazine, 1998]

2004: 2nd place

(North—

…

♠Q)

…

PlayCard(P₃; S, R₃)

PlayCard(P₂; S, R₂) PlayCard(P₄; S, R₄)

FinesseFour(P₄; S) PlayCard(P₁; S, R₁)

StandardFinesseTwo(P₂; S) LeadLow(P₁; S)

PlayCard(P₄; S, R₄’) StandardFinesseThree(P₃; S)

EasyFinesse(P₂; S) BustedFinesse(P₂; S) FinesseTwo(P₂; S)

StandardFinesse(P₂; S) Finesse(P₁; S)

Us:East declarer, West dummy

Opponents:defenders, South & North Contract:East – 3NT

On lead:West at trick 3 East:♠KJ74 West: ♠A2 Out: ♠QT98653

(North— 3) West— ♠2

Games

(9)

Conceptual Model 1. Environment

State transition system Σ = (S,A,E,γ)

S = {states}

A = {actions}

E = {exogenous events}

γ = state-transition function System Σ

(10)

Σ = (S,A,E,γ)

z S = {states}

z A = {actions}

z E = {exogenous events}

z State-transition function γ: S x (A ∪ E) → 2^S

S = {s₀, …, s₅}

A = {move1, move2, put, take, load, unload}

E = {}

γ: see the arrows

State Transition System

take

put

move1 put

take

move1 move1

move2

unload load

move2

location 1 location 2

s₀

s₁

s₄

s₅

location 1 location 2 location 1 location 2

s₃

s₂

The Dock Worker Robots (DWR) domain

(11)

Observation function h: S → O

s₃

Given observation o in O, produces action a in A

Conceptual Model 2. Controller

Controller

(12)

Omit unless

planning is online

Planning problem Planning problemPlanning problem

Conceptual Model 3. Planner’s Input

Planner

(13)

Planning Problem

take

put

move1 put

take

move1 move1

move2

unload load

move2

s₀

s₁

s₄

s₅

s₃

s₂

Description of Σ

Initial state or set of states Initial state = s₀

Objective

Goal state, set of goal states, set of tasks,

“trajectory” of states, objective function, …

Goal state = s₅

(14)

Instructions to the controller

Conceptual Model 4. Planner’s Output

Planner

(15)

Plans

take

put

move1 put

take

move1 move1

move2

unload load

move2

s₀

s₁

s₄

s₅

s₃

s₂

Classical plan: a sequence of actions

〈take, move1, load, move2〉

Policy: partial function from S into A

{(s₀, take), (s₁, move1), (s₃, load), (s₄, move2)}

take

move1

load

move2

(16)

z A0: Finite system:

finitely many states, actions, events

z A1: Fully observable:

the controller knows Σ’s current state

z A2: Deterministic:

each action has only one outcome

z A3: Static (no exogenous events):

no changes but the controller’s actions

z A4: Attainment goals:

a set of goal states S_g(no states to be avoided, etc.)

z A5: Sequential plans:

a plan is a linearly ordered sequence of actions (a₁, a₂, … a_n)

z A6: Implicit time:

no time durations; linear sequence of instantaneous states

z A7: Off-line planning:

planner doesn’t know the execution status

Restrictive Assumptions

(17)

z Classical planning requires all eight restrictive assumptions

Offline generation of action sequences for a deterministic, static, finite system, with complete knowledge, attainment goals, and implicit time

z Reduces to the following problem:

Given (Σ, s₀, S_g)

Find a sequence of actions (a₁, a₂, … a_n) that produces a sequence of state transitions (s₁, s₂, …, s_n)

such that s_n is in S_g.

z This is just path-searching in a graph

Nodes = states

Edges = actions

z Is this trivial?

Classical Planning

(18)

Classical Planning

z Generalize the earlier example:

Five locations, three robot carts, 100 containers, three piles

» Then there are 10²⁷⁷ states

z Number of particles in the universe is only about 10⁸⁷

The example is more than 10¹⁹⁰ times as large!

z Automated-planning research has been heavily dominated by classical planning

Dozens (hundreds?) of different algorithms

We will only describe the simplest (state-space planning)

z In this course we will focus in planning with uncertainty (non- determinism)

s₁

take put

move1 move2

(19)

z Generalization of the earlier example

A harbour with several locations

» e.g., docks, docked ships, storage areas, parking areas

Containers

» going to/from ships

Robot carts

» can move containers

Cranes

» can load and unload containers

A running example: Dock Worker Robots

(20)

A running example: Dock Worker Robots

z Locations: l1, l2, …

z Containers: c1, c2, …

can be stacked in piles, loaded onto robots, or held by cranes

z Piles: p1, p2, …

fixed areas where containers are stacked

pallet at the bottom of each pile

z Robot carts: r1, r2, …

can move to adjacent locations

carry at most one container

z Cranes: k1, k2, …

each belongs to a single location

move containers between piles and robots

if there is a pile at a location, there must also be a crane there

(21)

A running example: Dock Worker Robots

z Fixed relations: same in all states

adjacent(l,l’) attached(p,l) belong(k,l)

z Dynamic relations: differ from one state to another occupied(l) at(r,l)

loaded(r,c) unloaded(r) holding(k,c) empty(k) in(c,p) on(c,c’) top(c,p) top(pallet,p)

z Actions:

take(c,k,p) put(c,k,p)

load(r,c,k) unload(r) move(r,l,l’)

(22)

PART II

Representations for Classical Planning

(23)

Representations: Motivation

z In most problems, far too many states to try to represent all of them explicitly as s₀, s₁, s₂, …

z Represent each state as a set of features

e.g.,

» a vector of values for a set of variables

» a set of ground atoms in some first-order language L

z Define a set of operators that can be used to compute state- transitions

z Don’t give all of the states explicitly

Just give the initial state

Use the operators to generate the other states as needed

(24)

Outline

z Representation schemes

Classical representation

Set-theoretic representation

State-variable representation

Examples: DWR and the Blocks World

Comparisons

(25)

Classical Representation

z Start with a function-free first-order language

Finitely many predicate symbols and constant symbols, but no function symbols

z Example: the DWR domain

Locations: l1, l2, …

Containers: c1, c2, …

Piles: p1, p2, …

Robot carts: r1, r2, …

Cranes: k1, k2, …

(26)

Classical Representation

z Atom: predicate symbol and arguments

Use these to represent both fixed and dynamic relations adjacent(l,l’) attached(p,l) belong(k,l)

occupied(l) at(r,l)

loaded(r,c) unloaded(r) holding(k,c) empty(k)

in(c,p) on(c,c’)

top(c,p) top(pallet,p)

z Ground expression: contains no variable symbols - e.g., in(c1,p3)

z Unground expression: at least one variable symbol - e.g., in(c1,x)

z Substitution: θ = {x₁← v₁, x₂← v₂, …, x_n← v_n}

Each x_i is a variable symbol; each v_i is a term

z Instance of e: result of applying a substitution θ to e

Replace variables of e simultaneously, not sequentially

(27)

States

z State: a set s of ground atoms

The atoms represent the things that are true in one of Σ’s states

Only finitely many ground atoms, so only finitely many possible states

(28)

Operators

z Operator: a triple o=(name(o), precond(o), effects(o))

name(o) is a syntactic expression of the form n(x₁,…,x_k)

» n: operator symbol - must be unique for each operator

» x₁,…,x_k: variable symbols (parameters)

• must include every variable symbol in o

precond(o): preconditions

» literals that must be true in order to use the operator

effects(o): effects

» literals the operator will make true

(29)

Actions

z Action: ground instance (via substitution) of an operator

(30)

Notation

z Let S be a set of literals. Then

S⁺ = {atoms that appear positively in S}

S^– = {atoms that appear negatively in S}

z More specifically, let a be an operator or action. Then

precond⁺(a) = {atoms that appear positively in a’s preconditions}

precond^–(a) = {atoms that appear negatively in a’s preconditions}

effects⁺(a) = {atoms that appear positively in a’s effects}

effects^–(a) = {atoms that appear negatively in a’s effects}

effects⁺(take(k,l,c,d,p) = {holding(k,c), top(d,p)}

effects^–(take(k,l,c,d,p) = {empty(k), in(c,p), top(c,p), on(c,d)}

(31)

Applicability

z An action a is applicable to a state s if s satisfies precond(a),

i.e., if precond⁺(a) ⊆ s and precond^–(a) ∩ s = ∅

z Here are an action and a state that it’s applicable to:

(32)

Result of Performing an Action

z If a is applicable to s, the result of performing it is γ(s,a) = (s – effects^–(a)) ∪ effects⁺(a)

Delete the negative effects, and add the positive ones

(33)

z Planning domain:

language plus operators

Corresponds to a

set of state-transition systems

Example:

operators for the DWR domain

(34)

Planning Problems

z Given a planning domain (language L, operators O)

Statement of a planning problem: a triple P=(O,s₀,g)

» O is the collection of operators

» s₀ is a state (the initial state)

» g is a set of literals (the goal formula)

The actual planning problem: P = (Σ,s₀,S_g)

» s₀ and S_g are as above

» Σ = (S,A,γ) is a state-transition system

» S = {all sets of ground atoms in L}

» A = {all ground instances of operators in O}

» γ = the state-transition function determined by the operators

z I’ll often say “planning problem” when I mean the statement of the problem

(35)

Plans and Solutions

z Plan: any sequence of actions σ = 〈a₁, a₂, …, a_n〉 such that each a_i is a ground instance of an operator in O

z The plan is a solution for P=(O,s₀,g) if it is executable and achieves g

i.e., if there are states s₀, s₁, …, s_n such that

» γ (s₀,a₁) = s₁

» γ (s₁,a₂) = s₂

» …

» γ (s_n–1,a_n) = s_n

» s_n satisfies g

(36)

Example

z Let P₁ = (O, s₁, g₁), where

O is the set of operators given earlier

g₁={loaded(r1,c3), at(r1,loc2)}

(37)

Example (continued)

z Here are three solutions for P₁:

〈take(crane1,loc1,c3,c1,p1), move(r1,loc2,loc1), move(r1,loc1,loc2), move(r1,loc2,loc1), load(crane1,loc1,c3,r1), move(r1,loc1,loc2)〉

〈take(crane1,loc1,c3,c1,p1), move(r1,loc2,loc1), load(crane1,loc1,c3,r1), move(r1,loc1,loc2)〉

〈move(r1,loc2,loc1), take(crane1,loc1,c3,c1,p1), load(crane1,loc1,c3,r1), move(r1,loc1,loc2)〉

z Each of them produces the state shown here:

(38)

Example (continued)

z The first is redundant: can remove actions and still have a solution

〈take(crane1,loc1,c3,c1,p1), move(r1,loc2,loc1), move(r1,loc1,loc2), move(r1,loc2,loc1), load(crane1,loc1,c3,r1), move(r1,loc1,loc2)〉

〈take(crane1,loc1,c3,c1,p1), move(r1,loc2,loc1), load(crane1,loc1,c3,r1), move(r1,loc1,loc2)〉

〈move(r1,loc2,loc1), take(crane1,loc1,c3,c1,p1), load(crane1,loc1,c3,r1), move(r1,loc1,loc2)〉

z The 2nd and 3rd are irredundant and shortest

(39)

Set-Theoretic Representation

z Like classical representation, but restricted to propositional logic

z States:

Instead of a collection of ground atoms …

{on(c1,pallet), on(c1,r1), on(c1,c2), …, at(r1,l1), at(r1,l2), …}

… use a collection of propositions (boolean variables):

{on-c1-pallet, on-c1-r1, on-c1-c2, …, at-r1-l1, at-r1-l2, …}

(40)

z Instead of operators like this one,

take all of the operator instances,

e.g., this one,

and rewrite ground atoms as propositions

take-crane1-loc1-c3-c1-p1

precond: belong-crane1-loc1, attached-p1-loc1, empty-crane1, top-c3-p1, on-c3-c1

delete: empty-crane1, in-c3-p1, top-c3-p1, on-c3-p1 add: holding-crane1-c3, top-c1-p1

(41)

Comparison

z A set-theoretic representation is equivalent to a classical representation in which all of the atoms are ground

z Exponential blowup

If a classical operator contains n atoms and each atom has arity k,

then it corresponds to c^nk actions where c = |{constant symbols}|

(42)

z Use ground atoms for properties that do not change, e.g., adjacent(loc1,loc2) z For properties that can change, assign values to state variables

Like fields in a record structure

z Classical and state-variable representations take similar amounts of space

Each can be translated into the other in low-order polynomial time

State-Variable Representation

{top(p1)=c3, cpos(c3)=c1, cpos(c1)=pallet, holding(crane1)=nil, rloc(r1)=loc2,

loaded(r1)=nil, …}

(43)

Example: The Blocks World

z Infinitely wide table, finite number of children’s blocks

z Ignore where a block is located on the table

z A block can sit on the table or on another block

z Want to move blocks from one configuration to another

e.g.,

initial state goal

z Can be expressed as a special case of DWR

But the usual formulation is simpler

z I’ll give classical, set-theoretic, and state-variable formulations

For the case where there are five blocks

c a c b

a b e

d

(44)

Classical Representation: Symbols

z Constant symbols:

The blocks: a, b, c, d, e

z Predicates:

ontable(x) - block x is on the table

on(x,y) - block x is on block y

clear(x) - block x has nothing on it

holding(x) - the robot hand is holding block x

handempty - the robot hand isn’t holding anything

c

a b e

d

(45)

unstack(x,y)

Precond: on(x,y), clear(x), handempty Effects: ~on(x,y), ~clear(x), ~handempty,

holding(x), clear(y) stack(x,y)

Precond: holding(x), clear(y) Effects: ~holding(x), ~clear(y),

on(x,y), clear(x), handempty pickup(x)

Precond: ontable(x), clear(x), handempty Effects: ~ontable(x), ~clear(x),

~handempty, holding(x) putdown(x)

Precond: holding(x)

Effects: ~holding(x), ontable(x), clear(x), handempty

Classical Operators

^c

a b

a b c

c

a b

c

a b

c

a b

(46)

z For five blocks, there are 36 propositions

z Here are 5 of them:

ontable-a - block a is on the table on-c-a - block c is on block a

clear-c - block c has nothing on it

holding-d - the robot hand is holding block d

handempty - the robot hand isn’t holding anything

c

a b

d

e

Set-Theoretic Representation: Symbols

(47)

Set-Theoretic Actions

Fifty different actions Here are four of them:

unstack-c-a

Pre: on-c,a, clear-c, handempty Del: on-c,a, clear-c, handempty Add: holding-c, clear-a

stack-c-a

Pre: holding-c, clear-a Del: holding-c, ~clear-a

Add: on-c-a, clear-c, handempty pickup-c

Pre: ontable-c, clear-c, handempty Del: ontable-c, clear-c, handempty Add: holding-c

putdown-c

Pre: holding-c Del: holding-c

Add: ontable-c, clear-c, handempty

c

a b

a b c

c

a b

c

a b

c

a b

(48)

z Constant symbols:

a, b, c, d, e of type block 0, 1, table, nil of type other

z State variables:

pos(x) = y if block x is on block y pos(x) = table if block x is on the table pos(x) = nil if block x is being held

clear(x) = 1 if block x has nothing on it

clear(x) = 0 if block x is being held or has another block on it holding = x if the robot hand is holding block x

holding = nil if the robot hand is holding nothing

c

a b e

d

State-Variable Representation: Symbols

(49)

State-Variable Operators

unstack(x : block, y : block)

Precond: pos(x)=y, clear(y)=0, clear(x)=1, holding=nil Effects: pos(x)=nil, clear(x)=0, holding=x, clear(y)=1

stack(x : block, y : block)

Precond: holding=x, clear(x)=0, clear(y)=1

Effects: holding=nil, clear(y)=0, pos(x)=y, clear(x)=1

pickup(x : block)

Precond: pos(x)=table, clear(x)=1, holding=nil Effects: pos(x)=nil, clear(x)=0, holding=x

putdown(x : block) Precond: holding=x

Effects: holding=nil, pos(x)=table, clear(x)=1

c

a b

a b c

c

a b

c

a b

c

a b

(50)

Expressive Power

z Any problem that can be represented in one representation can also be represented in the other two

z Can convert in linear time and space, except when converting to set-theoretic (where we get an exponential blowup)

Classical representation

State-variable representation Set-theoretic

representation

trivial

P(x₁,…,x_n) becomes f_P(x₁,…,x_n)=1

write all of the ground

instances

f(x₁,…,x_n)=y becomes P_f(x₁,…,x_n,y)

(51)

Comparison

z Classical representation

The most popular for classical planning, partly for historical reasons

z Set-theoretic representation

Can take much more space than classical representation

Useful in algorithms that manipulate ground atoms directly

» e.g., planning graphs and satisfiability

Useful for certain kinds of theoretical studies

z State-variable representation

Equivalent to classical representation

Less natural for logicians, more natural for engineers

Useful in non-classical planning problems as a way to handle numbers, functions, time

(52)

PART III

State-Space Planning

(53)

Outline

z State-space planning

Forward search

Backward search

STRIPS

(54)

Forward Search

take c3

move r1

take c2 …

…

(55)

Properties

z Forward-search is sound

for any plan returned by any of its nondeterministic traces, this plan is guaranteed to be a solution

z Forward-search also is complete

if a solution exists then at least one of Forward-search’s nondeterministic traces will return a solution.

(56)

Deterministic Implementations

z Some deterministic implementations of forward search:

breadth-first search

depth-first search

best-first search (e.g., A*)

greedy search

z Breadth-first and best-first search are sound and complete

But they usually aren’t practical because they require too much memory

Memory requirement is exponential in the length of the solution

z In practice, more likely to use depth-first search or greedy search

Worst-case memory requirement is linear in the length of the solution

In general, sound but not complete

» There are infinite branches

» Thus, can make depth-first search complete by doing loop-checking s₀

s₁

s₂

s₃ a₁

a₂

a₃

s₄

s₅ s_g

a₄

a₅ …

(57)

Branching Factor of Forward Search

z Forward search can have a very large branching factor

E.g., many applicable actions that don’t progress toward goal

z Why this is bad:

Deterministic implementations can waste time trying lots of irrelevant actions

z Need a good heuristic function and/or pruning procedure

a₃ a₁ a₂ a₁ a₂ a₃ … a₅₀

initial state goal

(58)

Backward Search

z For forward search, we started at the initial state and computed state transitions

new state = γ(s,a)

z For backward search, we start at the goal and compute inverse state transitions

new set of subgoals = γ^–1(g,a)

z To define γ^-1(g,a), must first define relevance:

An action a is relevant for a goal g if

» a makes at least one of g’s literals true

• g ∩ effects(a) ≠ ∅

» a does not make any of g’s literals false

• g⁺ ∩ effects^–(a) = ∅ and g^– ∩ effects⁺(a) = ∅

(59)

Inverse State Transitions

z If a is relevant for g, then

γ^–1(g,a) = (g – effects(a)) ∪ precond(a)

z Otherwise γ^–1(g,a) is undefined

z Example: suppose that

g = {on(b1,b2), on(b2,b3)}

a = stack(b1,b2)

z What is γ^–1(g,a)?

(60)

g₀ g₁

g₂

g₃

a₁

a₂

a₃ g₄

g₅ s₀

a₄

a₅

(61)

Efficiency of Backward Search

z Backward search can also have a very large branching factor

z As before, deterministic implementations can waste lots of time trying all of them

a₁ a₁ a₂ a₃ … a₅₀

initial state goal

(62)

STRIPS

z π ← the empty plan

z do a modified backward search from g

instead of γ^-1(s,a), each new set of subgoals is just precond(a)

whenever you find an action that’s executable in the current state, then go forward on the current search path as far as possible, executing actions and appending them to π

repeat until all goals are satisfied

g g₁

g₂

g₃

a₁

a₂ a₃ g₄

g₅ g₃

a₄

a₅ current search path a₆

π = 〈a₆, a₄〉

s = γ(γ(s₀,a₆),a₄)

g₆

a₃

satisfied in s₀