Goal Regression Planning, Constraint Automata and Causal Graphs

Transcription

1 Goal Regression Planning, Constraint Automata and Causal Graphs Contributions: Dan Weld Seung Chung Erez Karpas David Wang Prof Brian Williams MIT CSAIL courtesy of JPL New Text J/6.834J Cognitive Robotics Leap Day - February 29 th, 2016 This image is in the public domain. 1

2 After lecture you will know how to Generate plans by working backwards from goals. Generate least commitment, partial order plans. Encode actions with indirect effects as concurrent automata with constraints (CCA). Analyze action dependence using causal graphs. Generate plans with out search, for CCA that have tree structured causal graphs. Use causal graph planning as a heuristic to HFS. 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 2

3 Assignments Today: D. Weld, An introduction to least commitment planning, AI Magazine 15(4):27-61, B. Williams and P. Nayak, "A reactive planner for a model-based executive, IJCAI, Next: D. Wang and B. Williams, tburton: A Divide and Conquer Temporal Planner, AAAI, Homework: PSet #3 PDDL Modeling, out today, due Wed, March 9 th. Advanced Lecture Pset #1, out today, due Fri, March 4 th. 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 3

4 Outline Review: programs on state Planning as goal regression (SNLP) Goal regression planning with causal graphs (Burton) Appendix: HFS planning with the causal graph heuristic (Fast Downward) 4

5 A single cognitive system language and executive. Enterprise Programs that: Uhura User Goals & models in RMPL Collaboratively resolves goal failures Kirk Make choices Sketches mission and assigns tasks Burton Pike Plans actions Adjust timing Control Commands Coordinates and monitors tasks Sulu Bones Plans paths Diagnoses likely failures Monitor conditions 2/29/2016 source unknown. All rights reserved. This content is excluded from our Creative Commons license. For more information, see Goal Regression and Causal Graph Planning 5

6 In a State-Space Program, Activity Planning Maps Desired States to Actions no-fly zone lake2 fire2 no-fly zone fire1 base2 no-fly zone lake1 uav1 base1 class Main{ UAV uav1; Lake lake1; Lake lake2; Fire fire1; Fire fire2;... Main (){ uav1 = new UAV(); uav1.location= base_1_location; uav1.flying = no; uav1.loaded = no; lake1 = new Lake(); lake1.location = lake_1_location; lake2 = new Lake(); lake2.location = lake_2_location; class UAV { Roadmap location; Boolean flying; Boolean loaded; primitive method takeoff() flying == no => flying == yes; primitive method land() flying == yes => flying == no; primitive method load_water(lake lakespot) ((flying == yes) && (loaded == no) && (lakespot.location == location)) => loaded == yes; method run() { sequence{ }}} (fire1 == out); (fire2 == out); (uav1.flying == no && uav1.location == base_1_location); } fire1 = new Fire(); fire1.location = fire_1_location; fire1 = high; fire2 = new Fire(); fire2.location = fire_2_location; fire2 = high; } primitive method drop_water_high_altiture(fire firespot) ((flying == yes) && (loaded == yes) && (firespot.location == location) && (firespot == high)) => ((loaded == no) && (firespot == medium)); primitive method drop_water_low_altiture(fire firespot) ((flying == yes) && (loaded == yes) && (firespot.location == location) && (firespot == medium)) => ((loaded == no) && (firespot == out)); #MOTION_PRIMITIVES(location, fly, flying==yes) 6

7 Planning maps desired states to actions no-fly zone fire2 no-fly zone base2 no-fly zone lake1 lake2 fire1 uav1 base1 Roadmap Path Planning Classical Action Planning 7

8 A single cognitive system language and executive. Enterprise State Space Programs that: Uhura User Goals & models in RMPL Collaboratively resolves goal failures Kirk Sketches mission and assigns tasks Burton Plan to achieve discrete states. Plans actions Control Commands Pike Coordinates and monitors tasks Sulu Plan to achieve continuous states. Bones Plans paths Diagnoses likely failures Monitor continuous states. source unknown. All rights reserved. This content is excluded from our Creative Commons license. For more information, see 2/29/2016 Goal Regression and Causal Graph Planning 8

9 Last Week: Classic Activity Planning Representation (STRIPS/PDDL) Action Model: Objects (things): Predicates (used to create true or false statements about the world, facts ): (On A, B ) (Clear A ) Actions (used to change truth of predicates): (Clear ) A (Clear ) B Stack (Clear B ) (On A, B ) Initial: Goal: preconditions (Clear ) (Clear ) (Clear ) (On, ) (On, ) effects 9

10 Last Week: Forward Search action3 s 2 0 action1 s 1 0 Initial State (s o ) action2 s 1 0 action4 goal action2 s 2 1 action3 s 1 0 action3 s 2 2 Time 10

11 This Week: Goal-Regression Search action3 S t 0 action1 Initial State (s o ) action2 action2 S t 1 action4 goal action3 S t 2 action2 Time 11

12 This Week: Planning to Control Complex Devices no-fly zone fire2 no-fly zone base2 no-fly zone lake1 lake2 fire1 uav1 base1 Roadmap Path Planning Classical Action Planning Thrust Goals Delta_V(direction=b, magnitude=200) Power This image is in the public domain. Attitude Point(a) Turn(a,b) Point(b) Turn(b,a) Device Reconfiguration & Repair Engine Off Warm Up Thrust (b, 200) O ff Time-line Planning 12

13 This Week: Automata Representation Action Model: Concurrent Constraint Automata (CCA) (like a state-machine): Initial: Goal: Note: This is a very simple example, there are usually many automata, and guards on the transitions. We will use CCA to support indirect effects, concurrency and metric time. Algorithms exist to map between the two representations. 13

14 Outline Review: programs on state Planning as goal regression (SNLP/UCPOP) Partial Order Planning Overview Partial Order Planning Problem Partial Order Plan Generation Goal regression planning with causal graphs (Burton) Appendix: HFS planning with the causal graph heuristic (Fast Downward) 14

15 Classical Planning Problem Propositions: P i Initial Conditions: P 1 P 2 P 3 P 4 Operators: pre 1 pre 2 pre 3 Op eff 1 eff 2 Goals: Goal 1 Goal 2 Goal 3 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 15

16 Simple Spacecraft Problem Observation-1 target instruments Observation-2 Observation-3 Observation-4 pointing calibrated This image is in the public domain. Propositions: Operators: Target Pointed To, Camera Calibrated?, Has Image? Calibrate, Turn to Y, and Take Image. 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 16

17 Example Init Actions Goal p C p C C c I A p y p x T y p x c Im I x p x Propositions: Operators: Target Pointed To, Camera Calibrated?, Has Image? Calibrate, Turn to Y, and Take Image. 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 17

18 Actions in the Planning Domain Description Language (PDDL) (:action TakeImage :parameters (?target,?instr) :precondition (and (Status?instr Calibrated) (Pointing?target)) :effect (Image?target)) By convention, parameters start with?, as in?var. (:action Calibrate :parameters (?instrument) :precondition (and (Status?instr On) (Calibration-Target?target), (Pointing?target) :effect (and (not (Status?inst On)) (Status?instr Calibrated))) (:action Turn :parameters (?target) :precondition (and (Pointing?direction)?direction?target :effect (and (not (Pointing?direction) (Pointing?target))) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 18

19 Partial Order Plan p C C c S p A Im I A F p C T A p C 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 19

20 Planning from Goals: Partial Order Causal Link Planning (SNLP, UCPOP) I A F 1. Select an open condition; 2. Choose an op that can achieve it: Link to an existing instance or Add a new instance; 3. Resolve threats. p C c I Im A F p A C c p A Im I A F S p C p C C c p A T A Im p C Alternatives: Forward Search, CSP I A S F S p C C c p A p A p C T A 2/29/ J / 6.834J Goal Regression & Causal Graph Planning p C Im I A p C C c Im I A F F 20

21 Planning as Goal Regression Partial Planning Overview Partial Order Planning Problem Problem Encoding Partial Order Plans Plan Correctness Partial Order Plan Generation 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 21

22 Example Problem Initial State: At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) Goal: Have(Milk) At(Home) Have(Ban.) Have(Drill) Operators: At(HWS) Go(SM) At(SM) At(SM), Sells(SM,Ban.) Buy(Ban.) Have(Ban) At(Home) Go(HWS) At(HWS) At(SM) Go(Home) At(Home) At(HWS) Sells(HWS,Drill) Buy(Drill) Have(Drill) At(SM), Sells(SM,Milk) Buy(Milk) Have(Milk) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 22

23 Initial and Goal States Encoded as Operators Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) Why encode as operators? Don t need to introduce (partial) states as separate objects. Keeps theory minimal. Have(Milk) At(Home) Have(Ban.) Have(Drill) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 23

24 Partial Order Plan <Actions,Orderings,Links> Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(HWS) Sells(HWS,Drill) Buy(Drill) At(HWS) Go(SM) At(SM), Sells(SM,Milk) Buy(Milk) At(SM), Sells(SM,Ban.) Buy(Ban.) At(SM) Go(Home) Have(Milk) At(Home) Have(Ban.) Have(Drill) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 24

25 Partial Order Plan <Actions,Orderings,Links> Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(HWS) Sells(HWS,Drill) Buy(Drill) At(HWS) Go(SM) At(SM), Sells(SM,Milk) Buy(Milk) At(SM), Sells(SM,Ban.) Buy(Ban.) At(SM) Go(Home) Have(Milk) At(Home) Have(Ban.) Have(Drill) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 25

26 Partial Order Plan <Actions,Orderings,Links> Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(HWS) Sells(HWS,Drill) Buy(Drill) At(HWS) Go(SM) At(SM), Sells(SM,Milk) Buy(Milk) At(SM), Sells(SM,Ban.) Buy(Ban.) At(SM) Go(Home) Have(Milk) At(Home) Have(Ban.) Have(Drill) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 26

27 Partial Order Plan <Actions,Orderings,Links> Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(HWS) Sells(HWS,Drill) Buy(Drill) At(HWS) Go(SM) At(SM), Sells(SM,Milk) Buy(Milk) At(SM), Sells(SM,Ban.) Buy(Ban.) At(SM) Go(Home) Have(Milk) At(Home) Have(Ban.) Have(Drill) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 27

28 Planning as Goal Regression Partial Planning Overview Partial Order Planning Problem Problem Encoding Partial Order Plans Plan Correctness Partial Order Plan Generation 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 28

29 What Constitutes a Correct Plan? Complete Plan Achieves all Goals Achieves all preconditions No actions intervene to undo a needed precondition Consistent Plan There exists an execution sequence that is consistent with the ordering At(SM), Sells(SM,Milk) Buy(Milk) Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(HWS) Sells(HWS,Drill) Buy(Drill) At(HWS) Go(SM) At(SM) Go(Home) Finish At(SM), Sells(SM,Ban.) Buy(Ban.) Have(Milk) At(Home) Have(Ban.) Have(Drill) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 29

30 Why is an ordering needed? At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Buy(Milk) Have(Milk) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 30

31 Why is an ordering needed? Suppose the other order is allowed, what happens? At(HWS) Go(SM) At(SM), not At(HWS) Threatened Precondition At(SM) Buy(Milk) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 31

32 Why is an ordering needed? Link indicates protected time interval for the precondition. Suppose the other order is allowed, what happens? At(SM) Buy(Milk) At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 32

33 Why is an ordering needed? The ordering resolves the threat. At(SM) Buy(Milk) At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 33

34 Solution: A Complete and Consistent Plan Complete Plan IFF every precondition of every step is achieved. A step s precondition is achieved iff its the effect of some preceding step, no intervening step can undo it. Consistent Plan At(SM), Sells(SM,Milk) Buy(Milk) IFF there is no contradiction in the ordering constraint. i.e., never s i < s j and s j < s i, or the causal links + orderings are loop free. Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(HWS) Sells(HWS,Drill) Buy(Drill) At(HWS) Go(SM) At(SM) Go(Home) Finish At(SM), Sells(SM,Ban.) Buy(Ban.) Have(Milk) At(Home) Have(Ban.) Have(Drill) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 34

35 Planning as Goal Regression Partial Planning Overview Partial Order Planning Problem Partial Order Plan Generation Derivation from Completeness and Consistency Backward Chaining Threat Resolution The POP algorithm 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 35

36 Partial Order Planning Algorithm The algorithm falls out of Consistency and Completeness Completeness: Must achieve all preconditions Backward chain from goals to initial state, by inserting actions and causal links. Must avoid intervening actions that threaten After each action is inserted, find any action that threatens its effects, and impose ordering to resolve. Consistent: Ordering must be consistent After each causal link and ordering is inserted, check for loops. 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 36

37 Planning as Goal Regression Partial Planning Overview Partial Order Planning Problem Partial Order Plan Generation Derivation from Completeness and Consistency Backward Chaining Threat Resolution The POP algorithm 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 37

38 Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 38

39 Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(HWS) Sells(HWS,Drill) Buy(Drill) At(SM), Sells(SM,Milk) Buy(Milk) At(SM), Sells(SM,Ban.) Buy(Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 39

40 Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(HWS) Sells(HWS,Drill) At(SM), Sells(SM,Milk) At(SM), Sells(SM,Ban.) Buy(Drill) Buy(Milk) Buy(Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 40

41 Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(?x) Go(HWS) At(?x) Go(SM) At(HWS) Sells(HWS,Drill) At(SM), Sells(SM,Milk) At(SM), Sells(SM,Ban.) Buy(Drill) Buy(Milk) Buy(Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 41

42 Planning as Goal Regression Partial Planning Overview Partial Order Planning Problem Partial Order Plan Generation Derivation from Completeness and Consistency Backward Chaining Threat Resolution The POP algorithm 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 42

43 After adding a causal link/action, find threat with any existing action/link At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Buy(Milk) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 43

44 To remove threats At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Buy(Milk) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 44

45 To remove threats promote the threat or At(SM) Buy(Milk) At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Go(Home) At(Home), not At(SM) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 45

46 To remove threats promote the threat or demote the threat At(HWS) Go(SM) At(SM), not At(HWS) At(SM) Go(Home) At(Home), not At(SM) At(SM) Buy(Milk) But only allow demotion/promotion if schedulable consistent = loop free no action precedes initial state 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 46

47 Go(SM) threatens At(HWS) Promote Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(Home) Go(SM) At(HWS) Sells(HWS,Drill) At(SM), Sells(SM,Milk) At(SM), Sells(SM,Ban.) Buy(Drill) Buy(Milk) Buy(Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 47

48 Go(HWS) threatens At(Home) Try Promote Start Inconsistent! At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(Home) Go(SM) At(HWS) Sells(HWS,Drill) At(SM), Sells(SM,Milk) At(SM), Sells(SM,Ban.) Buy(Drill) Buy(Milk) Buy(Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 48

49 Use Go(HWS) to produce At(?x) Finish by Go(Home). Start At(Home) Sells(HWS,Drill) Sells(SM,Milk) Sells(SM,Ban.) At(Home) Go(HWS) At(?x) Go(SM) HWS At(HWS) Sells(HWS,Drill) At(SM), Sells(SM,Milk) At(SM), Sells(SM,Ban.) Buy(Drill) Buy(Milk) Buy(Ban.) Have(Drill) Have(Milk) Have(Ban.) at(home) Finish At(SM) Go(Home) 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 49

50 POP(<A,O,L>, agenda, actions) 1. Termination: If agenda is empty, return plan <A,O,L>. 2. Goal Selection: Select and remove open condition <p, a need > from agenda. 3. Action Selection: Choose new or existing action a add that can precede a need and whose effects include p. Link and order actions. 4. Update Agenda: If a add is new, add its preconditions to agenda. 5. Threat Detection: For every action a threat that might threaten some causal link from a produce to a consume, choose a consistent ordering: a) Demote: Add a threat < a produce b) Promote: Add a consume < a threat 6. Recurse: on modified plan and agenda Choose is nondeterministic 2/29/ J / 6.834J Goal Regression & Causal Graph Planning Select is deterministic 50

51 Lets Start with Goal-Regression Search action3 S t 0 action1 Initial State (s o ) action2 action2 S t 1 action4 goal Time action3 S t 2 action2 Why can Goal Regression be slow? Multiple actions can achieve goals. Many possible (sub-)goal orderings. Dead-ends can be discovered late. We try a real-world example next! 51

52 What assumptions are implied by the STRIPS representation? TakeImage (?target,?instr): Pre: Status(?instr, Calibrated), Pointing(?target) Eff: Image(?target) Calibrate (?instrument): Pre: Status(?instr, On), Calibration-Target(?target), Pointing(?target) Eff: Status(?inst, On), Status(?instr, Calibrated) Turn (?target): Pre: Pointing(?direction),?direction?target Eff: Pointing(?direction), Pointing(?target) STRIPS Assumptions: Atomic time, Agent is omniscient (no sensing necessary), Agent is sole cause of change, Actions have deterministic effects, and No indirect effects. One action at a time. No metric time. No goals over time. 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 52

53 Outline Review: programs on state Planning as goal regression (SNLP/UCPOP) Goal regression planning with causal graphs (Burton) Appendix: HFS planning with the causal graph heuristic (Fast Downward) 53

54 DS1 Revisited: Planning to Control Interacting Devices over Time Thrust Goals Delta_V(direction=b, magnitude=200) Power Time-line Planning (tburton) Attitude Point(a) Turn(a,b) Point(b) Turn(b,a) Engine Off Warm Up Thrust (b, 200) O ff This image is in the public domain. Reactive Device Reconfiguration and Repair (Burton) 54

55 Domain Independent Planning in the Real-World classical off-line Speed Heuristic-based Searches & Symbolic Search (not presented) real-time Burton (real-time planning) Expressivity tburton (planning with time and indirect effects) Both Burtons use Causal Graphs to analyze weak and tight couplings and to Decompose. 55

56 Planning with Indirect Effects: DS 1 Attitude Control System Example Flight Computer Data Commands PDE Propulsion Drive Electronics SRU Stellar Reference Unit PDU Power Distribution Unit z facing thrusters He N 2 H 4 x facing thrusters Bus Controller GDE Gimbal Drive Electronics PASM Power Activation & Switching Module DSEU Diagnostics Sensors Electronics Unit Dynamics 1553 bus ACE Attitude Control Electronics control mode xatt error yatt error zatt error 56

57 Why is Controlling an Engineered Device Easier than a Puzzle? Propulsion Drive Electronics Stellar Reference Unit Power Distribution Unit Gimbal Drive Electronics Power Activation & Switching Module Diagnostics Sensors Electronics Unit Attitude Control Electronics Flight Computer PDE He Data Commands SRU PDU z facing thrusters N 2 H 4 x facing thrusters Bus Controller GDE 3 5 PASM DSEU Dynamics 1553 bus ACE control mode xatt erro r yatt erro r zatt erro r 1. Actions are reversible. 2. Devices hold state. 3. Causes and effects form a tree. 1. Actions are reversible. 2. Devices hold state. 3. Causes and effects form tight cycles. 57

58 Outline Review: programs on state Planning as goal regression (SNLP/UCPOP) Goal regression planning with causal graphs (Burton) Constraint automata planning problem Causal graphs Using causal graphs to order goals Computing policies for selecting actions Appendix: Planning for cyclic causal graphs Appendix: HFS planning with the causal graph heuristic (Fast Downward) 58

59 Concurrent Constraint Automata Planning Problem flow in dcmd out = vcmd in dcmd in Variables and Domains: flow out State: driver in {on, off, resettable, failed}, valve in {open, closed, stuck-open, stuck-closed}. Control: dcmd in in {idle, on, off, reset, open, close} Dependent: flow in, flow out in {pos, neg, zero} dcmd out, vcmd in in Domain{dcmd in } Initial assignment: {driver = on, valve = closed} Goal assignment: {driver = off, valve = open} 59

60 Concurrent Constraint Automata Planning Problem (cont.) flow in dcmd out = vcmd in dcmd in Constraint automata (one per state variable): flow out driver dcmd in = dcmd out valve flowin in = flow out On dcmd in = reset Resettable Open Stuck Open dcmd in = on dcmd in = off dcmd in = off vcmd in = open vcmd in = close Off Failed Closed Stuck Closed Assume: transitions independently controlled, each location can idle. State constraints: {dcmd out = vcmd in } 60

61 Outline Review: programs on state Planning as goal regression (SNLP/UCPOP) Goal regression planning with causal graphs (Burton) Constraint automata planning problem Causal graphs Using causal graphs to order goals Computing policies for selecting actions Appendix: Planning for cyclic causal graphs Appendix: HFS planning with the causal graph heuristic (Fast Downward) 61

62 Observation: Engineered systems are largely loop free. Computer Bus Control Remote Terminal Remote Terminal Driver Driver Valve Valve 62

63 Causal Graph G of concurrent automata S: Vertices are control and state variables of automata. Edge from v i to v j if v j s transition is conditioned on v i. Driver On dcmd in = dcmd out dcmd in = reset Resettable dcmd in Driver Valve dcmd in = on dcmd in = off Off dcmd in = off Failed Valve Open flowin in = flow out Stuck Open dcmd in vcmd in = open vcmd in = close dcmd out = vcmd in Closed Stuck Closed 63

64 Outline Review: programs on state Planning as goal regression (SNLP/UCPOP) Goal regression planning with causal graphs (Burton) Constraint automata planning problem Causal graphs Using causal graphs to order goals Computing policies for selecting actions Appendix: Planning for cyclic causal graphs Appendix: HFS planning with the causal graph heuristic (Fast Downward) 64

65 Idea: use causal graph analysis to eliminate ALL forms of search (Burton) POP(<A,O,L>, agenda, actions): 1. Termination: If agenda is empty, return plan <A,O,L>. 2. Goal Selection: Select and remove open condition <p, a need > from agenda. 3. Action Selection: Choose new or existing action a add that can precede a need and whose effects include p. Link and order actions. 4. Update Agenda: If a add is new, add its preconditions to agenda. 5. Threat Detection: For every action a threat that might threaten some causal link from a produce to a consume, choose a consistent ordering: a) Demote: Add a threat < a produce b) Promote: Add a consume < a threat 6. Recurse: on modified plan and agenda Burton [Williams & Nayak, IJCAI 1997] 2/29/ J / 6.834J Goal Regression & Causal Graph Planning 65

66 Why do goal orderings matter? 1. An achieved goal can be clobbered by a subsequent goal. command Example: Current State: Goal State: driver = on, valve = closed. driver = off, valve = open. Achieving (driver = off), followed by (valve = open) clobbers (driver = off). Achieve valve goal before driver goal. Effect Cause 66

67 Goal Ordering for Causal Graph Planning Require: The CCA causal graph to be acyclic. Causal Graph Driver dcmd in Valve Idea: Achieve conjunctive goals upstream within the causal graph, from effects to causes (i.e., children to parents). 67

68 Property: Safe to achieve goals in an upstream order The only variables used to set some variable v 7 is its ancestors. Variable v 7 can be changed without affecting its descendants. Affected Unaffected Exploits: Each transitions independently controlled, each location can idle. Simple check: 1. Number the causal graph depth-first from leaves. Child has lower number than parents 2. Achieve goals in the order of increasing depth-first number. 68

69 Why do goal orderings matter? 2. Two goals can compete for the same variable associated with their sub-goals. data Switc h Latch 1 Latch 2 Example: Latch Data at Latches 1 and 2 If Latch1 and Latch2 goals achieved at same time, Latch1 and Latch2 compete for Switch position. 1 2 Solve one goal completely before the other (serially). 69

70 Property: Safe to achieve one goal before starting next sibling (serialization). Sibling goals v 7 and v 4 may both need shared ancestors. Not Shared Shared Unaffected Competition gone once sibling goal v 7 is satisfied. Not Shared Unaffected 70

71 Goal regression, CU causal graph planning example Driver 1 Driver 2 Valve 1 Valve 2 Burton s goal ordering rules: 1. Achieve effect goals before their causes, with control actiions last. 2. Achieve goals serially (like Hacker[Sussman PhD]). Valve 1 = open Valve 2 = open Driver 1 = off Driver 2 = off Open Open Turn-Off Turn-Off Driver 1 = on Driver 2 = on CU = on CU = on Turn-On CU = on CU = on CU = on CU = on Turn-On No Search! First action generated first! Turn-On Worst Case per action: Depth * Sub-goal branch factor. CU = off Average Cost per action: Sub-goal branch factor. 71

72 To select actions reactively, convert constraint automata to lookup policies Driver 2 Valve 1 dcmd out = vcmd in dcmd in = dcmd out inflow = outflow on dcmd in = reset resettable open stuck open cmd in = on dcmd in = off dcmd in = off vcmd in = open vcmd in = close off failed closed stuck closed 72

73 To select actions reactively, convert constraint automata to lookup policies Algorithm: Instance of APSP Driver 2 Valve 1 dcmd out = vcmd in dcmd in = dcmd out dcmd on in = reset resettable Goal Current On Off On idle cmd = off cmd in = on dcmd in = off dcmd in = off Off cmd = on idle off failed Resettable cmd = reset cmd = off 73

74 To select actions reactively, convert constraint automata to lookup policies Driver 2 Valve 1 dcmd out = vcmd in Goal inflow = outflow Current Open Open idle Closed driver = on cmd = close open stuck open Closed driver = on cmd = open idle vcmd in = open vcmd in = close Stuck fail fail closed stuck closed 74

75 Plan by passing sub-goals up causal graph Goal: Driver = off, Valve = closed Current: Driver = off, Valve = open Driver 2 Algorithm: see [williams and nayak, IJCAI97] Valve 1 Goal Current On Off Goal Current Open Closed On idle cmd = off Open idle driver = on cmd = close Off cmd = on idle Closed driver = on cmd = open idle Resettable cmd = reset cmd = off Stuck fail fail 75

76 Plan by passing sub-goals up causal graph Goal: Driver = off, Valve = closed Current: Driver = off, Valve = open Driver 2 Algorithm: see [williams and nayak, IJCAI97] Valve 1 Goal Current On Off Goal Current Open Closed On idle cmd = off Open idle driver = on cmd = close Off cmd = on idle Closed driver = on cmd = open idle Resettable cmd = reset cmd = off Stuck fail fail 76

77 Plan by passing sub-goals up causal graph Goal: Driver = off, Valve = closed Current: Driver = off, Valve = open Send: cmd = on Driver 2 Algorithm: see [williams and nayak, IJCAI97] Valve 1 Goal Current On Off Goal Current Open Closed On idle cmd = off Open idle driver = on cmd = close Off cmd = on idle Closed driver = on cmd = open idle Resettable cmd = reset cmd = off Stuck fail fail 77

78 Plan by passing sub-goals up causal graph Goal: Driver = off, Valve = closed Current: Driver = resettable, Valve = open Driver Failed Resettable 2 Algorithm: see [williams and nayak, IJCAI97] Valve 1 Goal Current On Off Goal Current Open Closed On idle cmd = off Open idle driver = on cmd = close Off cmd = on idle Closed driver = on cmd = open idle Resettable cmd = reset cmd = off Stuck fail fail 78

79 Plan by passing sub-goals up causal graph Goal: Driver = off, Valve = closed Current: Driver = resettable, Valve = open Driver 2 Algorithm: see [williams and nayak, IJCAI97] Valve 1 Goal Current On Off Goal Current Open Closed On idle cmd = off Open idle driver = on cmd = close Off cmd = on idle Closed driver = on cmd = open idle Resettable cmd = reset cmd = off Stuck fail fail 79

80 Plan by passing sub-goals up causal graph Goal: Driver = off, Valve = closed Current: Driver = resettable, Valve = open Send cmd = reset Driver 2 Algorithm: see [williams and nayak, IJCAI97] Valve 1 Goal Current On Off Goal Current Open Closed On idle cmd = off Open idle driver = on cmd = close Off cmd = on idle Closed driver = on cmd = open idle Resettable cmd = reset cmd = off Stuck fail fail 80

81 What if the causal graph G contains cycles? Computer Bus Control K-band Transmitter Amplifier Antenna K-band Transmitter Problem: Plan is no longer serializable. Solution: Amplifier Antenna Isolate the cyclic components (compute Strongly Connected Components). compose each cycle into a single component. New causal graph G is acyclic. Goals of G are serializable. 81

82 Action Policy for Composed Components Transmitter Amplifier on on on T on A cmd A = off cmd A = on on T off A A = off cmd T = on A = off cmd T = off T = on cmd A = on cmd A = off cmd T = on cmd T = off off off off T on A cmd A = off off T off A 82

83 Action Policy for Composed Components Problem: Composition grows exponential in space usage. Solution: Use BDD encoding. [Chung and Williams, Self-adaptive SW 03] on T on A cmd A = off cmd A = on on T off A cmd T = on cmd T = off Current Goal On T, On A On T, Off A Off T, Off A Off T, On A off T on A cmd A = off off T off A On T, On A idle cmd A = off cmd A = off fail On T, Off A cmd A = on idle cmd T = off fail Off T, Off A cmd T = on cmd T = on idle fail Off T, On A fail fail cmd A = off idle 83

84 Outline Review: programs on state Planning as goal regression (SNLP/UCPOP) Goal regression planning with causal graphs (Burton) Appendix: HFS planning with the causal graph heuristic (Fast Downward) 84

85 Causal Graph Heuristic for PDDL Recall: The Fast Forward (FF) Heuristic is computed over a Relaxed Planning Graph. Likewise: The Causal Graph (CG) Heuristic is computed over a Causal Graph. Map PDDL to concurrent automata, and extract causal graph (called domain transition graph (DTG). 2/29/2016 Goal Regression and Causal Graph Planning 85

86 Problem Reformulation Original Representation: STRIPS or PDDL Init Goal TruckIn(t, c 2 ) BoxIn(b,c 1 ) BoxIn(b, Paris) Operators, e.g. Drive(t, c 1, c 2 ) Pre: TruckIn(c 1 ) Add: TruckIn(c 2 ) Del: TruckIn(c 1 ) New Representation: Multi-valued Planning Task Variables: truck := {c 1, c 2, Paris} box := {ontruck, c 1, c 2, Paris} Init truck = c 2 box = c 1 Goal box = Paris Operators, e.g. Drive(t, c 1, c 2 ) Pre: truck = c 1 Post: truck = c 2 86

87 Domain Transition Graphs (DTG) One DTG per variable. Edges represent possible transitions (actions) and are guarded by preconditions truck c 1 Causal Edge Drive Drive Drive Drive Drive c 2 Drive Paris A causal edge between the DTG represents that the box DTG has preconditions that depend on the truck DTG. box Load truck=c 1 Unload truck=paris c 1 Unload truck=c 1 Load truck=c 2 Paris t c 2 Load truck=paris Unload truck=c 2 87

88 Calculating the Heuristic Initial: BoxIn(b, c 1 ) TruckIn(t, c 2 ) Goal: BoxIn(b, Paris) truck c 1 Drive Drive Drive Drive Paris Drive Drive c 2 box c 1 Load truck=c 1 Unload truck=paris Unload truck=c 1 Load truck=c 2 Paris t c 2 Load truck=paris Unload truck=c 2 88

89 Calculating the Heuristic Initial: BoxIn(b, c 1 ) TruckIn(t, c 2 ) Goal: BoxIn(b, Paris) truck c 1 Drive Drive Drive Drive Paris # of transitions to get the box to Paris: 2 Drive c 2 Drive box c 1 Load truck=c 1 Unload truck=paris Unload truck=c 1 Load truck=c 2 Paris t c 2 Load truck=paris Unload truck=c 2 89

90 Calculating the Heuristic Initial: BoxIn(b, c 1 ) TruckIn(t, c 2 ) Goal: BoxIn(b, Paris) truck c 1 Drive Drive Drive Drive Paris # of transitions to get the box to Paris: 2 Drive c 2 Drive # of transitions to get the truck to c 1 : 1 box c 1 Load truck=c 1 Unload truck=paris Unload truck=c 1 Load truck=c 2 Paris t c 2 Load truck=paris Unload truck=c 2 90

91 Calculating the Heuristic Initial: BoxIn(b, c 1 ) TruckIn(t, c 2 ) Goal: BoxIn(b, Paris) truck c 1 Drive Drive Drive Drive Paris # of transitions to get the box to Paris: 2 Drive c 2 Drive # of transitions to get the truck to c 1 : 1 # of transitions to get the truck to Paris: 1 box c 1 Load truck=c 1 Unload truck=paris Unload truck=c 1 Load truck=c 2 Paris t c 2 Load truck=paris Unload truck=c 2 91

92 Calculating the Heuristic Initial: BoxIn(b, c 1 ) TruckIn(t, c 2 ) Goal: BoxIn(b, Paris) truck c 1 Drive Drive Drive Drive Paris # of transitions to get the box to Paris: 2 Drive c 2 Drive # of transitions to get the truck to c 1 : 1 # of transitions to get the truck to Paris: 1 box c 1 Sum the transitions to get the heuristic value for the initial state h CG = 4 Load truck=c 1 Unload truck=paris Unload truck=c 1 Load truck=c 2 Paris t c 2 Load truck=paris Unload truck=c 2 92

93 Causal Graph Heuristic h CG ( s) v dom( s cost * ) v ( s( v), s * ( v)) The cost of getting from the current state, s, to the goal state, s* Idea: Sum the domain transition costs to get to the goal. 1. Identify each variable involved in the goal. 2. Recurse from child to parent in the causal graph. For each variable, sum the cost of along the shortest path to change the current value to the goal value. If changing that variable s value has preconditions, also add the cost of changing its parent variable. 93

94 Causal Graph Heuristic Notes Can not handle cyclic causal graphs Relax some links until the graph is acyclic Calculation performed differently in practice Modified Dijkstra algorithm Each cost calculation can over estimate. Not admissible Assumes no helpful interactions between subgoals 94

95 Breaking Cycles Break action with multiple effects into multiple unary effect actions If there are still cycles, ignore some preconditions of actions Pickup(d1, cream, w1,5) Pre: d1_location=w1, w1_cream=5 Eff: w1_cream=4, d1_payload=cream Pickup1(d1, cream, w1,5) Pre: d1_location=w1, w1_cream=5 Eff: d1_payload=cream Pickup2(d1, cream, w1,5) Pre: d1_location=w1, w1_cream=5 Eff: w1_cream=4 95

96 Breaking Cycles d1_loca tion d1_loca tion Actions split into unary effects d1_payl oad w1_inven tory d1_payl oad w1_inven tory c1_wa nt c1_wa nt 96

97 Causal Graph Heuristic Summary Graph capture dependence between variables Requires re-achieving conditions Multi-valued formulation reveals structure h CEA heuristic extends to cyclic graphs Performs extremely well on classical problems 97

98 Heuristic Families There are many other heuristics, we ve only covered two Type Relaxation Heuristics Estimates Relaxed planning graph Ignore deletes h max, h add, h ff, h cs, h m, ADHG h+ (cost of relaxed plan) Causal Graph Limited interactions h CG, h cea h+ (cost of relaxed plan) Projection Abstractions Projection h PDB, h STAN, h FORK h (cost of optimal) 98

99 Key Ideas Heuristic forward search is one of the fastest current planning techniques Domain independent heuristic design still problematic Fast Forward Heuristic Solve a relaxed problem in a Planning Graph Causal Graph Heuristic Consider problem s structure in a Causal Graph 99

100 MIT OpenCourseWare J / 6.834J Cognitive Robotics Spring 2016 For information about citing these materials or our Terms of Use, visit: