Lecture 5- Hybrid Agents 2015/2016 Ana Paiva * These slides are based on the book by Prof. M. Woodridge An Introduction to Multiagent Systems and the slides online compiled by Professor Jeffrey S. Rosenschein.. Modifications introduced by Prof. Ana Paiva, are her sole responsibility.
What is an Agent? Agent sensors actuators ENVIRONMENT The main point about agents is they are autonomous: capable of acting independently, exhibiting control over their internal state Thus: an agent is a computer system capable of autonomous action in some environment in order to meet its design objectives
Limitations of Reactive Agents Agents without environment models must have sufficient information available from local environment If decisions are based on local environment, how does it take into account non-local information (i.e., it has a short-term view) Difficult to make reactive agents that learn Since behavior emerges from component interactions plus environment, it is hard to see how to engineer specific agents (no principled methodology exists) It is hard to engineer agents with large numbers of behaviors (dynamics of interactions become too complex to understand) 5-3
Criticisms to Deliberative Agents The symbolic approach The grounding to the real world Speed of decisions to deal with the real world.
Hybrid Architectures Many researchers have argued that neither a completely deliberative nor completely reactive approach is suitable for building agents They have suggested using hybrid systems, which attempt to marry classical and alternative approaches 5-5
Hybrid Architectures An obvious approach is to build an agent out of two (or more) subsystems: a deliberative one, containing a symbolic world model, which develops plans and makes decisions in the way proposed by symbolic AI a reactive one, which is capable of reacting to events without complex reasoning 6
Requirements for Hybrid Architectures Situated behaviour: agents need to respond to the events in the world in real time Goal oriented behaviour: agents need to select their actions taking into account the resources they have to achieve them and plan for how to achieve them. Efficiency: the tasks should be done in an efficient way, even with time constraints. Coordination: agents should be able to coordinate their activities with other agents.
Hybrid Architectures Often, the reactive component is given some kind of precedence over the deliberative one This kind of structuring leads naturally to the idea of a layered architecture. In such an architecture, an agent s control subsystems are arranged into a hierarchy, with higher layers dealing with information at increasing levels of abstraction 5-8
Hybrid Architectures A key problem in such architectures is what kind of control framework to embed the agent s subsystems in, to manage the interactions between the various layers. Horizontal layering Layers are each directly connected to the sensory input and action output. In effect, each layer itself acts like an agent, producing suggestions as to what action to perform. Vertical layering Sensory input and action output are each dealt with by at most one layer each 5-9
Hybrid Architectures: horizontal layering Each layer can be seen as one agent Advantages: simplicity Disadvantages: the global behaviour may not be coherent. difficult to build Layer n P input.. Layer 2 action Layer 1
Hybrid Architectures: vertical layering Vertical (1 pass control) Vertical (2 passes control) ) Action output Layer n Layer n Layer 2 Layer 2 Layer 1 Layer 1 Perceptual input Perceptual input Action output
Hybrid Architectures m possible actions suggested by each layer, n layers m n interactions Introduces bottleneck in central control system m 2 (n-1) interactions Not fault tolerant to layer failure 5-12
Ferguson- TOURING MACHINES 13
TOURINGMACHINES (Ferguson) The TOURINGMACHINES architecture consists of perception and action subsystems, which interface directly with the agent s environment, and three control layers, embedded in a control framework, which mediates between the layers 5-14
TOURINGMACHINES 5-15
TOURINGMACHINES The reactive layer is implemented as a set of situation-action rules, a la subsumption architecture Example: rule-1: kerb-avoidance if is-in-front(kerb, Observer) and speed(observer) > 0 and separation(kerb, Observer) < KerbThreshHold then change-orientation(kerbavoidanceangle) 5-16
TOURINGMACHINES The planning layer constructs plans and selects actions to execute in order to achieve the agent s goals (pro-active behaviour) Responsible for generating, executing and dynamically adapt the partial plans of the plan library. These plans aim at achieving tasks that the agent needs to execute on a long term basis. Plans are stored as partial hierarchical plans in a plan library. 5-17
TOURINGMACHINES The modeling layer contains symbolic representations of the cognitive state of other entities in the agent s environment It maintains a representation of the world through 4-tuples <C,B,D,I> indexed to a time stamp C- configuration: describes the place of the objects of the environment, velocity, acceleration, etc.; B- set of beliefs attributed to the entity; D- desires, represented through a list of goals and their priorities I- represented as an hierarchic plan that represents the intentional structure. 5-18
TOURINGMACHINES The three layers communicate with each other and are embedded in a control framework, which use control rules Control rules are defined by the programmer guaranteeing that: - At least one will fire in a given situation (bottleneck) - At least one action needs to remain in the action subsystem. 5-19
TOURINGMACHINES Control rules Example: censor-rule-1: if entity(obstacle-6) in perception-buffer then remove-sensory-record(layer-r, entity(obstacle-6)) 5-20
TOURINGMACHINES Suppressor: Conditional part: tests the existence of actions in the action submodule Action: operations to restrict that certain commands are executed by the action subsystem.. rule suppressor-rule-3 : if action-command(layer-r, rule-6, change-orientation()) in action-buffer and current-intention(start-overtake) then remove-action-command(layer-r, change-orientation()) and remove-action-command(layer-m, _). 5-21
Müller InteRRaP 22
Müller InteRRaP Vertically layered, two-pass architecture (CC) cooperation layer social knowledge (PBC) plan layer planning knowledge (BBC) behavior layer world model (WIF) world interface perceptual input action output 5-23
InteRRaP- Components of the architecture WIF (World Interface) contains the perception facilities of the system, as well as the action and communication facilities. BBC- (Behaviour- Based Component) executes and controls the basic recative behaviour. This element of based on a set of patterns of behaviour PBC (Plan Based Component): this component contains the planning mechanism of the agent allowing for the agent to have local plans. KB (Knowledge Base)- is the internal structure that is organised in an hierarchical manner composed by 4 levels associated with the different control levels. CC (Cooperation Component): allows for the cooperation between agents
InteRRaP- Knowledge Base Cooperation Level: knowledge about the other agents, common goals and intentions, and shared plans. Local Plans Level knowledge about itself including plans, intentions and actions. World Model Level- raw information coming from the environment. This hierarchic division of the knowledge base contrasts InteRRaP and TOURINGMACHINES
InteRRaP-Behaviour Based Component (BBC) Goals To incorporate the reactive activities of the agent that allows it to deal with emergency situations and real time Provide the agent with procedural knowledge that will allow it to execute routine tasks..
InteRRaP-Behaviour Based Component (BBC) Patterns of Behaviour connect the perception to action Types: Reactors: these rules are activated by external events that cause the execution of an action by the agent (eg. Stop). Control modifiers- they extend the control of the planning system make a call to the PBC (eg. Treat_order) Knowledge modifiers- change the internal state of the agent and are activated by changes in the world detected by the agent. Procedures- execute actions specified by the planner.
InteRRap: PoBs
InteRRap: Planning Component Includes the planning capabilities of the agent. Functions: (1) obtain a plan for a given goal (2) control the execution of that plan Components PBC-control- makes the interface with the BBC by receiving messages and sending messages to the other level. Plan-generator- generates plans through the selection of a set of plans from the plan library. Plan-evaluator- used for selecting the best generated plan.
InteRRap: Use of Plans of Behaviour (PoBs)
A summary 31
InteRRap: Interaction between layers Bottom-up activation: Perceptions start to be handled in the reactive layer. If this layer cannot handle it, the information goes up one layer to be handled. Top-down execution: Actions are sent from the top layers down to the bottom layers for execution.
InteRRap: Control
InteRRap: Application: the Loading Dock
InteRRap: Dealing with conflicts
Application: robots
Stanley 37
The DARPA Grand Challenge Launched by DARPA in 2003 Goal: Develop an autonomous robot capable of traversing unrehearsed off-road terrain Requirement: travel a 175 mile long course through the Mojave desert in no more than 10 hours Prizes: 2004: $1M, 2005: $2M, Teams: 2004, 107 teams participated (15 raced), 2005, 195 teams (23 raced, 5 finished) The route was kept secret from all teams until 2h before the race. At this time the teams were given a description of the course on a CD-ROM in a route definition data format (RDDF) In 2005 the RDDF contained 2,935 waypoints. 38
Stanley Volkswagen Touared R5 (4WD) 39
Stanley: Guiding principles Treat autonomous navigation as a software problem 40
Stanley Sensory Equipment Environment Perception: roof rack that houses (environment sensor group): 5 SICK laser range finders A color camera for long rage perception 2 RADAR sensors Two antennae Plus, 2 additional antennae (one for GPS and one for GPS compass) 41 and a radio antennae
Stanley control Three main actuators: Brakes Throttle Steering 42
Stanley: a layered architecture Sensor interface layer- this layer is responsible for receiving and time-stamping all the data (also contains the database server with the course coordinates in RDDF) Perception layer- maps sensor data into internal models. The internal models include information about the environment and information about the vehicle s own state (with 15 variables, such as position, orientation, velocity) 43
Stanley: a layered architecture Planning and Control Layer- responsible for regulating the steering, throttle and brake. A key module here is the path planner, which sets the trajectory of the vehicle. The trajectory created is passed to two controllers: one for the steering and the other for the throttle and brake. Vehicle Interface Layer- responsible for interfacing with the concrete actuators. 44
Stanley: a layered architecture The User Interface Layer- comprises the remote E-stop and a touch screen for starting up the software. The Global Services Layer- provides services to be used by all the modules. 45
Stanley Free Template from www.brainybetty.com 46
Stanley in action http://www.youtube.com/watch?v=m2acmnfzpng 47
DARPA Urban Grand Challenge The DARPA Urban Challenge was held on November 3, 2007 in Victorville, Calif. This event required teams to build an autonomous vehicle capable of driving in traffic, performing complex maneuvers such as merging, passing, parking and negotiating intersections 48
DARPA Urban Challenge 49
Since then 50
C4 51
C4 Create an agent architecture to generated intelligent behaviour for synthetic creatures (eg. a dog). Including: Reactive behaviour Goal and expectation based behaviour Learning 52
C4: The architecture 53
C4:Working Memory Structure mirroring the psychological concept of a working memory, providing a sensory history of the objects in the world. Incorporates time Location matching is used to combine perception information of different modalities. 54
C4: Layers Sensory System- is a filter through which all world events pass, keeping the creature s virtual sensation honest. This layer converts visual location information to local space. Events in the world can be of different type, such as acoustic patterns or visual symbolic events. Perception System in this system what is sensed from the world is interpreted and its meaning is given to the creature. 55
C4: Layers Prediction and Action Creatures act upon events that are likely to occur (prediction) Prediction mechanisms can extend the knowledge of the world by making associations Action system What to do? (PrimitiveActions) When do it? (TriggerContext What do I do it to? (ObjectContext) How long do it for? (DoUntilContext) What is it worth (IntrinsicValue) 56
C4: Layers Navigation layer Allows the creature to have spatial competencies Overrides the motor commands passed down by the action system with a more immediately appropriate motor command Motor System Executes the motor actions of the agent. 57
C4 in Action 58
Advantages of the Hybrid Architectures This approach is one of the most adopted now. It allows for a real time response of the agent combined with goal oriented behaviour. The reactivity can be privileged in relation to the deliberative level. Knowledge about the world can be also subdivided into layers with different levels of abstraction. The amount of knowledge in its layer can be restricted.
Disadvantages of Hybrid Architectures Lack of semantic clarity in the layered approach. Lack of conceptual clarity in the layered approach The interactions between layers are difficult to program and to test one will need to analyze all the possible interactions between these layers.
Extra Bibliography Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back, Ed. P. Maes, MIT Press, 1990. The Design of Intelligent Agents: a Layered Approach, Jörg Müller, Springer, 1996. Stanley: The Robot that Won the DARPA Grand Challenge, S. Thrun, et. Al., Journal of Field Robotics 23(9), 661-691, 2006 A Layered Brain Architecture for Synthetic Creatures, D. Isla, R. Burke, M. Downie and B. Blumberg, IJCAI 2001.
Discussion 62