Intelligent Agents. Instructor: Tsung-Che Chiang

Size: px

Start display at page:

Download "Intelligent Agents. Instructor: Tsung-Che Chiang"

Wilfrid Franklin
5 years ago
Views:

1 Intelligent Agents Instructor: Tsung-Che Chiang Department of Computer Science and Information Engineering National Taiwan Normal University Artificial Intelligence, Spring, 2010 Outline Agents & Environments Rationality Task Environments Agent Structures Summary 2 1

2 What is AI Thinking humanly: The cognitive modeling approach Acting humanly: The Turing Test approach Thinking rationally: The laws of thought (logical) approach Acting rationally: The rational agent approach A rational agent is one that acts so as to achieve the best outcome or, when there is uncertainty, the best expected outcome. It is more general than laws of thought approach. It is more amenable than approaches based on human behavior. 3 Agents & Environments An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment thorough actuators. examples: robot agent perceives through cameras, infrared, etc. acts through arms, legs, etc. software agent perceives through keyboard, mouse, etc. acts through monitors, speakers, etc. 4 2

3 Agents & Environments The percept sequence is the complete history of everything the agent has ever perceived. An agent s behavior is described by the agent function that maps any given percept sequence to an action. It is an abstract mathematical description. The agent function is an abstract mathematical description; the agent program is a concrete implementation. 5 Agents & Environments Vacuum-cleaner world What is the right way to fill out the table? Tabulation of a simple agent function for the vacuum-cleaner world. Artificial Intelligence: A Modern Approach, 2nd ed., Figure 2.2 &

4 Rationality A rational agent is one that does the right thing. The right action is the one that will cause the agent to be the most successful. We need some way to measure success. 7 Rationality A performance measure embodies the criterion for success of an agent s behavior. The amount of dirt cleaned up in a single 8-hour shift? The number of clean square at each time step? The selection of a performance measure is not always easy. Maximum? Average? Multiobjective decision making/optimization 8 4

5 Rationality What is rational depends on four things: Performance measure Agent s prior knowledge of the environment The actions that the agent can perform The agent s percept sequence to date 9 Rationality Definition of a rational agent: For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has. 10 5

6 Rationality Is this a rational agent? (Dirty then suck; otherwise, move to the other square.) It depends! Artificial Intelligence: A Modern Approach, 2nd ed., Figure 2.2 & Rationality Under the following assumptions, it is. The performance measure awards 1 point for each clean square at each time step over a lifetime of 1000 time steps. The geography of the environment is known a priori. Clean squares stay clean. (knowledge) The only available actions are Left, Right, Suck, and NoOp. The agent correctly perceives its location and whether that location contains dirt. 12 6

7 Rationality In the following situations, it is not. The performance measure includes a penalty of 1 point for each movement. What if the clean squares become dirty again? The geography is unknown. 13 Omniscience, Learning, & Autonomy We need to distinguish between rationality and omniscience. e.g. Final Destination series The rational choice depends only on the percept sequence to date. Doing actions to modify future percepts (information gathering) is an important part of rationality. Rationality maximizes expected performance, while perfection maximizes actual performance. 14 7

8 Omniscience, Learning, & Autonomy The rational agent needs not only to gather information but also to learn from what it perceives. Successful agents split the task of computing the agent function into three periods: Design phase Acting phase Learning phase 15 Omniscience, Learning, & Autonomy A rational agent should be autonomous it should learn what it can do to compensate for partial or incorrect prior knowledge. Just as evolution provides animals with enough built-in reflexes, it would be reasonable to provide an agent with some initial knowledge and the learning ability. 16 8

9 Task Environments Task environments are essentially the problems to which rational agents are solutions. In designing an agent, the first step must always be to specify the task environment as fully as possible. PEAS: Performance, Environment, Actuators, and Sensors. 17 Task Environments Example: Automated taxi driver Agent Type Performance Measure Environment Actuators Sensors Taxi driver Safe, fast, legal, comfortable, profits, etc. Roads, pedestrians, animals, customers, etc. Steering, accelerator, brake, signal, horn, display Cameras, sonar, speedometer, GPS, odometer, accelerometer, engine sensors, keyboard 18 9

Task Environments Some other examples Agent Type Performance Measure Environment Actuators Sensors Part-picking robot Percentage of parts in correct bins Conveyor belt with parts; bins Jointed arm

10 Task Environments Some other examples Agent Type Performance Measure Environment Actuators Sensors Part-picking robot Percentage of parts in correct bins Conveyor belt with parts; bins Jointed arm and hand Camera, joint angle sensors Interactive English tutor Maximize student s score on test Set of students, testing agency Display exercises, suggestions, corrections Keyboard entry 19 Properties of Environments Fully observable vs. partially observable Fully observable: the sensors detect all aspects that are relevant to the choice of action. The agent doesn t need internal states in fully observable environments. An environment might be partially observable due to missing data noise & inaccuracy 20 10

11 Properties of Environments Deterministic vs. stochastic Deterministic: the next state of the environment is completely determined by the current state and the action executed The aforementioned vacuum world is deterministic, whereas taxi driving is stochastic. Strategic: the environment is deterministic except for the actions of other agents 21 Properties of Environments Episodic vs. sequential Each episode consists of the agent perceiving and then performing an action. In episodic environments, the choice of action in each episode depends only on the episode itself. E.g. agent spotting defective parts on an assembly line Episodic environments are much simpler because the agent does not need to look ahead

12 Properties of Environments Dynamic vs. static Dynamic: the environment changes while an agent is deliberating. Semidynamic: the environment itself does not change but the performance score does. Static environments are easy to deal with because the agent need not keep looking at the world nor need it worry about the passage of time. 23 Properties of Environments Discrete vs. continuous The discrete/continuous distinction can be applied to the state of the environment, the way time is handled, and the percepts and actions of the agent. Chess has discrete states, percepts, and actions. Taxi driving has continuous state, time, and actions

13 Properties of Environments Single agent vs. multiagent When does an agent A treat an object B as an agent? when B s behavior is to maximize a performance measure whose value depends on A s behavior Chess is a competitive multiagent environment. Taxi driving is partially competitive and partially cooperative. 25 Properties of Environments 可見度掌握度關聯性變動度 Task Environment Observable Determin -istic Episodic Static Discrete Agents Crossword puzzle Fully D Static Discrete Single Chess with a clock Fully Strategic Semi Discrete Poker Static Discrete Taxi driving Part-picking robot Episodic Single 26 13

14 Properties of Environments In the table on previous slide, the answers are not always fixed. e.g. Exceptions to observability of chess Some other answers depend on how the task environment is defined. e.g. Each game can be viewed as an episode, but decision making within a single game is certainly sequential. 27 Agent Structure Agent = architecture + program The job of AI is to design the agent program that implements the agent function mapping percepts to actions. The architecture 硬體軟體 makes the percepts from the sensors available to the program, runs the program, and feeds the program s action choices to the actuators

15 Agent Structure Agent program vs. agent function Agent program takes the current percept as input, and agent function takes the entire percept history. If the agent s actions depend on the entire percept sequence, the agent will have to remember the percepts. 29 Agent Structure A trivial agent program that keeps track of the percept sequence, and then uses it to index into a table of actions Artificial Intelligence: A Modern Approach, 2nd ed., Figure

16 Agent Structure Why is the table-driven approach doomed to failure? There will be P + P 2 + P P T entries, where P is the number of possible percepts and T is the lifetime of the agent. e.g. The table of chess has at least entires. The daunting size of these tables means no space to store the table no time to create the table no agent could ever learn the table from experience no guidance about how to fill in the table entries 31 Agent Structure The TABLE-DRIVEN-AGENT does implement the desired agent function. However, the key challenge of AI is to find out how to write programs that produce rational behavior from a small amount of code rather than from a large number of table entries

17 Agent Structure Four basic kinds of agent programs Simple reflex agents Model-based reflex agents Goal-based agents Utility-based agents After introducing the above four kinds of agent programs, we then explain how to convert all these into learning agents. 33 Simple Reflex Agents They select actions based only on the current percept, ignoring the rest of the percept history. Example: Artificial Intelligence: A Modern Approach, 2nd ed., Figure

18 Simple Reflex Agents The simple reflex vacuum agent program is very small compared to the table-driven agent. Reduction1: Ignoring the percept history. Reduction2: Ignoring the location when it is dirty. Artificial Intelligence: A Modern Approach, 2nd ed., Figure 2.3 & Simple Reflex Agents Condition-action rules (if-then rules) if car-in-front-is-braking then initiate-braking Human have many such connections, some of which are learned responses (driving) some of which are innate reflexes (blinking when something approaches the eye) 36 18

19 Simple Reflex Agents Schematic diagram Artificial Intelligence: A Modern Approach, 2nd ed., Figure Simple Reflex Agents Agent program Artificial Intelligence: A Modern Approach, 2nd ed., Figure

20 Simple Reflex Agents They are simple but of very limited intelligence. 看到黑影就開槍 They will work only if the environment is fully observable. Infinite loops are often unavoidable for them in partially observable environments. e.g. the vacuum agent without a location sensor Artificial Intelligence: A Modern Approach, 2nd ed., Figure Simple Reflex Agents Escape from infinite loop is possible if the agent can randomize its actions. A randomized simple reflex agent might outperform a deterministic simple reflex agent. The randomized behavior can be rational in some multi-agent environment. But in most single-agent cases we can do much better with more sophisticated deterministic agents

21 Model-based Reflex Agents The most effective way to handle partial observability is for the agent to keep track of the part of the world it can t see now maintain some sort of internal state that depends on the percept history. 41 Model-based Reflex Agents Updating the internal state requires the knowledge about how the world evolves independently of the agent how the agent s own actions affect the world This knowledge about how the world works is called a model of the world

Model-based Reflex Agents Schematic diagram Artificial Intelligence: A Modern Approach, 2nd ed., Figure 2.

22 Model-based Reflex Agents Schematic diagram Artificial Intelligence: A Modern Approach, 2nd ed., Figure Model-based Reflex Agents Agent program Artificial Intelligence: A Modern Approach, 2nd ed., Figure

23 Goal-based Agents Knowing about the current state is not always enough to decide what to do. The agent needs some sort of goal information that describes situations that are desirable. 45 Goal-based Agents Schema diagram Artificial Intelligence: A Modern Approach, 2nd ed., Figure

24 Goal-based Agents The goal-based decision making is different from the condition-action rules in that the future is considered. What will happen if I do that? Will that make me satisfied? 47 Goal-based Agents Reflex vs. goal-based The reflex agent brakes when it sees brake lights. A goal-based agent could reason that if the car in front has its brake lights on, it will slow down. Given the way the world usually evolves, the only action that will achieve the goal of not hitting other cars is to brake

25 Goal-based Agents Although the goal-based agent appears less efficient, it is more flexible because the knowledge that supports its decisions is represented explicitly and can be modified. The goal-based agent s behavior can easily be changed to go to a different location, but the reflex agent s rules work only for a single destination. 49 Goal-based Agents Sometimes goal-based selection is not trivial when the agent has to find a long sequence of actions to achieve the goal. Search and planning are devoted to this

26 Utility-based Agents Goals alone are not really enough to generate high-quality behavior in most environments. Goals just roughly distinguish between happy and unhappy. A performance measure should reflect the degree of happiness. Utility functions tradeoff between conflicting objectives likelihood of success 51 Utility-based Agents Schema diagram Artificial Intelligence: A Modern Approach, 2nd ed., Figure

27 Learning Agents Learning is now the preferred method for creating state-of-the-art systems. Manual programming is too slow and tiring. Learning allows the agent to operate in initially unknown environments and to become more competent. 53 Learning Agents Four conceptual components learning element: making improvement performance element: aforementioned agent critic: evaluating how the agent is doing problem generator: suggesting actions that will lead to new and informative experiences 54 27

28 Learning Agents Schema diagram Artificial Intelligence: A Modern Approach, 2nd ed., Figure Learning Agents The design of the learning element depends very much on the design of the performance element. Given an agent design, learning mechanism can be constructed to improve every part of the agent. how the world evolves what my actions do if-then rules etc

29 Learning Agents The critic is necessary because the percepts themselves provide no indication of the agent s success. The performance standard distinguishes part of the incoming percept as a reward (or penalty). The problem generator is responsible for suggesting actions that will lead to new and informative experiences. 57 Learning Agents Example: the automated driver Shocking language Bad action Try out the brakes on different road surfaces Make a quick left turn across three lanes 58 29

30 Learning Agents Learning is a process of modification of each component to bring them into closer agreement with the available feedback information. The simplest case is to learn from the percept sequence. Observation of pairs of successive states of the environment can allow the agent to learn How the world evolves. 59 Summary An agent is something that perceives and acts in an environment. The agent function specifies the action in response to any percept sequence. A rational agent acts to maximize the expected value of the performance measure given the percept sequence and candidate actions

31 Summary A task environment specifies the performance measure, environment, actuators, and sensors. Task environments vary along several dimensions. The agent program implements the agent function. 61 Summary There exists a variety of agent-program designs. simple reflex agents model-based agents goal-based agents utility-based agents All agents can improve their performance through learning

Intelligent Agents. Instructor: Tsung-Che Chiang

Intelligent Agents. Instructor: Tsung-Che Chiang Intelligent Agents Instructor: Tsung-Che Chiang tcchiang@ieee.org Department of Computer Science and Information Engineering National Taiwan Normal University Artificial Intelligence, Spring, 2010 Outline