Artificial Intelligence Lecture 7

Size: px

Start display at page:

Download "Artificial Intelligence Lecture 7"

Charles Summers
5 years ago
Views:

1 Artificial Intelligence Lecture 7

2 Lecture plan AI in general (ch. 1) Search based AI (ch. 4) search, games, planning, optimization Agents (ch. 8) applied AI techniques in robots, software agents,... Knowledge representation (ch. 2) semantic networks, frames, logic, resolution Expert systems (ch. 3) forward/backward chaining, uncertainty, Bayesian networks Natural language processing (ch. 5) Machine learning (ch. 7) version spaces, decision trees, classification, neural networks 2

3 What is an agent An agent interacts with its environments Perceive through sensors Human agent: eyes, ears, nose etc. Robotic agent: cameras, infrared range finder etc. Soft agent: receiving keystrokes, network packages etc. Act through actuators Human agent: hands, legs, mouse etc. Robotic agent: arms, wheels, motors etc. Soft agent: display, sending network packages etc. A rational agent is One that does the right thing Or one that acts so as to achieve best expected outcome 3

4 Agents Agent can be described as a mapping from percept sequences to actions. Specifying which action to take for each possible percept sequence provides the ideal agent. cf. game playing, optimal strategies Table in memory mapping percept sequences to actions Size of table too large To difficult to build that table Lack of generality 4

5 Rational agents Select action expected to maximize the performance measure given evidence provided by percept sequence to date and any prior knowledge. Performance measure defined the success criteria Percept sequence required to not just react to local percepts Prior knowledge of environment 5

6 Rational agents Select action expected to maximize the performance measure given evidence provided by percept sequence to date and any prior knowledge. Rational thinking may still fail due to lack of information Selecting the right performance measure and collecting information part of the problem Example: build a vacuum cleaning robot and give the criteria that all visible dirt should be gone when I get home Scenario 1: it doesn't see any dirt so it does nothing Scenario 2: it hides the dirt Success criteria: collect as much dirt as possible Scenario 3: It goes outdoors and vaccums up a big pile of dirt 6

7 Rational agents What is rational? The performance measure The percept sequence What an agent knows about the environment take all previous percepts into account. Model the environment The actions that the agent can perform consider also future effects of these actions. An ideal rational agent should do whatever action is expected to maximize it's performance measure, on the basis of the evidence provided by the percept sequence and whatever built in knowledge the agent has. Not forgetting to do actions in order to obtain useful information! 7

8 Rationality versus perfection How and when should we evaluate the performance measure? Is it better to take an immediate action giving a small increase in performance, or a an action that will/might give a larger increase in performance later? Example: should I withdraw my money from the bank and have fun, or wait until next year when I have more interest? Or 100 years after that?... Amortized costs and utilities Autonomy The utility of getting a reward in the distant future is less (but not zero!) than compared to getting it right now. How much less? Rationality requires exploration, learning and autonomy 8

9 Agent environment PEAS when designing rational agents Performance Environment Actuators Sensors 9

10 Agent environment Properties of different environments Fully observable vs. partially observable Deterministic vs. stochastic Episodic vs. sequential Static vs. dynamic Discrete vs. continous Single agent vs. multiagent 10

11 Fully observable vs. partially observable Fully observable Agent can access the complete state of the environment at all times Agent can detect all aspects that are relevant to the choice of action Partially observable Eg. vacuum cleaner with only local dirt sensor, doesn't know the situation at the other square(s) Eg. taxi driver, cannot see where all the other drivers are or their destinations. 11

12 Deterministic vs. stochastic Deterministic Next state of the environment is completely determined by the current state and the agents current action This is suitable for eg. the types of state space search, planning etc. that we have looked at so far Eg. vacuum world is deterministic (if we clean a square, it will be clean). Stochastic Next state in environment affected by current state and unobservable factors (eg. other agents, random outcomes of actions) It is stochastic if eg. the suction mechanism sometimes requires multiple tries to remove the dirt. 12

13 Episodic vs. sequential Episodic Agents experience is divided into episodes Each episode consists of agent perceiving and performing a single action Next episode does not depend on the actions in this episode. Example: Classification task: spotting and removing defective parts on an assembly line are episodic. Chess playing and taxi driving are sequential 13

14 Static vs. dynamic environments Dynamic The environment can change while the agent is deliberating Agent is continuously asked what to do next Examples Thinking means do nothing Taxi driving is dynamic. Other cars and ourselves keep moving while thinking and planning the route Playing chess is static Semi dynamic Environment doesn't change, but agents performance score does. Eg. chess playing with a clock 14

15 Discrete vs. continuous Environment states, actions and/or perceptions are discrete (symbols, integers,...) or continuous (real values). Eg. chess playing is discrete Taxi driving is continuous (position is a real value, continuous control signals, continuous percepts). The real physical world usually is continuous.. Common properties of continuous environments Large amounts of data Often sensor noise 15

16 Single agent vs. multi agents Single agent eg. vacuum cleaner example Multi agent multiple agents exists in environment Cooperative eg. Multiple vacuum cleaners are cooperative, taxi driving is partially cooperative (avoid collisions). Communication may be required Competitive Eg. chess playing 16

17 Agent environment Examples The hardest case partially observable, stochastic, sequential, dynamic, continuous, multi agent 17

18 Structure of agents How does the inside of agents work? Especially, in addition to their behaviours? How can they model the environment How to interact with the environment... 18

19 environments Assumption: agents can perceive their own actions 19

20 environments Percept (P) Agents perceptual input at any given time Percept sequence (P*) Complete history of anything it has ever perceived Agent function A mapping from any given percept sequence to actions Agent function is implemented by an agent program Agent program Runs on the physical agent, to implement f 20

21 What is an agent? Agent = Architecture + Program Hardware and environment (or simulation) Inputs/outputs from the program to the hardware What do we need to know to design an agent Possible percepts and actions Primitive control signals, or high level actions Goals or performance measure Model of the environment Static vs. dynamic environment Dealing with uncertainty 21

22 Different kinds of agents Informal classification of agent programs Table driven agents Doesn't work very well Simple based reflex agents Always does the same thing in the same situation Model based reflex agents Always does the same thing when the whole environment and state is the same Goal based agents Deliberation (eg. search, planning...) to reach the goal Utility based agents Maximize the performance (utility) function 22

23 Table driven agents Agents select actions based on entire percept sequence Requires a huge table. Compare with search trees for games. Grows exponentially over length of percept sequences. Problems Memory/space requirement Non generic Time for constructing the table This approach generally does not work! 23

24 Simple reflex based agents Select actions based on the current percept, ignoring rest of percept history Memory less. Respond directly to pecercepts Can get stuck in local optima 24

vs deterministic simple reflex agent Eg.

25 Simple reflex agents Problems Work properly if environment is fully observable Limited range of application Randomized vs deterministic simple reflex agent Eg. vacuum cleaning is deprived of location sensor Compare roomba vs. hauzen vc re70 25

26 Example Robot navigating towards goal in continous env. Inputs: Goal position, My position, Obstacles in front Act(Goal, My pos, Obst): Angle = atan2(goal, my pos) If len(goal my pos) < 0.1: return none If Obst: return 'turn right Else if Angle > 0.1: return 'turn right Else if angle < 0.1: return 'turn left Else return 'just forward 26

Model based reflex agents Agent maintain internal state to track aspects of the world that are not evident in current state Parts of percept history kept to reflect some unobserved

27 Model based reflex agents Agent maintain internal state to track aspects of the world that are not evident in current state Parts of percept history kept to reflect some unobserved aspects of current state Updating internal state information require knowledge: which perceptual information is significant how the world evolves independently how our actions affect the world 27

Goal based agents The action decision process involve some sort of goal information describing situations that are desirable Combine goal information with

28 Goal based agents The action decision process involve some sort of goal information describing situations that are desirable Combine goal information with possible actions proposed by internal state to choose actions leading to goal. Gives flexibility in what the agent can do. Search and planning based techniques 28

29 Utility based agents Crisp goal provides a crude distinction between happy and unhappy states Utility: maximize the agents expected happiness eg. choose the quicker, safer more reliable route rather than justs one that works. Utility function Map a state (or sequence of states) onto a real number denoting degree of happiness Explicit utility functions provide appropriate tradeoff or uncertainties to be reached of several goals Deals with conflicting goals (eg. speed vs. safety) Likelihood of success between different plans 29

30 Utility based agents 30

31 Sense Model Deliberate Act Sensing the environment Interpret the percepts, handle noise, uncertainty... Model the environment Handle the partial observability of environment Deliberation Act Eg. planning, search, heuristics,... Implement the actions as control signals M P Environment D C 31

32 Sense Model Deliberate Act Sensing the environment Ex: Filter the sonar readings Model the environment Ex: building a map of the room Deliberation Act Ex: plan a path that brings robot to dirty or unknown part of room Ex: Use low level actions to implement path following M P Environment D C 32

33 Learning agents Learning allows the agent to operate in unknown environments and become more competent than it's initial knowledge allows Learning algorithms Learning agent consists of: Learning element: making improvements Performance element: selecting external actions Critic: determine how performance element should be modified according to learning standard Supervised vs. unsupervised learning Problem generator: suggest actions that lead to new and informative experiences if agent is willing to explore a little. 33

34 Learning agents 34

35 Learning agents Example: taxi driver makes quick left turn over multiple lanes critic observes the shocking language from other drivers learning element formulates a rule that this was a bad choice of action performance element makes sure to never do it again Problem generator may suggest certain actions needing to be investigated in different situations. Eg. testing the brakes when driving on a road under new circumstances (eg. wet, icy...) Critic may also observe behaviors of others, to learn from them. 35

36 Simulating agents Use simulations of robots/agents during development / testing Cheaper development. No risk for dangerous misstakes / costs Simplified problem, perfect sensors, actions 36

37 Do we really want intelligent devices? 37

Agents and Environments

Agents and Environments Berlin Chen 2004 Reference: 1. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Chapter 2 AI 2004 Berlin Chen 1 What is an Agent An agent interacts with its