Autonomous Multi-Criteria Decision-Making in Ambient Intelligence

Transcription

1 Autonomous Multi-Criteria Decision-Making in Ambient Intelligence Petr Tucnik Department of Information Technologies University of Hradec Kralove, Czech Republic

2 1 Introduction The multi-criterial decision making (MCDM) is an approach typically used in managerial and planning decisions or for decision support purposes. It allows mathematically precise consideration of a large number of factors before decision is made. This work is focused on use of such approach in autonomous entities (agents) which requires some modifications to be made. Standard MCDM is using pre-prepared data while agent is acting autonomously and all information needed for making decision must be in a correct format in order to be processed mechanically (by software in computer). This somehow changes the perspective of MCDM theory as it is normally used because it is not primarily intended to be applied in autonomous decision making or autonomous systems. The intended use of MCDM in agent decision making processes is to function as a control mechanism for high-level decisions. Similarly to company practice where MCDM is used by management, previous experience with application in soccer robotics (Tucnik, 2007) and other areas has shown maximum effectiveness of MCDM approach in high-level decision making and reasoning. Although, in principle, application to lower-levels of behavior is possible (movement, pathfinding, manipulation with objects, etc.), other methods (artificial neural networks, genetic algorithms, tools of theory of control, etc.) are often more suitable or more effective. Therefore, this work will be focused on application of MCDM in high-level decision making. This is the level where strategic decisions and planning are made. Proposed approach was already used (with certain modifications) in area of soccer robotics (Tucnik, 2007). But as our domain or interest, area of ambient intelligence will be intended for future application of MCDM as its control mechanism. The notion of ambient intelligence (AmI) refers (Augusto & McCullagh, 2007, Cook & Das, 2004) to a digital environment that proactively but sensibly supports people in their everyday efforts. Historically, this approach is extension of area of artificial intelligence (AI) and many of its principles were built upon both its theoretical and practical basis. In pursuit of achieving state of intelligent environment (i.e. environment with presence of elements that exhibits ambient intelligence in some way), it is often both necessary and useful to use ideas from AI area. The agent paradigm will be most useful concept for our purposes here. The most obvious advantages of MCDM application arise from its mathematical basis. Possibly most important advantage lies in possibility to include large number of factors (theoretically unlimited) into decision making process. As a consequence, even very complex decision making situation may be processed and reasonable decisions obtained given several design rules are followed. Other advantages are: adjustable level of abstraction of task environment representation, speed of decision making and modularity of design and others. These matters will be explained in more detail in following text. 2 Autonomous Approach Shift to autonomy with use of MCDM (autonomous decision making is not a standard use of this approach) requires change of perspective in perception of both agent and environment. In this part, more precise specification of these terms will be offered which will clarify the perspective of all related aspects introduced in the rest of this chapter.

3 2.1 Agent The intended use of MCDM approach is in autonomous entities which are in the artificial intelligence called agents. As it is a good practice to start with basics, we will use following definition of an agent (adopted from (Russell & Norvig, 2009)): A (computer) agent is something that acts and it is expected to operate autonomously, perceive its environment, persist over a prolonged period of time, adapt to change, and create and pursue goals. Furthermore, this conception may be extended to cover the perspective of efficiency of behavior (again (Russell & Norvig, 2009)): A rational agent is one that acts so as to achieve the best outcome or, when there is uncertainty, the best expected outcome. The first definition quite accurately covers all important features of agents and they will be perceived as such in the following text. Such concept is easily applicable in area of ambient intelligence (AmI). For our purposes now, objects in (ambient) environment capable of any type of action will be perceived as agents and - as presumed - will be capable of functioning on their own and following respective internal routines, procedures or algorithms. Other aspect, often mentioned in relation to agent s behavior, is proactivity. Proactivity means agent is capable of acting on its own, without direct stimulus from the surrounding environment. There are several basic agent architectures which differ in complexity of features they are capable to use in decision making, ranging from environment representation (deliberative agents) to social behavior (social agents) or any of its combinations (hybrid agents). The proactivity is a feature of more complex agent architectures; simplest case of agent which is purely reactive (responses to external stimuli only) and without internal representation of surrounding world is not considered here. It is not useful in AmIE because environment perception plays essential role in decision making with this method of control. In the second definition mentioned at the beginning (rational agent), there is a notion of rationality mentioned in connection to the architecture of agents. It is the fundamental aim of MCDM principle to find most effective or optimal solution to the every given decision-making situation, and therefore it is a status of rational agent we would like to achieve in our agents. It is matter of reasonable and correct design to achieve this. Decisions are only as good as information they are based upon. Sub-optimal performance may not be caused by wrong decision-making method but rather insufficient information. Rationality of behavior is a complex problem which could extend our text to areas of cognitive science (as well as other domains). As it is not purpose of this text to focus on this issue in detail, definition mentioned above will be considered sufficient for our purposes at this moment. Reader should bear in mind there are more ideas hidden in the notion of rational agent than will be discussed here, for more information on related topics, see (Pfeifer & Scheier, 2001). 2.2 Agent Definition From the perspective of MCDM approach, every piece of important information has to be included into model of the system. As the agent is inseparable part of the environment, it has to be described in a similar way as the environment surrounding it. Before environment itself, an agent will be defined in more detail, formally. Agent is defined as a 3-tuple: AGENT = {P, G, V}, (1) where AGENT is self-explanatory, P is the set of perceptive functions (each sensor has one perceptive function and agent may have more than one sensor), G is the set of goals, and V is the set of actions agent is able to perform.

4 According to definition (1), an agent consists of three important parts determining what it is able to sense (perceive), what does it want to do, and what it is capable of doing. These parts will be discussed in more detail here, but altogether they form a cycle called sense-think-act cycle in literature. This is shown in the Figure 1. This loop allows agent to perceive its environment, decide and act. Cycle is theoretically infinite. This is a universal concept describing agent`s behavior. But the fact has to be pointed out that the cyclical behavior shown in the Figure 1 is intended for agents which have all the components necessary to be autonomous on their own. Agents in our perspective are independent on the user, not individually on each other. In principle, this is not an issue, because the multiagent system is intended to be cooperative. Figure 1: Sense-think-act cycle of an agent (adopted from: (Russell & Norvig, 2009) Agent s Perception The perception plays crucial role in agent`s design because it directly constrains agent`s functionality. The importance of context when providing services in described in (Bottaro et. al., 2007). An agent has to be able to perceive at least impact of its own actions (for formal specification, see below) in order to function properly and optimize its behavior towards some ideal state, whichever it may be. Without this feedback, loop action-impact-correction-action will never be created. Machine learning methods may not be successfully implemented in such systems. Keep in mind that it is unsupervised learning that is to be considered here (see (Russell & Norvig, 2009) for more detailed description of machine learning techniques). Supervised learning paradigm is not suitable because AmI system should be able to adapt to user`s needs as these occur and this cannot be made in advance (because it is unknown). Also, the real world environment is the intended application environment here and as such, it is more difficult to prepare agents beforehand for their tasks. Supervised learning is typically used in limited, well-structured problem domains, preferably with deterministic nature. This is distant from the environment of ambient intelligence. Our ideal agent is capable at least of perceiving impact of its own actions. There are two fundamental sources of information available to an agent, regardless of its construction, tasks, decision making algorithm, etc. These sources are either internal (agent`s sensors) or external. From decision-making perspective, there is no difference (at least for purpose of this text there will be none) between information scanned from the environment directly by an agent`s sensors and information obtained from the outside, e.g. from other agents, operator, database, etc. However, external information may always be flawed by unintentional or intentional error and both these cases are out of agent`s control.

5 The information verification is quite difficult to design and implement. Especially in multi-agent systems where antagonism exists between agents, disinformation may occur frequently. This is typically not the case of multi-agent systems in environment with ambient intelligence, but it should be noted that this poses some level of risk (e.g. in case of security attack, etc.). Agents in AmIE are typically cooperating with each other, providing information freely and correctly (as much as they are capable). The construction of agents in AmIE is very often uncomplicated and quite simple. Active objects (i.e. capable of actions) in the environment are taken as agents and they are very frequently severely limited in number of activities they are capable of doing. This is derived from the limited number of tasks they are intended to do, at the first place. One probably won`t expect a coffee maker to move freely around apartment or a vacuum cleaner to bring newspaper. It is a matter of common sense design where purpose defines set of actions for every agent present in the system. If we establish term of a complete agent as an agent capable of sensing, decision making, performing actions and moving, situated in adequate environment, suitably embodied - then it is most probably not a typical AmIE agent. The typical case of agent in AmIE is agent incapable of perception by itself (or have only limited equipment allowing it to perceive its environment), immovable, with limited actuator capabilities or possibly even combination of some of these limitations. Most likely, AmI multi-agent system will be highlighting mutual cooperation, free exchange of information and emergence of desired behavior from interactions between agents instead of their full functionality. Social dimension of agent interactions has to fill the functionality gap resulting from simplicity of design. In other words, specialization and information sharing is supposed to compensate missing full functionality in AmIE system`s agents. Special cases of agents may often be present in the AmIE: action-agents capable of performing actions but without any sensor equipment or sensor-agents capable of perception but incapable of actions. These extremes, in fact, do not establish need to modify theoretical structure as it is presented here because whole multi-agent system is presumed to be working together in a coordinated, cooperative way. Information is shared freely here. Set P (from Eq. 1) contains perceptive functions used by agent. At least one sensor must be present in the set P in order to get the necessary information from the environment to the agent. This can be extended to sensor agents as well they substitute sensors directly attached to agents performing actions. If the agent has no sensor apparatus on its own, it has to be connected to at least one sensor input provided by the sensor agent. Case of only one sensor is a minimal case here. In most cases, a combination of several sensors (as sources of external information) will be very probably used in real world applications. The perceptive functions p from the set P transform scanned data about external world W into attribute representation E of the environment. It can be formally written in a formula: W ρ A X, X E. (2) The perceptive function p A transforms part of the world into attributes belonging to E. We can also describe perception alternatively: ρ A (W ) = X, X E. (3) Eqs. (2) and (3) have the same meaning, but in some cases, one form of notation may be more useful or transparent than the other. This is the reason why both possibilities are presented here. The X in Eqs. (2) and (3) represents a limited part of the environment within the scanning range of the sensor with perceptive function p A. Scanning range of any single sensor usually does not cover the

6 whole environment (even if this was the case, the theoretical description has to be universally valid); therefore X is a subset of E. Area which can be influenced by agent s actions can be formally described as: ν A (Y ),Y E (4) Y is part of environment modifiable by agent s actuators. It is a matter of good design to ensure that for the following formula holds: ρ A (ν A (Y )),where Y X E, (5) i.e., the modifiable part is perceptible by the agent. Agent must be able to perceive effects of its actions, for more details see (Russell & Norvig, 2009) and (Tucnik, 2007). The condition in Eq. (5) allows the agent to learn from its own actions and, as a consequence, it will allow implementation of machine learning method of designer s choice. The Eq. (5) also represents minimal range of sensors. The range of sensors will be very probably wider than that, but this is a minimal case. The influence of exogenous factors cannot be always prevented (in the learning process), but this is a common problem in all real world applications and it does not matter which architecture is used. The perception of agent`s actions has to be available at least to some reasonable level of accuracy in order for agent to learn. This is a matter of proper design and preliminary testing to be done correctly when implementing sensors. For every attribute a in the environment, the following formula holds: A E C : ρ A (X), X E. (6) The whole environment (apart from calculated attributes which are not obtained via sensors, see below) must be perceptible. It may be useful at this point to mention again that level of abstraction is chosen by designer of the system in design phase and environment description covers all that is needed for the multiagent system to function properly. As a result of formula (6), the entire environment (in a form of attributes) is perceptible by the multiagent system. Individual perceptive functions of agents may overlap (this means they are scanning/perceiving the same attributes). For our case, we will assume that same attributes scanned by different sensors will have the exactly same values (we will neglect the case of wrong sensor calibration or noise effect here). Scanning of the environment is assumed to be continuous (real-time) or done in short time intervals. Although term real-time is somehow vague, we will simply assume that when something new happens in the environment, there should be no significant delay between the occurrence of such event and obtaining this information via sensors. Normally, the agent acts in repeating cycles (as was indicated in the Figure 1) and there is often some time-consuming task involved in this cycle every time it runs. This task (e.g. video recognition and processing which is typically very time-consuming, etc.) defines one cycle of an agent. Its length varies from very short time intervals (like sec, even less) in simple systems to longer delays (like 1 sec). Agent has generally better reflexes if its cycle`s length is short, like in the case of robotic soccer (where faster team usually wins). In case of AmIE, expectations for quick reactions are not so high and even slower actions are sufficient Agent s Goals Second part of agent`s definition from Eq. (1) is the set of goals G. This part of agent is closely related to the notion of rationality of behavior, as it has been already mentioned in the introduction. When making

7 decision, it is necessary to differentiate possible actions and their outcomes, to recognize bad and good solutions. Goals provide measure to differentiate. Those actions that bring agent closer to goal state are preferred above others. Idea is to allow continuous improvement of functionality. This is corresponding with general requirements of agent`s learning component which are listed in (Russell & Norvig, 2009): Noise immunity learning component of an agent should be able to work with filtered noised signal from sensors. Fast convergence (to optimal values). On-line learning agent is capable of learning while performing its task. Incremental learning learning process is continuous. Tractability learning has to be computable in real-time. Groundedness solution is based on information from the environment only, or information obtained during time. Learning process is realized from the perspective of agent. In real world applications, it is often complicated to express explicitly desired goal-state of the environment, even in case of a single criterion used. It is simple to imagine situations where the explicit interpretation may be used: e.g. in task (A): Bring 10 items X from area A to area B. This is easy to be interpreted in description of a goal state (10 items X in area B). Other ideal solutions may not be so easy to describe or find. Consider this task (B): Move from A to B with minimal energy consumption. When both scenarios are considered to be taking place in real world, environment will be stochastic and nondeterministic and many things may happen during the task processing. It would be generally easier to work with task (A), as it is better for measuring progress, than task (B). In scenario (B), an ideal solution will obviously be one which does not require energy consumption at all, and this is not possible. Therefore, the goal state is unreachable in this case solutions will be considered more efficient (better) when they will be near the goal state, but none of them could meet the precise definition of ideal state. In case of (B) type task, a principle of maximization is used, see (Ramik, 1999) we always try to maximize the attribute, not to minimize it. In fact, this is just matter of used perspective instead of consumption, terms for preservation or saving are used instead, etc. This reformulation makes whole work easier as we do not have to differentiate positive or negative aspects of solutions all are always positive. For goal definition, other approaches may also be applicable and worth considering. E.g. one of possible ways to prepare problem definition in suitable form is use of preferences in goal specification, see (Ennaceur et al., 2011, Fürnkranz & Hüllermeier, 2010). This would require some modifications in formal description, but otherwise these are valid, applicable approaches. Since this goes beyond the scope of this text, we will not discuss these matters here any further Agent s Actions The last part of agent`s definition (1) is the set V of possible actions. This reflects ability of agent to influence environment. Actuator set should be designed in a way which allows agent to pursue its goals. In case of standard (hardware) agents, actuators are mechanical arms, movement apparatus, servos etc. Given the design limitations of AmIE agents, as were described above, i.e. they are typically not complete agents, actuator set is typically closely related to specific agent`s function, according to its specialization.

8 There exists a dependency between agent`s perception and actuator functions. Agent is supposed to be able to perceive direct impact of its own actions. In practice, this means that immediate change in environment, emerged as a result of agent`s action, is to be noticed. It is the prerequisite of machine learning and continuous improvement of agent`s performance, i.e. adaptation to changes in environment. But realworld problem domains often come with presence of exogenous factors as the model of the world is created by/for an agent. Therefore, accuracy of environment perception decreases over time, as these exogenous factors increase their impact over time. Situation of agent in context of its action is captured in the Figure 2. Agent is able to make changes in its environment but, in general, not the whole impact of its own actions is perceived. Also, it is only a limited part of the real world the agent is able to perceive. This poses no problem, though, as people (as an example of intelligent beings capable of manifesting intelligence) are also limited in their perception (people are unable to see part of color spectrum or hear ultrasound or infrasound, etc.). The rational behavior can be achieved without capability to perceive everything in the environment. It just has to be taken into consideration that part influenced by actions from the Figure 2 should be included in part covered by sensors as much as it is possible, possibly whole. Figure 2: Perception and activity of agent Inaccuracies resulting from exogenous factors cannot be entirely prevented or eliminated from the model. Every real-world application requires certain level of abstraction when describing environment, objects, their mutual dependencies and relations etc. Also, phenomena previously unknown or not anticipated by designer of the model effectively function as exogenous factors. As a result, loop between actions and perception apparatus will contain errors resulting from unknown factors which change state of environment independently on agent and/or agent is unaware of them. As a consequence of above mentioned, agent is capable to perceive actuator impact with reliable precision only partially or over limited period of time. 2.3 Environment From the MCDM perspective, the whole environment (including agent, etc.) is perceived as a set of attributes. Attributes are numerically represented properties of the environment or factors describing

9 agent`s characteristics or condition. Whole description is domain-dependent and is created before agent begins its functioning in the environment. The term attribute is sometimes used by some other authors in different meaning, see (Keeney, 1993, Howard, 2007, von Winterfeld & Edwards, 2007) for comparison. In this case, term attribute will be used to describe environment. The environment has to allow machine processing of information. In order to do so, numerical representation of information (attributes) is required. This may not be obvious, but requirement of numerical representation is a significant constraint for description process. It is easy to come up with examples of adjectives with non-specific meaning (young/old, slightly used, heavy, etc.) and other obstacles related to natural language and designer of the system has to numerically clarify these terms (possibly with help of an expert in the field, if it can be arranged). Fuzzy set theory offers one of possible tools to handle such information capture ; other means are e.g. repertory grids, cluster analysis, designer-defined scales for diversification of term, etc., see (Awad & Ghaziri, 2004). This is, in fact, matter of proper handling of knowledge/information capture. As such, this topic is not closely related to main problem discussed here. When creating an attribute, following questions should be ask: Is the attribute necessary for proper function of an agent? Is the attribute unambiguous? (If yes, can it be renamed?) Is attribute correctly represented? (appropriate scale, units of measurement) Is attribute related to actuator function? Is attribute related to goal definition? Answering these questions helps to consider each attribute carefully and assign its description value accordingly to given circumstances. It is also possible to leave out attributes with little or low importance, i.e. in other words set suitable level of abstraction to model in question. While designing environment description, there are also several advantages to be considered: The problem domain and its description are limited there exists only a final number of important factors needed in model which are difficult to be represented numerically. Most of description would very probably pose no problem. Description is prepared in advance system is not active, therefore description problems do not interfere with control. New components (or pieces of description of model) are introduced into system only after this preparation phase. Level of detail may be determined by designer of the system this is related to the abstraction mentioned above designer may choose the level of detail accordingly to his/her needs. Also, it is often possible to split one attribute to several sub-attributes, creating hierarchy of attributes and making description easier. Similar mechanism can be used to merge attributes together if needed. Formally, the environment E can be defined as follows: E = A F C U, (7)

10 where E represents environment, A is set of agent-related attributes, F is set of environmental factors, C is set of calculated attributes and U are user-related attributes (problem definition). Each part of the environment E will be described in more detail now. Agent-related attributes of the set A are used to describe physical state of agent like remaining energy/fuel, condition of individual parts, situatedness in the environment (see (Russell & Norvig, 2009)), internal states, etc. In this context, situatedness is technical term and expresses the fact that an agent is not an isolated entity but exists in an (adequate) environment. Environmental factors in the set F contains attributes describing environment room size, appliance location, number of persons, temperature, etc. This set of attributes is usually easy to work with. Calculated attributes in the set U contains attributes that are used when needed and are calculated separately. Typically, there is no need to watch all attributes existing in environment all the time. Many of them have to be calculated only when needed. Short-term predictions of position of important elements in environment (persons, agents, objects) can be mentioned as typical example of calculated attributes. Also, when additional information is necessary in order to improve quality of decision, it is possible to run additional procedures, for example enhancing quality of video image recognition, etc. The set C has support role but, as the previous research in this area indicated, see (Tucnik, 2007), it is very important. Large part of information needed for qualified decisions is in the form of attributes of the set C. The last set of user-related attributes U contains information about users and their preferences which helps agent find appropriate actions towards the user. U = U for individual user`s attribute sets U 1, U 2, etc. At least one user must be defined (U is non-empty set). Typically, a preference-neutral default user description is also present in this set, which is useful when dealing with new users in AmIE, etc. For definition of environment E, as it was presented, holds following expressions: A = {a 1,,a k },F = {a k+1,,a l },C = {a l+1,,a m },U = {a m+1,,a n }, (8) E = A F C U = {a 1,,a n }, (9) A φ,f φ,u φ. (10) Set C can be empty (it means no calculated attributes are used), but this is not usual case. Number of n attributes together forms up environment E. 2.4 Data Preparation and Structure Another possible classification of attributes is to differentiate source of their origin (from agent s point of view). From this perspective, three groups of attributes exist in the model: external, internal and combined. External attributes are obtained though scanning the environment around agent by sensors. Internal attributes are related to agent`s problem definition and task information etc. These are information stored inside the agent (in a form of data). Combined attributes are derived from external information but modified by additional agent-specific information layer. Example can be map created by agent. Source of information is important in the means of reliability but in other aspects this differentiation can be neglected. While internal attributes are considered to be completely reliable regarding noise, quality of data transfer, etc., external data are to be captured and scanned by agent`s sensors (or other

11 agent`s sensors). This creates some risk of inaccuracy because of wrong sensor calibration or changing conditions on the outside of the agent`s body for using sensors (changes in temperature, moisture, pressure, EM interference, etc.). For the purpose of this text, an assumption will be made that sensors are always calibrated correctly and obtained data are without any noise. However, in real world application, it is important to take this into consideration when evaluation agent`s performance. All attributes are normalized in order to ensure their mutual comparability. Implication of this is obvious it allows values in different measurement units to be used together. For normalization, following formula is used: norm(a) = (a actual value a min ) / (a max a min ). (11) Boundaries a min and a max are defining range of attribute a, limiting its possible values. After normalization, the range of values is norm(a) 0;1. The normalization procedure is handled by sensor module of respective agent (attributes of the set C are to be normalized as well). Every attribute in the set E is normalized. The normalization procedure will take place before attributes are introduced into decisionmaking process. In practice, the normalization filters all decision-related data, i.e. all attributes. Since one sensor may be scanning several attributes at once (at least one), the situation is similar to one shown in the Figure 3. Figure 3: Scanning attribute from the environment The sensor provides only such information from the environment which it is able to process. However, the range of attributes may be quite wide, depending on the sensor. Also, one agent may be equipped with more than one type of sensor, providing even more information at the same time. In case of cooperative multi-agent system (such as agents in our ambient environment), information may be provided by another agent. When multiple sources (sensors, sensor agents) are used at the same time, situation is similar to the one in the Figure 4.

12 Figure 4: Multiple scanners used by agent. Grid on the right side contains all attributes in the environment. The Figure 4 represents situation when multiple sensors (or sensor agents) are used together to provide necessary information. The grid on the right side represents attributes, even those which are not used for actual decision and therefore eliminated from the decision making procedure for the moment. The normalization procedure, as it was defined by Eq. 11, requires setting of attribute`s boundaries specific value must be chosen for both maximum and minimum of attribute. There is an issue related to this because maximum and minimum may not be known or obvious. Solution is simple: set maximum, resp. minimum, value high enough, resp. low enough, to be sure that they will not be exceeded. However, following situation may occur (see Figure 5). Figure 5: Setting attributes range wrong case (circles represent examples of attribute values) In case shown in the Figure 5, range of attribute is wrong. In all cases (indicated by numbers), value of attribute was always very far from minimum and maximum. After normalization, all values of this attribute will be probably very near to each other, making all but extreme changes unnoticeable. Possible solutions: (A) Increase weight of attribute this will increase impact of normal change in attribute`s value. Risk is in occurrence of extreme value along with high value of weight it will very probably disrupt decision-making process, making system unreliable and unstable, with inconsistentcy in behavior.

13 (B) Adjust minimum and maximum of attribute according to real observation when functioning. Solution (B) is better from the long-term perspective. This requires introducing a testing phase into agent`s lifecycle. When attribute is scanned and used several times, minimum and maximum values are adjusted more accurately. However, during this phase - because attribute`s definition is modified and its values from previous decisions are changed - machine learning may not be used during this time. Modified attribute range is shown in the Figure 6. Figure 6: Setting attributes range adjusted case (circles represent examples of attribute values) Similar solution may be used when maximum and minimum are set to close to each other by accident when attribute is defined. When maximum or minimum is exceeded, the limit should be modified to cover new case. To avoid similar situation in future, some tolerance should be applied (5-10%). 3 Decision Making Attributes, agents, environment and basic principles were presented in the previous part. The decisionmaking principle will be described in this part of the chapter. 3.1 Decision Making Procedure The basic component in decision making process (DMP) is an attribute. Attribute, as a piece of information, is important only in context of an intended plan, an activity. As descriptor of the environment, it serves purpose of showing how would the environment look if certain action is taken, or how does it look at the moment. Every attribute is only a piece of mosaic, together they form image of the whole world of an agent. Using knowledge engineering perspective where data, information and knowledge usually form three-level hierarchy, see (Awad & Ghaziri, 2004), the attribute stands at the level of information, because it is always situated in some context. Attributes also describe environment in time line. Present, past and future configuration of environment - all these are described in a form of attributes, allowing agent to learn from past decisions as well as predict the future (to a limited extent). What we are trying to do when making decision is not undergoing immediate action action is only consequence and implementation of reached decision but rather future configuration of an environment. This is shown in the Figure 7. We may distinguish between two planes of existence: reality, named act space in the Figure 7, in which the agent is situated at the moment of decision making, and several

14 possible outcomes of actions in the consequence space. Each of n arrows represents solution to given decision making situation and whole sequence of actions (plan) is hidden behind it. Figure 7: Act and consequence space When making decision, following formula is used for every considered variant V: y ( ) c(v ) = w x norm(a x ), (12) x=1 where c(v) represents suitability of the variant V. Each variant`s attributes` values are calculated, using Eq. (12) and the importance of each attribute a is expressed by weight w. The solution we are looking for is the variant with highest c(v) value: solution = max{ c(v 1 ),c(v 2 ),,c(v Z )}. (13) In order to function properly, there is a threshold value defined for agent. Its purpose is to allow agent to work without interruption unless something serious happens. Minor disturbances should be ignored. Setting of specific level of threshold is designer`s decision. It may be modified later by using appropriate method of machine learning. In decision making procedure, a universal sequence of steps shown in the Figure 8 takes place. The sequence in the Figure 8 begins at the DMP (decision making point). This is a point in time when agent receives a stimulus to act. There are generally two types of such stimuli, usually implemented together either there is a change in agent`s environment which is possibly interfering with agent`s current state/activity, or its state can be checked periodically. Former can reflect presence of exogenous factors interfering with on-going agent`s activity. Each state has defined set of attributes for monitoring and these can interrupt current state, if the stimulus is strong enough. In any case, it is not necessary to monitor all attributes defined in environment. Strength of the stimulus is measured by threshold value (i.e. it is compared to it). If the impulse is not strong enough (i.e. lower or equal to threshold), agent remains in its original state, if it is exceeding threshold value, agent will leave its original state. To make decision making easier, number of variants is taken into consideration in following step. When there is only one option available, there is only behavioral shift towards another agent`s activity, because no decision making is required as the agent only needs to change its state to another one and only one option is available to it. In

15 case of more than one solution, decision making procedure takes place, as it is described above, using Eqs. (12) and (13). Suitability of each considered variant is computed using (12). Received value c(v) is stored and solution V x with highest c(v x )value is applied. Procedure described in the Figure 8 is taking advantage of state representation for agent`s behavior which is described in the following part. Figure 8: Decision making procedure 3.2 State Representation In proposed control method, notion of state refers to internal configuration of agent while it is performing action over given period of time. Two types of states are recognized: long and short. Distinction is made according to timeframe of agent`s activity. Long states represent continuous activity of an agent with unreachable goal state or an activity that is to be maintained for indefinite amount of time. Typically, long state is a long-term activity. Long states will be using notation S 0 (letter S for state, subscript 0 is reserved for long states, this notation will be explained further in the following text). Short states represent activity of agent which is performed over limited time or when goal is reachable in a short period of time. For temporary states, following notation will be used: S n (letter S for state, subscript n is a number > 0 (0 is reserved for long states)). Concept of states is useful for several reasons. Most of all, it allows modularity in defining agent`s behavior. Each state is derived from the activity agent performs from its actuator`s activity. The performed activity can even be quite complex one, using more actuators at the same time, but typically it is aimed at modification of one or more of environment`s attributes (and as a result shifting the state of environment towards intended goal state). The presence of goal is crucial here agent is trying to reach its goal all the time and without its specification there is no way to decide which action should be taken at the given moment. Use of states allows form up agent`s behavior piece by piece, from small parts to complex blocks of actions. States are organized into segments. There is always exactly one long state and any number of short states (even none) present in single segment. Number of segments is unlimited, but at least one segment must be defined for the agent in order to function. This is, however, a minimal case. To distinguish individual segments, a letter in superscript is used, e.g. S ( S represents state, number in subscript position

16 is determining long/short state number, letter in superscript position is designation of segment). Segments are connected as it is needed (by designer of the system), but there always has to by a cycle present and all states should be reachable. This formal form of notation will sometimes be reduced in figures because relation of state to segment will be obvious from the context. For state representation, notation shown in the Figure 9 will be used. Figure 9: Graphical notation of state representation of agent`s behavior The long state is indicated by number 0 (there are no segments here at this figure, so no further description is needed) and double circle. Single circles with numbers > 0 represent short states. Figure 9 also shows allowed transitions between states. There is only one limitation here: there can be no transitions between short states in backwards manner. The rule is that from the short state, the only possible transition is forward or to the long state. There are two main reasons to establish such rule. Firstly, it is easier and more comprehensible for the designer of the system. Secondly, it prevents possible cycle deadlock between short states. It is also easier to find possible error or malfunction with such restriction. Figure 10 indicates layout or elements. It is divided into segments where near to the center are situated long states and short states are on the outside. Transitions between states are indicated in the segment C of the Figure 10. It was already mentioned before; one long state is in each segment along with any number of short states (even none). Figure 10: Organization of information

17 3.3 State Chart and its Description In this part, an example of the state chart will be presented along with description of some of its important characteristics. The example is in Figure 11. This is graph representation of agent`s behavior where nodes are states and edges indicate transitions between them. It is oriented graph. The example shown in the Figure 11 can be represented in a form of transition matrix. In this matrix, number one indicates existing transition between states, zero means no transition exists. Since the graph is oriented, matrix in not symmetric. Last column on the right in Table 1 is sum of outgoing edges, i.e. outgoing degree of a graph node. When decision procedure takes place, complete decision making is done only in nodes with a star * symbol. This is where more than one option is available to the agent. Figure 11: Example of state chart. DIRECTION FROM DIRECTION TO S S S S S S S S S S S * S S * S S * S S S * S S * Table 1: Representation of state transitions in matrix.

18 Number of variants taken into decision-making process is given by the sum in each row of the transition matrix (Tab. 1). If the sum in the row is equal to 1, then the agent is deciding whether to stay in the current stay or continue to the next one. If the goal state in such state is reached, agent moves on to another state and adopts another goal to reach. Since agents in AmIE represent appliances and equipment (indoors case of ambient intelligence in environment), it is usually good practice to create one more segment for maintenance purposes in case of any malfunction. This will allow agent to act correctly when malfunction occurs, log entry about event is to be created and possibly some user`s assistance requested, if necessary. As there are many types of possible failures, this matter will not be discussed in detail, but this maintenance segment can often be shared among many agents, as the procedure is same for all of them (or it probably will be very similar). Presented example is general (due to the space limitation), but practical example modeling practical application may be found in (Mls & Tucnik, 2010). 3.4 User-Oriented Approach One of the most desired features of ambient systems is to be helpful towards users and their needs. This is supposed to be realized discreetly while system requires only minimal interaction with user. Ambient elements in homes or offices often serve other functions as well maintaining optimal energy consumption, temperature, lights, security etc. Some of these features may not be so obvious. Nonetheless, the user-oriented features are important part of any such system, and they are expected to be incorporated. Proposed system allows adaptation of user needs through modularity of actions. Individual pieces of behavior can be arranged in many ways, fulfilling individual user`s needs. For every user, there can be behavioral pattern defined, similar to one from the example schematics in the Figure 11. This can be described in more detail and more formally, but it would be beyond intended scope of this text. When designing the system, the difficult part is to create attributes and implement atomic actions of agents. The more complex behavior is created by merging these simple parts together. It is expected that system acts according to user`s preferences. This requires testing period, during which system consults the user and makes required adjustments. After testing period, system acts independently on the user, but also allows modifications at any time. Proposed control method is capable of manifestation of some form of creativity in its activities. Since agents are trying to find the most appropriate reaction to each given situation, it is possible to perceive changes in system`s response to the same signal over time. Of course, this requires some form of machine learning to be implemented in the system, see (Dembczyński et al., 2010) for machine learning issues connected with multiple attribute ranking problems. Possibly the most appropriate form of machine learning for this type of control is reinforced learning, see (Russell & Norvig, 2009, Pfeifer & Scheier, 2001). In general, supervised learning methods may be used during testing period, unsupervised after. Use of supervised learning during normal use would require cooperation of user and could be possibly perceived as a malfunction or ineffectiveness of the system since it would interfere with every-day use. It is to be expected that effort to find the optimal solution would always require exploration of several suboptimal routes. In any case, this is to be a part of future work. As relevant source of information on MCDM creativity, see (Skulimowski, 2011). Setting system to act accordingly to single user`s preferences is not complicated. It is also nothing new in the area of ambient intelligence these features are widely available in commercial products of

19 today, see (Cook & Das, 2004). Conflicts of multiple users` goals (or users with different access rights towards system`s features, e.g. adults and children have different access rights) which frequently occur creates problems when they are living or working together. This is part of future work: to focus on autonomous solution of conflicts between multiple users` goals or preferences. 4 Conclusion and Future Work This text s purpose was to mediate the ideas about use of multi-criteria decision-making control of AmI in environment which consists of cooperating agents. The main portion of work lies in designing such system properly, before it is put into practice. When this is done, the control mechanism is able to act quickly and follow appropriate behavioral patterns, performing desired activities. Proposed system allows user-defined patterns of behavior to be easily implemented, once the background description of environment is complete. In practice, this is often the most required system`s capability to follow behavioral routines according to given situation. Multi-criteria decision-making allows system to recognize such situations correctly and apply convenient action without user`s intervention. This will increase comfort of the system`s user and creates user-friendly (possibly no-disturbing) interaction. There are open problems to be solved in future: collisions between users with different access rights, cooperation models/architectures of agents, recognition of user`s activity (which is necessary to arrange help in his/her efforts) and others. One of undeniable positive qualities of use of the MCDM control method is simple incorporation of new information (attributes), agents or other features into system. System is able to act quickly and when designed properly, fulfill important expectations users have of ambient technologies when creating smart homes or offices. Acknowledgements This work and contribution were supported by the project of Czech Science Foundation SMEW Smart Environments at Workplaces no. P403/10/1310, and Specific Research Project AmIRRA 2 Ambient Intelligence and Related Research Activities. References Augusto, J. C. & McCullagh, P. (2007). Ambient Intelligence: Concepts and Applications. In ComSIS Vol. 4, No. 1. Awad, E. M. & Ghaziri, H. M. (2004). Knowledge Management. New Jersey: Pearson Education. Bottaro, A., Bourcier, J., Escoffier, C. & Lalanda, P. (2007). Context-Aware Service Composition in a Home Control Gateway. In IEEE International Conference on Pervasive Services. Cook, D. J. & Das, S. (2004). Smart Environments: Technologies, Protocols and Applications. New Jersey: John Wiley and Sons.