INTELLIGENT control plays an important role when employing

Transcription

1 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER Behavior-Modulation Technique in Mobile Robotics Using Fuzzy Discrete Event System Rajibul Huq, Student Member, IEEE, George K. I. Mann, and Raymond G. Gosine Abstract This paper presents a novel behavior-modulation technique using a fuzzy discrete event system (FDES) for behavior-based robotic control. The method exploits the multivalued feature of fuzzy logic (FL) and event-driven property of a discrete event system (DES) to generate the activity of a behavior using fuzzy state vectors. State-based prediction of an activity is accomplished using fuzzily defined event matrices. A central arbiter employs priority-based arbitration among the activity state vectors and generates new event matrices to modify the activity states of the behaviors. The method combines aspects of both command fusion and behavior arbitration. Furthermore, the proposed approach has the ability to define state-based observability and controllability to handle sensory uncertainty and environmental dynamics. Observability describes decision vagueness associated with sensory data, whereas controllability specifies undesirable state-reach within the observed environment. Real-time results of FDES-based mobile robot navigation are presented and compared against four different modulation methods to validate its superior performance. Index Terms Behavior-based robotic control, behavior modulation, fuzzy discrete event system (FDES), mobile robot navigation, obstacle avoidance. I. INTRODUCTION INTELLIGENT control plays an important role when employing mobile robots in unstructured, unknown, and dynamic environments. The task complexity of intelligent control is greatly reduced by dividing the overall task into subtasks. These subtasks are modeled as perception-action units, called behaviors. The reduced task complexity in a behavior-based approach increases responsiveness to environmental dynamics. However, behaviors with different objectives may produce conflicting actions, which lead to system instability. Therefore, a major issue in the design of a behavior-based control system is the formulation of an effective behavior coordination mechanism that selects relevant behaviors at a particular moment to produce an appropriate system response. Manuscript received September 12, 2005; revised February 16, This paper was recommended for publication by Associate Editor G. Antonelli and Editor K. Lynch upon evaluation of the reviewers comments. This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC), in part by C-CORE, and in part by the Memorial University of Newfoundland. This paper was presented in part at the IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada, August The authors are with C-CORE and Memorial University of Newfoundland, St. John s, NL A1B 3X5, Canada ( rajib@engr.mun.ca; gmann@engr. mun.ca; rgosine@engr.mun.ca). This paper has supplementary multimedia material available at provided by the authors. This material contains three wmv files illustrating three navigation examples using the FDES-based coordinator. Digital Object Identifier /TRO The existing behavior coordination mechanisms can be broadly categorized into two groups. Behavior arbitration mechanisms [1] [12] choose one behavior at a time, whereas command fusion mechanisms [13] [21] allow combined or weighed multiple behavioral constraints at each time. While arbitration mechanisms are suitable for competitive behaviors, command fusion techniques are appropriate for cooperative behaviors [22], [23]. Behavior arbitration can cause instability [10] when the control of the robot alternates between two behaviors. It can also cause starvation [8] when a behavior does not gain control of the robot for a long period of time. Command fusion techniques partially remove this problem by activating all behaviors simultaneously. However, this approach has limited applicability when competing behaviors issue conflicting control commands. As an example, conflicting command fusion may lead to oscillation of the robot or stagnation during navigation [23], [24]. Several fuzzy logic (FL)-based command fusion approaches, such as fuzzy context-dependent blending of behaviors [25] [30], fuzzy behavior modulation [31], and a fuzzy decision-theoretic approach [32] have been proposed to overcome such problems. In context-dependent blending of behaviors, Saffiotti et al. [26] use a set of fuzzy rules to define a fuzzy behavior. Another set of fuzzy rules, called meta rules, are used to control the activity of individual fuzzy behaviors by detecting conflicting situations. In the behavior-modulation approach, Tunstel et al. [31] propose a similar concept, where a set of fuzzy rules is used to weigh (or modulate) the behavioral activity. In the fuzzy decision-theoretic approach [32], behaviors are first modulated using fuzzy rules. Then, a subset of modulated behaviors is selected, using the multiple-objective decision-making technique. Although fuzzy rules have the ability to cope with uncertainties in sensory data and environmental dynamics, they suffer the drawback of forming a large rules base for complex behavior-based systems. Moreover, these methods lack the capability of system analysis (e.g., controllability and observability that enables formal quantitative analysis of the system performance). On the other hand, behavior arbitration supports a predictive control structure (e.g., discrete event system (DES) [6]) that permits system analysis. Both behavior arbitration and command fusion constitute two paradigms in designing distributed systems: hierarchical [11], [20] and nonhierarchical [2], [16] approaches. In hierarchical approaches, a set of behaviors at the lowest level are activated using prior knowledge of the system, and ensures goal-directed decision making. On the other hand, in nonhierarchical approaches, all behaviors are set to be concurrently active, and eliminates the requirement of prior knowledge of the system, which makes the system more reactive /$ IEEE

2 904 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 Reliability is an important issue in designing a behavior coordinator that includes reactivity, error recovery, and uncertainty handling. Reactivity provides robustness against unpredictable environmental changes [12]. Error-recovery techniques continuously monitor the behavioral performances, and invoke a corrective action upon detection of an error or failure [3]. Uncertainty-handling techniques [21], [25] use approximate reasoning and predictive decision making to reason out a solution that increases the applicability of a behavior. In summary, for better control, a behavior coordinator should: 1) combine both behavior arbitration and command fusion; 2) facilitate the coordination of cooperative and competitive behaviors; 3) combine features of both hierarchical and nonhierarchical approaches to make the system goal-oriented as well as reactive; 4) be reliable; 5) provide adequate means to analyze the system characteristics, such as controllability and observability, and be scalable to a large behavior-based system. This paper attempts to develop a novel behavior-coordination mechanism that considers all the aforementioned requirements. The proposed approach employs a fuzzy discrete event system (FDES) [33] [37] to formulate the behavior coordinator. It combines the state-based formalism of DES with the deterministic vagueness of fuzzy decision making. DES is an effective tool to analyze complex systems that are difficult to model with differential equations, but can be described by a sequence of events [38]. The events record changes in the state of the system. When DES is employed for behavior-based robotic control, the system states are updated using the events constructed from the sensory data. The DES theory also provides the formalism to analyze system characteristics. However, DES may lead to erroneous states if sensory information is faulty. The sensory uncertainty reflected into system states is better represented by FL. With partially known information, the states can be described using different memberships. The formalism of FDES, proposed in [33] [37], facilitates the integration of both state-based analysis of DES and the approximate reasoning of FL. Hence, this paper aims to extend the concept of FDES in the field of mobile robotics, which is a novel attempt with respect to the existing literature. The proposed methodology employs FDES to predict activity vectors of each behavior using multisensory information. It also employs FDES to analyze the observability and controllability of behavior-based decision making, which is further used for error/failure detection. Thus, this methodology addresses requirements 3 and 4. A central arbiter chooses a subset of behaviors having high (in fuzzy terms) activity for further assessment, thus incorporating the characteristics of hierarchical approaches (requirement 2). The central arbiter then assesses the activity vectors with respect to a predefined priority ranking of the behaviors, and generates a scheme for behavior modulation. The inclusion of explicit priority of behaviors denotes behavior arbitration, as found in Brooks Subsumption Architecture [1], whereas incorporation of behavior modulation indicates command fusion, as described in the FL-based approaches [31] (requirement 1). The method also includes the aspect of Fig. 1. Construction of F. nonhierarchical approaches, since each behavior is concurrently active (requirement 2). Furthermore, the proposed method will present a generalized approach that is scalable to a large behavior-based system (requirement 5). Therefore, the intended research attempts to address the aforementioned requirements for a behavior coordinator. Section II briefly describes the objective of the proposed behavior-modulation technique. Section III outlines modeling of FDES, and Section IV formulates the behavior-modulation mechanism. Section V presents an example of mobile robot navigation using FDES-based behavior modulation. It also demonstrates real-time navigation results to validate the performance of the proposed method. Finally, Section VI draws the conclusion. II. BEHAVIOR MODULATION A behavior having the th priority is defined as where is the identification label of a behavior (e.g., go-totarget). The order denotes the execution priority and is predefined ( has the highest priority and 1 has the lowest priority). The expected action to be accomplished by each behavior (e.g., velocity command) is denoted as. The related sensory information is defined as where is the number of different types of sensory information used in the activity prediction of. The FDES determines the state-based activity that modulates the action. The state-based prediction of is performed using a set of fuzzy event matrices, and are generated using another set of FDES,, as shown in Fig. 1. Each is placed to produce state-based prediction using single sensory information. The generates the activity of the th behavior based on the th sensory information. Hence, combines state-based predictions made by different sensory information to estimate the activity of behavior. Fig. 1 shows the construction of. Throughout the paper, the current state of an FDES is denoted as, whereas the expected state is specified as. The expected activity state is generated by while taking a set of fuzzy event matrices. These event matrices are generated using the (1) (2)

3 HUQ et al.: BEHAVIOR-MODULATION TECHNIQUE IN MOBILE ROBOTICS 905 sensory information. The set of event matrices is formed using, which is further used by to generate the combined activity state. The task of the proposed behavior modulator is to coordinate actions of different behaviors as the sensory data used in constructing is incorporated, is redefined as [33]. Hence, if observability (9) (3) where is a fuzzy OR operator (maximal) and is calculated as where the modulating factor is generated using. III. MODELING OF FDES The following sections describe the general formulation of FDES and modeling of and. A. General Formulation of FDES FDES is implemented using a fuzzy automaton [33] and is modified for this paper as where is a set of fuzzy state vectors, and is defined as (4) Here, is the identity matrix and is calculated as (10) times (11) where is the max-min fuzzy composition. The overall next state vector is estimated using (8). The observability described in [33] is interpreted to define state-based observability in order to provide state-based decision making for a physical agent like a mobile robot. Hence, state-based decision making increases reactivity of a robotic system. The state-based observability for an FDES is calculated using Here, is the number of states in the system, and is the degree of possibility of being in the th state. is a set of fuzzy event matrices, and is defined as Here, denotes the number of events in an FDES, and indicates the state transition possibility from the th state to the th state when the th event occurs. The state transition function,, generates the next state vector with respect to the current state vector upon occurrence of an event. Hence where vector (5) (6) (7) is the max-product operation. The overall next state is determined as 1) State-Based Observability: Each event in an FDES is associated with a user-defined degree of observability (certainty) and a degree of unobservability (uncertainty), where. They can usually be determined using experimental data. The parameters and can be interpreted as certainty and uncertainty, respectively, associated with (8) (12) where is used for transpose, and matrix is the inconsistency matrix defined by users. The degree of inconsistency between the th and th states in an FDES is denoted as.if, then an FDES is completely observable, and consistent decisions can be made based on the observations. 2) State-Based Controllability: Controllability of an FDES refers to the achievement of desired state transitions using an appropriate set of events. Supervisory control techniques employ a high-level decision maker (e.g., the Arbiter in the proposed method) that generates the appropriate set of events to achieve the desired state transitions. Hence, the measure of controllability quantifies the achievement of desired states at each state transition. For an FDES, if the current state is and the next state is, then the state-based controllability is measured as (13) where the user-defined matrix shows the undesired state transition between the th and th states in an FDES. The matrix component indicates an undesired transition between the th and th states, whereas indicates a desired one. If, the next state transition is completely controllable, and desired decisions can be made. B. Modeling of FDES The general formulation described in Section III-A is now modified to define the FDES that manipulates the th sen-

4 906 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 Fig. 2. FDES-based behavior modulation. sory information of the th behavior. We specify state 1 and state as the lowest and highest activity states, respectively. Here, is determined using if transition exists when otherwise occurs (14) where represents a fuzzy membership function (MF) that maps sensory information to a membership value over [0,1]. C. Modeling of FDES FDES is also defined using the formulation described in Section III-A. State 1 of refers to the lowest activity state, and state stands for the highest activity state. In, the number of events is equal to the number of states, i.e.,. Here, event is determined using if transition exists when otherwise occurs (15) (16) where is obtained by taking the element-wise average of the output state vectors of FDES. IV. BEHAVIOR-MODULATION MECHANISM Fig. 2 depicts the flow structure of the proposed FDES-based behavior-modulation technique. As explained in Section II, is used to predict the state-based activity of the th behavior. The activity of each behavior modeled by is represented using an activity state vector. The purpose of the Arbiter is to select the most appropriate behaviors based on activity states produced by different FDES components. While considering the selected behaviors, the Arbiter modifies their event matrices to produce the desired activity of a behavior for effective behavior modulation (see Fig. 2). The Arbiter performs the following steps to accomplish the modulation task. Step 1) While comparing the fuzzy membership values of the state vectors, the Arbiter first determines the best possible state for each behavior. Let the states be labeled state state state. Note the th state has the highest activity. The behaviors are grouped according to the highest degree of membership value of being in a given state. As an example, if the highest membership of the state vector is in state, the behavior will be grouped as being in the th activity state. Hence Step Step state where state is the set of behaviors having the highest memberships in state. 2) Find the group of behaviors (from nonempty groups) state having the highest activity state, say. In other words, state is selected if and state 3) Within this group, select the highest priority behavior, say.

5 HUQ et al.: BEHAVIOR-MODULATION TECHNIQUE IN MOBILE ROBOTICS 907 Step 4) The cooperative behaviors of are chosen while using heuristic knowledge. As an example, in force vector modulation, behaviors having directions within of the direction of can be considered as cooperative behaviors. To find them, an equivalence relation is defined, so that their actions do not conflict with the actions proposed by. Hence Step 5) Once the cooperative behaviors are chosen, the Arbiter redefines the set of event matrices as, based on the activity states of the behaviors. The modified event matrices, in turn, alter the state prediction of each. The Arbiter selects to stay in activity state, whereas it enforces to select the lowest activity state 1. Hence, for, the event is enabled to cause a transition to the th state, i.e., for and for if transition exists when otherwise if transition exists when otherwise occurs occurs (17) (18) where. Similarly, for, the event is emphasized that causes a transition to state 1, i.e., (17) is used for, and (18) is used for. The determines the final activity state vector using event matrices. The activity state is then defuzzified, using (19) to modulate the expected actions of (19) Finally, actions of different behaviors are combined using (3). Remark 1: The DES-based coordinator is a special form of the FDES-based coordinator, which requires two modifications: the MFs are changed to support either 1 or 0 membership; event matrix is changed so that the statetransition possibility can be either 0 or 1. Here, is calculated as if transition exists when otherwise where is calculated using (16). occurs and (20) Remark 2: Behavior arbitration is defined as a special form of DES-based coordinator where the following modifications are accomplished: assign, i.e., no cooperative behaviors are allowed with the highest-priority behavior in the most desirable state; to make the only active behavior, we set for. A. Computational Complexity The computational complexity of the modulation process is governed by the process of event and state generation in an FDES, which is on the order of or. Furthermore, the number of states and the number of events are usually equal in an FDES. Therefore, when, the computational complexity of an FDES will be. The overall complexity of the decision process depends on the number of FDES, which is, and the number of FDES, which is. Hence, the overall complexity is ( is counted twice, since FDES is used twice in the decision process). Therefore, to reduce the number of elementary operations in a large behavior-based system, it requires selection of fewer types of sensory information, as well as FDES s with a small number of states. B. Parameter Tuning The proposed FDES-based behavior modulator involves defining the following parameters: priority ranking of the behaviors; degree of observability of each event that reflects the reliability of the sensory information associated with the event; matrix that describes the inconsistency between the actions taken in different activity states; matrix that describes the undesired transitions between the activity states; ranges of MFs. We suggest the following general guidelines to assign priority ranking of the behaviors. Assign the highest priority to a behavior that models dynamic changes in the environment, e.g., avoid-obstacle behavior in mobile robot navigation. Next, assign a higher priority to a behavior that is to be executed first. The degree of observability is a probabilistic measure that describes reliability of sensory data used in an event. It is a ratio of the number of valid sensor readings to the total number of sensor readings taken in an experiment. If an event uses information that consists of more than one sensory data, the product of the reliability of each sensory data is taken as the value of. Matrix defines the inconsistency between the actions taken in different activity states. The inconsistency can be approximated by taking the difference between the estimated modulating factors of two different states. A state vector having possibility 1 of being in the th state modulates an action using the factor. Therefore, the action inconsistency between the th and th states can be approximated as. However, the MFs that

6 908 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 generate state transition possibilities are usually overlapping. This indicates that the activity states are also overlapping. An overlap between two states infers similarity between their actions. Hence, we suggest the following formula as a guideline for defining : (21) where is the overlap between the th and th states in terms of the intersected membership of their corresponding MFs. When two states are not adjacent and the corresponding MFs do not intersect, their actions are related in terms of the intermediate states. Hence, is determined using (22) where. The estimated value of can further be modified according to the expert knowledge. Matrix denotes the undesired transitions between the activity states. The expected state transitions between the th and th states are denoted using, whereas the undesired transitions are specified using. Tuning parameters of MFs is a common issue in FL-based applications. The tuning procedure mainly depends on expert knowledge. We suggest following guidelines to set the ranges of the MFs. The input range of the sensory information associated with an FDES is divided equally to define the MFs when the corresponding states are equally possible. When the possibility of being in a particular state is expected to increase, the range of the corresponding MF is widened. The peak values of the MFs are chosen so that the effects of the states become prominent at those points. An overlap between two MFs is determined according to the intended consistency of actions taken in the corresponding states. V. AN EXAMPLE: MOBILE ROBOT NAVIGATION This section validates the performance of the proposed method in the field of mobile robot navigation. The goal of the navigation task is to integrate global path planning with local motion planning. Global planning optimizes the overall traveled distance, and the local planning ensures safe navigation through obstacles. The experimental environment is mapped using laser range data, as shown in Fig. 3(a). It has a physical dimension of 9 12 m. When translated into an image plane, it represents pixels, and each pixel is scaled to represent a physical area of mm. For global path planning, the navigation is assumed to be in configuration space, i.e., obstacles are outgrown to the width of the robot [see Fig. 3(b)]. A Voronoi diagram is used to extract the safe path network through the known obstacles, and then the search algorithm is employed to specify the safe path between the initial position of the robot and the target. Voronoi vertices between the robot and the target are denoted as subgoals. Once the robot is closer to a subgoal point, the subgoal is discarded and the available closest Voronoi Fig. 3. (a) Original map generated by laser range data. (b) Processed map. Fig. 4. Motor schemas. vertex is redefined as the next subgoal. The following motor schemas [14] (or vector behaviors) are formed to generate the robot s heading direction. Go-to-target is used for path optimization, which is a unit vector directed to the second nearest subgoal with respect to the current robot s position (see Fig. 4). Route-follow is used to follow the safe path, which is a unit vector directed to the nearest subgoal with respect to the current robot s position (see Fig. 4). Avoid-obstacle is a unit vector that is normal to the direction of the resultant repulsive force obtained from the position vectors of obstacles, and it is biased towards the current orientation of the robot. This behavior employs a wall-following approach to avoid collisions (see Fig. 4). The schemas are modulated (or weighed) by,, and, and are coordinated using (23) where is the coordinated behavior and is the commanded heading direction. Fig. 5 shows the overall navigation architecture where the Global module generates the safe path and subgoals, then the Local module forms the schemas, which are further weighed by the Behavior coordination module to

7 HUQ et al.: BEHAVIOR-MODULATION TECHNIQUE IN MOBILE ROBOTICS 909 and 3 denote low, medium, and high activity, respectively. Event is determined using (14), as follows: (24) Fig. 5. Overall navigation architecture. The MFs,, and are defined as shown in Fig. 7. For state-based observability and controllability analysis, we set and, where and are defined as Event is calculated using (15) as follows: Fig. 6. Transition structure of FDESs. generate a safe heading direction. For performance comparison, four different behavior coordinators are employed: 1) unmodulated vector summation, i.e.,, ; 2) DES-based coordinator; 3) behavior arbitration; 4) FDES-based coordinator. The following sections describe formation of different behavior coordinators that generate the modulatory weights. A. FDES-Based Coordinator For this example three behaviors are chosen, namely, go-totarget, route-follow, and avoid-obstacle. The modulation technique defines a priority index for each behavior. In this case, goto-target behavior is given the lowest priority, and is defined as Here, and, where distance to the closest obstacle with respect to the current robot s position. Higher values of infer higher activity of ;, where and are the distances from the current robot s position to the nearest and the second-nearest subgoal, respectively. Higher values of infer lower activity of ;. Higher values of infer lower activity of. The FDES is composed of,, where each FDES has the transition structure shown in Fig. 6. States 1, 2, Here, is determined using (16). The next behavior, route-follow, is given a higher priority than go-to-target, and is defined as Here, and, where,, and. Higher values of and infer higher activity of, whereas lower values of infer higher activity of. The FDES constitutes,. Hence, and have the same transition structures as shown in Fig. 6. Event is determined using (14) with,, and. We have used and. Event is calculated using (15). Finally, avoid-obstacle behavior is given the highest priority, and is defined as Here, and, where,, and. Lower values of infer higher activity of, whereas higher values of and infer higher

8 910 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 Fig. 7. MFs. activity of. The FDES constitutes having the same transition structures as shown in Fig. 6. Event is determined using (14) with, and both. We have used and. Event is calculated using (15). Initial state vectors of all FDESs in are set to [ ]. The membership grade of medium activity is assigned the highest value, which enables a quick state transition from medium to low or high activity, depending on the sensory data. Hence, assignment of the highest value to medium activity reduces the time delay in reaching the expected activity state. For all and, we set and, which indicates that sensory information used in an event is 80% reliable (this value has been determined experimentally). The behavior modulation technique is implemented as described in Section IV, where the Arbiter defines the equivalence relation as if where represents the expected action of the highest priority behavior in the maximum activity state (see Section IV). B. DES-Based Coordinator and Behavior Arbitration A DES-based coordinator is obtained by modifying the FDES-based coordinator as mentioned in Remark 1. Hence, the MFs shown in Fig. 7 are changed in such a way that if, ; otherwise,. Behavior arbitration is obtained using Remark 2. C. Navigation Results An Active Media Pioneer 3-T robot quipped with a sonar ring is used in the experiments. The sonar sensors are employed to obtain range measurements to avoid obstacles. At each decision cycle, the robot is controlled by sending a rotational velocity command and a translational velocity command.for unmodulated coordination, the rotational velocity command is proportional to, i.e., /s, where. is set to 1 for the examples presented in this paper. The translational velocity command is adjusted proportional to rotational velocity, and is measured in mm/s. For unmodulated coordination, the sampling time period (i.e., the length of a decision cycle) is set at ms. For the FDES-based coordinator and its nonfuzzy forms, the velocity commands are weighed by the average observability and controllability of, computed using (12) and (13), respectively. Hence, the velocity commands are modified as follows: s ms mm/s The average observability gives an index of average vagueness of the decision made, and the average controllability gives an index of the average change of undesired behavioral activity (which is caused by the dynamic changes in the environment). The bottom-line idea is that the robot should move slowly, but take samples faster, in case of vague decisions and dynamic changes in the environment. The following performance measures are defined in identifying robustness of different coordinators proposed in this paper. 1) Average distance to the nearest obstacle is defined as mm where is the total number of decision cycles, and is the distance to the nearest obstacle in the th decision cycle. Higher values of indicate safer navigation. 2) Total traveled distance (in mm) is expected to be minimum to optimize the traveled distance. 3) Total navigation time (in ms) is expected to be minimum for fast navigation. 4) Total number of collisions (COL) should be zero for safe navigation. 5) Average rate of change of velocity is defined as s

9 HUQ et al.: BEHAVIOR-MODULATION TECHNIQUE IN MOBILE ROBOTICS 911 Fig. 8. Navigation results in Case I. (a) Navigation scenario. (b) Unmodulated coordinator. (c) DES-based coordinator. (d) Behavior arbitration. (e) FDES-based coordinator. Fig. 9. Weights generated by different coordinators in Case I. (a) DES-based coordinator. (b) Behavior arbitration. (c) FDES-based coordinator. where is the rotational velocity at the th decision cycle, and is the length of the th decision cycle. Here, only is considered, since varies linearly with, and for mm/s, is numerically the same for both velocities. Lower values of indicate consistent velocity of the robot. 6) Average radius of curvature is defined as mm where is the robot coordinate with respect to the world map in the th decision cycle. Higher values of indicate smoother trajectory of navigation. Using the navigation environment shown in Fig. 3, three different scenarios have been created. Case I: A box representing an obstacle is placed on the safe path [Fig. 8(a)]. Case II: A U-shaped obstacle is placed at the initial position of the robot, and a second obstacle is positioned on the way to the target [Fig. 11(a)]. Case III: The obstacles are placed in such a way that the robot is forced to pass through a narrow passage [Fig. 13(a)]. During navigation, the robot updates its current position (i.e., localizes itself) using the gyro-corrected odometry data. It is known that odometry error accumulates with the total traveled distance. Using the gyro-corrected odometry, the position uncertainty found in our experiments is approximately 4% of the traveled distance. With our current experimental settings, this accuracy will affect, in the worst case, less than 0.5 m overall. In a large-scale environment, the accumulated odometry error would be significant, and this may result in highly erroneous heading directions for the go-to-target and route-follow behaviors. This may cause oscillatory velocity commands and unstable robot motion. Hence, to reduce the positional uncertainty in a large-scale environment, simultaneous localization and mapping (SLAM) techniques (e.g., Kalman-filter-based methods [39], [40] or particle-filter-based methods [41], [42]) must be incorporated for online robot localization, where the robot determines its current position by finding the best match of the sensory information in a given map, and then updates the map using the sensory data to incorporate environmental changes. In each experiment, the robot path is traced, and the results are depicted in Figs The performance measures evaluated for each case are summarized in Table I. For the three cases, the

10 912 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 Fig. 10. Average observability and controllability. (a) Case I. (b) Case II. (c) Case III. Fig. 11. Navigation results in Case II. (a) Navigation scenario. (b) Unmodulated coordinator. (c) DES-based coordinator. (d) Behavior arbitration. (e) FDES-based coordinator. Fig. 12. Weights generated by different coordinators in Case II. (a) DES-based coordinator. (b) Behavior arbitration. (c) FDES-based coordinator. video clips corresponding to robot navigation under the FDESdriven coordinator are also attached to this paper (fdescasei. wmv, fdescaseii.wmv, and fdescaseiii.wmv) (see D. Results Comparison 1) Unmodulated Coordinator: In this case, the vectors are weighed equally, and the method has no flexibility to suppress go-to-target and route-follow behaviors to prioritize avoid-obstacle behavior. As a result, the system has experienced the maximum number of collisions in all three cases. In Case I [Fig. 8(b)], the robot has collided with the box at point, and in Cases II and III [Figs. 11(b) and 13(b)], the robot has encountered two collisions at points and. Consequently, this method produces poor performance in. In addition, with increased environmental complexity, this method undergoes higher values of (i.e., inconsistent velocity) and (i.e., irregular robot trajectory). Despite the collisions, the robot can travel faster using this approach, with the shortest traveled distance and navigation time (see Table I). In other words, in the absence of any obstacles or with no changes in environment, this method is easier and faster to implement. 2) DES-Based Coordinator: Table I reveals that all the performance measures associated with the DES-based coordinator degrades, compared with the FDES-based coordinator, as the environmental complexity increases in Cases II and III. This indicates inconsistent navigational performance of the DES-based

11 HUQ et al.: BEHAVIOR-MODULATION TECHNIQUE IN MOBILE ROBOTICS 913 Fig. 13. Navigation results in Case III. (a) Navigation scenario. (b) Unmodulated coordinator. (c) DES-based coordinator. (d) Behavior arbitration. (e) FDES-based coordinator. Fig. 14. Weights generated by different coordinators in Case III. (a) DES-based coordinator. (b) Behavior arbitration. (c) FDES-based coordinator. TABLE I PERFORMANCE MEASURES coordinator in complex environments. Moreover, in Case III, the system has failed to complete the navigational task [Fig. 13(c)]. In this coordination technique, the activity states are mutually exclusive, i.e., the activity of a behavior is either low, medium, or high. Hence, the modulating factor can take only three possible values, i.e., [see Figs. 9(a), 12(a), and 14(a)]. The use of discrete states (or equivalently, hard boundaries in MFs) results in frequent switching between behaviors, and produces a larger value of. According to the transition structure shown in Fig. 6, this approach always produces and. This indicates that the system has no velocity modulation. 3) Behavior Arbitration: This technique has failed to accomplish any navigational task in the experiments [see Figs. 8(d), 11(d), and 13(d)]. This is because once the robot finds an obstacle within a low distance, the control of the robot is completely given to obstacle-avoidance behavior (see Figs. 9(b), 12(b), and 14(b) where is dominating). As a result, the robot starts wall-following without considering the expected actions of go-to-target and route-follow behaviors, and this may cause a failure to reach the target. 4) FDES-Based Coordinator: The robot has successfully completed the navigational tasks using this method, and has

12 914 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 showed consistent performance against environmental complexities [see Figs. 8(e), 11(e), and 13(e)]. The effect of environmental dynamics on performance measures are minimal, compared with other coordinators (see Table I). In this case, the velocities and sampling frequency are modulated by and. Therefore, the robot has the ability to slow down in case of changing environments to assist safe navigation. On the other hand, this may result in slow navigation. However, the modulation through and helps to increase the sampling frequency. As an example, in Case I [Fig. 8(a)], the product becomes considerably lower around the 150th and 300th decision cycles [see Fig. 10(a)]. This refers to the cases where the robot is either approaching or leaving the unmodeled box placed within the navigational path. In Case II, the robot has detected three significant changes in the environment, while using the product at the 1st, 250th, and 450th decision cycles [see Fig. 10(b)]. The changes are corresponding to the points,, and in Fig. 11(e). In Case III, the robot has detected two significant changes in the environment, where the values of at the 180th and 280th decision cycles [see Fig. 10(c)] correspond to the points and, respectively, in Fig. 13(e). This particular feature is established through the state-based observability and controllability. This is the key advantage in using FDES systems over general FL-based navigational systems. Figs. 9(c), 12(c), and 14(c) describe modulating weights. Abrupt changes of are automatically restricted by the transition structure of, and this feature enables producing a consistent velocity and smooth trajectory. As discussed in Section I, reliability includes reactivity, error recovery, and uncertainty handling. Reactivity provides robustness against unpredictable environmental changes. Table I reveals that the FDES-based approach provides consistent values of the performance measures, compared with other coordinators, when the environmental complexity increases (e.g., in Cases II and III). As an instance, only the FDES-based approach provides 100% successful navigation without collisions in the presence of unpredictable obstacles. Error recovery includes continuous monitoring of the behavioral performance and taking corrective actions, if necessary. The proposed FDES-based approach accommodates monitoring of behavioral performance in terms of state-based observability and controllability measures. They describe the uncertainty in sensory information and environmental dynamics. In order to provide corrective actions, we propose velocity and samplingfrequency modulations, which result in slower speed and faster sampling in changing environments. Velocity modulation prevents sharp turning of the robot in the presence of dynamic obstacles. On the other hand, a higher sampling rate enables faster perceptions. As an instance, we have performed a simple experiment where an obstacle periodically comes closer and moves away from the robot. Without velocity and frequency modulation, the robot has experienced oscillation near the obstacle [see Fig. 15(a)]. However, application of velocity and frequency modulation can slow down the robot s movement near obstacles, which provides enough time to analyze the updated sensory information and avoids occurrence of oscillations [see Fig. 15(b)]. Fig. 15. Effect of velocity and frequency modulation. (a) Without modulation. (b) With modulation. Uncertainty handling employs predictions using approximate reasoning to process faulty sensory information. This capability is accomplished in the proposed FDES-based approach using the graded membership (or soft boundary) of FL. For example, in Section V-D.2, we have shown that the use of hard boundaries for event generation in the DES-based approach shows frequent behavioral switching [e.g., see Fig. 12(a)], which is reduced in the FDES-based approach [see Fig. 12(c)]. VI. CONCLUSION This paper devises a novel behavior-modulation technique using FDES. In the proposed method, inclusion of the execution priority of each behavior indicates behavior arbitration, whereas activation of all behaviors with different membership grades addresses command fusion. At each decision cycle, behaviors are categorized either as cooperative or noncooperative. The cooperative behaviors are heavily weighted to control the robot. This categorization of behaviors indicates the nature of hierarchical approaches. Again, the final action is generated using the weighed sum of all behavioral actions as found in nonhierarchical approaches. State-based observability provides a measure of vagueness of the estimated activity state, which, in turn, provides an indication of decision uncertainty for behavior selection. The state-based controllability provides a measure to detect sudden changes in behavioral activity, which can be caused by a dynamic environment. Consequently, analysis of state-based observability and controllability helps make a reliable behavior selection for robot control. The proposed behavior-modulation technique is scalable to a large behavior-based system, since the computational complexity is tractable with respect to an increased number of behaviors. Three mobile robot navigation examples are presented to validate the performance of the proposed method. Navigation results are shown for four behavior coordinators. It has been observed that the performance measures of the FDES-based system are unaffected, even under changing or complex environments. The FDES-based system is able to produce collision-free navigation in all three cases. Unmodulated vector summation is prone to collision in the presence of unknown obstacles. The DES-based coordinator and behavior arbitration

13 HUQ et al.: BEHAVIOR-MODULATION TECHNIQUE IN MOBILE ROBOTICS 915 produce oscillation in behavior selection. Furthermore, both of them suffer the starvation problem, due to selection of the same behavior for several decision cycles. The hard boundaries selected for generating event matrices in the DES-based coordinator result in oscillations. The state-based observability and controllability phenomena in FDES make it possible to produce modulated velocity and sampling frequency. This allows the system to produce safe navigation. Lower values of observability and controllability result in lower velocity and higher sampling frequency. The main drawback of this method is that it requires handling of a larger number of variables and tuning parameters for achieving robust operations, and has restricted adaptability for different surrounding conditions. This requires a sensitivity analysis of the system against different environmental conditions and development of an adaptive mechanism to change parameters. Hence, our future work will focus on developing suitable methods for online parameter adaption. REFERENCES [1] R. A. Brooks, A robust layered control system for a mobile robot, IEEE J. Robot. Autom., vol. RA-2, no. 1, pp , Feb [2] P. Agre and D. Chapman, Pengi: An implemetation of a theory of activity, in Proc. 6th Nat. Conf. Artif. Intell., Seattle, WA, Jul. 1987, pp [3] D. W. Payton, An architecture for reflexive autonomous vehicle control, in Proc. IEEE Int. Conf. Robot. Autom., San Francisco, CA, Apr. 1986, pp [4] L. P. Kaelbling, An architecture for intelligent reactive systems, in Reasoning About Actions and Plans, M. P. Georgeff and A. L. Lansky, Eds. San Mateo, CA: Morgan Kaufmann, 1987, pp [5] P. Maes, Situated agents can have goals, Robot. Auton. Syst., vol. 6, pp , [6] J. Koŝecká and R. Bajcsy, Discrete event systems for autonomous mobile agents, Robot. Auton. Syst., vol. 12, pp , [7] R. Arkin and D. MacKenzie, Temporal coordination of perceptual algorithms for mobile robot navigation, IEEE J. Robot. Autom., vol. 10, no. 3, pp , Jun [8] M. K. Sahota, Action selection in robotics in dynamic environments through inter-behavior bidding, in Proc. 3rd Int. Conf. Simul. Adapt. Behav., Brighton, U.K., 1994, pp [9] S. Kristensen, Sensor planning with Bayesian decision theory, Robot. Auton. Syst., vol. 19, pp , Mar [10] M. Egerstedt, Behavior-based robotics using hybrid automata, in Proc. 3rd Int. Workshop Hybrid Syst.: Comput., Control, 2000, pp [11] R. G. Simmons, Structured control for autonomous robots, IEEE J. Robot. Autom., vol. 10, no. 1, pp , Feb [12] E. Gat and G. Dorais, Robot navigation by conditional sequencing, in Proc. IEEE Int. Conf. Robot. Autom., 1994, pp [13] O. Khatib, Real-time obstacle avoidance for robot manipulator and mobile robots, Int. J. Robot. Res., vol. 5, no. 1, pp , [14] R. C. Arkin, Motor schema-based mobile robot navigation, Int. J. Robot. Res., vol. 8, no. 4, pp , [15] G. Schöner, M. Dose, and C. Engels, Dynamics of behavior: Theory and application for autonomous robot architectures, Robot. Auton. Syst., vol. 16, no. 2, pp , [16] J. K. Rosenblatt, The distributed architecture for mobile navigation, J. Exp. Theoret. Artif. Intell., vol. 9, pp , [17] J. Reikki and J. Roning, Reactive task execution by combining action maps, in Proc. IEEE Int. Conf. Intell. Robot. Syst., Sep. 1997, pp [18] J. Yen and N. Pfluger, A fuzzy logic-based extension to Payton and Rosenblatt s command fusion method for mobile robot navigation, IEEE Trans. Syst., Man, Cybern., vol. 25, no. 6, pp , Jun [19] G. Hoff and G. Bekey, An architecture for behavior coordination learning, in Proc. IEEE Int. Conf. Neural Netw., Nov. 1995, pp [20] E. Tunstel and M. Jamshidi, Fuzzy logic and behavior coordination control strategy for autonomous mobile robot mapping, in Proc. IEEE 4th Int. Conf. Fuzzy Syst., Jun. 1994, pp [21] J. K. Rosenblatt, Optimal selection of uncertain actions by maximizing expected utility, Auton. Robots, vol. 9, pp , [22] F. Hoffmann, An overview on soft computing in behavior-based robotics, in Proc. Int. Fuzzy Syst. Assoc. World Congr., Istanbul, Turkey, 2003, pp [23] P. Pirjanian, Behavior coordination mechanisms State-of-the-art Univ. Southern California, Tech. Rep. IRIS , Oct [24] J. K. Rosenblatt and J. Handler, Architectures for mobile robot control, Adv. Comput., vol. 48, pp , [25] A. Saffiotti, E. Ruspiniand, and K. Konolige, Blending reactivity and goal-directedness in a fuzzy controller, in Proc. IEEE 2nd Int. Conf. Fuzzy Syst., 1993, pp [26] A. Saffiotti, K. Konolige, and E. Ruspini, A multivalued-logic approach to integrating planning and control, Artif. Intell., vol. 76, pp , [27] F. Michaud, Selecting behaviors using fuzzy logic, in Proc. IEEE 6th Int. Conf. Fuzzy Syst., Jul. 1997, pp [28] A. Abreu and L. Correia, Behavior-based decision control in autonomous vehicles: A fuzzy approach using Khepera, in Proc. IEEE 9th Int. Conf. Fuzzy Syst., May 2000, pp [29] A. Bonarini, G. Invernizzi, T. H. Labella, and M. Matteucci, An architecture to coordinate fuzzy behaviors to control an autonomous robot, Fuzzy Sets Syst., vol. 134, no. 1, pp , [30] P. Vadakkepat, O. C. Miin, X. Peng, and T. H. Lee, Fuzzy behaviorbased control of mobile robots, IEEE Trans. Fuzzy Syst., vol. 12, no. 4, pp , Aug [31] E. Tunstel, Coordination of distributed fuzzy behaviors in mobile robot control, in Proc. IEEE Int. Conf. Syst., Man, Cybern., Oct. 1995, pp [32] P. Pirjanian and M. Mataric, A decision-theoretic approach to fuzzy behavior coordination, in Proc. IEEE Int. Symp. Comput. Intell. Robot. Autom., Nov. 1999, pp [33] F. Lin and H. Ying, Modeling and control of fuzzy discrete event systems, IEEE Trans. Syst., Man, Cybern. B, vol. 32, no. 4, pp , Aug [34] F. Lin, H. Ying, X. Luan, R. D. MacArthur, J. A. Cohn, D. C. Barth- Jones, and L. R. Crane, Control of fuzzy discrete event systems and its applications to clinical treatment planning, in Proc. 43rd IEEE Conf. Decision Control, Paradise Island, Bahamas, Dec. 2004, pp [35] X. Luan, H. Ying, F. Lin, R. D. MacArthur, J. A. Cohn, D. C. Barth- Jones, H. Ye, and L. R. Crane, A fuzzy discrete event system for HIV/ AIDS treatment, in Proc. 14th IEEE Int. Conf. Fuzzy Syst., May 2005, pp [36] Y. Cao and M. Ying, Supervisory control of fuzzy discrete event system, IEEE Trans. Syst., Man, Cybern. B, vol. 35, no. 2, pp , Apr [37] D. W. Qiu, Supervisory control of fuzzy discrete event systems: A formal approach, IEEE Trans. Syst., Man, Cybern. B, vol. 35, no. 1, pp , Feb [38] C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems. Norwell, MA: Kluwer, [39] R. Smith, M. Self, and P. Cheeseman, Estimating uncertain spatial relationships in robotics, Auton. Robot Veh., vol. 8, pp , [40] J. E. Guivant and E. M. Nebot, Optimization of the simultaneous localization and map-building algorithm for real-time implementation, IEEE Trans. Robot. Autom., vol. 17, no. 3, pp , Jun [41] D. Fox, W. Burgard, and S. Thrun, Markov localization for mobile robots in dynamic environments, J. Artif. Intell. Res., vol. 11, pp , Nov [42] S. Thrun, A probabilistic online mapping algorithm for teams of mobile robots, J. Robot. Res., vol. 20, no. 5, pp , Rajibul Huq (S 01) received the bachelor s degree in electrical and electronic engineering from Bangladesh University of Engineering and Technology, Dhaka, Bangladesh, in Currently, he is working toward the Ph.D. degree in mobile robotics at the Memorial University of Newfoundland, St. John s, NL, Canada. Previously, he held a teaching position at Ahsanullah University of Science and Technology, Dhaka, Bangladesh. He is currently a research student in the Intelligent Systems Group at C-CORE, Memorial University of Newfoundland. His areas of research and academic interests include behavior-based robotic control, mobile robot navigation, machine learning, and applications of thermal images in robotics.

14 916 IEEE TRANSACTIONS ON ROBOTICS, VOL. 22, NO. 5, OCTOBER 2006 George K. I. Mann received the B.Sc. (Hons.) engineering degree from the University of Moratuwa, Moratuwa, Sri Lanka, the M.Sc. degree in computer-integrated manufacture from Loughborough University, Leicestershire, U.K., and the Ph.D. degree in 1999 from the Memorial University of Newfoundland, St. John s, NL, Canada. After completing his doctoral studies, he was with C-CORE, Memorial University of Newfoundland, for two years as a Research Engineer. In 2001, he joined the Mechanical Engineering Department, Queen s University, Kingston, ON, Canada. Before he came to Canada, he was a Lecturer with the Department of Mechanical Engineering, University of Moratuwa. He is currently an Associate Professor in the Faculty of Engineering and Applied Science, Memorial University of Newfoundland. He also holds the C-CORE Junior Chair position in Intelligent Systems at Memorial University. His main research areas are intelligent control, robotics, and machine vision. Dr. Mann was awarded the NSERC Postdoctoral Fellowship award in Raymond G. Gosine received the B.Eng. degree from the Memorial University of Newfoundland, St. John s, NL, Canada, in 1986, and the Ph.D. degree from Cambridge University, Cambridge, U.K., in From 1990 to 1991, he was a Research Associate with Cambridge University and a Bye-Fellow of Selwyn College, Cambridge, U.K. From 1991 to 1993, he was the NSERC Junior Chair of Industrial Automation and an Assistant Professor, Department of Mechanical Engineering, with the University of British Columbia, Vancouver, BC, Canada. In 1994, he joined the Faculty of Engineering and Applied Science, Memorial University of Newfoundland, and served as the Director for the Intelligent Systems Group at C-CORE in the same university. Currently, he is a Professor and Dean of the Faculty of Engineering and Applied Science with Memorial University of Newfoundland. His research interests are in the areas of industrial automation and application of intelligent systems for resource industries in Canada.