An Overview on Soft Computing in Behavior Based Robotics

An Overview on Soft Computing in Behavior Based Robotics Frank Hoffmann Fakultät Elektrotechnik und Informationstechnik Universität Dortmund D-44221 Dortmund (Germany) E-mail: hoffmann@esr.e-technik.uni-dortmund.de Abstract. This paper provides an overview on the contribution of soft computing to the field of behavior based robotics. It discusses the role of pure fuzzy, neuro-fuzzy and genetic fuzzy rule-based systems for behavior architectures and adaptation. It reviews a number of applications of soft computing techniques to autonomous robot navigation and control. 1 Introduction In many robotic applications, such as autonomous navigation in unstructured environments, it is difficult if not impossible to obtain a precise mathematical model of the robot s interaction with its environment. Even if the dynamics of the robot itself can be described analytically, the environment and its interaction with the robot through sensors and actuators are difficult to capture in a mathematical model. The lack of precise and complete knowledge about the environment limits the applicability of conventional control system design to the domain of autonomous robotics. What is needed are intelligent control and decision making systems with the ability to reason under uncertainty and to learn from experience. It is unrealistic to assume that any learning algorithm is able to learn a complex robotic task, in reasonable learning time starting from scratch without prior knowledge about the task or the environment. The situation is analogous to software design in which the design process is constrained by the three mutually conflicting constraints cost, time and quality. Optimization of one or two of the objectives, often results in a sacrifice on the third objective. In robot learning the three conflicting objectives are complexity of the task, number of training examples or episodes and prior knowledge. Learning a complex behavior in an unstructured environment without prior knowledge requires a prohibitively long exploration and training phase and therefore creates a serious bottleneck to realistic robotic applications. Task complexity can be reduced by a divide and conquer approach, which attempts to break down the overall problem into more manageable subtasks. This course is advocated by hierarchical behavior architectures, in that they separate the design or adaptation of primitive behaviors from the task of learning

a supervisory policy for behavior coordination [3, 12, 14]. The designer biases the learning algorithm towards solutions consistent with the problem specific, prior expert knowledge. Fuzzy control offers a means to integrate explicit domain knowledge in form of linguistic rules that describe the behavioral mapping from perception to action. These rules constitute an initial, sub-optimal behavior that is later refined through experiences gathered from the robot s interaction with the environment [3, 7, 13, 15]. Artificial neural networks and evolutionary algorithms draw inspiration from the capabilities of animals and humans to adapt and learn in dynamic environments under varying conditions, situations and tasks. Fuzzy logic is inspired by the approximate type of reasoning that allows humans to make decisions under uncertain and incomplete information. In the context of the above mentioned trade-offs imposed on robot learning, fuzzy techniques offer a means to sacrifice optimal performance for a reduction in complexity, elimination of unnecessary details and increased robustness of solutions. Section 2 describes hierarchical approaches and methodologies for fuzzy behavior design and coordination. Section 3 recounts neuro-fuzzy techniques for supervised behavior adaptation. Section 4 discusses the role of evolutionary algorithms for learning primitive fuzzy behaviors and behavior coordination schemes. 2 Fuzzy Behaviors for Robot Control Behavior coordination architectures can be divided into two categories: arbitration and command fusion schemes. In arbitration, the selected dominant behavior solely controls the robot until the next decision cycle, whereas the motor commands of the suppressed behaviors are completely ignored. The subsumption architecture is a prototypical representative of behavior arbitration [5]. The alternative to arbitration are command fusion approaches, such as dynamical systems [2], which aggregate the control actions of multiple concurrently active behaviors into a consensual decision. Fuzzy rule-based hierarchical architectures offer an alternative approach to robotic behavior coordination [11, 4, 14, 12, 16, 17]. A set of primitive, self-contained behaviors is encoded by fuzzy rule bases that map perceptions to motor commands. Reactive behaviors in isolation are incapable of performing autonomous navigation in complex environments. However, more complex tasks can be accomplised through combination and cooperation among primitive behaviors. A composite behavior is implemented as a supervisory fuzzy controller that activates and deactivates the underlying primitive behaviors according to the current robot s context and goals. Fuzzy behavior coordination is similar to voting, with the main difference that the action selection process is based on fuzzy inference. A fuzzy coordination mechanism offers the advantage that behaviors are active to a certain degree, rather than being either switched on or off. In contrast to mere voting, the weight with which a behavior contributes to the overall decision depends on its current applicability and desirability.

The advantage of fuzzy based behavior fusion turns into a drawback when competing behaviors issue conflicting control commands. In fuzzy fusion the resulting motor command is caused by the weighted average of the decisions proposed by the currently active behaviors. In case of a conflict among active behaviors, this compromise decision might be sub-optimal or even worse than any of the individual commands. What is needed are extensions to mere fuzzy command fusion schemes capable of resolving conflicts among contradicting actions supported by dissenting behaviors. In context dependent blending of behaviors, originally proposed by Saffiotti et al in [12], higher level supervisory fuzzy rules regulate the activation and deactivation of individual fuzzy behaviors, thereby reintroducing some form of behavior arbitration into fusion. The hierarchical behavior architecture is composed of two distinct layers. On the deliberative level a planner defines a sequence of intermediate goals. At the lower level a fuzzy controller reconciles the abstract goals obtained from the planner, with the constraints and affordances arising from the immediate environmental context. The objective of the coordinating controller is to generate commands that achieve the long-term goals such as reaching a target location while simultaneously satisfying innate goals such as obstacle avoidance. Behavior coordination is achieved by means of supervisory fuzzy rules of the form: if context then behavior. The context describes the desirability and applicability of a particular behavior, for example collision avoidance is desirable when obstacles are close. The context also reflects the needs of higher level goals, for example wall-following is applicable when the robot is located in a corridor and desirable if the goal is to navigate to the other end of the corridor. Yen et al propose an improved defuzzification scheme for fuzzy command fusion in case of conflicting behaviors [16]. Their fusion approach substitutes the center of gravity defuzzification method for multi-modal fuzzy sets by centroid of largest defuzzification. Centroid of largest defuzzification only considers the output fuzzy set with the largest area, whereas fuzzy sets in minor modes are ignored. In a sense, centroid of largestdefuzzification becomes similar to majority voting schemes. It guarantees that the final crisp control command is supported by at least one of the active behaviors. The final decision represents the best compromise among those active behaviors that already vote for a sufficiently similar control action, while ignoring minority votes for deviating actions. Tunstel et al present a fuzzy behavior hierarchy for autonomous navigation and multi-robot control [14]. They distinguish between primitive behaviors that implement a distinct control policy and composite behaviors for behavior coordination. Primitive behaviors interact by means of cooperation and/or competition. Behavior modulation is a process that regulates the activation levels of primitive behaviors, thus determining the impact of a certain behavior on the overall behavior in light of the current situation and goal. The same architecture is applicable not only to multi-behavior coordination but to multi-robot cooperation as well. The authors describe experiments with homogeneous and heterogeneous groups of mobile robots that collectively perform foraging and

area coverage. A central agent acquires and processes sensor information and in turn issues directives to individual robots. Bonarini et al developed a behavior management system for fuzzy behavior coordination [4]. The approach is an extension to the fuzzy behavioral architecture in [12]. Goal-specific strategies are realized by means of conflict resolution among multiple objectives. Behaviors obtain control over the robot according to fuzzy activation conditions and motivations that reflect the agent s goals and situation. An activation condition, named cando condition, is a fuzzy predicate that verifies whether a behavior is applicable in the current context. The fuzzy predicates for motivations, denoted as want conditions, are responsible for behavior coordination as they determine whether an applicable behavior is also desirable for achieving the current goal. The cando and wantdo predicates are matched with the information provided by the sensors and an internal world modeler. The world modeler interprets the sensor information and abstracts higher level features described by means of symbolic concepts such as the door is on the left. A planner module generates goals as symbolic inputs to the behavior coordinator, which are transformed into the corresponding want conditions responsible for selecting and blending actions advanced by primitive behaviors. Pirjanian et al describe an integration of fuzzy logic with multiple objective decision theory for behavior-based control [11]. The quality of an action-objective pair reflects the desirability of an alternative with regard to a specific goal. The behavior coordination scheme is based on the notion of Pareto-optimality and satisficing solutions. A satisficing solution is the subset of feasible alternatives that achieve each of the objectives to a sufficient degree. The Pareto-optimal subset contains those solutions that can not be further improved on any objective without simultaneously sacrificing the quality of at least one other objective. The multi-objective behavior coordination mechanism first determines the set of feasible actions, second removes inferior feasible solutions to obtain the set of Pareto-optimal actions, third incorporates subjective knowledge such as weights, priorities and goals to find a set of satisficing alternatives and finally determines the most preferred action based on additional criteria. The process systematically condenses the set of possible solutions by incrementally imposing additional decision criteria. Based on the behavioral objectives, the alternative actions are categorized into permissible, Pareto-optimal, satisficing and preferred actions. The authors compare a fuzzy command fusion approach with their method on a mobile robot navigation task, that involves the coordination among three basic behaviors, obstacle avoidance, maintain target heading and move fast forward. The emergent coordinated behaviors are evaluated in terms of safety, velocity, number of successful runs and combined deviation from the ideal values. The results demonstrate that the multiple-objective behavior coordination outperforms the fuzzy approach in terms of safety, velocity, compromise and number of successful runs. Zhang et al present a fuzzy modular framework for integrating deliberative with reactive strategies [17]. The approach explicitly takes the connection between sensing, planing and control into account. Deliberative behaviors depend

on geometric information about the environment and achieve their objectives by means of path planning. Reactive behaviors avoid explicit plans altogether and instead rely on feed-back control to meet their demands. The integration of the beneficial features of deliberative and reactive strategies contributes to solve autonomous navigation tasks in partially unknown environments. The fuzzy control scheme blends deliberative subgoal oriented behaviors with a local obstacle avoidance behavior that copes with unknown objects. 3 Neuro-Fuzzy Systems From a historic perspective, neuro-fuzzy systems became the first representative of hybridization in soft computing. Neuro-fuzzy systems incorporate the knowledge representation of fuzzy logic with the learning capabilities of artificial neural networks. Both methodologies are concerned with the design of intelligent systems albeit from different directions. The power of neural networks stems from the distributed processing capability of a large number of computationally simple elements. In contrast fuzzy logic is closer related to reasoning on a higher level. Pure fuzzy systems do not possess the capabilities of learning, adaptation or distributed computing that characterize neural networks. On the other hand, neural networks lack the ability to represent knowledge in a manner comprehensible to humans, a key feature of fuzzy rule based systems. Neuro-fuzzy systems bridge the gap between both methodologies, as they synthesize the adaptation mechanisms of neural networks with the symbolic components of fuzzy inference systems, namely membership functions, fuzzy connectives, fuzzy rules and aggregation operators. Ahrns et al apply neuro-fuzzy control to learn a collision avoidance behavior [1]. Their approach relies on reinforcement learning for behavior adaptation. The learner incrementally adds new fuzzy rules as learning progresses and simultaneously tunes the membership functions of the fuzzy RBF-network. Godjavec et al present a neuro-fuzzy approach to learn an obstacle avoidance and wall-following behavior on a small size robot [6]. Their scheme allows it to seed an initial behavior with expert rules, which are refined throughout the learning process. During training the robot is controlled either by a human or a previously designed controller. The recorded state-action pairs serve as training examples during supervised learning of neuro-fuzzy control rules. The robot successfully imitates the demonstrated behavior after 1500 iterations. Ye et al propose a neuro-fuzzy system for supervised and reinforcement based learning of an obstacle avoidance behavior [15]. The scheme follows a two-stage tuning approach, in a first phase supervised learning determines the coarse structure of input-output membership functions. The second reinforcement learning stage fine-tunes the output membership functions. The authors emphasize that the pre-tuned rule-base obtained in the initial supervised learning phase facilitates the subsequent reinforcement learning phase. In conjunction with an improved exploration scheme, the reduced search space complexity accelerates the learning process and leads to more robust control behaviors.

4 Genetic Fuzzy Systems Evolutionary robotics is concerned with the design of intelligent systems with life-like properties by means of simulated evolution [10]. The basic idea is to automatically synthesize behaviors that enable the robot to perform useful tasks in complex environments. The evolutionary algorithm searches through the space of parameterized controllers that map sensory perceptions to control actions, thus realizing a specific robotic behavior. The evolutionary algorithm maintains and improves a population of candidate behaviors by means of selection, recombination and mutation. A scalar fitness function evaluates the performance of the resulting behavior according to the robot s task or mission. The approaches in evolutionary robotics can be categorized according to the control structures that represent the behavior and the parameters of the controller that undergo adaptation. The following presentation is restricted to evolutionary optimization of fuzzy behaviors. Notice, that combinations of evolutionary with neural techniques even though beyond the scope of this article, play a highly visible role within evolutionary robotics [9, 10]. Neuro-genetic approaches are of particular interest from an ethological perspective as they provide a framework to integrate individual based learning with population based evolutionary adaptation. The evolutionary algorithm either encodes the synaptic weights, the topology or the learning rules of the neural network. Several authors proposed evolutionary algorithms for learning and tuning of fuzzy controllers for robotic behaviors [3, 7, 8, 13]. In a genetic fuzzy system the evolutionary algorithm evolves a population of parameterize fuzzy controllers. Candidate controllers share the same fuzzy inference mechanism, but establish different input-output mappings according to their genetically encoded knowledge base. The evolutionary algorithm adapts all or part of the components that constitute the knowledge base, namely membership functions, scaling factors and fuzzy rules. The same mechanism is used for the adaptation of primitive behaviors [3, 8] and learning of supervisory fuzzy controllers [4, 7, 13]. The performance of the fuzzy controller while it governs the behavior of the robot is described by a scalar fitness function. Those fuzzy controllers that demonstrate behaviors of higher fitness compared to their competitors are selected as parents for reproduction. Novel candidate behaviors are generated through recombination and mutation of parent fuzzy rules and membership functions. The cycle of selection, reproduction and fitness evaluation progressively leads to improved fuzzy controllers that enable the robot to achieve the behavior design goals in its environment. In the evolutionary learning of fuzzy rules (ELF) scheme proposed in [3] each chromosome represents a single fuzzy rule rather than an entire rule-base. The population is partitioned into sub-populations, such that fuzzy rules that trigger in similar situations belong to the same sub-population. Rules within one subpopulation compete with each other, whereas rules in different sub-populations cooperate to achieve the behavior objectives. A reinforcement scheme, similar to temporal difference learning, distributes the reward among those fuzzy rules that were active during the past training episode. The fitness of a rule is adjusted

according to the observed reward and the extent to which the rule contributed to the control actions taken in that episode. Since competition is restricted to subpopulations, fuzzy rules are selected based on the action of the state for which they trigger, rather than the desirability of the state itself. A covering mechanism guarantees that new fuzzy rules are automatically generated for states that are currently not matched by any other rule in the population. ELF has been successfully applied to learn individual robotic behaviors, the coordination of primitive behaviors within a single autonomous agent and cooperation among multiple agents. Tunstel et al apply genetic programming for off-line identification of supervisory fuzzy rules that coordinate primitive fuzzy behaviors [13]. Composite behaviors are evaluated according to their success in orchestrating the primitive behaviors such that they eventually navigate the robot to its goal location. The fitness is averaged over several trials in different simulated environments in order to obtain robust and reliable behaviors. The generalization capability of the highest scoring behaviors is tested in a more general environment unrelated to the test environments used during evolution. The off-line evolved behaviors are transfered to the physical robot for verification. The robot successfully navigates within close proximity to the goal. In [7] Hagras et al present a fuzzy classifier systems that utilizes a genetic algorithm for online-learning of a goal seeking and a wall following behavior. The fuzzy classifier system maintains a rule cache to store suitable fuzzy rules, that might be of benefit in future situations and which serve to seed the initial population when a new genetic learning process is invoked. This technique substantially speeds up the learning process, thus making the entire approach feasible for on-line learning of robotic behaviors. In one scenario, the robot acquired the goal seeking and wall following behavior after 96 seconds. 5 Conclusions Soft computing approaches are preferable over conventional control system design, for problems that are difficult to describe by analytical models. Autonomous robotics is such a domain in which knowledge about the environment is inherently weak and incomplete. Therefore, the features of fuzzy control, neural networks and evolutionary algorithms are of particular benefit to the type of problems emerging in behavior based robotics. The references in the text on fuzzy control, neuro-fuzzy and genetic-fuzzy approaches in robotics do not claim to be complete but rather intend to provide insight into the general utility of soft-computing techniques for behavior based robotics. Fuzzy behavior hierarchies, neuro-fuzzy and genetic fuzzy system are valuable methodologies for the design and adaptation of complex robotic behaviors. The knowledge representation of fuzzy rule based systems combined with the learning capabilities of neural networks and evolutionary algorithms opens a promising avenue towards more intelligent and robust robotic systems. Soft computing techniques, such as design based on expert knowledge, hierarchical

behavior architectures, fuzzy behavior command fusion and evolutionary and neural adaptation contribute to one of the long term goal in robotics, namely that intelligent, autonomous robots demonstrate and acquire complex skills in unstructured real-world environments. References 1. I. Ahrns, J. Bruske, G. Hailu, and G. Sommer. Neural fuzzy techniques in sonarbased collision avoidance. In Soft Computing for Intelligent Robotic Systems, pages 185 214. Physica, 1998. 2. P. Althaus, H. I. Christensen, and F. Hoffmann. Using the dynamical system approach to navigate in realistic real-world environments. In IROS 2001, 2001. 3. A. Bonarini. Evolutionary learning of fuzzy rules: competition and cooperation. In W. Pedrycz, editor, Fuzzy Modelling: Paradigms and Practice, pages 265 284. Kluwer Academic Press, Norwell, MA, 1996. 4. A. Bonarini, G. Invernizzi, Th. H. Labella, and M. Matteucci. An architecture to coordinate fuzzy behaviors to control an autonomous robot. Fuzzy Sets and Systems, 134(1):101 115, 2003. 5. R. Brooks. A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, RA-2(1):14 23, 1986. 6. J. Godjavec and N. Steele. Neuro-fuzzy control for basic mobile robot behaviors. In Fuzzy Logic Techniques for Autonomous Vehicle Navigation, pages 97 117. Springer, 2000. 7. H.Hagras, V. Callaghan, and M.Colley. Learning fuzzy behaviour co-ordination for autonomous multi-agents online using genetic algorithms and real-time interaction with the environment. In Fuzzy IEEE, 2000. 8. F. Hoffmann. Evolutionary algorithms for fuzzy control system design. Proceedings of the IEEE, 89(9):1318 33, September 2001. 9. J.A. Meyer. Evolutionary approaches to neural control in mobile robots. In IEEE Int. Conference on Systems, Man and Cybernetics, 1998. 10. S. Nolfi and D. Floreano. Evolutionary Robotics The Biology, Intelligence, and Technology of Self-Organizing Machines. MIT Press, 2000. 11. P. Pirjanian and M. Mataric. A decision theoretic approach to fuzzy behavior coordination. In CIRA 99. 1999. 12. A. Saffiotti, K. Konolige, and E.H. Ruspini. A multivalued-logic approach to integrating planning and control. Artificial Intelligence, 76(1-2):481 526, 1995. 13. E. W. Tunstel. Fuzzy-behavior synthesis, coordination, and evolution in an adaptive behavior hierarchy. In Fuzzy Logic Techniques for Autonomous Vehicle Navigation, pages 205 234. Springer, 2000. 14. E. W. Tunstel, M. A. A. de Oliveira, and S. Berman. Fuzzy behavior hierarchies for multi-robot control. Int. Journal of Intelligent Systems, 17:449 470, 2002. 15. C. Ye, N. H. C. Yung, and D. Wang. A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance. IEEE Transactions on Systems, Man and Cybernetics Part B, 33(1):17 27, 2003. 16. J. Yen and N. Pfluger. A fuzzy logic based extension to payton and rosenblatt s command fusion method for mobile robot navigation. IEEE Transactions on Systems, Man and Cybernetics, 25(6):971 978, 1995. 17. J. Zhang and A. Knoll. Integrating deliberative and reactive strategies via fuzzy modular control. In Fuzzy Logic Techniques for Autonomous Robot Navigation. Springer, 2000.