Robot Learning Letter of Intent

Size: px

Start display at page:

Download "Robot Learning Letter of Intent"

Rudolph Small
5 years ago
Views:

1 Research Proposal: Robot Learning Letter of Intent BY ERIK BILLING

2 SUMMARY The proposed project s aim is to further develop the learning aspects in Behavior Based Control (BBC) and Sequence Learning (SL) by combining these approaches with resent findings in the field of cognitive and neural science.while the project primarily will focus on developing models for learning in situated intelligent systems, the intention is to implement models that are of interest both for computing science, cognitive science and neuroscience. PROJECT AIM Robots are slowly entering human domains. Recent years witnessed a promising development of intelligent robotic systems, ranging from autonomous lawn movers and vacuum cleansers to reconnaissance airplanes and naturally walking humanoids. However impressive these systems are, they generally require elite groups to setup and control. In contrast, customer focused robots tend to work out of the box, with limited support for customization and optimization. Bellow, the proposed project will be described on a conceptual level, including a short motivation and identification of research areas that I find relevant and interesting. Suggestions for precise research directions, as well as a full project description, are found in the complete research proposal. Why learning? A common feature of all living organisms is the ability to adapt to its environment. Any species that do not, will sooner or later die or be replaced by another species, more fit to the environment. When designing an intelligent robot, a programmer has to explicitly implement the robot s behaviors so that the system knows what to do in each situation. For a robot aimed to work in the real world, this is practically impossible. A programmer has no chance to predict every possible situation the robot may experience, and can not prepare the robot for these situations. From this point of view, the ability to learn from experience, adapt to the present environment, stands out as a key feature for success, not only for living organisms, but also for robots, or any situated intelligent system. Even though the robot must be able to learn, it does not necessarily posts that it should figure out everything by itself. A human that likes to know a new thing generally ask, or quietly watches someone who knows, simply because 2

3 this is the easiest way to gain new knowledge. Similarly, it is probably a good way to let a robot learn from others, be able to ask, watch and imitate. In a very general sense, the programming of the robot may be viewed as a kind of learning. Even though the robot does not explicitly ask, the programmer does tell the robot what to do. But the end user of a robot designed for, let's say, home environments, can not be expected to program the robot by hand. Instead, the programming, or teaching, of the robot has to be done in a more natural way for the user. The learning problem may in this way be viewed as an interface problem. How to transfer the knowledge possessed by the user to the system? These aspects of learning are well covered in the literature under labels such as Learning from demonstration and Learning from imitation. This approach has generally taken support by the assumption that people imitate each other, that the parent demonstrates how to do, and the child replicates. While this with no doubt is true in many situations, it has been reported that parents instinctively imitate their infants [Meltzoff, 1996], not the other way. One hypothesis is that the information gained from the parents reactions is used by the child to create a mapping from perception to production. In turn, it helps the child predict the consequences of its own actions. These findings have previously been used in the field of robotics to create natural face and body positions for a life-like robot [Brooks et. al. 2004]. Infants does not only watch there parents, but also make an effort to watch themselves. It appears that infants instinctively watch their own hands while moving [van der Meer et al., 1995], [van der Meer, 1997], which may be a way for the child to, by closing the visual-manual loop, explore the relationship between commands and movements [von Hofsten, 2004]. Recent research on eye-hand coordination proposes that humans learn new motor tasks in multiple stages [R. S. Johansson, 2005]. Subjects were asked to control a cursor on a screen with a bimanual joystick while tracking their eye movements. The joystick had no obvious mapping to the cursor movements which resulted in an initially very poor performance by the subjects, when chasing squares with the cursor. In the early stage of training, the performance remained poor and no obvious skill was gained. After a while it suddenly increased fast for a period of about 400 seconds. From this point, the rapid skill acquisition stopped and performance increased much slower. In the first stage of learning, when the performance remained poor, the gaze clearly tracked the cursor movements. It is proposed that this reflects a mapping between manual actions and cursor movements. In the second stage, when the performance increased dramatically, the gaze behavior successively changes from pursuing to prediction. In the later part of the second stage, as well as in the third stage of learning, gaze clearly marked future goal positions for the cursor. 3

4 While the application of these results in the field of robotics my seam unclear, they do stage the importance of prediction in learning of human motor control. At the same time it is striking how rarely prediction is mentioned in conjunction with learning in intelligent systems. It is my strong belief that prediction is a key to successive learning, in humans and animals, as well as robots. We must learn to learn, and so must the robot. Behavior-Based Control Behavior-based control (BBC) [Arkin, 1998] is a distributed approach to robotic control that is based on multiple simultaneously executed processes, or behaviors. The definition of a behavior is taken from the behaviorist school of psychology, which simply states a behavior as a response to a stimulus. Implemented in the robotic domain, a behavior may be more clearly described as a process that take information from the sensors, and/or memory, and use the actuators to achieve or uphold some certain goal. Behaviors are typically reactive, meaning that they must possess real-time responses. To avoid the interaction problems often appearing in hybrid systems, all behaviors must operate on a compatible time scale, i.e., slowloop and general modeling behaviors are avoided. This allows the behaviorbased systems to maintain stable, real-time responses similar to thaws of purely reactive systems (RS), without binding to the limitations of RS. A key feature of BBS is the ability to construct distributed representations that allow complex control structures to be modeled in a divide-and-conquer manner, where the parts, the behaviors, are often very loosely connected. However, the ability of BBS is yet underused, mainly because behaviors lack an abstract, symbolic, representation. This makes it hard to model more complex behaviors that, for example, require greater temporal distribution. A suitable symbolic representation would allow the behaviors to be combined at a higher level [Nicolescu 2003]. One attempt to approach this problem is the Hierarchical Abstract Behavior Based architecture (HABB) [Nicolescu, 2003]. This approach divides the behaviors into two parts, one abstract perceptual component, and one acting component. The abstract behavior encapsulates preconditions and goals while the active part, referred to as the primitive behavior, performs actions leading to the goal under the given conditions. An abstract behavior depends not only on preconditions in terms of sensor data, but also on other behaviors. Behaviors depending on each other can be encapsulated into behavior networks that in turn can work as a single component in another behavior network. This allows behaviors to be combined and reused in much more powerful ways than in the original BBS approach. 4

5 Behavior based systems often state there close relation to biological systems. On a behavioral level the biological connection is made mainly in terms of reactive policies inspired by animal behavior. On a topological level BBC has a modular and parallel structure similar to many neurological systems, such as the human brain. Despite the obvious effort to create systems with biologically plausible structure, the differences are seldom discussed. Many differences are of course obvious and may not constitute a meaningful subject for discussion, but other may be a mistake to neglect. One key property of BBC is real time response. Fast connections between sensors and actuators result in robust systems that instantly react to changes in the environment. Biological systems on the other hand have very few sensor-motor loops faster than a couple of hundred milliseconds. Still, the biological systems do not suffer the problems of slow-loop systems so often stated in the field of robotics. Even though the slowness of biological systems does not imply that slowloop systems are a good approach for robotic control, it may at least be interesting to evaluate the possibility. By further looking at how biology solved the problem of time-delays in motor control, we may very well find new ways to solve the same problem in robots. 5

Behavior Architectures

Behavior Architectures 5 min reflection You ve read about two very different behavior architectures. What are the most significant functional/design differences between the two approaches? Are they compatible