Application of a model of instrumental conditioning to mobile robot control

Lisa M. Saksida; D. S. Touretzky

doi:10.1117/12.287655

22 September 1997 Application of a model of instrumental conditioning to mobile robot control

Lisa M. Saksida, D. S. Touretzky

Proceedings Volume 3209, Sensor Fusion and Decentralized Control in Autonomous Robotic Systems; (1997) https://doi.org/10.1117/12.287655
Event: Intelligent Systems and Advanced Manufacturing, 1997, Pittsburgh, PA, United States

Abstract

Instrumental conditioning is a psychological process whereby an animal learns to associate its actions with their consequences. This type of learning is exploited in animal training techniques such as 'shaping by successive approximations,' which enables trainers to gradually adjust the animal's behavior by giving strategically timed reinforcements. While this is similar in principle to reinforcement learning, the real phenomenon includes many subtle effects not considered in the machine learning literature. In addition, a good deal of domain information is utilized by an animal learning a new task; it does not start from scratch every time it learns a new behavior. For these reasons, it is not surprising that mobile robot learning algorithms have yet to approach the sophistication and robustness of animal learning. A serious attempt to model instrumental learning could prove fruitful for improving machine learning techniques. In the present paper, we develop a computational theory of shaping at a level appropriate for controlling mobile robots. The theory is based on a series of mechanisms for 'behavior editing,' in which pre-existing behaviors, either innate or previously learned, can be dramatically changed in magnitude, shifted in direction, or otherwise manipulated so as to produce new behavioral routines. We have implemented our theory on Amelia, an RWI B21 mobile robot equipped with a gripper and color video camera. We provide results from training Amelia on several tasks, all of which were constructed as variations of one innate behavior, object-pursuit.

Citation Download Citation

Lisa M. Saksida and D. S. Touretzky "Application of a model of instrumental conditioning to mobile robot control", Proc. SPIE 3209, Sensor Fusion and Decentralized Control in Autonomous Robotic Systems, (22 September 1997); https://doi.org/10.1117/12.287655

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available