Robotics | More Info | Notesale | Buy and Sell Study Notes Online | Extra Student Income | University Notes

Search for notes by fellow students, in your own course and all over the country.

Browse our notes for titles which look like what you need, you can preview any of the notes via a sample of the contents. After you're happy these are the notes you're after simply pop them into your shopping cart.

My Basket

Buy These Notes

You have nothing in your shopping cart yet.

Title: Robotics
Description: Robotic

Buy These Notes Preview

Document Preview

Extracts from the notes are below, to see the PDF you'll receive please use the links above

I N T R O D U C T I O N

T O

AI ROBOTICS

R O B I N R
...
Arkin, editor

Behavior-Based Robotics, Ronald C
...
Murphy, 2000

Introduction
to
AI
Robotics
Robin R
...
No part of this book may be reproduced in any form by any
electronic or mechanical means (including photocopying, recording, or information
storage and retrieval) without permission in writing from the publisher
...

Printed and bound in the United States of America
...
Murphy
...
cm
...
A Bradford Book
...

ISBN 0-262-13383-0 (hc
...
paper)
1
...
2
...
I
...
II
...
M865 2000
00-033251
629
...
and Carlyle Ramsey, Monroe Swilley, Chris Trowell

Brief Contents

I Robotic Paradigms
1
2
3
4
5
6
7
8

II Navigation
9
10
11
12

1

From Teleoperation To Autonomy
13
The Hierarchical Paradigm
41
Biological Foundations of the Reactive Paradigm
67
The Reactive Paradigm
105
Designing a Reactive Implementation
155
Common Sensing Techniques for Reactive Robots
195
The Hybrid Deliberative/Reactive Paradigm
257
Multi-agents
293

315

Topological Path Planning
325
Metric Path Planning
351
Localization and Map Making
375
On the Horizon
435

Contents

Preface

xvii

I Robotic Paradigms

1

1 From Teleoperation To Autonomy
1
...
2
1
...
4

1
...
6
1
...
8
1
...
3
...
4
...
4
...
5
...
5
...
1
2
...
2
...
2
...
2
...
3
2
...
5
2
...
7
2
...
9

Closed World Assumption and the Frame Problem
53
Representative Architectures
54
2
...
1
Nested Hierarchical Controller
54
2
...
2
NIST RCS
57
2
...
3
Evaluation of hierarchical architectures
59
Advantages and Disadvantages
61
Programming Considerations
62
Summary
62
Exercises
63
End Notes
64

3 Biological Foundations of the Reactive Paradigm
3
...
2
3
...
4

3
...
6
3
...
8
3
...
1
...
1
...
2
...
3
...
3
...
4
...
4
...
4
...
4
...
5
...
1
4
...
2
...
2
...
2
...
3

4
...
5
4
...
7
4
...
3
...
3
...
4
...
4
...
4
...
4
...
4
...
4
...
4
...
4
...
1
5
...
3
5
...
5

5
...
7
5
...
2
...
2
...
2
...
5
...
5
...
5
...
5
...
5
...
1
6
...
1
...
3

Designing a Sensor Suite
202
6
...
1
Attributes of a sensor
203
6
...
2
Attributes of a sensor suite
206
6
...
4
...
4
...
5 Proximity Sensors
210
6
...
1
Sonar or ultrasonics
210
6
...
2
Infrared (IR)
216
6
...
3
Bump and feeler sensors
217
6
...
6
...
6
...
6
...
6
...
7 Range from Vision
231
6
...
1
Stereo camera pairs
232
6
...
2
Light stripers
235
6
...
3
Laser ranging
239
6
...
4
Texture
241
6
...
9 Summary
250
6
...
11 End Notes
254
7 The Hybrid Deliberative/Reactive Paradigm
7
...
2

7
...
4

7
...
2
...
2
...
3
...
3
...
4
...
4
...
5
...
6

Model-Oriented Architectures
277
7
...
1
Saphira
278
7
...
2
Task Control Architecture (TCA)
280
7
...
8 Evaluation of Hybrid Architectures
284
7
...
10 Summary
288
7
...
12 End Notes
291
8 Multi-agents
8
...
2

8
...
4
8
...
6

8
...
8
8
...
2
...
2
...
2
...
6
...
6
...
1
9
...
3

9
...
3
...
3
...
4
...
4
...
5

9
...
7
9
...
5
...
5
...
5
...
1 Objectives and Overview
351
10
...
3 Cspace Representations
354
10
...
1 Meadow maps
354
10
...
2 Generalized Voronoi graphs
357
10
...
3 Regular grids
358
10
...
4 Quadtrees
359
10
...
5 Wavefront Based Planners
365
10
...
7 Summary
371
10
...
9 End Notes
374
11 Localization and Map Making

375

11
...
2 Sonar Sensor Model
378
11
...
3
...
3
...
3
...
4 Dempster-Shafer Theory
386
11
...
1 Shafer belief functions
387
11
...
2 Belief function for sonar
389
11
...
3 Dempster’s rule of combination
390
11
...
4 Weight of conﬂict metric
394
11
...
5
...
5
...
6 Comparison of Methods
403
11
...
1 Example computations
403
11
...
2 Performance
411
11
...
3 Errors due to observations from stationary robot
11
...
4 Tuning
413
11
...
7
...
7
...
8 Exploration
424
11
...
1 Frontier-based exploration
425
11
...
2 Generalized Voronoi graph methods
427
11
...
10 Exercises
431
11
...
1 Overview
435
12
...
3 Applications and Expectations
442
12
...
5 Exercises
445
12
...
The
reader is not expected to have taken a course in artiﬁcial intelligence (AI),
although the book includes pointers to additional readings and advanced
exercises for more advanced students
...
These programming structures lend themselves well to laboratory
exercises on commercially available robots, such as the Khepera, Nomad 200
series, and Pioneers
...

A background in digital circuitry is not required, although many instructors
may want to introduce laboratory exercises for building reactive robots from
kits such as the Rug Warrior or the Handy Board
...
Although machine
perception is a separate ﬁeld of endeavor, the book covers enough computer
vision and sensing to enable students to embark on a serious robot project
or competition
...
Part I deﬁnes what are
intelligent robots and introduces why artiﬁcial intelligence is needed
...
The bulk of the seven chapters is concerned with the
Reactive Paradigm and behaviors
...
Also, Part I covers the coordination
and control of teams of multi-agents
...
The book concludes with an overview of
how advances in computer vision are now being integrated into robots, and
how successes in robots are driving the web-bot and know-bot craze
...
The guiding principle has
been to include only material that clearly illuminates a speciﬁc topic
...
Behavior-based
Robotics 10 provides a thorough survey of the ﬁeld and should be an instructor’s companion
...
I’d like to thank my parents (I think this is the equivalent of scoring a goal and saying “Hi Mom!” on national TV) and my family
(Kevin, Kate, and Allan)
...
I have tried to maintain the intellectual rigor of his course and excellent book while trying to distill the material
for a novice audience
...
David Kortenkamp suggested that I write this book after using my course notes for a
class he taught, which served as a very real catalyst
...
I would like to speciﬁcally thank Leslie Baski,
John Blitch, Glenn Blauvelt, Ann Brigante, Greg Chavez, Aaron Gage, Dale
Hawkins, Floyd Henning, Jim Hoffman, Dave Hershberger, Kevin Gifford,
Matt Long, Charlie Ozinga, Tonya Reed Frazier, Michael Rosenblatt, Jake
Sprouse, Brent Taylor, and Paul Wiebe from my CSM days and Jenn Casper,
Aaron Gage, Jeff Hyams, Liam Irish, Mark Micire, Brian Minten, and Mark
Powell from USF
...
Karen Sutherland and her robotics class at the University
of Wisconsin-LaCrosse (Kristoff Hans Ausderau, Teddy Bauer, Scott David
Becker, Corrie L
...
Colby III, Mark Erickson, Chris Falch, Jim Fick, Jennifer Fleischman, Scott Galbari, Mike Halda,
Brian Kehoe, Jay D
...
Zander) painstakingly reviewed an early draft of the book and made extensive
suggestions and added review questions
...

Ken always comes to my rescue
...
Most of the case studies came from work or through equipment
sponsored by NSF
...
Michael Mason also provided encouragement,
in many forms, to hang in there and focus on education
...
Katherine and especially Judy were very
patient and nice— despite knowing that I was calling with Yet Another Crisis
...
Chris Manning provided the
A
L TEX 2" style ﬁles, with adaptations by Paul Anagnostopoulos
...

Besides the usual suspects, there are some very special people who indirectly helped me
...

They taught me that learning isn’t a place like a big university but rather a
personal discipline
...
He worked hard to make sure I could spend the time on this
book without missing time with the kids or going crazy
...
I dedicate the
book to these four men who have inﬂuenced my professional career as much
as any academic mentor
...
These paradigms characterize the ways in which intelligence is organized in robots
...

What Are Robots?
One of the ﬁrst questions most people have about robotics is “what is a robot?” followed immediately by “what can they do?”
In popular culture, the term “robot” generally connotes some anthropomorphic (human-like) appearance; consider robot “arms” for welding
...
” The word “robot” came into the popular consciousness on January 25, 1921, in Prague with the ﬁrst performance
of Karel Capek’s play, R
...
R
...
37 In R
...
R
...
Capek described the workers as robots, a term derived from the Czech

Part I

INTELLIGENT ROBOT

3

word “robota” which is loosely translated as menial laborer
...

This attitude towards robots has disastrous consequences, and the moral of
the rather socialist story is that work deﬁnes a person
...
Three classic ﬁlms, Metropolis (1926), The Day the Earth
Stood Still (1951), and Forbidden Planet (1956), cemented the connotation that
robots were mechanical in origin, ignoring the biological origins in Capek’s
play
...
Industrial automation conﬁrmed this suspicion as robot arms were installed which would go
through the motions of assembling parts, even if there were no parts
...
The notion of
anthropomorphic, mechanical, and literal-minded robots complemented the
viewpoint taken in many of the short stories in Isaac Asimov’s perennial favorite collection, I, Robot
...
Susan Calvin, or two erstwhile trouble shooters,
Powell and Donovan, diagnosing robots who behaved logically but did the
wrong thing
...
While robots are mechanical, they don’t have to
be anthropomorphic or even animal-like
...
And the HelpMate Robotics,
Inc
...

It should be clear from Fig
...
1 that appearance does not form a useful definition of a robot
...

“Intelligent” implies that the robot does not do things in a mindless, repetitive way; it is the opposite of the connotation from factory automation
...
It also emphasizes that a robot is not the same as a computer
...

b
...
1 Two views of robots: a) the humanoid robot from the 1926 movie
Metropolis (image courtesty Fr
...
(Photograph courtesy of the National Institute for Standards
and Technology
...
A computer doesn’t move around under its own power
...

Autonomy means that a robot can adapt to changes in its environment (the
lights get turned off) or itself (a part breaks) and continue to reach its goal
...
Even after losing one camera (eye) and having all external coverings (skin, ﬂesh) burned off, it continued to pursue its target (Sarah Connor)
...
9, 1996, article in the Denver Post
...

Part I

5

What are Robotic Paradigms?
PARADIGM

ROBOTIC PARADIGMS

ROBOT PARADIGM
PRIMITIVES

A paradigm is a philosophy or set of assumptions and/or techniques which characterize an approach to a class of problems
...
No one paradigm is right;
rather, some problems seem better suited for different approaches
...
There are problems that could be solved
by differentiating in cartesian (X Y Z ) coordinates, but are much easier to
solve if polar coordinates (r ) are used
...
Both produce the correct answer, but one
takes less work for certain problems
...
Therefore,
knowing the paradigms of AI robotics is one key to being able to successfully
program a robot for a particular application
...

There are currently three paradigms for organizing intelligence in robots:
hierarchical, reactive, and hybrid deliberative/reactive
...

1
...
The functions of a robot can be divided
into three very general categories
...
If the function is
taking in information (either from sensors or its own knowledge about
how the world works) and producing one or more tasks for the robot to
perform (go down the hall, turn left, proceed 3 meters and stop), that function is in the PLAN category
...
2mps)
...
I
...

2
...
How much a person or robot or animal is inﬂuenced by what it
senses
...
In some paradigms, sensor information is restricted
to being used in a speciﬁc, or dedicated, way for each function of a robot;

6

Part I

ROBOT PRIMITIVES

INPUT

OUTPUT

SENSE

Sensor data

Sensed information

PLAN

Information (sensed
and/or cognitive)

Directives

ACT

Sensed information
or directives

Actuator commands

Figure I
...

SENSING
ORGANIZATION IN
ROBOT PARADIGMS

in that case processing is local to each function
...

Overview of the Three Paradigms

H IERARCHICAL
PARADIGM

In order to set the stage for learning details, it may be helpful to begin with
a general overview of the robot paradigms
...
I
...

The Hierarchical Paradigm is the oldest paradigm, and was prevalent from
1967–1990
...
I
...
This was based on an introspective view of how people think
...
” (Unfortunately, as many cognitive psychologists now know, introspection is not always a good way of getting an accurate assessment of
a thought process
...
) Under the Hierarchical
Paradigm, the robot senses the world, plans the next action, and then acts
(SENSE, PLAN, ACT)
...
At each step,
the robot explicitly plans the next move
...
Constructing generic global world models

7

Part I

SENSE

PLAN

ACT

a
...

PLAN

ACT

SENSE

c
...
3 Three paradigms: a
...
) Reactive, and c
...

turns out to be very hard and brittle due to the frame problem and the need
for a closed world assumption
...
I
...
I
...
Unfortunately, the ﬂow of events ignored biological evidence that sensed information can be directly coupled to an action, which is why the sensed information input is blacked out
...
4 Another view of the Hierarchical Paradigm
...
It was heavily used in robotics starting
in 1988 and continuing through 1992
...
The Reactive Paradigm
was made possible by two trends
...
Another was the rapidly decreasing
cost of computer hardware coupled with the increase in computing power
...

The Reactive Paradigm threw out planning all together (see Figs
...
3b and
I
...
It is a SENSE-ACT (S-A) type of organization
...

If the sensor is directly connected to the action, why isn’t a robot running
under the Reactive Paradigm limited to doing just one thing? The robot has
multiple instances of SENSE-ACT couplings, discussed in Ch
...
These couplings are concurrent processes, called behaviors, which take local sensing
data and compute the best action to take independently of what the other
processes are doing
...
5 The reactive paradigm
...
The robot will do a combination of both behaviors, swerving off course temporarily at a 45 angle to
avoid the collision
...

While the Reactive Paradigm produced exciting results and clever robot
insect demonstrations, it quickly became clear that throwing away planning
was too extreme for general purpose robots
...
F
...
It explained how some animals
accomplished tasks, but was a dead end in explaining the entire range of
human intelligence
...
As a result,
the Reactive Paradigm serves as the basis for the Hybrid Deliberative/Reactive
Paradigm, shown in Fig
...
3c
...
Under the Hybrid Paradigm, the
robot ﬁrst plans (deliberates) how to best decompose a task into subtasks
(also called “mission planning”) and then what are the suitable behaviors to
accomplish each subtask, etc
...
This type of organization is PLAN, SENSE-ACT (P, S-A),
where the comma indicates that planning is done at one step, then sensing
and acting are done together
...
Sensor data gets routed
to each behavior that needs that sensor, but is also available to the planner

10

Part I

ROBOT PRIMITIVES

PLAN

SENSE-ACT
(behaviors)

INPUT

OUTPUT

Information (sensed
and/or cognitive)

Directives

Sensor data

Actuator commands

Figure I
...

for construction of a task-oriented global world model
...
e
...
Each function performs computations at its own rate; deliberative
planning, which is generally computationally expensive may update every
5 seconds, while the reactive behaviors often execute at 1/60 second
...

Architectures

ARCHITECTURE

Determining that a particular paradigm is well suited for an application is
certainly the ﬁrst step in constructing the AI component of a robot
...
In order to visualize how to apply these paradigms to real-world
applications, it is helpful to examine representative architectures
...

What is an architecture? Arkin offers several deﬁnitions in his book, Behavior-Based Robots
...
Following Mataric, 89 an
architecture provides a principled way of organizing a control system
...
Following Dean and Wellman, 43 an architecture describes a set of architectural components and how they interact
...
It also is interested in
the principles and rules of thumb for connecting these components together
...
There is no “right” design for a house, although most houses share the
same components (kitchens, bathrooms, walls, ﬂoors, doors, etc
...
This is similar to cars
designed by different manufacturers
...
The internal combustion (IC) engine car is a paradigm (as contrasted to the paradigm of an
electric car)
...
The car manufacturers may make slight modiﬁcations to the architecture for sedans, convertibles, sport-utility vehicles,
etc
...
The point is: by studying representative robot
architectures and the instances where they were used for a robot application,
we can learn the different ways that the components and tools associated
with a paradigm can be used to build an artiﬁcially intelligent robot
...
This skill
will save both time spent on re-inventing the wheel and avoid subtle problems that other people have encountered and solved
...
The set that will be used in this book is adapted from BehaviorBased Robotics: 10
MODULARITY

NICHE TARGETABILITY

1
...
Niche targetability: how well does it work for the intended application?

PORTABILITY

3
...
Robustness: where is the system vulnerable, and how does it try to reduce that vulnerability?
Note that niche targetability and ease of portability are often at odds with
each other
...
The generic structures, however, often introduce undesirable computational and storage overhead, so in
practice the designer must make trade-offs
...
Ch
...
Ch
...
Ch
...
It also offers deﬁnitions of primitive robot behaviors
...
5 provides
guidelines and case studies on designing robot behaviors
...
At this point, the reader should be
almost able to design and implement a reactive robot system, either in simulation or on a real robot
...
Ch
...
Ch
...
Up until this point, the emphasis is towards programming a single
robot
...
8 concludes the section by discussing how the principles of the
three paradigms have been transferred to teams of robots
...

While the SENSE, PLAN, ACT primitives are generally accepted, some researchers
are suggesting that a fourth primitive be added, LEARN
...

1

From Teleoperation To Autonomy

Chapter Objectives:
Deﬁne intelligent robot
...

Be able to describe the difference between telepresence and semi-autonomous
control
...

1
...
At
ﬁrst, that may appear redundant; aren’t robots intelligent? The short answer is “no,” most robots currently in industry are not intelligent by any
deﬁnition
...

The chapter begins with an overview of artiﬁcial intelligence and the social
implications of robotics
...
1
...
One
way of viewing robots is that early on in the 1960’s there was a fork in the
evolutionary path
...
The key to success in
industry was precision and repeatability on the assembly line for mass production, in effect, industrial engineers wanted to automate the workplace
...
As a result, the emphasis was

14

1 From Teleoperation To Autonomy

AI Robotics

planetary rovers

vision

telesystems

telemanipulators

Industrial
Manipulators

manufacturing

1960

1970

Figure 1
...

placed on the mechanical aspects of the robot to ensure precision and repeatability and methods to make sure the robot could move precisely and
repeatable, quickly enough to make a proﬁt
...
The standards for mass production would make
it more economical to devise mechanisms that would ensure parts would be
in the correct place
...

Robotics for the space program took a different fork, concentrating instead
on highly specialized, one-of-a-kind planetary rovers
...

Consider that on Apollo 17, astronaut and geologist Harrison Schmitt found
an orange rock on the moon; an orange rock was totally unexpected
...
Since it couldn’t be preprogrammed to handle all possible contingencies, it had to be able to notice
its environment and handle any problems that might occur
...
And the need to sense and adapt to a partially unknown
environment is the need for intelligence
...
In fact, as will be seen in Ch
...
So
what happened when someone had an application for a robot which needed

1
...
These mechanisms are
generally referred to under the umbrella term: teleoperation
...
In practice, intelligent robots such as the Mars Sojourner are controlled with some form of
teleoperation
...

The chapter concludes by visiting the issues in AI, and argues that AI is imperative for many robotic applications
...
However, it has served as a reasonable
patch
...
Manufacturing is now shifting to a “mass customization” phase, where companies which can economically make short runs of
special order goods are thriving
...
As a result, AI techniques are migrating to industrial manipulators
...
2
ARTIFICIAL
INTELLIGENCE

How Can a Machine Be Intelligent?
The science of making machines act intelligently is usually referred to as artiﬁcial intelligence, or AI for short
...
One of the ﬁrst textbooks on AI deﬁned it as “the study
of ideas that enable computers to be intelligent,” 143 which seemed to beg the
question
...
” 120 This
deﬁnition is interesting because it implies that once a task is performed successfully by a computer, then the technique that made it possible is no longer
AI, but something mundane
...
making
machines that will be proud of us
...
As Roger Penrose notes
in his book, The Emperor’s New Mind: “Nevertheless, it would be fair to
say that, although many clever things have indeed been done, the simulation of anything that could pass for genuine intelligence is yet a long way
off
...
As a result of such
vehement criticisms, many researchers often label their work as “intelligent
systems” or "knowledge-based systems” in an attempt to avoid the controversy surrounding the term “AI
...
AI
robotics is the application of AI techniques to robots
...
An article in the May 5, 1997 issue
of Newsweek, “Actually, Chess is Easy,” discusses why robot applications
are more demanding for AI than playing chess
...
So by studying AI robotics, a reader interested in AI is getting exposure to the general issues in
AI
...
3

THE

3 D’ S

What Can Robots Be Used For?
Now that a working deﬁnition of a robot and artiﬁcial intelligence has been
established, an attempt can be made to answer the question: what can intelligent robots be used for? The short answer is that robots can be used for just
about any application that can be thought of
...
Or as the well-worn joke among roboticists
goes, robots are good for the 3 D’s: jobs that are dirty, dull, or dangerous
...

Many of the same technologies developed for the nuclear industry for processing radioactive ore is now being adapted for the pharmaceutical indus-

1
...

Another example of a task that poses signiﬁcant risk to a human is space
exploration
...
, but only at great economic expense
...

Worse yet, having people in space necessitates more people in space
...
Even more people would have to be
sent into space, requiring a larger structure
...
A study
by Dr
...
51 (Interestingly enough, a team of three robots
which did just this were featured in the 1971 ﬁlm, Silent Running, as well as
by a young R2D2 in The Phantom Menace
...

The most obvious use of robots is manufacturing, where repetitious activities in unpleasant surroundings make human workers inefﬁcient or expensive to retain
...
One reason that welding is now largely robotic is
that it is an unpleasant job for a human (hot, sweaty, tedious work) with
a low tolerance for inaccuracy
...
One such activity is janitorial work, especially maintaining public
rest rooms, which has a high turnover in personnel regardless of payscale
...

Agriculture is another area where robots have been explored as an economical alternative to hard to get menial labor
...
The De-

18

1 From Teleoperation To Autonomy

partment of Mechanical and Material Engineering at the University of Western Australia developed a robot called Shear Majic capable of shearing a live
sheep
...
Possibly the most creative use of robots for agriculture is a mobile automatic milker developed in the Netherlands and in Italy
...

Finally, one of the most compelling uses of robots is for humanitarian purposes
...
Humanitarian land demining is a challenging task
...
Various types of robots are being tested
in the ﬁeld, including aerial and ground vehicles
...
3
...
Don’t robots put people out of work? One of the pervasive
themes in society has been the impact of science and technology on the dignity of people
...
”
Robots appear to amplify the tension between productivity and the role of
the individual
...
People who object to robots, or technology in general, are often called Luddites, after Ned Ludd, who is often credited with leading a
short-lived revolution of workers against mills in Britain
...
Mechanization of the weaving process
changed the jobs associated with weaving, the status of being a weaver (it
was a skill), and required people to work in a centralized location (like having your telecommuting job terminated)
...
After escalating vi-

1
...
The rebelling workers were persecuted
...
The
connotation is that Luddites have an irrational fear of technological progress
...
The HelpMate Robotics, Inc
...
Cleaning ofﬁce buildings is menial and
boring, plus the hours are bad
...
The Robotics Industries Association, a trade group, produces annual
reports outlining the need for robotics, yet possibly the biggest robot money
makers are in the entertainment and toy industries
...
While the sheep
shearing robots in Australia were successful and were ready to be commercialized for signiﬁcant economic gains, the sheep industry reportedly rejected the robots
...
But human shearers accidently kill
several sheep, while the robots had a demonstrably better rate
...
4

A Brief History of Robotics
Robotics has its roots in a variety of sources, including the way machines are
controlled and the need to perform tasks that put human workers at risk
...
The theory for the nuclear bomb had
existed for a number of years in academic circles
...

One of the ﬁrst problems that the scientists and engineers encountered
was handling and processing radioactive materials, including uranium and

20

1 From Teleoperation To Autonomy

Figure 1
...
The upper portion of the device is placed
in the ceiling, and the portion on the right extends into the hot cell
...
)

TELEMANIPULATOR

plutonium, in large quantities
...
One of the ﬁrst solutions
was the glove box
...
A person
stood (or sat) behind a leaded glass shield and stuck their hands into thick
rubberized gloves
...

But this was not an acceptable solution for highly radioactive materials,
and mechanisms to physically remove and completely isolate the nuclear
materials from humans had to be developed
...

A popular telemanipulator is shown in Fig
...
2
...
Telemanipulators are similar in principle to the power gloves now used in computer games, but much
harder to use
...
Often the opera-

1
...
Likewise, the telemanipulators had challenges in providing
force feedback so the operator could feel how hard the gripper was holding
an object
...
Operators might take years of
practice to reach the point where they could do a task with a telemanipulator
as quickly as they could do it directly
...
The USA and Soviet Union also entered into a nuclear arms race
...
Machines that
looked more like and acted like robots began to emerge, largely due to advances in control theory
...

1
...
1

INDUSTRIAL
MANIPULATOR

Industrial manipulators
Successes with at least partially automating the nuclear industry also meant
the technology was available for other applications, especially general manufacturing
...
37 The two most common types of robot technology that have evolved
for industrial use are robot arms, called industrial manipulators, and mobile
carts, called automated guided vehicles (AGVs)
...
The emphasis
in industrial manipulator design is being able to program them to be able
to perform a task repeatedly with a high degree of accuracy and speed
...
1
...
The MOVEMASTER arm has ﬁve degrees
of freedom, because it has ﬁve joints, each of which is capable of a single
rotational degree of freedom
...
3 An RT3300 industrial manipulator
...
)

BALLISTIC CONTROL
OPEN LOOP CONTROL

CLOSED - LOOP
CONTROL

bow, and wrist), two of which are complex (shoulder and wrist), yielding six
degrees of freedom
...
Rapidly
moving around a large tool like a welding gun introduces interesting problems, like when to start decelerating so the gun will stop in the correct location without overshooting and colliding with the part to be welded
...
Another interesting problem is
the joint conﬁguration
...
Redundant degrees
of freedom means there are multiple ways of moving the joints that will accomplish the same motion
...
In ballistic control, the position trajectory
and velocity proﬁle is computed once, then the arm carries it out
...
In order to accomplish a precise task with ballistic control,
everything about the device and how it works has to be modeled and ﬁgured
into the computation
...
4 A Brief History of Robotics

a
...

Figure 1
...
) the robot arm and b
...

FEEDBACK

and a new trajectory and proﬁle is computed and executed, then modiﬁed on
the next update, and so on
...

In general, if the structural properties of the robot and its cargo are known,
these questions can be answered and a program can be developed
...
The dynamics (how the mechanism moves
and deforms) and kinematics (how the components of the mechanism are
connected) of the system have to be computed for each joint of the robot, then
those motions can be propagated to the next joint iteratively
...
To move the gripper in Fig 1
...
The coordinate transformations
often have singularities, causing the equations to perform divide by zeros
...

One simplifying solution is to make the robot rigid at the desired velocities,
reducing the dynamics
...
However, a robot is made rigid by making it

24

1 From Teleoperation To Autonomy

TEACH PENDANT

AUTOMATIC GUIDED
VEHICLES

heavier
...
Another simplifying solution is to
avoid the computations in the dynamics and kinematics and instead have the
programmer use a teach pendant
...
The robot remembers these motions and
creates a program from them
...
Many programmers have to
direct the robot to perform delicate tasks, and have to get physically close
to the robot in order to see what the robot should do next
...
You don’t want to have your head next to a 2 ton robot arm if it
suddenly spins around!
Automatic guided vehicles, or AGVs, are intended to be the most ﬂexible conveyor system possible: a conveyor which doesn’t need a continuous belt or
roller table
...
For example, an AGV might
receive a bin containing an assembled engine
...
As it returned, it might be diverted by the central computer and instructed to pick up a defective part and take it to another area of the shop for
reworking
...
The AGV has
to know where it is, plan a path from its current location to its goal destination, and to avoid colliding with people, other AGVs, and maintenance
workers and tools cluttering the factory ﬂoor
...
Various solutions converged on creating a trail for the AGV to
follow
...
Unfortunately, changing the path of an AGV required ripping up the
concrete ﬂoor
...
Another method is to put down a strip of photochemical tape for
the vehicle to follow
...
Regardless of the guidance method, in
the end the simplest way to thwart an AGV was to something on its path
...
A

1
...
If the AGV
did have range sensors, it would stop for anything
...
Even better from a disgruntled worker’s perspective, many AGVs
would make a loud noise to indicate the path was blocked
...

From the ﬁrst, robots in the workplace triggered a backlash
...
This was particularly true of manufacturing facilities which were unionized
...
Future models were modiﬁed
to have some mechanisms to prevent malicious acts
...
A black factory is a factory that has no lights turned on because there are no workers
...

But two unanticipated trends undermined industrial robots in a way that
the Luddite movement could not
...
Often industrial
manipulators were applied to the wrong application
...
The plant was
built with a high degree of automation, and the designers wrote numerous
articles on the exotic robot technology they had cleverly designed
...
While the plant’s failure wasn’t the fault of robotics,
per se, it did cause many manufacturers to have a negative view of automation in general
...
Customers were demanding “mass customization
...
(Mass customization is also referred to as “agile
manufacturing
...
The lack of adaptability, combined with concerns over
worker safety and the Luddite effect, served to discourage companies from
investing in robots through most of the 1990’s
...

b
...
5 Motivation for intelligent planetary rovers: a
...
) Astronaut Jim Irwin stopping the lunar rover as it slides down a hill on Apollo 15
...
)

1
...
2

Space robotics and the AI approach
While the rise of industrial manipulators and the engineering approach to
robotics can in some measure be traced to the nuclear arms race, the rise
of the AI approach can be said to start with the space race
...
Kennedy announced that United States would put a man
on the moon by 1970
...
There were concerns about the Soviets setting up military bases
on the Moon and Mars and economic exploitation of planetary resources
...
And even then, it would most likely be
with experimental spacecraft, posing a risk to the human astronauts
...
Fig
...
5a shows astronaut John
Young on Apollo 16 collecting samples with a lunar rake
...

Planetary rovers were a possible solution, either to replace an astronaut or
assist him or her
...

Because of the time delays, a human would be unable to safely control a rover
over the notoriously poor radio links of the time, even if the rover went very

1
...
Therefore, it would be desirable to have a robot that was autonomous
...
, and radio back the results
...
The robot would receive commands from Earth
to explore a particular region
...
The robot
might even be smart enough to regulate its own energy supply, for example,
by making sure it was sheltered during the planetary nights and to stop what
it was doing and position itself for recharging its solar batteries
...

Getting a mobile robot to the level of a trained dog immediately presented
new issues
...
Fig
...
5b shows astronaut Jim Irwin rescuing the lunar rover during an extra-vehicular activity (EVA) on Apollo 15
as it begins to slide downhill
...
Furthermore, an autonomous rover would have
no one to rescue it, should it make a mistake
...
If the robot was moving based on a map taken
from a telescope or an overhead command module, the map could still contain errors or at the wrong resolution to see certain dangers
...
What if the robot did
something broke totally unexpected or all the assumptions about the planet
were wrong? In theory, the robot should be able to diagnose the problem
and attempt to continue to make progress on its task
...

Clearly, developing a planetary rover and other robots for space was going to require a concentrated, long-term effort
...
They were in a position to take advantage of the outcome of the Dartmouth Conference
...
DARPA was interested in hearing what the potential

28

1 From Teleoperation To Autonomy

uses for computers were
...
This in turn suggested that computers might mimic the capabilities of animals and humans sufﬁciently for a planetary rover to survive
for long periods with only simple instructions from Earth
...
NASA also introduced the notion
that AI robots would of course be mobile, rather than strapped to a factory
ﬂoor, and would have to integrate all forms of AI (understanding speech,
planning, reasoning, representing the world, learning) into one program—a
daunting task which has not yet been reached
...
5
TELEOPERATION

LOCAL
REMOTE

SENSORS
DISPLAY
COMMUNICATION LINK

Teleoperation
Teleoperation is when a human operator controls a robot from a distance (tele
means “remote”)
...
The operator and robot have
some type of master-slave relationship
...
1
...

The control interface could be a joystick, virtual reality gear, or any number of innovative interfaces
...

The local must have some type of display and control mechanisms, while the
remote must have sensors, effectors, power, and in the case of mobile robots,
mobility
...
g
...
g
...

Therefore, the sensors which acquire information about the remote location,
the display technology for allowing the operator to see the sensor data, and
the communication link between the local and remote are critical components
of a telesystem
...
5 Teleoperation

REMOTE
Sensor

LOCAL
Display
Control

Communication

Mobility
Effector
Power

Figure 1
...
(Photographs courtesy of Oak Ridge National Laboratory
...
One example of teleoperation is the exploration of underwater sites such as the Titanic
...
R
...
141
Humans can also perform dextrous manipulation (e
...
, screwing a nut on a
bolt), which is very difﬁcult to program a manipulator to do
...
7 Sojourner Mars rover
...
)

Another example is the Sojourner robot (shown in Fig
...
7) which explored
Mars from July 5 to September 27, 1997, until it ceased to reply to radio commands
...
It is important that any
unusual rocks or rock formations (like the orange rock Dr
...
Humans are particularly adept at
perception, especially seeing patterns and anomalies in pictures
...
Humans are also adept
at problem solving
...
When the
petals of the lander opened, an airbag was in the way of Sojourner
...
That type of problem solving is extremely difﬁcult for the
current capabilities of AI
...
Many tasks
are repetitive and boring
...
Now imagine trying to control the car while only looking
through a small camera mounted in front
...
5 Teleoperation

COGNITIVE FATIGUE

SIMULATOR SICKNESS

TELEOPERATION
HEURISTIC

PREDICTIVE DISPLAYS

31

because of the limited ﬁeld of view; essentially there is no peripheral vision
...
Most people quickly experience cognitive fatigue; their attention wanders and they may
even experience headaches and other physical symptoms of stress
...
141
Another disadvantage of teleoperation is that it can be inefﬁcient to use for
applications that have a large time delay
...
Or, an unanticipated event such as a rock fall might occur
and destroy the robot before the teleoperator can see the event and command
the robot to ﬂee
...
A teleoperation task which took 1 minute for a teleoperator to guide a
remote to do on the Earth might take 2
...
142 Fortunately, researchers have made some progress with
predictive displays, which immediately display what the simulation result of
the command would be
...
A recent example of an application of teleoperation are unmanned aerial vehicles (UAV)
used by the United States to verify treaties by ﬂying overhead and taking
videos of the ground below
...
In this case of the Darkstar UAV (shown in Fig
...
8), human operators were available to assume teleoperation control of the vehicle should
it encounter problems during take-off
...
Darkstar no
...
As a result, it earned the unofﬁcial nickname “Darkspot
...
The Predator unmanned aerial vehicle has
been used by the United States for veriﬁcation of the Dayton Accords in
Bosnia
...
Other UAVs have teams composed of up to four teleoperators plus a
ﬁfth team member who specializes in takeoffs and landings
...
8 Dark Star unmanned aerial vehicle
...
)

TASK
CHARACTERISTICS

erators may have over a year of training before they can ﬂy the vehicle
...

According to Wampler, 142 teleoperation is best suited for applications
where:
1
...

2
...

3
...

4
...

5
...

6
...

1
...
1

Telepresence
An early attempt at reducing cognitive fatigue was to add more cameras with
faster update rates to widen the ﬁeld of view and make it more consistent
with how a human prefers to look at the world
...
5 Teleoperation

TELEPRESENCE

VIRTUAL REALITY

1
...
2
SEMI - AUTONOMOUS
CONTROL
SUPERVISORY CONTROL

SHARED CONTROL

33

for many applications because of limited bandwidth
...
In
these instances, the physical restrictions on how much and how fast information can be transmitted result in image updates much slower than the rates
human brains expect
...
So adding more cameras only exacerbates the
problem by adding more information that must be transmitted over limited
bandwidth
...
Telepresence aims for what is popularly called virtual
reality, where the operator has complete sensor feedback and feels as if she
were the robot
...
If the operator pushes on a joystick for the robot to
move forward and the wheels are slipping, the operator would hear and feel
the motors straining while seeing that there was no visual change
...
It also still requires
one person per robot
...

Semi-autonomous control
Another line of research in teleoperation is semi-autonomous control, often
called supervisory control, where the remote is given an instruction or portion of a task that it can safely do on its own
...

In continuous assistance systems, the teleoperator and remote share control
...
If the teleoperator delegates the task to the robot,
the human must still monitor to make sure that nothing goes wrong
...
The operator can
relax (relatively) while the robot arm moves into the speciﬁed position near
a panel, staying on alert in case something goes wrong
...

Shared control helps the operator avoid cognitive fatigue by delegating boring, repetitive control actions to the robot
...
6

human to perform delicate operations
...

An alternative approach is control trading, where the human initiates an action for the robot to complete autonomously
...

The overall scheme is very much like a parent giving a 10-year old child a
task to do
...
g
...
They have a common deﬁnition (clean room means
go to the bedroom, make the bed, and empty the wastebaskets)
...
g
...
Control
trading assumes that the robot is capable of autonomously accomplishing
certain tasks without sharing control
...
A single operator could control multiple
robots because they would not require even casual monitoring while they
were performing a task
...
Data such as video
images need to be transferred only when the local is conﬁguring the remote
for a new task, not all the time
...
5 minute delay in communication is irrelevant; the robot either wrecked itself or it didn’t
...
Which brings us back
to the need for artiﬁcial intelligence
...
It was primarily
programmed for traded control, where the geologists could click on a rock
and Sojourner would autonomously navigate close to it, avoiding rocks, etc
...
A difﬁculty with most forms
of shared control is that it is assumed that the human is smarter than the
robot
...

The Seven Areas of AI
Now that some possible uses and shortcomings of robots have been covered,
it is motivating to consider what are the areas of artiﬁcial intelligence and

1
...
The Handbook of Artiﬁcial Intelligence 64 divides up the ﬁeld into seven main areas: knowledge
representation, understanding natural language, learning, planning and problem
solving, inference, search, and vision
...
Knowledge representation
...
Suppose a robot
is scanning a pile of rubble for a human
...

Of course, what happens if only a portion of the human is visible?
2
...
Natural language is deceptively challenging, apart from the issue of recognizing words which is now being
done by commercial products such as Via Voice and Naturally Speaking
...
The story goes that after Sputnik
went up, the US government needed to catch up with the Soviet scientists
...
Therefore,
the US decided to use these newfangled computers to create translation
programs
...
It was given the proverb: the spirit is willing, but the ﬂesh is weak
...

LEARNING

3
...
Imagine a robot that could be programmed by just watching a
human, or by just trying the task repeatedly itself
...
Planning and problem solving
...
One of the earliest childhood fables, the
Three Pigs and the Big, Bad Wolf, involves two unintelligent pigs who
don’t plan ahead and an intelligent pig who is able to solve the problem
of why his brothers’ houses have failed, as well as plan an unpleasant
demise for the wolf
...
Inference
...
Consider a planetary rover looking at a dark region on the
ground
...
Assume that depth information can’t be extracted from

36

1 From Teleoperation To Autonomy

the camera
...
g
...

SEARCH

6
...
Search doesn’t necessarily mean searching a large physical space
for an object
...
Deep Blue, the computer that beat the World Chess master Gary
Kasparov, won by searching through almost all possible combinations of
moves to ﬁnd the best move to make
...

VISION

7
...
Vision is possibly the most valuable sense humans have
...
As such, AI researchers have pursued creating vision
systems both to improve robotic actions and to supplement other work in
general machine intelligence
...
But that is not the
case
...

Breakthroughs in methods for planning (operations research types of problems) came after the paradigm shift to reactivity in robotics in the late 1980’s
showed how unpredictable changes in the environment could actually be exploited to simplify programming
...
These programs are called
software agents: autonomous programs which can interact with and adapt to
their world just like an animal or a smart robot
...
Even animation is being
changed by advances in AI robotics
...
4
...
7 Summary

1
...
Industrial robots has concentrated on control theory issues, particularly solving the dynamics and kinematics of a robot
...
AI robotics has concentrated on how a mobile
robot should handle unpredictable events in an unstructured world
...

Teleoperation arose as an intermediate solution to tasks that required automation but for which robots could not be adequately programmed to handle
...
Telepresence techniques attempt to
create a more natural interface for the human to control the robot and interpret what it is doing and seeing, but at a high communication cost
...

1
...
1
List the four attributes for evaluating an architecture
...

Exercise 1
...

Exercise 1
...

Exercise 1
...
5
What is a Luddite?
Exercise 1
...

Exercise 1
...

Exercise 1
...

Exercise 1
...

Exercise 1
...
Give
at least two examples of potentially good applications for teleoperation not covered
in the chapter
...
11

[World Wide Web]

Search the world wide web for sites that permit clients to use a robot remotely (one
example is Xavier at Carnegie Mellon University)
...

Exercise 1
...

Exercise 1
...
Harrison “Jack” Schmitt is a vocal proponent for space mining of Near Earth
Objects (NEOs) such as mineral-rich asteroids
...
Search
the web for more information on space mining, and give examples of why robots are
needed
...
14

[Programming]

(This requires a robot with an on-board video camera and a teleoperation interface
...
Now looking only at the output of the video camera, repeat the obstacle course
...
Which
viewpoint led to faster completion of the course? Fewer collisions? Why?

1
...
15

39
[Advanced Reading]

Read “Silicon Babies,” Scientiﬁc American, December 1991, pp 125-134, on the challenges of AI robotics
...

Exercise 1
...

Exercise 1
...
The military uses a teleoperated vehicle (in reality, NASA
Jet Propulsion Laboratory’s Hazbot) to ﬁrst go through the star gate and test the environmental conditions on the other side
...

Exercise 1
...
The movie has
several nerve wracking scenes as the scientists try to telemanipulate an unknown,
deadly organism as fast as possible without dropping it
...
9

End Notes
Finding robots in the popular press
There is no one-stop-shopping publication or web site for robot applications
...
Popular
Mechanics and Popular Science often contain short pieces on new robots and applications, although those short bits are often enthusiastically optimistic
...
In addition to surﬁng the web, annual proceedings from the IEEE
International Conference on Robotics and Automation (ICRA) and IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) contain scientiﬁc articles
on the newest robots emerging from university and government laboratories
...
by S
...
Tzafestas, Marcel Dekker Inc, NY, 1991
...
The work is a
bit dated now, but gives some feel for the variety of applications
...
” His impressive resume includes having formed Unimation
...
Engleberger participates in many robotics

40

1 From Teleoperation To Autonomy

forums, where his sartorial style (he always wears a bow tie) and verbal style (stridently pro-robotics, and that his company should get more money) make him easily
recognizable
...
This entertaining CD provides a very detailed, crossreferenced look at robots as a theme in science ﬁction and a lengthy list (and review)
of movies and books with robots in them
...
It was directed by Douglas Trumbull who gained fame
for his work in special effects, including 2001: A Space Odyssey, Close Encounters of the
Third Kind, and Blade Runner
...
The robots and how they interact with Dern
and their environment are very realistic and consistent with AI robotics
...
Well worth watching for the
‘bots, especially if the audience is curious about the hippie movement
...
That Marvin is widely assumed to be named after the
cantankerous AI researcher, Dr
...

Have Spacesuit, Will Travel
...

2

The Hierarchical Paradigm

Chapter Objectives:
Describe the Hierarchical Paradigm in terms of the three robot paradigms
and its organization of sensing
...

Solve a simple navigation problem using Strips, given a world model, operators, difference table, and difference evaluator
...

Understand the meaning of the following terms as applied to robotics:
precondition, closed world assumption, open world, frame problem
...

List two advantages and disadvantages of the Hierarchical Paradigm
...
1

Overview
The Hierarchical Paradigm is historically the oldest method of organizing intelligence in mainstream robotics
...
2
...

This chapter begins with a description of the Hierarchical Paradigm in
terms of the SENSE, PLAN, ACT primitives and by its sensing representation
...
1 Shakey, the ﬁrst AI robot
...
(Photograph courtesy of SRI
...
Strips will serve to motivate the reader as to the computer challenges
inherent in even as simple a task as walking across a room
...
Two representative architectures are
presented, NHC and RCS, that serve as examples of robot architectures popular at the time
...

2
...

The Hierarchical Paradigm is sequential and orderly, as shown in Figs
...
2
and 2
...
First the robot senses the world and constructs a global world map
...
2 Attributes of the Hierarchical Paradigm

SENSE

PLAN

ACT

Figure 2
...

ROBOT PRIMITIVES

INPUT

OUTPUT

SENSE

Sensor data

Sensed information

PLAN

Information (sensed
and/or cognitive)

Directives

ACT

Sensed information
or directives

Actuator commands

Figure 2
...

WORLD MODEL

A PRIORI

goal
...
After the robot has
carried out the SENSE-PLAN-ACT sequence, it begins the cycle again: eyes
open, the robot senses the consequence of its action, replans the directives
(even though the directives may not have changed), and acts
...
2
...
The global data structure is generally referred to as a world
model
...
In the Hierarchical
Paradigm, the world model typically contains
1
...
g
...
sensing information (e
...
, “I am in a hallway, based on where I’ve traveled, I must be in the northwest hallway”), plus
3
...
g
...

44

2 The Hierarchical Paradigm

Creating a single representation which can store all of this information can
be very challenging
...
However, as roboticists in the
1980’s began to study biological intelligence, a consensus arose that even
with increased computing power, the hierarchical, logic-based approach was
unsatisfactory for navigational tasks which require a rapid response time to
an open world
...
2
...
(An algorithm is a procedure which is correct
and terminates
...

The method ﬁnally selected was a variant of the General Problem Solver
method, called Strips
...
g
...
g
...
This is inspired by cognitive behavior in humans; if you can’t see how
to solve a problem, you try to solve a portion of the problem to see if it gets
you closer to the complete solution
...
Unless the robot is at SAIL (represented in Strips as a
variable goal state), some sort of transportation will have to arranged
...
The robot may
represent the decision process of how to get to a location as function called an
operator which would consider the Euclidean distance (a variable named
difference) between the goal state and initial state
...
For
example using an arbitrary frame of reference that put Tampa at the center
of the world with made-up distances to Stanford:
initial state:
goal state:
difference:

Tampa, Florida (0,0)
Stanford, California (1000,2828)
3,000

45

2
...

A mode of transportation, fly, ride_train, drive, walk, in the table
is really a function in the robot’s program
...

A robot following this difference table would begin by ﬂying as close as it
could to SAIL
...
It’d be within
100 miles of SAIL, so the robot appears to have made an intelligent decision
...
It examines the
difference table with a new value of difference
...
Drive what? A car? Ooops: if the robot did have a
personal car, it would be back in Tampa
...
This is done by listing
the preconditions that have to be true in order to execute that particular operator
...
In practice, the list of
preconditions is quite lengthy, but for the purposes of this example, only
drive_rental, drive_personal will be shown with preconditions
...
But this introduces a new issue: how does the robot know where
it is at? This is done by monitoring the state of the robot and its world
...
Its initial state is now at the San Francisco airport, and no

46

2 The Hierarchical Paradigm

ADD - LIST
DELETE - LIST

longer Tampa
...
This two lists are stored in the difference
table so that when the robot picks an operator based on the difference and
operator, it can easily apply the appropriate modiﬁcations to the world
...

difference

operator

d 200

fly

100
train

d 100
d<1

drive_rental
drive_personal
walk

preconditions

addlist

deletelist

at
at
at
at

at X

Y
airport
Y
station

at X

at airport
at home

Of course, the above difference table is fairly incomplete
...
(And that
the robot have a waiver from the state highway patrol to drive as an experimental vehicle and a satisfactory method of payment
...
Which is Very Bad from a programming standpoint
...
It should be
apparent that a recursive function can be written which literally examines
each entry in the table for the ﬁrst operator that reduces the difference
...
The robot actually
constructs the plan before handing it off to another program to execute
...
So perhaps the criticisms of Strips is because the example used too
complicated a task to be realistic
...

2
...
2
CONSTRUCTING A
WORLD MODEL

More realistic Strips example
The ﬁrst step in creating a Strips planner is to construct a Strips-based representation of the world, or world model
...
2 Attributes of the Hierarchical Paradigm

AXIOMS
PREDICATES

47

relevant to the problem is represented by facts, or axioms, in predicate logic
...
By years of AI programming convention, predicates are usually written in uppercase
...
2
...
In order to solve this
problem using Strips, the robot has to be given a way of representing the
world, which will in turn inﬂuence the difference table, a difference evaluator, and how the add and delete lists are written
...

A world model is generally built up from static facts (represented as predicates) from a set of candidates, and things in the world, like the robot
...
Lowercase identiﬁers indicate that the thing is a variable, that a real thing hasn’t
been assigned to that placeholder yet
...
In a programming sense,
there would be only three types of things in the world: movable_object
(such as the robot, boxes it should pick up), room, and door
...
2
...
4 An example for Strips of two rooms joined by an open door
...
A door D1
connects a room named R1 to R2 and it connects R2 to R1
...
) A door called D1 has the enumerated value of being
OPEN
...

Under this style of representation, the world model for the goal state
would be:

goal state:
INROOM(IT,R2)
INROOM(B1,R2)
CONNECTS(D1, R1, R2)
CONNECTS(D1, R2, R1)
STATUS(D1,OPEN)

CONSTRUCTING THE
DIFFERENCE TABLE

Once the world model is established, it is possible to construct the difference table
...
2 Attributes of the Hierarchical Paradigm

operator

preconditions

add-list

OP1:
GOTODOOR(IT,dx)
OP2:
GOTHRUDOOR(IT,dx)

INROOM(IT,rk)
CONNECT(dx,rk,rm)
CONNECT(dx,rk,rm)
NEXTTO(IT,dx)
STATUS(dx, OPEN)
INROOM(IT,rk)

NEXTTO(IT,dx)
INROOM(IT,rm)

delete-list

INROOM(IT,rk)

This difference table says the robot is programmed for only two operations: go to a door, and go through a door
...

CONNECT(dx, rk, rm) There is a door, which will be assigned to the
identiﬁer dx, which connects rk to some other room called rm
...
Notice that only the variables dx and rk get bound when GOTODOOR is called
...

If GOTODOOR is executed, the robot is now next to the door called dx
...
The only thing that has
changed is that the robot is now in a noteworthy position in the room: next
to the door
...
In this case, predicates must
be added and deleted from the world model when the operator executes
...

So far, the world model and difference table should seem reasonable, although tedious to construct
...
(Notice that there wasn’t a
column for the difference in the above table
...
In this example, the evaluator is
predicate calculus, where the initial state is logically subtracted from
the goal state
...
In
order to reduce the difference, Strips looks in the difference table, starting at
the top, under the add-list column for a match
...
If Strips ﬁnds an operator that produces the goal
state, then that operator eliminates the existing difference between the initial
and goal states
...
If rm=R2, then
the result of OP2 would be INROOM(IT, R2)
...

Before the OP2 can be applied, Strips must check the preconditions
...

OP2 has two preconditions, only CONNECTS(dx, rk, rm) is affected
...
Until dx and rk are bound, the predicate doesn’t have a true or false value
...
To ﬁll in values for these variables, Strips looks at
the current state of the world model to ﬁnd a match
...
D1 is now bound to dx and R1 is bound to rk
...
NEXTTO(IT, D1) is FALSE because the predicate is
not in the current world state
...
An informal interpretation is that GOTHRUDOOR(IT, D1) will
get the robot to the goal state, but before it can do that, IT has to be next to
D1
...
It marks the original goal state as
G0, pushes it on a stack, then it creates a new sub-goal state, G1
...
Indeed, OP1: GOTODOOR(IT,
dx) has a match in the add-list of NEXTTO(IT, dx)
...

2
...
This time rk=R1 and rm=R2
can be matched with CONNECTS(dx, rk, rm), and all preconditions are
satisﬁed (that is, they evaluate to true)
...

(Note that this is the equivalent of a “mental operation”; the robot doesn’t
actually physically go to the door, it just changes the state to imagine what
would happen if it did
...
There is only a predicate on the add-list and none on the deletelist
...
It resumes where it
left off in evaluating the preconditions for OP2 with dx=D1, rm=R2 and
rk=R1, only now the world model has changed
...

Strips puts OP2 on its plan stack and changes the world model by applying
the add-list and delete-list predicates
...

2
...
3

Strips summary
Strips works recursively; if it can’t reach a goal directly, it identiﬁes the problem (a failed precondition), then makes the failed precondition a subgoal
...
Strips plans rather than execute: it creates a list of operators
to apply; it does not apply the operator as it goes
...
Compute the difference between the goal state and the initial state using
the difference evaluation function
...

2
...

3
...
If not, take the ﬁrst false precondition,
make it the new goal and store the original goal by pushing it on a stack
...

4
...
Then return to
the operator with the failed precondition so it can apply its operator or
recurse on another failed precondition
...
3 Closed World Assumption and the Frame Problem

2
...
As deﬁned earlier, the closed world
assumption says that the world model contains everything the robot needs to
know: there can be no surprises
...
But, on the other hand, it is
very easy to forget to put all the necessary details into the world model
...

But even assuming that the programmer did come up with all the cases,
the resulting world model is likely to be huge
...
And there
were no obstacles! People began to realize that the number of facts (or axioms) that the program would have to sort through for each pass through
the difference table was going to become intractable for any realistic application
...
The opposite of the closed world assumption is known as the open world assumption
...

The above example, although trivial, shows how tedious Strips is (though
computers are good at tedious algorithms)
...
It also illustrates the advantage of a closed-world assumption:
imagine how difﬁcult it would be to modify the planning algorithm if the
world model could suddenly change
...
The example should also bring home the meaning of the frame
problem: imagine what happens to the size of the world model if a third
room is added with boxes for the robot to move to and pick up! And this is
only for a world of rooms and boxes
...

One early solution was ABStrips which tried to divide the problem into
multiple layers of abstraction, i
...
, solve the problem on a coarse level ﬁrst
...
The
two ﬁelds became distinct, and by the 1980’s, the planning and robotics researchers had separate conferences and publications
...
to a goal location
...
4

Representative Architectures
As mentioned in Part I an architecture is a method of implementing a paradigm, of embodying the principles in some concrete way
...

Possibly the two best known architectures of the Hierarchical period are
the Nested Hierarchical Controller (NHC) developed by Meystel 93 and the
NIST Realtime Control System (RCS) originally developed by Albus, 1 with
its teleoperation version for JPL called NASREM
...
4
...
2
...
The
robot begins by gathering observations from its sensors and combining those
observations to form the World Model data structure through the SENSE
activity
...
After the World
Model has been created or updated, then the robot can PLAN what actions
it should take
...
Each of
these modules has access to the World Model in order to compute their portion of planning
...
g
...
6 meters per second)
...
g
...
Together, the Low-Level Controller and actuators form the ACT
portion of the architecture
...
As shown in Fig
...
6, the Mission Planner either receives a mission from a human or generates a mission

55

2
...
5

drive

ACT

steer

Nested Hierarchical Controller
...
The Mission Planner is responsible for operationalizing, or translating, this mission into terms
that other functions can understand: box=B1; rm=ROOM311
...
The Navigator takes this information and generates a
path from the current location to the goal
...
The path is passed to the Pilot
...
For instance, the robot may need
to turn around to face the way point before it can start driving forward
...

After the Pilot gives the Low-Level Controller commands and the controller

56

2 The Hierarchical Paradigm

5

6

10

Mission
Planner
goal

Navigator
path

Pilot
path subsegment:
Turn 83 degrees
move 5 meters

Figure 2
...

sends actuator signals, the robot polls its sensors again
...
However, the entire planning cycle does not repeat
...
Instead, the Pilot checks the World Model to see if the robot has drifted
off the path subsegment (in which case it generates a new control signal),
has reached the waypoint, or if an obstacle has appeared
...
If the waypoint isn’t
the goal, then there is another path subsegment for the robot to follow, and
so the Navigator passes the new subsegment to the Pilot
...
The Mission Planner may then issue a new goal, e
...
, Return to the starting place
...
The Navigator must compute
a new path, and subsegments, based on the updated World Model
...

2
...
It differs from Strips in that it interleaves
planning and acting
...
Notice that the decomposition is inherently hierarchical in intelligence and scope
...
The
Mission Planner is responsible for a higher level of abstraction then the Navigator, etc
...

One disadvantage of the NHC decomposition of the planning function is
that it is appropriate only for navigation tasks
...
The role of a Pilot in controlling end-effectors is not
clear
...

2
...
2

NIST RCS
Jim Albus at the National Bureau of Standards (later renamed the National
Institute of Standards and Technology or NIST) anticipated the need for intelligent industrial manipulators, even as engineering and AI researchers were
splitting into two groups
...
This made industry and equipment manufacturers leery of AI, for fear of buying an expensive robot that would not
be compatible with robots purchased in the future
...
RCS used NHC in its design, as shown in Fig
...
7
...
The output of the sensors is passed off to the world
modeling module which constructs a global map using information in its
associated knowledge database about the sensors and any domain knowledge (e
...
, the robot is operating underwater)
...
The main difference is that the sensory perception module introduces a useful preprocessing step between the sensor and the fusion into a
world model
...
6, sensor preprocessing is often referred
to as feature extraction
...

SENSE

PLAN
Value
Judgment

changes
and
events

Sensory
Perception

simulated
plans

perception,
focus of
attention

World
Modeling
Knowledge
Database

observed
input

task
goals

Behavior
Generation

plans,
state of
actions

ACT
commanded
actions

b
...
7 Layout of RCS: a
...
) functional decomposition
...
4 Representative Architectures

NASREM

2
...
3

59

The Value Judgment module provides most of the functionality associated
with the PLAN activity: it plans, then simulates the plans to ensure they will
work
...
Notice that the Behavior Generation module is
similar to the Pilot in NHC, but there appears to be less focus on navigation
tasks
...
(This use of “behavior” in RCS is a bit of retroﬁt,
as Albus and his colleagues at NIST have attempted to incorporate new advances
...
) There is another module,
operator interface, which is not shown which allows a human to “observe”
and debug what a program constructed with the architecture is doing
...
RCS serves as a blueprint for saying:
“here’s the types of sensors I want, and they’ll be fused by this module into a
global map, etc
...
Fig
...
8 shows
three of the diverse mobile robots that have used RCS
...
The human operator could
provide the world model (via eyes and brain), decide the mission, decompose it into a plan, and then into actions
...
As robotics advanced, the robot could replace
more functions and “move up” the autonomy hierarchy
...
As AI advanced, the human would
only have to serve as the Mission Planner: “go to the White House
...
Albus noted this and worked with JPL to develop a version of RCS for
teleoperating a robot arm in space
...

Evaluation of hierarchical architectures
Recall from Part I that there are four criteria for evaluating an architecture:
support for modularity, niche targetability, ease of portability to other domains, and robustness
...

a
...

Figure 2
...
) a commercial
ﬂoor cleaning robot, b
...
) a submersible or underwater robot
...
)

to decompose a robot program into intuitive modules
...
Both have been used recently for successful
vehicle guidance, with RCS being used to control mining equipment, submarines, and cars
...
The ease of
portability to other domains is unclear
...
” Architectures which are more speciﬁc, “there should be one bathroom for every two bedrooms,” and which have associated techniques simplify portability
...
5 Advantages and Disadvantages

61

could be reused for a submarine, especially since RCS is not object-oriented
...
In particular, it assumes the Value Judgment module simulates a
plan to conﬁrm that it should be successful when deployed
...
The most notable example is a
nuclear processing cell
...

This is a very limited form of robustness
...

Simulation may not be appropriate for all actions; if a piece of the ceiling is
falling on the robot, it needs to get out of the way immediately or risk coming
up with the best place to move too late to avoid being crushed
...
5

Advantages and Disadvantages
Robots built in the time period before 1990 typically had a Hierarchical style
of software organization
...

The robots are interesting because they illustrate the diversity and scope of
applications being considered for mobile robots as far back as 15 or 20 years
ago
...
The
primary disadvantage was planning
...
The sensing and planning algorithms of the day were extremely slow (and many still
are), so this introduced a signiﬁcant bottleneck
...
This effectively eliminated any stimulusresponse types of actions (“a rock is crashing down on me, I should move
anywhere”) that are seen in nature
...

In Strips, in order to do something as simple as opening a door, the robot had
to reason over all sorts of details that were irrelevant (like other rooms, other
doors)
...
Unfortunately, these decomposi-

62

2 The Hierarchical Paradigm

tions appear to be dependent on a particular application
...

Another issue that was never really handled by architectures in the Hierarchical Paradigm was uncertainty
...
Another important aspect of uncertainty is action completion:
did the robot actually accomplish the action? One robotics researcher said
that their manipulator was only able to pick up a cup 60% of the attempts;
therefore they had to write a program to check to see if it was holding a cup
and then restart the action if it wasn’t
...

2
...
These languages were developed
by AI researchers speciﬁcally for expressing logical operations
...
However, during the 1960’s the dominant scientiﬁc and engineering
language was FORTRAN IV which did not support recursion
...
The use of special AI languages for robotics may have aided the split
between the engineering and AI approaches to robotics, as well as slowed
down the infusion of ideas from the the two communities
...

The Hierarchical Paradigm tends to encourage monolithic programming,
rather than object-oriented styles
...
In particular, NHC and RCS don’t provide much guidance on how to build modular, reusable components
...
7

Summary
The Hierarchical Paradigm uses a SENSE then PLAN then ACT (S,P,A)
...
Global data structures often ﬂag that an ar-

2
...
The Hierarchical Paradigm
was introduced in the ﬁrst mobile robot, Shakey
...
Concepts and terms which emerged that
continue to play an important role in deﬁning robotics are: preconditions,
the closed and open world assumptions, and the frame problem
...
The decline in popularity is due in part to its focus on
strong niche targetability at the expense of true modularity and portability
...
One often overlooked property of most hierarchical architectures is that they tend to
support the evolution of intelligence from semi-autonomous control to fully
autonomous control
...
8

Exercises
Exercise 2
...

Exercise 2
...
Give an example of how each
arises in a robotic system:
a
...
closed world assumption
c
...
frame problem
Exercise 2
...
Suppose the World Model for a Strips-based robot consisted of 100 facts
...
Every time a new object
is added to the world model, it increases the model by 100 (a linear increase)
...
How many objects
would ﬁll 64KB of memory?
Exercise 2
...
One object, 100 facts, 1KB, two objects,
200 facts, 200KB, three objects, 400 facts, 400KB
...
5
Describe the mission planner, navigator, pilot organization of the Nested Hierarchical
Controller
...
2
...
2
...
6
List 2 advantages and disadvantages of the Hierarchical Paradigm
...
7
Solve the following navigation problem using Strips
...
2
...

The robot will move to the box B1 and pick it up
...
Add a new operator pickup to the difference table
...
Use the world model, difference table, difference evaluator to construct a plan
...

c
...

Exercise 2
...

Exercise 2
...

2
...

Robot vehicles do in fact need special authorization to drive on public roads
...
The Navlab (a modiﬁed Saturn station wagon) was reportedly pulled over by
the Kansas State Highway Patrol for driving an experimental vehicle without permission
...
As impressive as the Carnegie Mellon feat was, a group of German
researchers under the direction of Ernst Dickmanns and Volker Graefe have been
ﬁelding even more advanced autonomous highway driving vehicles since 1988
...

It can be debated whether Shakey is really the ﬁrst mobile robot
...
See
Behavior-Based Robots 10 for details
...
9 End Notes

65

Robot name trivia
...

Strips
...
Barr and E
...
1, William Kaufmann,
Inc
...

Jim Albus
...
Although his RCS architecture is a representative of the Hierarchical Paradigm, Albus was heavily by cognitive studies—so
much so that he wrote Brains, Behaviors and Robots (1981)
...
The RCS architecture was never meant to be totally static, and in recent
years, it has begun to resemble what will be referred to as a model-oriented style of
the Hybrid Paradigm
...

Explain in one or two sentences each of the following terms: reﬂexes, taxes,
ﬁxed-action patterns, schema, affordance
...

Given a description of an animal’s sensing abilities, its task, and environment, identify an affordance for each behavior
...

3
...
The most inﬂuential robot was
the Stanford Cart developed by Hans Moravec, which used stereo vision to
see and avoid obstacles protruding from the ground
...
While many roboticists had been fascinated
by animals, and many, including Moravec, had used some artifact of animal
behavior for motivation, no one approached the ﬁeld with the seriousness
and dedication of Arbib
...
It was a series of
gedanken or entirely hypothetical thought experiments, speculating as to
how machine intelligence could evolve
...
The next, more complex vehicle had two thermal sensor-motor pairs, one on each side of the vehicle
...

Throughout the book, each vehicle added more complexity
...

Vehicles became a cult tract among roboticists, especially in Europe
...
They began exploring the biological sciences in search
of new organizing principles and methods of obtaining intelligence
...
This
chapter attempts to set the stage for the Reactive Paradigm by recapping inﬂuential studies and discoveries, and attempting to cast them in light of how
they can contribute to robotic intelligence
...
Next, it covers the work of Lorenz and Tinbergen in
deﬁning how concurrent simple animal behaviors interact to produce complex behaviors through Innate Releasing Mechanisms (IRMs)
...
The previous chapter on the Hierarchical Paradigm showed how early
roboticists attempted to fuse all sensing into a global world map, supplemented with a knowledge base
...
J
...
Gibson refuted the necessity of global
world models, a direct contradiction to the way perception was handled in
the hierarchical paradigm
...
Later
work by Neisser attempts to deﬁne when global models are appropriate and
when an affordance is more elegant
...
In order to address this concern, the chapter discusses speciﬁc principles and how they can be applied
to robotics
...
Finally, the chapter covers schema theory, an attempt in cognitive

3
...
Schema theory has been used
successfully by Arbib to represent both animal and robot behavior
...

3
...
1

OPEN WORLD
ASSUMPTION
CLOSED WORLD
ASSUMPTION
FRAME PROBLEM

Why explore the biological sciences?
Why should roboticists explore biology, ethology, cognitive psychology and
other biological sciences? There is a tendency for people to argue against
considering biological intelligence with the analogy that airplanes don’t ﬂap
their wings
...
The advances in aeronautics came as the Wright Brothers and others extracted aerodynamic principles
...
The “planes don’t ﬂap their wings” argument turns out to be even less convincing for computer systems: animals
make use of innate capabilities, robots rely on compiled programs
...
Animals and man provide existence proofs of different aspects of
intelligence
...
For example, the issue of how to
combine information from multiple sensors (sensor fusion) has been an open
question for years
...
Additional
research showed that animals (including man) do perform sensor fusion, although with surprisingly different mechanisms than most researchers had
considered
...
Animals live in an open world, and roboticists would like to overcome
the closed world assumption that presented so many problems with Shakey
...
Therefore, they must be doing something
that avoids the frame problem
...
1
...
Consider something we’ll call an agent
...
It has
its own “brains” and can interact with the world to make changes or to sense
what is happening
...
Under this deﬁnition, a person is
an agent
...
More importantly,
an intelligent robot would be an agent, even certain kinds of web search engines which continue to look for new items of interest to appear, even after
the user has logged off
...
In OOP terms,
“agent” is the superclass and the classes of “person” and “robot” are derived
from it
...
One helpful way of seeing correspondences is to decide the level at
which these entities have something in common
...
Marr was a neurophysiologist who tried to recast biological vision
processes into new techniques for computer vision
...
Suppose a roboticist
is interested in building a robot to search for survivors trapped in a building after an earthquake
...
As anyone who has been camping knows, mosquitoes are
very good at ﬁnding people
...
At Level 1, agents can share a commonality of purpose or
functionality
...
This level can be thought of as creating a ﬂow chart of “black
boxes
...

Returning to the example of a mosquito, the roboticist might realize from
biology that the mosquito ﬁnds humans by homing on the heat of a hu-

3
...
If the mosquito senses a hot area, it
ﬂies toward it
...
The “black box” is how the mosquito transforms the input into the output
...
If the hot patch moves,
the thermal image will change with the next sensory update, and a new
steering command will be generated
...
Also notice that by focusing on the process
rather than the implementation, a roboticist doesn’t have to worry about
mosquitoes ﬂying, while a search and rescue robot might have wheels
...

Level 3: How to implement the process
...
For example, in a mosquito, the steering commands might be implemented with a special type of neural network, while in a robot, it might
be implemented with an algorithm which computes the angle between the
centroid of heat and where the robot is currently pointing
...

It should be clear that Levels 1 and 2 are abstract enough to apply to any
agent
...
Some roboticists actively attempt to emulate biology, reproducing the physiology and neural mechanisms
...
) Fig
...
1 shows work at Case Western Reserve’s Bio-Bot
Laboratory under the direction of Roger Quinn, reproducing a cockroach on
a neural level
...
Most roboticists do not strive to precisely
replicate animal intelligence, even though many build creatures which resemble animals, as seen by the insect-like Genghis in Fig
...
2
...

72

3 Biological Foundations of the Reactive Paradigm

a
...

Figure 3
...
) Robot I, an earlier version, and b
...
(Photographs courtesy of Roger Quinn
...
2 Genghis, a legged robot built by Colin Angle, IS Robotics, which imitates
an insect at Levels 1 and 2
...
)

73

3
...
3 Graphical deﬁnition of a behavior
...
2
BEHAVIOR

REFLEXIVE BEHAVIOR
STIMULUS - RESPONSE

REACTIVE BEHAVIOR

CONSCIOUS BEHAVIOR

What Are Animal Behaviors?
Scientists believe the fundamental building block of natural intelligence is
a behavior
...
For example, if a horse
sees a predator, it ﬂattens its ears, lowers its head, and paws the ground
...
The defensive motions make up a pattern because the actions and sequence is always the same, regardless of details which vary each
episode (e
...
, how many times the horse paws the ground)
...
3
...

Scientists who study animal behaviors are called ethologists
...
Often
the pattern of motor actions is easy to ascertain; the challenging part is to
discover the sensory inputs for the behavior and why the behavior furthers
the species survival
...
10 Reﬂexive behaviors
are stimulus-response (S-R), such as when your knee is tapped, it jerks upward
...
Reactive behaviors are learned, and then consolidated to
where they can be executed without conscious thought
...
g
...
Reactive behaviors can also be
changed by conscious thought; a bicyclist riding over a very narrow bridge
might “pay attention” to all the movements
...

The categorization is worthy of note for several reasons
...
Second,
the categorization can help a designer determine what type of behavior to
use, leading to insights about the appropriate implementation
...
In ethology, reactive behavior means learned behaviors or a
skill; in robotics, it connotes a reﬂexive behavior
...

3
...
1

Reﬂexive behaviors
Reﬂexive types of behaviors are particularly interesting, since they imply no
need for any type of cognition: if you sense it, you do it
...
Indeed, many kit or hobby robots work off of reﬂexes, represented
by circuits
...
reﬂexes: where the response lasts only as long as the stimulus, and the
response is proportional to the intensity of the stimulus
...
taxes: where the response is to move to a particular orientation
...
Until recently the brightest light would be the ocean reﬂecting the
moon, but the intrusion of man has changed that
...
Baby turtles
hatch at night, hidden from shore birds who normally eat them
...
The story goes that a volunteer left a ﬂashlight on the
sand while setting up an experiment intended to show that the baby turtles used magnetic ﬁelds to orient themselves
...

FIXED - ACTION

3
...
This is helpful for ﬂeeing predators
...

PATTERNS

The above categories are not mutually exclusive
...
3 Coordination and Control of Behaviors

IDIOTHETIC
ALLOTHETIC

3
...

The tight coupling of action and perception can often be quantiﬁed by
mathematical expressions
...
In
order to swim upright, an angelﬁsh uses an internal (idiothetic) sense of gravity combined with its vision sense (allothetic) to see the external percept of
the horizon line of the water to swim upright
...
On closer inspection, the angle that the angelﬁsh swims at is the
vector sum of the vector parallel to gravity with the vector perpendicular to
the perceived horizon line! The ability to quantify animal behavior suggests
that computer programs can be written which do likewise
...

Each man independently became fascinated not only with individual behaviors of animals, but how animals acquired behaviors and selected or coordinated sets of behaviors
...
Lorenz and Tinbergen’s work also helps with a computational theory Level 2 understanding of
how to make a process out of behaviors
...
to be born with a behavior (innate)
...
Arctic terns, as the name implies, live in the Arctic where
the terrain is largely shades of black and white
...
When babies are hatched and are hungry, they
peck at the beak of their parents
...
It
turns out that the babies do not recognize their parents, per se
...
Notice that the only red blobs in the ﬁeld of vision should
be the beaks of adult Arctic terns
...
This is a simple,
effective, and computationally inexpensive strategy
...
to be born with a sequence of innate behaviors
...
An example is the mating cycle in digger wasps
...
Once it sees
the nest, the female lays eggs
...
The nest isn’t built
until the female mates; that is a change in internal state
...
Notice that the wasp doesn’t have to “know” or understand the
sequence
...
This is very similar to Finite State Machines in computer
science programming, and will be discussed later in Ch
...

INNATE WITH MEMORY

3
...
An animal can be born with innate behaviors that need customizing
based on the situation the animal is born in
...

Bees are born in hives
...
It is believed that the curious behavior exhibited by baby
bees (which is innate) allows them to learn this critical information
...
This will get repeated, with the bee going a bit farther along
the straight line each time
...
Eventually, the bee will have
circumnavigated the hive
...
Furthermore, the bee can associate a view of the hive with a motor command
(“ﬂy left and down”) to get the bee to the opening
...

LEARN

4
...
Behaviors are not necessarily innate
...
An example of learned behaviors is hunting in lions
...
If they are not taught
by their mothers over a period of years, they show no ability to fend
for themselves
...
However, consider the complexity of hunting for food
...

Hunting may also require teamwork with other members of the pride
...
Imagine trying to write a program to cover all the possibilities!

3
...
It is just that the releasers and
actions are learned; the animal creates the program itself
...
3
...
It also suggests that behaviors can be innate but require memory
...
But as will be seen in Chapter 4, this is a common constraint
placed on many robot systems
...
Numerous BEAM robot web sites
guide adherents through construction of circuits which duplicate memoryless innate reﬂexes and taxes
...
Being hungry is a stimulus, equivalent to the pain
introduced by a sharp object in the robot’s environment
...
Motivations can stem from survival conditions (like being hungry) or more abstract
goals (e
...
, need to check the mail)
...
Something as complicated as mating and building a nest can be decomposed into primitives
or certainly more simple behaviors
...

Innate releasing mechanisms
Lorenz and Tinbergen attempted to clarify their work in how behaviors are
coordinated and controlled by giving it a special name innate releasing mechanisms (IRM)
...
The IRM activates the behavior
...
One way to think of IRMs is as a process of behaviors
...
Recall that behaviors take sensory input
and produce motor actions
...
4 Innate Releasing Mechanism as a process with behaviors
...
The releaser acts as a control signal to activate a
behavior
...
For example, if a baby arctic tern isn’t
hungry, it doesn’t peck at red, even if there is a red beak nearby
...
Imagine
the agent running a C program with a continuous while loop
...

enum
Releaser={PRESENT, NOT_PRESENT};
Releaser
predator;
while (TRUE)
{
predator = sensePredators();
if (predator == PRESENT)
flee();
}

In this example, the agent does only two things: sense the world and then
ﬂees if it senses a predator
...
flee is
released by the presence of a predator
...
If the agent does
not sense the releaser for the behavior, the agent does nothing
...

This example also shows ﬁltering of perception
...
The dedicated predator detection function could be a specialized sense (e
...
, retina is sensitive to the frequency of motions associated

3
...

Another important point about IRMs is that the releaser can be a compound
of releasers
...
If the releaser in the compound isn’t satisﬁed, the behavior isn’t triggered
...

enum
Releaser={PRESENT, NOT_PRESENT};
Releaser
food;
while (TRUE)
{
food = senseFood();
hungry = checkState();
if (food == PRESENT && hungry==PRESENT)
feed();
}

IMPLICIT CHAINING

The next example below shows what happens in a sequence of behaviors,
where the agent eats, then nurses its young, then sleeps, and repeats the
sequence
...

Once the initial releaser is encountered, the ﬁrst behavior occurs
...
If the behavior isn’t ﬁnished, the releasers remain unchanged and
no other behavior is triggered
...
When the original behavior has completed,
the internal state of the animal may have changed or the state of the environment may have been changed as a result of the action
...

enum
Releaser={PRESENT, NOT_PRESENT};
Releaser
food, hungry, nursed;
while (TRUE) {
food = sense();
hungry = checkStateHunger();
child = checkStateChild();
if (hungry==PRESENT)
searchForFood(); //sets food = PRESENT when done
if (hungry==PRESENT && food==PRESENT)
feed(); // sets hungry = NOT_PRESENT when done

80

3 Biological Foundations of the Reactive Paradigm

if (hungry== NOT_PRESENT && parent==PRESENT)
nurse(); // set nursed = PRESENT when done
if (nursed ==PRESENT)
sleep();
}

The example also reinforces the nature of behaviors
...

In the previous example, the agent’s behaviors allowed it to feed and enable the survival of its young, but the set of behaviors did not include ﬂeeing
or ﬁghting predators
...
But ﬂeeing is temporary, because it did
not change the agent’s internal state (except possibly to make it more hungry
which will show up on the next iteration)
...

One way around this is to inhibit, or turn off, any other behavior until
ﬂeeing is completed
...
3 Coordination and Control of Behaviors

81

while (TRUE) {
predator = sensePredator();
if (predator==PRESENT)
flee();
else {
food = senseFood();
hungry = checkStateHunger();

...
It doesn’t solve the problem with the predator releaser disappearing as the agents runs away
...
The agent will go back to foraging, feeding, nursing, or sleeping
...
The ﬁxed-pattern action effect can
be accomplished with:
#define T LONG_TIME
while (TRUE) {
predator = sensePredator();
if (predator==PRESENT)
for(time = T; time > 0; time--)
flee();
else {
food = senseFood();

...
An implementation of the same behaviors with an explicit sequence would be:

Releaser
while (TRUE) {

food, hungry, nursed, predator;

predator = sensePredator();
if (predator==PRESENT)

82

3 Biological Foundations of the Reactive Paradigm

flee();
food = senseFood();
hungry = checkStateHunger();
parent = checkStateParent();
if (hungry==PRESENT)
searchForFood();
feed();
nurse();
sleep();
}

The explicit sequence at ﬁrst may be more appealing
...
But this implementation is not
equivalent
...
Note that the agent cannot react to a predator until it has ﬁnished the sequence of behaviors
...
But every “ﬁx” makes the program less general purpose and harder to
add and maintain
...

3
...
2

Concurrent behaviors
An important point from the examples with the IRMs is that behaviors can,
and often do, execute concurrently and independently
...
However, some
behaviors may violate or ignore the implicit sequence when the environment
presents conﬂicting stimuli
...

Interesting things can happen if two (or more) behaviors are released that
usually are not executed at the same time
...
A squirrel will often appear to be visibly undecided as
to whether to go for the food or to stay away
...
4 Perception in Behaviors

83

DOMINANCE

Dominance of one (winner take all): you’re hungry and sleepy
...

CANCELLATION

Cancellation (the behaviors cancel each other out): Male sticklebacks (ﬁsh)
when their territories overlap get caught between the need to defend their
territory and to attack the other ﬁsh
...

Unfortunately, it doesn’t appear to be well understood when these different mechanisms for conﬂicting behaviors are employed
...
But it does emphasize that a roboticist who works with behaviors should pay close attention to how the behaviors will interact
...

3
...
Ulrich Neisser, who literally
created the term “cognitive psychology” in his book, Cognition and Reality,
argued that perception cannot be separated from action
...
J
...
The ecological
approach is the opposite of the top-down, model-based reasoning about the
environment approach favored by psychologists, including Neisser
...
J
...
Since then, Neisser has spent signiﬁcant
time and thought trying to reconcile the two views based on studies; this has
led to his identiﬁcation of two perceptual systems
...
4
...
A simple interpretation of the cycle is: When an agent acts, it
interacts with its environment because it is situated in that environment; it is
an integral part of the environment
...
g
...
Therefore the agent’s perception of the world is modiﬁed
...
5 Action-Perception Cycle
...
The term cognitive activity
includes the concepts of feedback and feedforward control, where the agent
senses an error in what it attempted to do and what actually happened
...
That activity can be something as straightforward as activating processes to look for
releasers, or as complex as looking for a particular face in a crowd
...
This is a type of selective
attention or focus-of-attention
...
If the agent actually acts in a way to gather more perception
before continuing with its primary action, then that is sometimes referred to
as active perception
...
Lorenz and Tinbergen might think of this as the agent having
a set of releasers for a task, and now is observing whether they are present
in the world
...
The action modiﬁes the environment, but it also modiﬁes the agent’s assessment of
the situation
...

In some regards, the action-perception cycle appears to bear a superﬁcial
resemblance to the Hierarchical Paradigm of SENSE, PLAN, ACT
...
Action is implicit

3
...
The agent may have to act to acquire more perception or to
accomplish a task
...

3
...
2
RELEASE

GUIDE

3
...
3

AFFORDANCES

Two functions of perception
Perception in behavior serves two functions
...
However, releasing a behavior isn’t necessarily the same as the second function: perceiving the information needed to
accomplish the behavior
...

The ﬁre activates the ﬂeeing
...
A frightened deer might bolt right past a hunter without apparently noticing
...
This is often referred to
as action-oriented perception by roboticists, when they wish to distinguish
their perceptual approach from the more hierarchical global models style of
perception
...
Some frogs which sit in water all day
with just half their eyes poking up have a split retina: the lower half is good
for seeing in water, the upper half in air
...
the world is its own best
representation
...
Gibson postulated (and proved) the existence of affordances
...
For example, to a baby arctic tern, the color red is perceivable and represents the potential for feeding
...
But like IRMs, an affordance is only a potential—
it doesn’t count until all the other conditions are satisﬁed (the baby tern is
hungry)
...
The
presence of red to a hungry baby arctic tern releases the feeding behavior
...
So in this case,
red is also the percept being used to guide the action, as well as release it
...
For example, a certain species of bees prefers one special
type of poppy
...
Smell?
Magnetism? Neither
...
In
the non-visible bands that type of poppy stood out from other poppy species
...
The bee and poppy had co-evolved, where the poppy’s
color evolved to a unique bandwidth while at the same time the bee’s retina
was becoming specialized at detecting that color
...
If that color was
present, the poppy was there
...
A ﬁshing lure attempts to emphasize those aspects of a ﬁsh’s desired food, presenting the strongest stimulus possible: if the ﬁsh is hungry, the stimulus of
the lure will trigger feeding
...
3
...

What makes Gibson so interesting to roboticists is that an affordance is directly perceivable
...
This means minimal computation, which usually translates to very rapid execution times (near instantaneous) on a computer or robot
...
And they’re deﬁnitely
not interpreting red in the sense of: “oh, there’s a red blob
...
” For baby arctic terns, it’s
simply: red = food, bigger red = better
...
You will most likely duck
...
The response happens too fast for any reasoning: “Oh
look, something is moving towards me
...
Balls are usually
hard
...
” Instead, you probably used a phenomena so basic that

3
...
6 A collection of artiﬁcial bait, possibly the ﬁrst example of humans exploiting affordances
...

OPTIC FLOW

TIME TO CONTACT

you haven’t noticed it, called optic ﬂow
...
Animals can determine time to contact quite easily
with it
...
When driving or riding in a car, objects in front seem to be in clear focus
but the side of the road is a little blurry from the speed
...
From that point outward,
there is a blurring effect
...
(They use this all the time in science ﬁction movies to simulate fasterthan-light travel
...
It is
straightforward, neurally, to extract the time to contact, represented in the
cognitive literature by
...
Gannets are large birds which dive from high
altitudes after ﬁsh
...
But they are plummeting so fast that if they hit the water with
their wings open, the hollow bones will shatter
...
Optic ﬂow allows the time to contact, , to be a stimulus: when the time to contact dwindles below a threshold, fold those wings!

88

STRUCTURAL MODELS

3 Biological Foundations of the Reactive Paradigm

Pole vaulters also make minute adjustments in where they plant their pole
as they approach the hurdle
...
It appears that pole vaulters use optic ﬂow rather
than reason (slowly) about where the best place is for the pole
...
)
In most applications, a fast computer program can extract an affordance
...
Neural mechanisms in
the retina have evolved to make the computation very rapid
...
Only recently have
we seen any algorithms which ran in real-time on regular computers
...

Affordances are not limited to vision
...
Think about ﬁlling a jug with
water or the fuel tank of a car
...
That change
in sound is directly perceivable; the person doesn’t need to know anything
about the size or shape of the volume being ﬁlled or even what the liquid is
...
135 A seemingly unsurmountable problem in computer
vision has been to have a computer recognize an object from a picture
...

The traditional way of approaching the problem has been to use structural
models
...
” But not all chairs ﬁt
the same structural model
...
Hanging baskets don’t have legs at all
...
So clearly the structural approach has problems: instead of one
structural representation, the computer has to have access to many different
models
...
If the robot is presented with a
new kind of chair (say someone has designed a chair to look like your toilet
or an upside down trash can), the robot would not be able to recognize it
without someone explicitly constructing another structural model
...
GRUFF identiﬁes chairs by function rather than form
...
4 Perception in Behaviors

a
...

Figure 3
...
) input, and b
...
(Figures courtesy of Louise Stark
...
And that affordance of sittability should be something that can be extracted from an image:
Without memory (the agent doesn’t need to memorize all the chairs in the
world)
...

Without an interpretation of the image (the robot doesn’t need to reason:
“there’s an arm rest, and a cushion,
...

90

3 Biological Foundations of the Reactive Paradigm

Stark and Bowyer represented sittability as a reasonably level and continuous surface which is at least the size of a person’s butt and at about the
height of their knees
...
) Stark and Bowyer wrote a computer program which accepted CAD/CAM drawings from students who tried to come up with nonintuitive things that could serve as chairs (like toilets, hanging basket chairs,
trash cans)
...

It should be noted that Stark and Bowyer are hesitant to make claims about
what this says about Gibsonian perception
...
But on the other hand, that level of inference and interpretation is signiﬁcantly different than that involved in trying to determine the
structure of the legs, etc
...
The robot would start
noticing that it could sit on a ledge or a big rock if a chair or bench wasn’t
around
...
4
...
A chair is a
chair because it affords sittability
...
Recognition deﬁnitely involves memory
(“my car is a blue Ford Explorer and I parked it in slot 56 this morning”)
...
(Imagine trying to duplicate Sherlock Holmes
in a computer
...
)
So while affordances certainly are a powerful way of describing perception
in animals, it is clearly not the only way animals perceive
...
direct perception
...

RECOGNITION

2
...
This is more recent perceptual track in the brain, which ties in
with the problem solving and other cognitive activities
...
5 Schema Theory

for the use of internal models to distinguish “your coffee cup” from “my
coffee cup
...

On a more practical note, Neisser’s dichotomy suggests that the ﬁrst decision in designing a behavior is to determine whether a behavior can be
accomplished with an affordance or requires recognition
...

3
...
6 Psychologists have used
schema theory since the early 1900’s
...
for vision
...
A schema consists both of the knowledge of how to act
and/or perceive (knowledge, data structures, models) as well as the computational process by which it is uses to accomplish the activity (the algorithm)
...
A schema class in C++ or Java would contain both data
(knowledge, models, releasers) and methods (algorithms for perceiving and
acting), as shown below
...
It is a template because a person can ride different bicycles without
starting the learning process all over
...
As with object-oriented programming,
the creation of a speciﬁc schema is called a schema instantiation (SI)
...
8 Behaviors decomposed into perceptual and motor schemas
...
For example, there could
be a move_to_food schema where the agent always heads in a straight line
to the food
...
However, it is just a
method; until the move_to_food schema is instantiated, there is no speciﬁc
goal to head for, e
...
, the candy bar on the table
...

3
...
1
COMPOSITION OF A
BEHAVIOR
MOTOR SCHEMA
PERCEPTUAL SCHEMA

Behaviors and schema theory
In the Arbibian application of schema theory towards a computational theory of intelligence, a behavior is a schema which is composed of a motor
schema and a perceptual schema
...
The motor schema and perceptual schema are like pieces of a puzzle; both pieces
must be together in place before there is a behavior
...
3
...

Essentially, the perceptual and motor schema concept ﬁts in with ethology
and cognitive psychology as follows:
A behavior takes sensory inputs and produces motor actions as an output
...
5 Schema Theory

A behavior can be represented as a schema, which is essentially an objectoriented programming construct
...

The transformation of sensory inputs into motor action outputs can be
divided into two sub-processes: a perceptual schema and a motor schema
...
A primitive behavior just has one motor and
one perceptual schema
...
In the case of a sequence of behaviors, the overall
behavior could be represented in one of two ways
...
This is probably the easiest way to express a “meta” behavior
...
For example, a
person usually uses vision (the default perceptual schema) to navigate out
of a room (motor schema)
...
In this
case, the schema-speciﬁc knowledge is knowing which perceptual schema

94

3 Biological Foundations of the Reactive Paradigm

to use for different environmental conditions
...
This will be discussed in more detail
in later chapters
...
They used schema theory to represent
the toad’s behavior in computational terms, and called their model rana computatrix (rana is the classiﬁcation for toads and frogs)
...
33 Toads and frogs can be characterized as responding visually
to either small, moving objects and large, moving objects
...
(If the object turns out not to be a ﬂy,
the toad can spit it out
...
The feeding behavior can be modeled as a
behavioral schema, or template, shown in Fig
...
9
...
Arbib’s group went one level
further on the computational theory
...
This direction and intensity (magnitude) was
represented as a vector
...
This is shown in Fig
...
10
...
In this case, each ﬂy releases a separate instance of
the feeding behavior
...
5 Schema Theory

Releaser
appearance of
small, moving object

toad’s vision

Sensory
Input

Perceptual
Schema

Motor
Schema

Get coordinates
of small, moving
object

Turn to
coordinates
of small,
moving object

toad’s legs

Pattern of
Motor Actions

Feeding Behavior

Figure 3
...

havior exists
...
What to do? Well,
rana computatrix summed the two vectors, resulting in a third vector in between the original two! The toad snaps at neither ﬂy, but in the middle
...

This example illustrates many important lessons for robotics
...
The concept of behaviors is Level 1 of the
computational theory, schema theory (especially the perceptual and motor
schemas) expresses Level 2, and Level 3 is the vector ﬁeld implementation
of the motor action
...
The example also shows how behaviors correspond to object-orienting programming principles
...
Recall that in reﬂex behaviors, the strength of the response is proportional to the strength of the stimulus
...

The motor schema can use the gain to compute a magnitude on the output
action
...

96

3 Biological Foundations of the Reactive Paradigm

fly1
activation
condition
motor schema

perceptual schema
locate_fly

snap

behavior
motor SI

percept,
gain

perceptual SI
locate_fly(fly1)

snap(fly1)
(x,y,z),
100%
action,
intensity
snap at (x,y,z)
with all strength

Figure 3
...

INHIBITION

Schema theory does not specify how the output from concurrent behaviors
is combined; that is a Level 3, or implementation, issue
...
The
winner-take-all effect is a type of inhibition, where one behavior inhibits the
instantiation of another behavior
...
7 Returning to the example of feeding and ﬂeeing, one possible way to
model this behavior is with two behaviors
...
The fleeing behavior would be
similar only with a motor schema for moving away from the perception of
large moving objects
...

The feeding behavior actually consists of moving toward any moving object
...
The frog
would try to eat anything, including predators
...
It ﬂees from them, but

97

3
...
11 Schema theory of a frog snapping at a ﬂy when presented with two ﬂies
equidistant
...
As a result, the inhibition
keeps the frog from trying to both ﬂee from predators and eat them
...
6
PRINCIPLES FOR

Principles and Issues in Transferring Insights to Robots
To summarize, some general principles of natural intelligence which may be
useful in programming robots:

PROGRAMMING

Programs should decompose complex actions into independent behaviors, which tightly couple sensing and acting
...

In order to simplify control and coordination of behaviors, an agent should
rely on straightforward, boolean activation mechanisms (e
...
IRM)
...

Direct perception (affordances) reduces the computational complexity of
sensing, and permits actions to occur without memory, inference, or interpretation
...

UNRESOLVED ISSUES

Unfortunately, studying natural intelligence does not give a complete picture of how intelligence works
...

Should the designer specify dominant behaviors? Combine? Let conﬂicting behaviors cancel and have alternative behavior triggered? Indeed, one
of the biggest divisions in robot architectures is how they handle concurrent behaviors
...
However, the ethological and cognitive literature is unsure of
the mechanisms for learning
...
One major
advantage that animal intelligence has over robotic intelligence is evolution
...
But robots are
expensive and only a small number are built at any given time
...
This puts tremendous pressure
on robot designers to get a design right the ﬁrst time
...

3
...
5 will provide a case study of a robot which was programmed to follow white lines in a path-following competition by using the affordance of
white
...
Fortunately
that design ﬂaw was compensated for when the robot got back on course by
reacting to a row of white dandelions in seed
...
One of the
most problematic attributes of the Reactive Paradigm, Ch
...

Since a psychologist can’t predict with perfect certainty what a human will
do under a stressful situation, it seems reasonable that a roboticist using principles of human intelligence wouldn’t be able to predict what a robot would
do either
...

3
...

A behavior is deﬁned as a mapping of sensory inputs to a pattern of motor
actions which then are used to achieve a task
...
IRMs model intelligence at
Level 2 of a computational theory, describing the process but not the implementation
...
A releaser may be either
an internal state (motivation) and/or an environmental stimulus
...

Perception in behaviors serves two roles, either as a releaser for a behavior
or as the percept which guides the behavior
...
In addition to the way in which perception is used, there appear to
be two pathways for processing perception
...
Affordances are particularly attractive to roboticists because they
can be extracted without inference, memory, or intermediate representations
...

Important principles which can be extracted from natural intelligence are:

100

3 Biological Foundations of the Reactive Paradigm

Agents programs should decompose complex actions into independent
behaviors (or objects), which tightly couple sensing and acting
...

In order to simplify control and coordination of behaviors, an agent should
rely on straightforward, boolean activation mechanisms (e
...
IRM)
...

Direct perception (affordances) reduces the computational complexity of
sensing, and permits actions to occur without memory, inference, or interpretation
...

Schema theory is an object-oriented way of representing and thinking about
behaviors
...

A behavioral schema is composed of at least one motor schema and at
least one perceptual schema, plus local, behavior-speciﬁc knowledge about
how to coordinate multiple component schemas
...

A behavior schema can have multiple instantiations which act independently, and are combined
...

3
...
1
Describe the three levels in a Computational Theory
...
8 Exercises

101

Exercise 3
...

Exercise 3
...

Exercise 3
...
The animal freezes when it sees
motion (an affordance for a predator) in an attempt to become invisible
...
(This explains why squirrels freeze
in front of cars, then suddenly dash away, apparently ﬂinging themselves under the
wheels of a car
...

Exercise 3
...
Identify the affordance for a warm-blooded mammal and the associated behavior
...

Exercise 3
...
If you have seen FSAs, consider a
FSA where the behaviors are states and releasers serve as the transitions between
state
...

Exercise 3
...
Build robots which:
a
...
Phototaxis: follow a black line (use the IR sensor to detect the difference between
light and dark)
c
...
8
What is the difference between direct perception and recognition?
Exercise 3
...
Do you
think the cockroach is using direct perception or recognition of a hiding place? Explain why
...
10
Describe how cancellation could happen as a result of concurrency and incomplete
FSA
...
11
Read the ﬁrst four chapters in Braitenberg’s Vehicles
...
List and describe the principles of behaviors for robotics in Ch
...

b
...
Be
speciﬁc, citing which vehicle illustrates what principle or attribute discussed in
the book
...
Discuss any ﬂaws in the reasoning or inconsistency between Vehicles with the biological foundations of reactivity or computer science
...
12

[Advanced Reading]

Read “Sensorimotor transformations in the worlds of frogs and robots,” by Arbib and
Liaw
...
List and describe how the principles of schema theory and potential ﬁelds for robotics given in Ch
...

b
...

3
...

Valentino Braitenberg’s Vehicles: Experiments in Synthetic Psychology 25 is the cult classic of AI roboticists everywhere
...

About David Marr
...
As is discussed in his book, Vision, 88 Marr had come from England to work in
the MIT AI lab on computer vision
...
His preface to the book
is heartbreaking
...

Howard Gardner’s The Mind’s New Science 56 gives a nice readable overview of cognitive psychology
...

3
...
J
...
J
...

While J
...
Gibson is very well-known, his wife Jackie (E
...
Gibson) is also a prominent
cognitive psychologist
...
She raised a family, ﬁnished a PhD, and publishes well-respected
studies on learning
...
J
...
J
...
The Picks are at the University of Minnesota, and Herb Pick
has been active in the mobile robotics community
...

Isaac Asimov’s robot stories often feature Dr
...
In the stories, Calvin is often the only person who can ﬁgure out the complex
interactions of concurrent behaviors leading to a robot’s emergent misbehavior
...

4

The Reactive Paradigm

Chapter Objectives:
Deﬁne what the reactive paradigm is in terms of i) the three primitives
SENSE, PLAN, and ACT, and ii) sensing organization
...

Describe the two dominant methods for combining behaviors in a reactive
architecture: subsumption and potential ﬁeld summation
...

Be able to program a behavior using a potential ﬁeld methodology
...

4
...
The Reactive Paradigm emerged in the late
1980’s
...

First, robotic systems in limited task domains are still being constructed using reactive architectures
...
7
...
1 Horizontal decomposition of tasks into the S,P,A organization of the Hierarchical Paradigm
...
Although various reactive systems may or may not strictly adhere to principles of biological intelligence, they generally mimic some aspect of biology
...
4
...

Instead, an examination of the ethological literature suggests that intelligence is layered in a vertical decomposition, shown in Fig
...
2
...
The parallel tracks can be thought of layers, stacked vertically
...
If anything happens to an advanced behavior, the lower level behaviors would still operate
...
Functions in the brain stem (such as
breathing) continue independently of higher order functions (such as counting, face recognition, task planning), allowing a person who has brain damage from a car wreck to still breathe, etc
...
Brooks took an approach now known as subsumption
and built insect-like robots with behaviors captured in hardware circuitry
...
1 Overview

build maps
explore

actuators

sensors
wander
avoid collisions

SENSE

ACT

SENSE

ACT

SENSE

ACT

SENSE

ACT

Figure 4
...

Arkin and Payton used a potential ﬁelds methodology, favoring software
implementations
...
The Reactive Paradigm
was initially met with stiff resistance from traditional customers of robotics, particularly the military and nuclear regulatory agencies
...
In particular,
reactive behaviors are not amenable to mathematical proofs showing they
are sufﬁcient and correct for a task
...

108

4 The Reactive Paradigm

The major theme of this chapter is that all reactive systems are composed
of behaviors, though the meaning of a behavior may be slightly different in
each reactive architecture
...
The two representative architectures, subsumption and potential
ﬁelds, are compared and contrasted using the same task as an example
...

4
...
As in ethological systems, behaviors are a
direct mapping of sensory inputs to a pattern of motor actions that are then used
to achieve a task
...
For
the purposes of this book, a behavior will be treated as a schema, and will
consist of at least one motor schema and one perceptual schema
...
Keep in mind that few reactive robot
architectures describe their behaviors in terms of schemas
...

The Reactive Paradigm literally threw away the PLAN component of the
SENSE, PLAN, ACT triad, as shown in Fig
...
3
...

The S-A organization does not specify how the behaviors are coordinated
and controlled; this is an important topic addressed by architectures
...
Each behavior has its own dedicated sensing
...
But in other
cases, more than one behavior can take the same output from a sensor and
process it differently (via the behavior’s perceptual schema)
...
Fig
...
4
graphically shows the sensing style of the Reactive Paradigm
...
Sensing is immediately available to the be-

4
...
3 S-A organization of the Reactive Paradigm into multiple, concurrent behaviors
...
If a computationally inexpensive
affordance is used, then the sensing portion of the behavior is nearly instantaneous and action is very rapid
...
In fact, Brooks
was fond of saying (loudly) at conferences, “we don’t need no stinking representations
...
For
example, extracting a red region in an image is non-trivial with a computer
compared with an animal seeing red
...

In early implementations of the reactive paradigm, the idea of “one sensor,
one behavior” worked well
...
This
type of sensor fusion is permitted within the reactive paradigm as long as
the fusion is local to the behavior
...
6
...
4 Behavior-speciﬁc sensing organization in the Reactive Paradigm: sensing
is local, sensors can be shared, and sensors can be fused locally by a behavior
...
2
...
This is a broad, vague deﬁnition
...

The primary connotation of a reactive robotic system is that it executes
rapidly
...
Behaviors can be
implemented directly in hardware as circuits, or with low computational
complexity algorithms (O(n))
...
Behaviors execute not only fast in their own right,
they are particularly fast when compared to the execution times of Shakey
and the Stanford Cart
...
In practice, many behaviors exhibit a

4
...
The main point is
that behaviors are controlled by what is happening in the world, duplicating
the spirit of innate releasing mechanisms, rather than by the program storing and remembering what the robot did last
...

The ﬁve characteristics of almost all architectures that follow the Reactive
Paradigm are:

SITUATED AGENT

ECOLOGICAL ROBOTICS

1
...
As seen earlier in
Part I, situated agent means that the robot is an integral part of the world
...
When a robot acts, it changes the
world, and receives immediate feedback about the world through sensing
...
Notice that situatedness is deﬁned by Neisser’s Action-Perception Cycle
...
To emphasize this, many robotic researchers say
they are working on ecological robotics
...
Behaviors serve as the basic building blocks for robotic actions, and the overall
behavior of the robot is emergent
...
The overall behavior is emergent: there
is no explicit “controller” module which determines what will be done, or
functions which call other functions
...
As with animals, the “intelligence” of the robot is in the eye of the beholder, rather than in a speciﬁc section of code
...
Recall from
Chapter 3 that these mechanisms include combination, suppression, and
cancellation
...
Only local, behavior-speciﬁc sensing is permitted
...
Any sensing which does require representation is expressed in ego-centric (robot-centric) coordinates
...
An ego-centric representation means that it
does not matter that an obstacle is in the world at coordinates (x,y,z), only

112

4 The Reactive Paradigm

where it is relative to the robot
...
g
...

4
...
The modularity of these behaviors supports the decomposition of a task into component behaviors
...

5
...
Unlike in the early days of AI robotics, where there was a
conscious effort to not mimic biological intelligence, it is very acceptable
under the reactive paradigm to use animals as a motivation for a collection
of behaviors
...
2
...
Programming by behavior has a number of
advantages, most of them consistent with good software engineering principles
...
e
...
Behaviors also support incremental
expansion of the capabilities of a robot
...
The behavioral decomposition results in an implementation that works in real-time and is usually computationally inexpensive
...
If the behaviors are implemented poorly, then a reactive implementation can be slow
...

Behaviors support good software engineering principles through decomposition, modularity and incremental testing
...
Low coupling means that the
modules can function independently of each other with minimal connections
or interfaces, promoting easy reuse
...

4
...
The examples in Sec
...
3 and 4
...

4
...
3

RULE ENCODING

4
...
The behaviors can either be new or use
existing behaviors
...
Therefore a reactive architecture must provide mechanisms for 1) triggering behaviors and 2) for determining what happens when
multiple behaviors are active at the same time
...
Keep in mind that the deﬁnitions presented in
Sec
...
2 are a generalization of the trends in reactive systems, and do not
necessarily have counterparts in all architectures
...
The two
best known and most formalized are the subsumption and potential ﬁeld
methodologies
...
Potential Field Methodologies require behaviors to be implemented as potential
ﬁelds, and the behaviors are combined by summation of the ﬁelds
...
The rules for combining behaviors are often ad hoc, and so will not be covered in this book
...

Subsumption Architecture
Rodney Brooks’ subsumption architecture is the most inﬂuential of the purely
Reactive Paradigm systems
...
As seen
in Fig
...
5, these robots actually looked like shoe-box sized insects, with
six legs and antennae
...
Furthermore, the robots were the ﬁrst to be able

114

4 The Reactive Paradigm

Figure 4
...
(Photograph courtesy of the MIT Artiﬁcial Intelligence Laboratory
...

The term “behavior” in the subsumption architecture has a less precise
meaning than in other architectures
...
The modules are augmented ﬁnite
state machines AFSM, or ﬁnite state machines which have registers, timers,
and other enhancements to permit them to be interfaced with other modules
...
In terms of schema theory,
a subsumption behavior is actually a collection of one or more schemas into
an abstract behavior
...
Four interesting aspects of subsumption in terms of releasing and control are:
LAYERS OF
COMPETENCE

1
...
The layers reﬂect a hierarchy of intelligence, or competence
...
3 Subsumption Architecture

115

more goal-directed actions such as mapping
...

LAYERS CAN SUBSUME
LOWER LAYERS

NO INTERNAL STATE

TASKABLE

4
...
1

L EVEL 0: AVOID

POLAR PLOT

2
...
The behavioral layers operate concurrently and independently, so there needs to be a mechanism to handle
potential conﬂicts
...

3
...
Internal state in this case means any
type of local, persistent representation which represents the state of the
world, or a model
...
If the robot depends on
an internal representation, what it believes may begin to dangerously diverge from reality
...

4
...
However, in practice, subsumption style systems are not easily taskable, that is, they can’t be ordered
to do another task without being reprogrammed
...
A
robot capable of moving forward while not colliding with anything could be
represented with a single layer, Level 0
...

Following Fig
...
6, the SONAR module reads the sonar ranges, does any
ﬁltering of noise, and produces a polar plot
...
As shown in
Fig
...
7, the polar plot can be “unwound
...
If the robot was moving forward, it now
stops
...

It treats each sonar reading as a repulsive force, which can be represented

116

4 The Reactive Paradigm

FEEL
FORCE

RUN
AWAY

force

TURN
heading

polar
plot

SONAR

heading

COLLIDE

encoders

FORWARD
halt

Figure 4
...

0

1
6
2
3
5

range reading

7

0 1 2 3 4
sonar number

4
a
...

Figure 4
...
) “robo-centric” view of range
readings along acoustic axes, and b
...

as a vector
...
FEELFORCE can be thought of as summing
the vectors from each of the sonar readings
...

The repulsive vector is then passed to the TURN module
...
TURN
also passes the vector to the FORWARD module, which uses the magnitude of
the vector to determine the magnitude of the next forward motion (how far
or how fast)
...

The observable behavior is that the robot will sit still if it is in an unoccupied space, until an obstacle comes near it
...
3 Subsumption Architecture

117

Figure 4
...

the robot, the robot will turn 180 the other way and move forward; essentially, it runs away
...
The robot
can react to an obstacle if the obstacle (or robot) is motionless or moving; the
response is computed at each sensor update
...
So it will stop, turn and begin to move forward again
...
Level 0 shows how a fairly complex set of actions
can emerge from very simple modules
...
4
...
Note how this looks like the vertical decomposition in Fig
...
2: the sensor data ﬂows through the concurrent behaviors to
the actuators, and the independent behaviors cause the robot to do the right
thing
...
For the purposes of this book, a behavior must
consist of a perceptual schema and a motor schema
...
For
Level 0, the perceptual schemas would be contained in the FEELFORCE and
COLLIDE modules
...
COLLIDE combines both perceptual processing (extracts the vector for
the sonar facing directly ahead) and the pattern of action (halt if there is a

118

4 The Reactive Paradigm

heading

WANDER

AVOID
modified
heading

force

FEEL
FORCE

force

RUN
AWAY

S

TURN

heading

SONAR

polar
plot

heading

COLLIDE

encoders

FORWARD
halt

Figure 4
...

reading)
...

One might be called the runaway behavior and the other the collide behavior
...

It should also be noticed that the behaviors used direct perception, or affordances
...

Consider building a robot which actually wandered around instead of sitting motionless, but was still able to avoid obstacles
...
4
...
In
this case, Level 1 consists of a WANDER module which computes a random
heading every n seconds
...

It needs to pass this heading to the TURN and FORWARD modules
...
That would sacriﬁce obstacle avoidance, because TURN only accepts one input
...
Adding a new avoid module offers an opportunity to create
a more sophisticated response to obstacles
...
This results in the actual
heading being mostly in the right direction rather than having the robot turn

119

4
...
10

Level 1 recast as primitive behaviors
...
(Notice also that the AVOID module was
able to “eavesdrop” on components of the next lower layer
...

The issue now appears to be when to accept the heading vector from which
layer
...
Subsumption is done in one of two
ways:
INHIBITION

1
...
In inhibition, the output of the subsuming module is connected
to the output of another module
...
” Inhibition acts like a faucet, turning an output stream on
and off
...
suppression
...
If the output of the subsuming module is on, it replaces the normal input to the subsumed module
...

In this case, the AVOID module suppresses (marked in the diagram with
a S) the output from RUNAWAY
...
Instead, the output from AVOID goes to TURN
...
This is good
software engineering, facilitating modularity and simplifying testing
...
The robot would at least be able to preserve
its self-defense mechanism of ﬂeeing from approaching obstacles
...
4
...
Note that FEELFORCE was
used by both RUNAWAY and AVOID
...
As is often the case, behaviors are usually named after the observable action
...
The ﬁgure does not show that the AVOID and RUNAWAY behaviors
share the same FEELFORCE perceptual schema
...

Now consider adding a third layer to permit the robot to move down corridors, as shown in Fig
...
11
...
) The LOOK module examines the sonar polar plot and identiﬁes a corridor
...
) Because identifying a corridor is more computationally expensive than just extracting range data, LOOK may take longer
to run than behaviors at lower levels
...

STAYINMIDDLE subsumes the WANDER module and delivers its output to
the AVOID module which can then swerve around obstacles
...

This gives an estimate of how far off course the robot has traveled since the
last update by LOOK
...
It serves to ﬁll in
the gaps in mismatches between updates rates of the different modules
...

INTEGRATE is an example of a module which is supplying a dangerous
internal state: it is actually substituting for feedback from the real world
...
3 Subsumption Architecture
distance, direction traveled

LOOK
corridor

STAYIN
MIDDLE

INTEGRATE
heading
to middle

WANDER

S

AVOID
modified
heading

force

FEEL
FORCE

force

RUN
AWAY

S

TURN

heading

SONAR

polar
plot

heading

COLLIDE

encoders

FORWARD
halt

Figure 4
...

erate without any sensor data forever
...
If the suppression from STAYINMIDDLE ran for longer than n seconds
with out a new update, the suppression would cease
...

Of course, a new problem is how does the robot know that it hasn’t started
going down the hallway it just came up? Answer: it doesn’t
...
If it’s not, the robot does not behave as intended
...
”

4
...
2

Subsumption summary
To summarize subsumption:
Subsumption has a loose deﬁnition of behavior as a tight coupling of sensing and acting
...
It groups schema-like modules into layers of
competence, or abstract behaviors
...
From a programming standpoint, this may seem strange
...
Recall that the ﬂeeing behavior in frogs (Ch
...

The design of layers and component behaviors for a subsumption implementation, as with all behavioral design, is hard; it is more of an art than
a science
...

There is nothing resembling a STRIPS-like plan in subsumption
...

Subsumption solves the frame problem by eliminating the need to model
the world
...
There may be some
perceptual persistence leading to a ﬁxed-action pattern type of behavior
(e
...
, corridor following), but there is no mechanism which monitors for
changes in the environment
...

Perception is largely direct, using affordances
...

Perception is ego-centric and distributed
...
A new polar plot was
created with each update of the sensors
...
Output from perceptual schemas can be
shared with other layers
...
4

Potential Fields Methodologies
Another style of reactive architecture is based on potential ﬁelds
...
4 Potential Fields Methodologies

VECTORS
VECTOR SUMMATION

4
...
1

ARRAY REPRESENTING
A FIELD

123

describe here, so instead a generalization will be presented
...

Visualizing potential ﬁelds
The ﬁrst tenet of a potential ﬁelds architecture is that the motor action of a
behavior must be represented as a potential ﬁeld
...
As described earlier, a vector is a mathematical construct
which consists of a magnitude and a direction
...
They are typically drawn as an arrow, where
the length of the arrow is the magnitude of the force and the angle of the
arrow is the direction
...
A vector can also be written as a tuple (m d), where m
stands for magnitude and d for direction
...
0 and 1, but the magnitude can be any real number
...
In most robotic applications, the
space is in two dimensions, representing a bird’s eye view of the world just
like a map
...
Each
element of the array represents a square of space
...
The force ﬁeld is analogous to a magnetic or gravitation ﬁeld
...
The
vector in each element represents the force, both the direction to turn and the
magnitude or velocity to head in that direction, a robot would feel if it were
at that particular spot
...

Fig
...
12 shows how an obstacle would exert a ﬁeld on the robot and make
it run away
...
If the robot is not within
range of the obstacle, it just sits there because there is no force on it
...
The ﬁeld isn’t
concerned with how the robot came to be so close to the obstacle; the robot
feels the same force if it were happening to move within range or if it was
just sitting there and someone put their hand next to the robot
...
12 Example of an obstacle exerting a repulsive potential ﬁeld over the radius of 1 meter
...
Another way is to think of them as a potential energy surface
in three dimensions (gravity is often represented this way) and the robot as
a marble
...
Hills in the surface cause the robot to roll away or
around (vectors would be pointing away from the “peak” of the hill), and
valleys would cause the robot to roll downward (vectors pointing toward
the bottom)
...
Fig
...
13 shows a uniform ﬁeld
...
No matter where it got set down
and at what orientation, it would feel a need to turn to align itself to the
direction the arrow points and to move in that direction at a velocity propor-

4
...
13 Five primitive potential ﬁelds: a
...
) perpendicular, c
...
) repulsion, and e
...

PERPENDICULAR FIELD

ATTRACTIVE FIELD

TANGENTIAL FIELD

tional to the length of the arrow
...
”
Fig
...
13b shows a perpendicular ﬁeld, where the robot is oriented perpendicular to some object or wall or border The ﬁeld shown is directed away
from the gray wall, but a perpendicular ﬁeld can be pointed towards an object as well
...
4
...
The circle at the center of the ﬁeld
represents an object that is exerting an attraction on the robot
...
Attractive ﬁelds
are useful for representing a taxis or tropism, where the agent is literally
attracted to light or food or a goal
...
4
...
Repulsive ﬁelds are commonly associated
with obstacles, or things the agent should avoid
...

The ﬁnal primitive ﬁeld is the tangential ﬁeld in Fig
...
13e
...
Tangential ﬁelds can “spin”
either clockwise or counterclockwise; Fig
...
13 shows a clockwise spin
...

4
...
2

MAGNITUDE PROFILE

(4
...
4
...
The way the magnitude of vectors in the ﬁeld change is called the
magnitude proﬁle
...
)
Consider the repulsive ﬁeld in Fig
...
12
...
This can be visualized with a plot of the magnitude
shown in Fig
...
14a
...
As soon as the robot gets out of range
of the obstacle, the velocity drops to 0
...
The ﬁeld is
essentially binary: the robot is either running away at a constant speed or
stopped
...
It leads
to jerky motion on the perimeter of the range of the ﬁeld
...
It runs away, leaving the ﬁeld almost immediately, and turns back to its
original path, encounters the ﬁeld again, and so on
...
They also
make it possible for a robot designer to represent reﬂexivity (that a response
should be proportional to the strength of a stimulus) and to create interesting
responses
...
4
...
It can be described as how

127

4
...

a
...

Figure 4
...
) constant
magnitude, b
...
) exponential drop off
...
4
...
Mathematically, this is called
a linear drop off , since the rate at which the magnitude of the vectors drops off
can be plotted as a straight line
...
b biases where the line starts,
and m is the slope (m = y )
...
If it is not
x
speciﬁed, m = 1 or -1 (a 45 slope up or down) and b = 0 in linear functions
...
4
...
But it shares the problem of the constant magnitude proﬁle in the sharp transition to 0
...

Therefore, another proﬁle might be used to capture the need for a strong
reaction but with more of a taper
...
The exponential proﬁle is shown in Fig
...
14c
...
The motivation for using magnitude proﬁles is to ﬁne-tune the
behavior
...
The ﬁgures display the entire ﬁeld for
all possible locations of the robot
...
Second, since
ﬁelds are continuous representations, it simpliﬁes conﬁrming that the ﬁeld is
correct and makes any abrupt transitions readily apparent
...
The strength of a potential ﬁeld can be a
function of the stimulus, regardless of distance
...
3 the feeding behavior of baby arctic terns where the feeding behavior
is guided by the stimulus “red
...

The bigger and redder an object in the baby’s ﬁeld of view, the stronger the
attraction, suggesting that a magnitude proﬁle using an increasing exponential function
...

4
...
4
...
The visualization of the entire ﬁeld may appear
to indicate that the robot and the objects are in a ﬁxed, absolute coordinate
system, but they are not
...
This should become clear through
the following examples
...
The
vector impacting the robot is computed each update
...
The designer has decided
that a repulsive ﬁeld with a linear drop off is appropriate
...
2)

Vdirection = ;180
( (D;d)
for d <= D
D
Vmagnitude =
0 for d > D
where D is the maximum range of the ﬁeld’s effect, or the maximum distance at which the robot can detect the obstacle
...
It can be the range at which the robot should respond to a stimulus
...
In practice, a roboticist might set a D of 2 meters
...
0 Vmagnitude 1
...

Below is a C code fragment that captures the repulsive ﬁeld
...
direction = -180; //turn around!
outputVector
...
direction=0
...
magnitude=0
...
The runaway behavior will use the repulsive() function as the motor schema, and a function
readSonar() as the perceptual schema
...
runaway is called by the robot on every update cycle
...
direction);
forward(Vrunaway
...
4
...
The second attribute is that it combines behaviors not by one subsuming others, but by vector summation
...
This section
provides two examples of how multiple behaviors arise and how they are
implemented and combined
...
3m in direction ”) and encounters an obstacle
...
4 Potential Fields Methodologies

131

Figure 4
...

schema of the move2goal behavior is represented with an attractive potential ﬁeld, which uses the shaft encoders on the robot to tell if it has reached
the goal position
...
Fig
...
15 shows a bird’s eye view
of an area, and shows a visualization of the potential ﬁeld
...
4
...
The runaway behavior in
Fig
...
15 exerts a repulsive ﬁeld in a radius around the obstacle (technically
the repulsive ﬁeld extends over all of the space as does the move2goal, but
the magnitude of the repulsion is 0
...
The combined ﬁeld
is shown in Fig
...
16c
...
4
...
At time t0 , the robot senses the
world
...
16 Potential ﬁelds from the world in Fig
...
15: a
...
) attractive from the goal, and c
...

of 0
...
It moves on a straight line for the goal
...
Both behaviors contribute
a vector; the vectors are summed and the robot now moves off course
...
At t4 , it resumes course and reaches the goal
...
Notice that the distance (length of the arrows) between updates is different
...
If the robot has a “shorter” vector, it travels more slowly and, therefore, covers less distance in the same amount of time
...
As a result, the path is jagged
with sharp lines
...
4 Potential Fields Methodologies

Figure 4
...

update interval
...
3 meters
from where the robot started)
...
(Programmers usually put
a tolerance around the goal location, for example instead of 10
...
3m+/- 0
...
)
Potential ﬁelds treat the robot as if it were a particle that could change velocity and direction instantaneously
...
Research
robots such as Kheperas (shown in Fig
...
18) can turn in any direction in
place, but they have to be stopped and there is a measurable amount of error
due to the contact between the wheels and the surface
...

A third problem is that the ﬁelds may sum to 0
...
Returning to Fig
...
16,
draw a line between the Goal and the Obstacle
...
This means that the magnitude is 0
...
This is called the
local minima problem, because the potential ﬁeld has a minima, or valley,
that traps the robot
...

134

4 The Reactive Paradigm

Figure 4
...
) a khepera on a ﬂoppy disk, b
...
) orientation of the IR sensors
...
4
...
Fig
...
18 shows a layout of the IR range sensors on a
Khepera robot
...
If a sensor receives a range
reading, something is in front of that sensor
...
The IR sensor isn’t capable
of telling that the obstacle may be a little off the sensor axis, so a reading is
treated as if an obstacle was straight in front of that sensor and perpendicular
to it
...
But what if, as in the case of a Khepera, the robot has

4
...
The common way is to have a RUNAWAY behavior for each
sensor
...
Below is
a code fragment showing multiple instantiations; all that had to be done is
add a for loop to poll each sensor
...

while (robot==ON) {
vector
...
dir=0
...
direction);
forward(vector
...
4
...
Each instance contributes
a vector, some of which have a X or Y component that cancels out
...
Recall from Ch
...
In this case, the
program is treating the robot as if it had 8 independent eyes!
From a robotics standpoint, the example illustrates two important points
...
Second, behavioral
programming is consistent with good software engineering practices
...
122 Functional cohesion is desirable, because it
means the function is unlikely to introduce side effects into the main program
or be dependent on another function
...
122 Data coupling
is good, because it means all the functions are independent; for example, the
program can be easily changed to accommodate more IRs sensors
...
19 Khepera in a box canyon a
...
) vectors from each
instance of Runaway (gray) and the summed output vector
...
4
...
One approach is to sum all 8 vectors internally
...
)
This is not as elegant from a software engineering perspective because the
code is now speciﬁc to the robot (the function is said to have procedural cohesion), 122 and can be used only with a robot that has eight range sensors at
those locations
...
This makes the behavior more selective
...
This can be

4
...
In the case of Level 0 in subsumption, if there are no obstacles
within range, the robot feels no repulsive force and is motionless
...

In the subsumption example, it could be imagined that these vectors were
summed in the RUNAWAY module as shown in Fig
...
20
...
The RUNAWAYpf behavior uses a repulsive potential ﬁeld
...

while (robot==ON)
{
vector
...
direction=0;
for (i=0; i<=numberSonars; i++) {
reading=readSonar();
//perceptual schema
currentVector=runaway(reading); // motor schema
vector = vectorSum(vector, currentVector);
}
turn(vector
...
magnitude*MAX-VELOCITY);
}

The COLLIDE module in subsumption does not map over to a behavior
in a potential ﬁelds methodology
...
This ﬁts the deﬁnition of a behavior: it has a sensory input
(range to obstacle = 0) and a recognizable pattern of motor activity (stop)
...
If it were treated as a behavior, the vector it contributes to would be summed with any other vectors contributed
by other behaviors
...
Instead,
collisions are often treated as “panic” situations, triggering an emergency
response outside the potential ﬁeld framework
...
The same functionality can
be accomplished by adding only a single instance of the WANDER behavior,
as shown in Fig
...
21
...
20 Level 0 redone as Potential Fields Methodology
...
This would be represented by a uniform ﬁeld
where the robot felt the same attraction to go a certain direction, regardless of
location, for n seconds
...
The WANDER vector is summed with the repulsive vectors,
and as a result, the robot moves both away from the obstacles and towards
the desired direction
...
4
...
The primary differences
in this example are that potential ﬁelds explicitly encapsulate sensing and
acting into primitive behaviors, and it did not have to subsume any lower
behaviors
...

Now consider how Level 3, corridor following, would be implemented in
a potential ﬁeld system
...
The robot would have two concurrent behaviors: RUNAWAYpf and follow-corridor
...
In the parlance of potential
ﬁelds, the task of following a corridor requires only two behaviors, while the
task of wandering requires two different behaviors
...
4 Potential Fields Methodologies

Figure 4
...

Figure 4
...

139

140

4 The Reactive Paradigm

Figure 4
...
) perpendical and b
...
) a
follow-corridor ﬁeld
...
As shown in Fig
...
23, it would be desirable for
the robot to stay in the middle of the corridor
...
Notice that both ﬁelds have a linear
decrease in magnitude as the ﬁeld nears the center of the corridor
...

Also notice that the two uniform ﬁelds are not sufﬁcient because they do
not permit the robot to move forward; the robot would move to the middle
of the corridor and just sit there
...
All three ﬁelds combined yield a smooth
ﬁeld which sharply pushes the robot back to the middle of the corridor as a
function of its proximity to a wall
...
The ﬁgure below shows the ﬁelds involved
...
4 Potential Fields Methodologies

SEQUENCING AND
PARAMETERIZING
POTENTIAL FIELDS

SELECTIVE
ATTRACTION

141

member, in this example, the robot is not projecting past or future boundaries
of the corridor; the visualization of the ﬁeld makes it appear that way
...
Why not just use a single uniform
parallel ﬁeld for follow-corridor? First, behaviors are independent
...
Second, the polar symmetry
of the repulsive ﬁelds may cause see-sawing, so there is a practical advantage
to having a separate behavior
...
If the robot knows the width of the hall a priori,
follow-corridor can suppress instances of avoid which are using obstacles it decides form the boundary of the wall
...
If there are no obstacles, follow-corridor
will produce a smooth trajectory
...

The motor schemas for a behavior may be sequenced
...
12 Docking is when a robot moves to a speciﬁc
location and orientation relative to a docking station
...
In order to accept a piece of material to carry, the robot has to be close enough to the correct side of the end
of a conveyor and facing the right way
...

That ﬁeld would have the robot make a bee-line for the dock, even if it was
coming from behind; the robot would stop at the back in the wrong position
and orientation
...
Here the robot only “feels” the attractive force when it is within a certain angular range
of the docking station, as shown in Fig
...
24
...
How does the robot move to
an area where the selective attraction ﬁeld can take effect? One way to do
this is to have tangential ﬁeld, which makes the robot orbit the dock until
it gets into the selective attraction area
...
4
...

142

4 The Reactive Paradigm

Figure 4
...

An interesting aspect of the docking behavior is that the robot is operating
in close proximity to the dock
...
In this
case, the docking behavior would lower the magnitude (gain) of the output
vector from an avoid instantiated in the dock area
...
Also, the magnitude or gain
can deﬁne the correct distance: the robot stops where the selective attraction
of the dock balances with the repulsion
...
If the robot can’t see the dock, it can’t deploy
the ﬁelds
...
Therefore an attractive force
attracts the robot to the vicinity of the dock, and then when the robot sees
the dock, it begins the funnel effect into the correct position and orientation,
even in the presence of obstacles, as seen in Fig
...
26
...

One is needed to extract the relative direction of the dock for the regular
attraction
...
The third perceptual schema is needed for the selective attention ﬁeld;
it has to be able to respond to the front of the dock and extract the robot’s
relative distance and orientation
...

A schema-theoretic representation indicates that the behavior has some co-

4
...
25
course
...
In the
case of the docking behavior, a type of ﬁnite state machine is a reasonable
choice for coordinating the sequence of perceptual and motor schemas
...
For instance, the
robot could be moving toward the dock under the tangential plus selective
attention ﬁelds when a person walks in front
...
26 Visualization of the docking behavior with obstacles
...
The general attraction motor schema would then be reactivate,
along with the associated perceptual schema to estimate the direction of the
dock
...

The docking behavior also illustrates how the sensing capabilities of the
robot impact the parameters of the motor schemas
...
Likewise the radius of
the tangential and selective attraction ﬁelds is determined by the distance at
which the robot can perceive the dock
...
4 Potential Fields Methodologies

4
...
8

NAVIGATION
TEMPLATES

145

Advantages and disadvantages
Potential ﬁeld styles of architectures have many advantages
...
As a result, it is easier for the designer to visualize the robot’s
overall behavior
...
The potential ﬁelds can be parameterized: their range of inﬂuence can be limited and any continuous function
can express the change in magnitude over distance (linear, exponential, etc
...

Building a reactive system with potential ﬁelds is not without disadvantages
...
Return to Fig
...
19, the box canyon
...
The box canyon problem is an example of reaching a local
minima
...
One
of the earliest was to always have a motor schema producing vectors with a
small magnitude from random noise
...

Another solution is that of navigation templates (NaTs), as implemented
by Marc Slack for JPL
...
The
minima problem would go away if the avoid potential ﬁeld was somehow
smarter
...
This vector represents the direction the robot would
go if there were no obstacles nearby
...
If the robot has a
strategic vector, that vector gives a clue as to whether an obstacle should be
passed on the right or the left
...
4
...
Note that the strategic vector deﬁnes what is left and
what is right
...
The repulsion ﬁeld is now supplemented with
a tangential orbit ﬁeld
...
27 Problem with potential ﬁelds: a
...
) the use of NaTs
to eliminate the problem
...
The output of the avoid behavior can be called a tactical
vector, because it carries out the strategic goal of the robot in the face of immediate challenges, just as a military general might tell a captain to capture
a fort but not specify each footstep
...
40 Potential ﬁelds implemented as harmonic
functions are guaranteed not to have a local minima of 0
...

To summarize the major points about potential ﬁelds architectures:
Behaviors are deﬁned as consisting of one or more of both motor and perceptual schemas and/or behaviors
...

All behaviors operate concurrently and the output vectors are summed
...
5, there may be abstract behaviors which internally sequence be-

4
...
The coordinated control program is not speciﬁed; the designer
can use logic, ﬁnite state machines, whatever is deemed appropriate
...

Although all behaviors are treated equally, behaviors may make varying
contributions to the overall action of the robot
...
This means that behaviors can inhibit or excite other
behaviors, although this is rarely used in practice
...

Perception can be shared by multiple behaviors
...

4
...

Essentially, they are equivalent
...
Subsumption seems to
favor a composition suited for a hardware implementation, while potential
ﬁelds methods have nice properties for a software-oriented system
...
Indeed, the use of direct perception
emphasizes that reactive robots are truly constructed to ﬁll a niche
...
Reactive systems are limited to applications
which can be accomplished with reﬂexive behaviors
...
(this led to the Hybrid Paradigm to be described in
Ch
...
In practice, very few of the subsumption levels can be ported to new
applications of navigating in an environment without some changes
...
The potential ﬁelds methodology performs a bit better in that the

148

4 The Reactive Paradigm

designer can create a library of behaviors and schemas to choose from, with
no implicit reliance on a lower layer
...
The layering of subsumption imparts some graceful degradation if an
upper level is destroyed, but it has no mechanisms to notice that a degradation has occurred
...
As with animals, a reactive robot will
also do something consistent with its perception of the world, but not always
the right thing
...
6

Summary
Under the Reactive Paradigm, systems are composed of behaviors, which
tightly couple sensing and acting
...
Sensing in the Reactive
Paradigm is local to each behavior, or behavior-speciﬁc
...
A
behavior may create and use its own internal world representation, but there
is no global world model as with the Hierarchical Paradigm
...

There are four major characteristics of robots constructed under the Reactive Paradigm
...
As a consequence of using behaviors, the overall behavior
of the robot is emergent
...

The use of explicit representations in perceptual processing, even locally, is
avoided in most reactive systems
...
Instead, reactive behaviors rely on the world to maintain state (as exempliﬁed by the gripper controlling whether the robot was looking for soda
cans or for the recycling bin)
...
Behaviors and groups of behaviors which
were inspired by or simulate animal behavior are often considered desirable and more interesting than hacks
...
Reactive systems are inherently modular from a software design
perspective
...
7 Exercises

149

ior is emergent
...

This supports good software engineering practices, especially low coupling
and high cohesion
...
Behaviors are
purely reﬂexive and may not use memory
...

The coordination of layers is done by higher layers, which have more speciﬁc
goal-directed behaviors, subsuming lower layers
...

Potential ﬁelds methodologies are another popular reactive system
...

All active behaviors contribute a vector; the vectors are summed to produce
a resultant direction and magnitude for travel
...
The ﬁelds can be readily implemented in software, and parameterized for ﬂexibility and reuse
...
The ﬁelds are often extensible to three dimensions, adding to the
re-usability and portability
...
Ch
...

Despite the differences, subsumption and potential ﬁelds appear to be
largely equivalent in practice
...
The ease of portability to other domains is relative to
the complexity of the changes in the task and environment
...

4
...
1
Deﬁne the reactive paradigm in terms of a
...
) sensing organization
...
2
Describe the difference between robot control using a horizontal decomposition and
a vertical decomposition
...
3
List the characteristics of a reactive robotic system
...
4
Describe the differences between two dominant methods for combining behaviors in
a reactive architecture, subsumption and potential ﬁeld summation
...
5
Evaluate the subsumption architecture in terms of: support for modularity, niche
targetability, ease of portability to other domains, robustness
...
6
Evaluate potential ﬁeld methodologies in terms of: support for modularity, niche
targetability, ease of portability to other domains, robustness
...
7
What is the difference between the way the term “internal state” was used in ethology
and the way “internal state” means in behavioral robotics?
Exercise 4
...

Exercise 4
...
10
Suppose you were to construct a library of potential ﬁelds of the ﬁve primitives
...
11
Use a spreadsheet, such as Microsoft Excel, to compute various magnitude proﬁles
...
12
Return to Fig
...
17
...

Exercise 4
...
4
...
What happens if an IR breaks and
always returns a range reading of N , meaning an obstacle is N cm away? What will
be the emergent behavior? and so on
...
7 Exercises

Exercise 4
...
15
An alternative RUNAWAY behavior is to turn 90 (either left or right, depending on
whether its “left handed” or “right handed” robot) rather than 180
...
16
Using rules, or if-then statements, is a method for representing and combining programming units which are often called behaviors; for example “if OBSTACLE-ONLEFT and HEADING-RIGHT, then IGNORE
...
17
Some researchers consider random wandering as a primitive potential ﬁeld
...
How can a wander ﬁeld be represented? Does the
array of the ﬁeld represent a physical area or time? Unlike regular potential ﬁelds, the
vector is computed as a function of time, every n minutes, rather than on the robot’s
relationship to something perceivable in the world
...
18

[Programming]

Design and implement potential ﬁelds:
a
...
Why won’t a simple attractive ﬁeld work? ANS: if the
robot is coming from a side, it will graze the door frame because the robot is not a
point, it has width and limited turning radius
...
What happens if a person is exiting the door as the robot enters? Design an appropriate “avoid” potential ﬁeld, and show the emergent potential ﬁeld when AVOID
and MOVE-THRU-DOOR are activated at the same time
...
Simulate this using the Khepera simulator for Unix systems found at:
http://www
...
com
...
Run this on a real khepera
...
19

[Programming]

Program two versions of a phototropic behavior using the Khepera simulator
...
In one version, the perceptual schema processes light from a single
sensor and the behavior is instantiated 8 times
...
Set up ﬁve interesting “worlds” in the simulator with different placements of lights
...

152

4 The Reactive Paradigm

Exercise 4
...

4
...

The subsumption architecture favors a hardware implementation using inexpensive
hardware
...
The straightforward circuits allowed any hobbyist to produce intelligent robots
...

About Rodney Brooks
...
) and anthropomorphic robots (Cog, Kismet) frequently appearing in the
media
...
The documentary is well worth watching, and
there are some gorgeous shots of robots walking over broken glass giving the terrain
a luminous quality
...
” The roboticist from the Massachusetts Institute of Robotics is a combination of Steven Hawking (the character is
disabled), Joe Engleberger (the bow tie), Marvin Minsky (stern, professorial manner),
and Rodney Brooks (the character says almost direct quotes from Brooks’ interviews
in science magazines)
...

Rodney Brooks’ 1986 paper never ofﬁcially named his architecture
...

Building a robot with petty cash
...

In the late 1980’s, Miller’s group wanted to build a small reactive robot to compare
with the traditional Hierarchical vehicles currently used for research and to get experience with subsumption
...
But they realized
that the individual electronics were cheap
...
8 End Notes

153

mobile robot in batches of $50 or less, which could be reimbursed out of petty cash
...

Robot name trivia
...
Ron Arkin did his initial work on potential ﬁelds methodology
while pursuing his PhD at the University of Massachusetts
...
HARV stood for Hardly Autonomous Robot Vehicle morphing into Harmful Autonomous Robot Vehicle, when
military research sponsors were present
...

5

Designing a Reactive
Implementation

Chapter objectives:
Use schema theory to design and program behaviors using object-oriented
programming principles
...

For a given behavioral system, draw a behavioral table specifying the releasers, perceptual schemas, and motor schemas for each behavior
...
Be able
to represent a sequence of behaviors with a state diagram or with a pseudocode script
...
1

Overview
By this point, the sheer simplicity and elegance of reactive behaviors tend
to incite people to start designing and implementing their own robots
...
However, the new issues are how to program more intelligence into the software
and how to exploit better sensors than come with kits
...
First,
designing behaviors tends to be an art, not a science
...
The second deﬁcit is more subtle
...
4 was characterized by robots running a very small set of
behaviors which were combined internally to produce an overall emergent
behavior
...

However, many applications are best thought of as a series of behaviors,
each operating in a recognizable sequence
...
This involved search for a can, move toward the can
when it is found, pick up the can, search for the recycle bin, move toward
the recycle bin, drop the can
...
(There is certainly the possibility of
concurrency, for example avoiding obstacles while moving to the soda can
or recycle bin
...
Most of these techniques are conceptually the
equivalent of constructing a macro behavior, where the schema structure is
used recursively to simplify programming the control program
...
First, an objectoriented programming approach to designing behaviors is introduced
...
3
...
The case study emphasizes the importance of establishing the ecological niche of a reactive robot
...
As could be expected from the material presented in Ch
...
Finally, the chapter shows how these techniques
were applied to entries in the “Pick Up the Trash” events of the 1994 AAAI
and 1995 IJCAI Mobile Robot Competitions
...
The use of schemas to “factor out” and program a
small set of generic behaviors rather than designing a large set of specialized
behaviors is emphasized throughout the chapter
...
2 Behaviors as Objects in OOP

Schema

Behavior

coordinated control program

coordinated control program

*
Perceptual
Schema

Motor
Schema

Behavior

a
...
1

5
...

Classes: a
...
) behavior
...
Schema theory is well suited for transferring theoretical concepts to OOP
...

Recall from software engineering that an object consists of data and methods, also called attributes and operations
...
3, schemas contain speciﬁc knowledge and local data structures and other schemas
...
5
...
Following Arbib, 6 a schema as a
programming object will be a class
...
The coordinated control program is a
function that coordinates any methods or schemas in the derived class
...
Behaviors are composed of at least one Perceptual
Schema and one Motor Schema; these schemas act as the methods for the
Behavior class
...
A
Motor Schema has at least one method which transforms the percept into a
vector or other form of representing an action
...
The Perceptual Schema is linked to the sensor(s), while the Motor Schema is linked to the robot’s actuators
...

158

5 Designing a Reactive Implementation

PRIMITIVE BEHAVIOR

ABSTRACT BEHAVIORS

5
...
1

Using the Uniﬁed Modeling Language representation, 55 the Schema and
Behavior classes look like Fig
...
1
...
Another way of stating this is that the deﬁnition of a behavior is
recursive
...
Sec
...
2
...

Recall that a primitive behavior is composed of only one perceptual schema
and one motor schema; there is no need to have any coordinated control
program
...
Because they are usually a simple mapping
from stimulus to response, they are often programmed as a single method,
not composed from multiple methods or objects
...

Behaviors which are assembled from other behaviors or have multiple
perceptual schema and motor schema will be referred to as abstract behaviors, because they are farther removed from the sensors and actuators than
a primitive behavior
...

Example: A primitive move-to-goal behavior
This example shows how a primitive behavior can be designed using OOP
principles
...
129;66 The basic idea was for a robot to be placed
in an empty arena about the size of an ofﬁce
...
In two
of the four corners, there would be a blue recycling bin; in the other two,
a different colored trash bin
...
In most
years, the strategy was to ﬁnd and recycle the Coca-Cola cans ﬁrst, because
it was easier for the robot’s vision processing algorithms to perceive red and
blue
...
When the robot sees a red can, it
must move to it
...

5
...
The goal
for the current instance can be passed in at instantiation through the object
constructor
...

From a software engineering perspective, writing two behaviors which do
the same thing is an opportunity to introduce a programming bug in one
of them and not notice because they are supposed to be the same
...
Consider simplifying the equation 45x2 + 90x + 45
...
In this case, 45 can be factored
and the equation rewritten as 45(x + 1)2
...

Modular, generic code can be handled nicely by schemas as shown in
Fig
...
2
...
attraction
...
This
information forms the percept of the goal; the affordance of the Coke can is
the color, while the information extracted from the perception is the angle
and size
...
It
can do this easily by using an attractive ﬁeld, where the larger the region, the
stronger the attraction and the faster the robot moves
...
attraction(goal_angle, goal_strength)

The above table implies some very important points about programming
with behaviors:

160

5 Designing a Reactive Implementation

The behavior is the “glue” between the perceptual and motor schemas
...
Instead, the behavior puts the percept created by the perceptual schema in a
local place where the motor schema can get it
...
The pﬁelds sufﬁx on
the pfields
...
The ﬁve primitive potential ﬁelds
could be encapsulated into one class called PFields, which any motor
schema could use
...
Once the potential
ﬁelds in PFields were written and debugged, the designer doesn’t ever
have to code them again
...
In this example, the move to
goal behavior was written to accept a structure (or object) deﬁning a color
and then moving to a region of that color
...

5
...
2

Example: An abstract follow-corridor behavior
The move to goal example used a single motor schema with a single perceptual schema
...
In the corridor following example in Ch
...
The follow_corridor ﬁeld could be implemented in schemas in
at least two different ways as shown in Fig
...
2
...
The follow corridor motor schema would consist of the three primitives and the coordinated control program
...
They were summed together by the coordinated control program in the behavioral schema to produce a single output vector
...
The perceptual schema would return the distance to the left
wall and the right wall
...
2 Behaviors as Objects in OOP

follow corridor
vector summation of MS

Motor
Schema 1:
perpendicular

Motor
Schema 2:
uniform

Perceptual
Schema 1:
find-walls
(polar plot)

a
...

Figure 5
...
) use of primitive ﬁelds, and b
...

162

5 Designing a Reactive Implementation

wall (left) and follow_wall(right)
...
The associated class diagram is shown on the right in Fig
...
2
...
Since there were multiple motor schemas, the coordinated control
program for follow-corridor is not null as it was for move-to-goal
...

5
...
3

TWO PURPOSES OF
PERCEPTION

Where do releasers go in OOP?
The previous examples showed how behaviors can be implemented using
OOP constructs, such as classes
...
As was discussed in Ch
...
Perceptual schemas are clearly used for
guiding the behavior, either moving toward a distinctively colored goal or
following a wall
...
It can execute independently of whatever else is happening
with the robot; it is a perceptual schema not bound to a motor schema
...
One way to implement this is when the schema sees
red, it can signal the main program that there is red
...
move_to_goal
can instantiate a new instance of extract_color or the main program
can pass a pointer to the currently active extract_color
...
attraction, since the attraction motor schema wouldn’t be running
...
This is awkward to program
...
This
programming style requires a coordinated control program
...
The behavior returns an identity function,

5
...
0,0
...
This style of programming can tie up some
resources, but is generally a simple, effective way to program
...
5
...

Either way, once the robot saw red, the observable aspect of move to goal
(e
...
, moving directly toward the goal) would commence
...
This percept would then be available to
the motor schema, which would in turn produce a vector
...
5
...
Depending where the robot was in the sequence of activities, the robot uses move to goal to move to a red Coke can or a blue recycling
bin
...
There is nothing in the OOP design to prevent that from
happening—in fact, OOP makes it easy
...
” Notice that the move to goal behavior can use any perceptual
schema that can produce a goal angle and goal strength
...
This is an example of software reusability
...
3

Steps in Designing a Reactive Behavioral System
Fig
...
3 shows the steps in designing a reactive behavioral system, which
are taken from Behavior-Based Robotics 10 and a case study by Murphy
...

The methodology in Fig
...
3 assumes that a designer is given a task for the
robot to do, and a robot platform (or some constraints, if only budgetary)
...
Therefore, the ﬁrst three
steps serve to remind the designer to specify the ecological niche of the robot
...
It asks the question: what does the robot do? Deﬁning the ecological niche deﬁnes constraints and opportunities but doesn’t
necessarily introduce major insights into the situatedness of the robot: how it
acts and reacts to the range of variability in its ecological niche
...
Sometimes,

164

5 Designing a Reactive Implementation

1
...
Describe the robot
3
...
Describe how the robot should
act in response to its environment

5
...
Test each behavior independently

7
...
3 Steps in designing a reactive behavioral system, following basic software
engineering phases
...
For example, in the 1994 and 1995 Pick Up the
Trash events, most of the teams used a partitioning along the lines of: random search until see red, move to red, pick up can, random search until see
blue, move to blue, drop can
...
This,
of course, sidesteps the question of how the roboticist knew what class of animal tasks the robot task is similar to, as well as implies a very linear thinking process by roboticists
...

Steps 5-7 are less abstract
...
4 Case Study: Unmanned Ground Robotics Competition

165

ifying its motor and perceptual schemas
...
The designer usually programs each schema independently, then integrates them into a behavior and
tests the behavior thoroughly in isolation before integrating all behaviors
...

The list of steps in implementing a reactive system can be misleading
...
5
...

In practice, it is iterative
...

The single source of iteration may be testing all the behaviors together in the
“real world
...

5
...
98 The objective of the competition was to have a small unmanned vehicle (no larger
than a golf cart) autonomously navigate around an outdoor course of white
lines painted on grass
...

Each design step is ﬁrst presented in boldface and discussed
...
This case study illustrates
the effective use of only a few behaviors, incrementally developed, and the
use of affordances combined with an understanding of the ecological niche
...

Step 1: Describe the task
...

The task was for the robot vehicle to follow a path with hair pin turns, stationary
obstacles in the path, and a sand pit
...
The
maximum speed was 5 mph
...
If the robot
hit an obstacle enough to move it, another distance penalty was levied
...
Entrants were given three runs on one day and two days to prepare and
test on a track near the course; the times of the heats were determined by lottery
...
The purpose of this step is to determine the
basic physical abilities of the robot and any limitations
...
In practice, most roboticists work
with either a commercially available research platform which may have limitations on what hardware and sensors can be added, or with relatively inexpensive kit type of platform where weight and power restrictions may impact what it can reasonably do
...

In this case, the competition stated that the robot vehicle had to have a footprint
of at least 3ft by 3
...
Furthermore, the robot had to
carry its own power supply and do all computing on-board (no radio communication
with an off-board processor was permitted), plus carry a 20 pound payload
...
Fig
...
4 shows Omnibot
...
The base met the minimum footprint exactly
...
The vehicle had a 22 turning
angle
...
The sensor suite consisted of three devices: shaft encoders on the drive and steer motors for dead reckoning, a video camcorder mounted
on a mast near the center of the vehicle and a panning sonar mounted below the
grille on the front
...
The sonar was a Polaroid lab grade ultrasonic transducer
...
All coding was done in C++
...
5 mph
...
It also meant that the steering had to have at least a 150ms update rate
or the robot could veer out of bounds without ever perceiving it was going off course
...
Worse yet, the update
rate of the framegrabber was almost 150ms; any vision processing algorithm would
have to be very fast or else the robot would be moving faster than it could react
...
5 ft to
about 10 ft
...
4 Case Study: Unmanned Ground Robotics Competition

167

Figure 5
...

Step 3: Describe the Environment
...

First, it is a key factor in determining the situatedness of the robot
...
Recall from Chapter 4 that the Reactive Paradigm favors direct perception or affordance-based perception because it has
a rapid execution time and involves no reasoning or memory
...
The course consisted
of a 10 foot wide lane marked in US Department of Transportation white paint,
roughly in the shape of a kidney (see Fig
...
5)
...
Obstacles were all
stationary and consisted of bales of hay wrapped in either white or red plastic
...
The sonar was able to reliably detect the plastic covered bales at most angles of
approach at 8 feet away
...
In addition to the visual challenges
of changing lighting due to clouds, the bales introduced shadows on the white lines
between 9–11am and 3–5pm
...

The analysis of the environment offered a simpliﬁcation of the task
...
Since Omnibot was only 3 ft wide, the course
could be treated as having no obstacles if the robot could stay in the center of the lane

168

5 Designing a Reactive Implementation

IV

SAND PIT
FINISH

START

I

III

II

Milestone (red bale)
Rock (white bale)
Obstacle (white bale)

Figure 5
...

with a 0
...
This eliminated the need for an avoid obstacle behavior
...
The only object of interest to the robot was the white line, which should have
a high contrast to the green (dark gray) grass
...
However, if the camera was pointed directly
at one line, instead of trying to see both lines, the majority of the brightest points
in the image would belong to the line (this is a reduction in the signal to noise ratio
because more of the image has the line in it)
...
Therefore, if the
robot tried to keep the centroid of the white points in the center of the image, it would
stay in the center of the lane
...
The purpose of this step is to identify the set of one or more candidate
primitive behaviors; these candidates will be reﬁned or eliminated later
...
It should be emphasized that the point of this step is to concen-

5
...

In the case of the CSM entry, only one behavior was initially proposed: follow-line
...

In terms of expressing the behaviors for a task, it is often advantageous to
construct a behavior table as one way of at least getting all the behaviors on a
single sheet of paper
...
3
is undesirable)
...
For example, consider what happens if an implementation
has a purely reﬂexive move-to-goal motor schema and an avoid-obstacle behavior
...

Behavior Table
Releaser
always on

Behavior
follow-line()

Motor Schema
stay-on-path(c_x)

Percept
c_x

Perceptual Schema
compute-centroid(image,white)

As seen from the behavior table above, the CSM team initially proposed only one
behavior, follow-line
...
The perceptual schema, compute-centroid(image,white),
extracted an affordance of the centroid of white from the image as being the line
...

Step 5: Reﬁne each behavior
...

This step concentrates on the design of each individual behavior
...
g
...

The follow-line behavior was based on the analysis that the only white things
in the environment were lines and plastic covered bales of hay
...
As the robot was following the white line down the course, one of the judges

170

5 Designing a Reactive Implementation

stepped into view of the camera
...
The
CSM team captain, Floyd Henning, realized what was happening and shouted at the
judge to move
...
Suddenly, right before the leftmost rear wheel was about to leave the boundary, Omnibot
straightened up and began going parallel to the line! The path turned to the right,
Omnibot crossed back into the path and re-acquired the line
...
The crowd went wild, while the
CSM team exchanged confused looks
...
When it
wandered onto the grass, Omnibot went straight because the reﬂection on the grass
was largely random and the positions cancelled out, leaving the centroid always in
the center of the image
...
Next to the path was a parallel swatch of uncut grass
loaded with dandelion seed puffs
...
It was sheer luck that the path curved so that when the dandelions ran out,
Omnibot continued straight and intersected with the path
...

Step 6: Test each behavior independently
...
Ideally, testing occurs
in simulation prior to testing on the robot in its environment
...
However, it is important to remember that simulators often only
model the mechanics of the robot, not the perceptual abilities
...

Step 7: Test with other behaviors
...
This also includes testing the behaviors in the actual
environment
...
The obstacles,
shiny plastic-wrapped bales of hay sitting near the line, were often brighter than the
grass
...
Invariably the robot became ﬁxated on the bale,

5
...
The bales were referred to as “visual
distractions
...
If the robot could “close its eyes” for
about two seconds and just drive in a straight line, it would stay mostly on course
...
It used the direction of the robot (steering
angle, dir) to essentially produce a uniform ﬁeld
...

The approach to the issue of when to ignore vision was to use the sonar as a releaser
for move-ahead
...
Due to the difﬁculties in working
with DOS, the CSM entry had to use a ﬁxed schedule for all processes
...
As a result the sonar releaser for move-ahead essentially inhibited followline, while the lack of a sonar releaser inhibited move-ahead
...
Fig
...
6 shows this
inhibition, while the new behavioral table is shown below
...
It went
all the way around the track until about 10 yards from the ﬁnish line
...
The sand pit was of some concern
since sand is a light color, and might be interpreted as part of the line
...
In the end,
the team decided that since the sand pit was only half the length of a bale, it wouldn’t
have enough effect on the robot to be worth changing the delicate schedule of existing
processes
...
However, they forgot about the issue of traction
...
Once in the sand, the robot spun its wheels inside the tires
...
Indeed
it did
...

It is clear that a reactive system was sufﬁcient for this application
...
6 The behavioral layout of the CSM entry in the 1994 Unmanned Ground
Vehicle Competition
...

There are several important lessons that can be extracted from this case
study:
The CSM team evolved a robot which ﬁt its ecological niche
...
The behaviors would not work for a similar domain, such as following sidewalks, or even a path of white lines with an
intersection
...
The robot behaved “correctly”
to features of the open world, but not in a desirable fashion
...
The sonar was used to inhibit the behaviors
...

5
...

5
...
In that case, there were a trivial number of behaviors
...
The issue is how
to formally represent the releasers and their interactions into some sort of
sequencing logic
...

This raises the issue of how to encapsulate the set of behaviors and their
sequencing logic into a separate module
...
The same OOP schema
structure used to collect a perceptual schema and a motor schema into a behavior can be used to collect behaviors into an abstract behavior, as shown
by a behavior being composed of behaviors in Fig
...
1
...

This leaves the issue of how to formally represent the releasers in a way
that both the robot can execute and the human designer can visualize and
troubleshoot
...
Finite state
automata and scripts are logically equivalent, but result in slightly different ways about thinking about the implementation
...
FSA-type of behavioral coordination and control were used successfully by the winning Georgia Tech team 19
in the 1994 AAAI Pick Up the Trash event, and the winning LOLA team in the
1995 IJCAI competition for the Pick Up the Trash event
...
Those three teams used at most
eight behaviors, even though LOLA had a more sophisticated gripper than
the Georgia Tech team
...

b
...
7 Two award-winning Pick Up the Trash competitors: a
...
) University of North Carolina’s LOLA (photograph courtesy of Rich LeGrand and
AAAI)
...
53;54 Since in general skills lead to a more
complex implementation than FSA and scripts, they will not be covered here
...

5
...
1
FINITE STATE
AUTOMATA
STATE DIAGRAM

Finite state automata
Finite state automata (FSA) are a set of popular mechanisms for specifying
what a program should be doing at a given time or circumstance
...
Most designers do both
...
5 Assemblages of Behaviors

time-remaining

range far

time-elapsed
followline

M :K

moveahead
range near

a
...

Figure 5
...
) a diagram and b
...

STATES

S TART STATE

FSA, but each works about the same way
...
86
To begin with, the designer has to be able to specify a ﬁnite number of discrete states that the robot should be in
...
In the case of the UGR competition, there were only two states:
following the line and moving forward
...
(See Fig
...
8
...
The Start
state is often written as s or qo and drawn with a double circle
...

The next part of the FSA is the inputs (also called the alphabet)
...
Unlike
the IRM diagrams, the FSM table considers what happens to each state q for

176

5 Designing a Reactive Implementation

range far
range far
Start

time-remaining
time-elapsed
moveahead

followline

range near

see finish
line

victorydance

see finish
line

range near

M: K

a
...

Figure 5
...
) a diagram and b
...

TRANSITION FUNCTION

all possible inputs
...
5
...

The third part of the FSM is the transition function, called , which speciﬁes
what state the robot is in after it encounters an input stimulus,
...
These stimuli are represented by arrows
...
The new behavior triggered by the releaser
depends on the state the robot is in
...

5
...
3 that the
agent “noticed” releasers every second
...
In the next iteration, it
might still be hungry and re-enter the state of feeding
...

For the example in Fig
...
8, the robot starts in the state of following a line
...
If it does, the program may fail because the FSA clearly shows it won’t respond to the distraction for at least
one update cycle
...
Starting in the following-line state ﬁne for the UGR competition, where
it was known in advance that there were no bales of hay near the starting
line
...
5
...
The FSA doesn’t make it
clear that if the robot starts by a bale, it better be pointed straight down the
path!
The fourth piece of information that a designer needs to know is when the
robot has completed the task
...
In the UGR example,
the robot is never done and there is no ﬁnal state—the robot runs until it is
turned off manually or runs out of power
...

If the robot could recognize the ﬁnish line, then it could have a ﬁnish state
...
Notice that this adds more rows to the FSA
table, since there must be one row for each unique state
...

The resulting table is known as a ﬁnite state machine and abbreviated M for
machine
...
There must be one arrow
in the state diagram for every row in the table
...
Avoidance should always be active, so it is often implicit in the FSA
...
This implicit
grouping of “interesting task-related behaviors” and “those other behaviors
which protect the robot” will be revisited in Ch
...

Another important point about using FSA is that they describe the overall behavior of a system, but the implementation may vary
...
5
...
The control
for the behaviors could have been implemented exactly as indicated by the
FSA: if following-line is active and range returns range near, then moveahead
...
The following examples will show
how the FSA concept can be implemented with subsumption and schematheoretic systems
...
5
...
Assume that the robot is a small vehicle, such as the ones
used by Georgia Tech shown in Fig
...
7 or the Pioneer shown in Fig
...
10,
with a camera, and a bumper switch to tell when the robot has collided with
something
...
Assume that the
robot can tell if the gripper is empty or full
...
When the IR returns a short range,
the gripper is full and it can immediately close, with a grasping reﬂex
...
As such,
there is always the possibility that the can will slip or fall out of the gripper
...

The Pick Up the Trash environment is visually simple, and there are obvious affordances
...
5 Assemblages of Behaviors

and trash cans are the only blue objects
...
All objects are on the ﬂoor, so the robot
only has to worry about where the objects are in the x axis
...

It should head straight for the center of the largest red blob until it scoops
the can in the forklift
...
There
should only be one blue blob in the image at a time because the two trash
cans are placed in diagonal corners of the arena
...
The robot should stop, let go of the can,
turn around to a random direction and resume the cycle
...

The behavior table is:
Releaser
always on
EMPTY=gripper_status()
EMPTY=gripper_status()
AND
SEE_RED=extract_color(red)
FULL=gripper_status()
FULL=gripper_status()
AND
NO_BLUE=extract_color(blue)
FULL=gripper_status()
AND
SEE_BLUE=extract_color(blue)
FULL=gripper_status()
AND
AT_BLUE=looming(blue, size=N)

Behavior
avoid()
wander()
move-to-goal(red)

Motor Schema
pﬁelds
...
random()
pﬁelds
...
random()

status
time_remaining

gripper_status()
countdown()

move-to-goal(blue)

pﬁelds
...

The avoid behavior is interesting
...
It may bump an arena wall
at several locations, but eventually a new wander direction will be set
...
This table shows that the design relies on the
gripper to maintain where the robot is in the nominal sequence
...
A full gripper means the robot is in
the deposit phase
...
If the region is greater than or equal

180

5 Designing a Reactive Implementation

Figure 5
...

(Photograph courtesy of ActivMedia, Incorporated
...

There are two problems with the behavior table
...
The second is how did the
designer come up with those behaviors? This is where a FSA is particularly
helpful
...

Fig
...
11 shows a FSA that is equivalent to the behavior table
...
It does so at the cost of not
showing precisely how the sequence would be implemented and encouraging the designer to create internal states
...

Many designers draw and interpret FSA as carrying forward previous releasers
...

This is a very dangerous mistake because it assumes that the implementation
will be keeping up with what internal state the robot is in (by setting a vari-

181

5
...

K = fwander for trash, move to trash, grab trash, wander for trash can,
move to trash can, drop trash g, = fon, EMPTY, FULL, SEE_RED,
NO_BLUE, SEE_BLUE, AT_BLUEg, s = Start, F = K

q
start
wander for trash
wander for trash
move to trash
move to trash
grab trash
grab trash
grab trash
wander for trash can
wander for trash can
move to trash can
move to trash can
drop trash

q

(

on
EMPTY, SEE_RED
FULL
FULL
EMPTY, NO_RED
FULL, NO_BLUE
FULL, SEE_BLUE
EMPTY
EMPTY
FULL, SEE_BLUE
EMPTY
FULL, AT_BLUE
EMPTY

)

wander for trash
move to trash
grab trash
grab trash
wander for trash
wander for trash can
move to trash can
wander for trash
wander for trash
move to trash can
wander for trash
drop trash
wander for trash

b
...
11 A FSA for picking up and recycling Coke cans: a
...
) table showing state transitions
...
Internal state is incompatible with reactivity
...
The avoid behavior was
always running, while the other behaviors were asynchronously being instantiated and de-instantiated
...
Other techniques, most especially Petri Nets, are used in software engineering but have not been commonly used in robotics
...
It was always running and the output of the avoid potential ﬁeld vector could be summed with the output of
whatever other behavior was active
...
5
...
If the robot’s sole task was to recycle soda cans, the controlling
logic could be placed in the main program
...
In that case,
the FSA logic would be placed in the coordinated control program slot of the
behavior schema
...
While the wander-togoal and move-to-goal behaviors can be easily implemented with a potential
ﬁelds methodology, drop-trash cannot
...
It ﬁts the overall proﬁle of a behavioral schema: it has an obvious
motor schema (open the gripper, turn the wheels), a perceptual schema (read
gripper encoders and wheel encoders), a coordinated control program (open
THEN turn), and a releaser (at trash can)
...

One advantage of FSA is that they are abstract, and can be implemented
in a number of ways
...
Fig
...
12 shows one way
it could be implemented in subsumption
...

In keeping with the idea of modularity and incrementally adding behaviors, the system starts with an explicit avoid behavior running on top of Level
0 (not shown)
...
Then

183

5
...
12 One possible Pick Up the Trash abstract behavior using subsumption
...
The move-to-trash behavior continues until the can is in
the gripper
...

Then the move-to-trash behavior is inhibited from executing
...

The next level of competency is to deposit trash in the trash can
...
If the gripper is empty, the output for moveto-trash can is inhibited
...

Dropping trash also is constantly executing
...
The
new heading suppresses any heading direction from move-to-trash can
...
The subsumption example produces
a much less complex system than a direct FSA mapping
...
5
...
5
...
They fall short as a programming tool for
abstract behaviors in a number of ways
...
In the Pick Up the Trash event, recycling Coke cans was only part of the
task; the robots were also supposed to ﬁnd white Styrofoam cups and deposit
them in yellow trash cans
...

Second, templates need to handle different initialization conditions
...
For example, the emergent behavior of the
robot described in the Unmanned Ground Vehicle competition in Sec
...
4
could be described as an abstract “follow-path” behavior
...
A more general purpose, reusable follow-path behavior would need to handle a broader range
of starting conditions, such as starting facing a bale or not perfectly lined up
with the line
...
In the Pick
Up the Trash competitions, several teams literally showed the robot the Coke
can and let it determine the best values of “red” for the current lighting conditions
...
In the case of the UGR competition, the termination behaviors was
NULL, but it could have been the victory dance
...
An exception might be when the robot does not pick up a soda
can in over 10 minutes
...

Scripts
Abstract behaviors often use scripts, 49;50;87;100 or a related construct called
skills, 53;54 to create generic templates of assemblages of behaviors
...
5 Assemblages of Behaviors

Script

Collection of Behaviors

Example

Goal
Places
Actors

Task
Applicability
Behaviors

Props, Cues
Causal Chain

Percepts
Sequence of Behaviors

Subscripts

Exception Handling

pick up and throw away a Coca-Cola can
an empty arena
WANDER _ FOR _ GOAL , MOVE _ TO _ GOAL ,
GRAB _ TRASH , DROP _ TRASH
red, blue
WANDER _ FOR _ GOAL ( TRASH ), MOVE _ TO _ GOAL ( TRASH ),
GRAB _ TRASH , WANDER _ FOR _ GOAL ( TRASH CAN ),
MOVE _ TO _ GOAL ( TRASH CAN ), DROP _ TRASH
if have trash and drop, try GRAB_ TRASH three times

Figure 5
...

iors
...
Scripts were originally used in natural language
processing (NLP) to help the audience (a computer) understand actors (people talking to the computer or writing summaries of what they did)
...
The script has more room for improvization though, if the
robot encounters an unexpected condition (an exception), the robot begins
following a sub-script
...
5
...
The main sequence of events is called a causal chain
...
It can be implemented in the same way
...
For
example, consider a computer trying to read and translate a book where the
main character has stopped in a restaurant
...
This missing,
but implied, information is easy to extract
...
” This is a clue that serves as an index into the current or relevant event of the script (eating at a restaurant), skipping over past
events (John arrived at the restaurant, John got a menu, etc
...
If the next sentence is “Armand brought out the
lobster and reﬁlled the white wine,” the computer can infer that Armand is
the waiter and that John had previously ordered and received white wine,
without having been explicitly told
...
Finite state automata force the designer to consider
and enumerate every possible transition, while scripts simplify the speciﬁcation
...
Effective implementations require asynchronous processing, so the implementation is beyond the scope of this book
...
The ﬁrst action on the causal chain is to look for the
Coke cans
...

Therefore, the designer wants the code to permit the robot to skip searching
the arena if it immediately sees a Coke can, and begin to pick up the can
without even calling the wander-for-goal(red) behavior
...

The resulting script for an abstract behavior to accomplish a task is usually
the same as the programming logic derived from an FSA
...

\\ look for props and cues first: cans, trash cans, gripper
rStatus=extract_color(red, rcx, rSize); \\ ignore rSize
if (rStatus==TRUE)
SEE_RED=TRUE;
else
SEE_RED=FALSE;
bStatus=extract_color(blue, bcx, bSize);
if (bStatus==TRUE){
SEE_BLUE=TRUE; NO_BLUE=FALSE;
} else {
SEE_BLUE=FALSE; NO_BLUE=TRUE;
}
AT_BLUE=looming(size, bSize);
gStatus=gripper_status();
if (gStatus==TRUE) {
FULL=TRUE; EMPTY=FALSE;
} else {
FULL=FALSE; EMPTY=TRUE;
}

5
...
6

Summary
As deﬁned in Ch
...

While an architectural style (subsumption, potential ﬁelds) may specify the
structure of the implementation, the designer must invest signiﬁcant effort
into developing individual behaviors and into assembling them into a sequence or an abstract behavior
...
A
behavior is derived from the schema class; it is a schema that uses at least one
perceptual and motor schema
...
Finite State
Automata offer a formal representation of the coordination logic needed to
control a sequence of behaviors
...

The steps in designing robot intelligence under the Reactive Paradigm are:
1
...
Describe the robot,
3
...
Describe how the robot should act in response to its environment,

188

5 Designing a Reactive Implementation

5
...
Test each behavior independently,
7
...
A behavior table serves as a means of representing the component schemas and functions of the behavioral system
...

These steps emphasize the need to fully specify the ecological niche of the
robot in order to design useful behaviors
...
Regardless of the implementation of the coordination
program , the control should rely on the world to inform the robot as to what
to do next, rather than rely on the program to remember and maintain state
...
7

Exercises
Exercise 5
...
2
Deﬁne:
a
...
causal chain

y

c
...
3

Can the perceptual schema and the motor schema for a behavior execute asynchronously, that is, have different update rates?
Exercise 5
...
5
...
A third way is to have two instances of a move-away-from-wall (perpendicular) behavior with a move-parallel-to-wall behavior
...
5
List and describe the steps in designing a reactive system
...
7 Exercises

Exercise 5
...
5
...
2
...
What would happen if there were
chairs and tables present? Could the gripper accidently scoop a chair or table leg?
How would the system react? What changes, if any, would need to be made to the
behavior table and FSA?
Exercise 5
...
2)
...

y

Exercise 5
...
2)
...

Exercise 5
...
10

What is special about a primitive behavior in terms of perceptual and motor schema?

y

Exercise 5
...

y

Exercise 5
...
13
Make up a task for a robot to complete
...

y

Exercise 5
...

y

Exercise 5
...
Make a Behavior Table to include this new behavior
...
16
Assuming the state of the robot in question 1, describe how the coordinated control
program should handle concurrent behaviors
...
17
Recall how some mammals freeze when they see motion (an affordance for a predator) in an attempt to become invisible, and persist in this behavior until the predator
is very close, then ﬂee
...

y

Exercise 5
...
4 has been modiﬁed so that a
hay bale can be placed directly on the white line
...
Create a behavior table that not only follows
the line but correctly responds to the bale so that it can continue following the line
...
19

[World Wide Web]

Search the web for the International Association for Unmanned Systems competition
home page and brieﬂy describe three unmanned vehicle projects linked to the IAUS
site
...
20

[World Wide Web]

Identify at least 5 robot competitions, and for each describe the basic task
...
21

[Programming]

Using Rug Warrior kits, Lego Mindstorms kits, or similar robotics tools, implement
your own schools version of an IAUS Unmanned Ground Robotics Competition
...
Groups should not begin construction of their robot without ﬁrst describing the
steps involved in designing a reactive behavioral system for the task at hand
...
22

[Programming]

Write a script in pseudo-code for following a hallway
...

Also the robot might start in the middle of a hall facing a wall, rather than having
forward point to the long axis of the hall
...
23

[Programming]

The Pick Up the Trash competitions were a bit different than actually presented in
this book
...
How would you integrate this into:
a
...
the script described in the book?
Exercise 5
...
8 End Notes

Exercise 5
...
Now consider the behavior
of the CSM robot around white shoes and dandelions
...
Is it possible?

Questions marked with a
Wisconsin Lacrosse
...
8

y were suggested by students from the University of

End Notes
For the roboticist’s bookshelf
...
The
book provides many insights into putting theory into practice
...

The International Association for Unmanned Systems (formerly known as the Association for Unmanned Vehicle Systems) sponsors three annual robot competitions
for student teams: Ground Robotics, Aerial Vehicle, and Underwater
...
As noted in a Robotics
Competition Corner column, 101 the competitions tend to reward teams for building
platforms rather than programming them
...

While researchers worried about picking up and disposing of soda cans, a popular
magazine began discussing what it would take to have a robot go to the refrigerator
and bring the user a beer
...

The 1995 IJCAI Mobile Robot Competition
...
The University of New Mexico
team spent the preparatory days of the competition frantically trying to locate where
their robot was
...
All the teams looked up
from their frantic testing and cheered in support of the New Mexicans
...
The New Mexicans didn’t even bother to un-crate the robot to
catalog the damage; they instead sought solace in Montreal’s wonderful night life
...

Cheering at robot competitions
...
Core Dump
...

Subsumption and Soda Cans
...
He used a special type of FSM called
an Augmented Finite State Machines (AFSMs), and over 40 behaviors to accomplish
the task
...

As often happens, robot competitions often pose problems that are a step beyond
the capability of current hardware and software technology
...
Participants, such as the Colorado School of Mines with an older robot and no
arm, could have a “virtual manipulator” with a point deduction
...
The virtual manipulator—a human team member, either Tyler Devore, Dale
Hawkins, or Jake Sprouse—would then physically either pick up the trash and place
it on the robot or remove the trash
...

The Pick Up the Trash event mutated in 1996 to picking up tennis balls in an empty
arena, and in 1997 into a variation on sampling Mars
...
Because of
weight considerations in shipping the rocks, the rocks were about the size of a couple
of textbooks and not that much bigger than the martians
...
It would misidentify a rock as a martian during a random search, navigate to it, grip it, and then
attempt to lift it
...
It would stay there, clutching the rock
...
The robot would then try to
grip the rock two more times
...
The robot would give up,
then resume a random search
...
Eventually the
judges would go over and move the robot to another location
...
8 End Notes

193

Documentaries
...

6

Common Sensing Techniques for
Reactive Robots

Chapter Objectives:
Describe the difference between active and passive sensors, and give examples of each
...

List the metrics for rating individual sensors and a sensor suite, and apply
these metrics to a particular application
...

Describe the types of behavioral sensor fusion and be able to apply to a realworld problem
...

If given a small interleaved RGB image and a range of color values for a
region, be able to 1) threshold the image on color and 2) construct a color
histogram
...

196

6 Common Sensing Techniques for Reactive Robots

6
...
3, perception in a reactive robot system has two roles: to release a behavior, and to support or guide the actions of the behavior
...
Also, the connotation of reactive robots
is that behaviors are most often stimulus-response, relying on direct perception rather than require memory
...
In order for a reactive robot to be successful in theory, it must have reliable perception since perception and action are tightly coupled
...
This chapter covers the most common sensing techniques for reactive robots
...

Regardless of sensor hardware or application, sensing and sensors can be
thought of interacting with the world and robots as shown in Fig
...
1
...
The term transducer is often used interchangeably with sensor
...
2 A sensor receives energy and transmits a signal to a display or a computer
...
) into
an analog or digital form capable of being used by a robot
...
This percept is then
used by the motor schema, which leads to an action
...

Passive sensors rely on the environment to provide the medium for observation, e
...
, a camera requires a certain amount of ambient light to produce a
usable picture
...
A sonar sends out sound, receives the echo,
and measures the time of ﬂight
...
Although a camera is a
passive device, a camera with a ﬂash is active
...
Active sensing connotes the system for using an
effector to dynamically position a sensor for a “better look
...

197

6
...
1 A model of sensing
...
1
...
This in turns leads to
different types of processing
...
A sensor modality
refers to what the raw input to the sensor is: sound, pressure, temperature,
light, and so on
...
A modality can be further subdivided, for instance, vision can be
decomposed into visible light, infrared light, X-rays, and other modalities
...
65 A logical sensor is a unit of sensing or module that
supplies a particular percept
...
A logical sensor can be easily
implemented as a perceptual schema
...
For example, a percept commonly used for obstacle avoidance is a polar plot of range data
...
The logical sensor would go further
and list all the possible ways the robot had for constructing a polar plot of
that form
...
Each of those modules would be logically equivalent; that is, they would
return the same percept data structure so they can be used interchangeably
...
As will be seen in this chapter, the sonar is liable to produce a noisy
percept in a second or two, while stereo vision may take minutes
...
Therefore, the logical sensor contains a selector function which
speciﬁes the conditions under which each alternative is useful and should be
selected
...
Also note that each individual method can
be implemented as a perceptual schema, leading to the recursive, buildingblock effect
...

“Logical sensor” is often used to connote information hiding, where the particular sensor and processing algorithm is hidden in the “package
...
An avoid behavior might use a polar plot of sonar range data, while a
panic-stop behavior might use the minimum of all the incoming sonar data
...

6
...
The motivation for sensor fusion
stems from three basic combinations of sensors: redundant (or competing),
complementary, and coordinated
...

In some cases multiple sensors are used when a particular sensor is too
imprecise or noisy to give reliable data
...
When a sensor leads the robot to believe that
a percept is present, but it is not, the error is called a false positive
...
Likewise, an
error where the robot misses a percept is known as a false negative
...
2 Behavioral Sensor Fusion

199

Figure 6
...

REDUNDANT
PHYSICAL
REDUNDANCY

LOGICALLY
REDUNDANT

COMPETING SENSORS

will often produce different false positive and false negative rates
...

When the sensors are both returning the same percept, the sensors are considered redundant
...
6
...
The sonar software returns the
minimum reading (shortest range) from the two, providing a more reliable
reading for low objects which would ordinarily specularly reﬂect the beam
from the upper sonar
...

An example is extracting a range image from stereo cameras and from a laser
range ﬁnder
...

Complementary sensors provide disjoint types of information about a percept
...
Both logical sensors return some aspect
of a “survivor,” but neither provides a complete view
...

A predator might see motion, causing it to stop and examine the scene more
closely for signs of prey
...
Early work in reactive systems
used robots with only a few simple sensors, a sonar or sonar ring for range
and a camera for color, texture, or motion affordances
...
Behaviors could share a sensor stream, but without knowing it
...
Instead sensor fusion was really multiple instances of the same behavior with
different sensor inputs being combined together
...

Brooks dubbed this sensor ﬁssion in part as a take off on the connotations of
the word “fusion” in nuclear physics
...
Fig
...
3a shows a diagram of sensor ﬁssion
...
99 The gist of the
studies was that sensor fusion does occur in behaviors
...
Only when the sensor signals routed to the
portion of the brain associated with a particular behavior arrive at that location does there appear to be any transformation
...
Any or all of these sensor streams can
be active and inﬂuence the resulting behavior
...
If a cat hears a noise and sees a movement, it will
react more strongly than if it has only a single stimulus
...
Fig
...
3b shows
a diagram of action-oriented sensor fusion
...
Sensor ﬁssion is by deﬁnition a competitive method,
though complementary sensors may be used to support a particular instance
of a behavior
...
This leaves the category of coordinated sensors untouched
...
A

201

6
...

sensor percept
sensor
sensor

percept

percept

fusion

behavior

action

percept
b
...

Figure 6
...
) sensor ﬁssion, b
...
) sensor fashion
...
6
...

Murphy incorporated pathways for all three types of behavioral sensor
fusion as well as other deliberative forms into the Sensor Fusion Effects (SFX)
architecture
...
7
...
3

PROPRIOCEPTION
EXTEROCEPTION
EXPROPRIOCEPTION

Designing a Sensor Suite
Historically, reactive robots used either inexpensive infrared (IR) or ultrasonic transducers to detect range
...
The percept for these behaviors all involve knowing the distance to an occupied area
of space
...
In agricultural and transportation applications of reactive robots, GPS technology has become popular as well
...
Because the sensor market is rapidly changing, the
chapter will focus on how to design a suite of sensors for use by a robot,
rather the device details
...
If it cannot observe the world and the effects of its
actions, it cannot react
...
14 This information can
be either from proprioception (measurements of movements relative to an internal frame of reference), exteroception (measurements of the layout of the
environment and objects relative to the robot’s frame of reference) or exproprioception (measurement of the position of the robot body or parts relative
to the layout of the environment)
...
5
...
In 1995, the follow-path behavior was expanded to track both lines of the path using a wide angle lens
on the camera
...
However, the camera for the robot was mounted on a panning mast, which was intended to turn to keep
the line in view, no matter what direction the path turned in
...
3 Designing a Sensor Suite

203

robot had to know where the camera was turned relative to the robot’s internal frame of reference in order to correctly transform the location of a white
line in image coordinates to a steering direction
...
(If the robot
was extracting the pose of its camera from exteroception, it would be clearly
exproprioceptive
...

The robot needed the exproprioception to determine where it should move
next: turn to follow the path in camera coordinates, plus the compensation
for the current camera pan angle
...
But the camera might be turned slightly to maintain the view of both
lines of the path through the pan-camera behavior
...
This subtle error surfaced as the robot went around
hair pin turns, causing the robot to go consistently out of bounds
...
3
...
Otherwise,
the robot cannot be considered reactive: there would be no stimulus from
the world to generate a reaction
...
Following Sensors for Mobile Robots, 52 in order to construct a sensor suite, the following attributes should be considered for each
sensor:
1
...
Every exteroceptive sensor has a region of
space that it is intended to cover
...
The ﬁeld of view is usually expressed in degrees; the number of degrees covered vertically may be
different from the number of degrees covered horizontally
...
A wide angle lens will often cover up to 70 , while a “regular” lens may only have a ﬁeld of view around 27
...

The ﬁeld of view (FOV) can be thought of in terms of egocentric spherical

204

6 Common Sensing Techniques for Reactive Robots

HORIZONTAL FOV
VERTICAL FOV
RANGE

RESOLUTION

HOTEL LOAD

coordinates, where one angle is the horizontal FOV and the other is the vertical
FOV
...
In spherical coordinates, this would be the values of r that
deﬁned the depth of the operating range
...
If the robot needs to be able to detect an obstacle when it’s 8 feet
away in order to safely avoid it, then a sensor with a range of 5 feet will not
be acceptable
...
Accuracy, repeatability, and resolution
...
But if a reading for the same conditions is
accurate only 20% of the time, then the sensor has little repeatability
...
If the
inaccuracy is random, then it will be difﬁcult to model and the applications
where such a sensor can be used will be limited
...

3
...
Most sensors have particular
environments in which they function poorly
...
g
...
As will
be seen below, sonar is often unusable for navigating in an ofﬁce foyer with
large amounts of glass because the glass reﬂects the sound energy in ways
almost impossible to predict
...

4
...
Power consumption is always a concern for robots
...
For example, the battery life on a Nomad 200, which carries
ﬁve batteries, was improved from four hours to six by shutting off all sensors
...
Sensors which require
a large amount of power are less desirable than those which do not
...

The amount of power on a mobile robot required to support a sensor package (and any other electronics such as a microprocessor and communications
links) is sometimes called the hotel load
...
3 Designing a Sensor Suite

LOCOMOTION LOAD

205

platform
...

Unfortunately, many robot manufacturers focus on only the locomotion load,
balancing power needs with the desire to reduce the overall weight and size
...

5
...
Sensors often have physical limitations on how
well they work
...
Other sensors have temperature
and moisture constraints which must be considered
...
Size
...
A
microrover on the order of a shoebox will not have the power to transport
a large camera or camcorder, but it may be able to use a miniature “QuickCam” type of camera
...
However, the sensors only provide observations; without the software perceptual schemas, the behaviors cannot use the sensors
...
7
...

Computational complexity is the estimate of how many operations an algorithm or program performs
...
x
is often a function itself
...
An algorithm that executes
with O(n) equally consuming operations is faster than one with O(n2 ) operations
...
) Computational complexity has become less critical
for larger robots, with the rapid advances in processors and miniaturization
of components
...

8
...
The designer should consider how reliable
the sensor will be for the ecological conditions and for interpretation
...
As a result the robot may “hallucinate” (think it is
seeing things that are not there) and do the wrong thing
...
If a sensor algorithm was not working properly in these modalities, the designer might not be skilled enough to notice
it
...

206

6 Common Sensing Techniques for Reactive Robots

a
...

c
...
4 Sequence showing a Denning mobile robot with redundant cameras, responding to sensing failures introduced by Dave Hershberger
...
3
...
Simplicity
...

2
...
In order to reconﬁgure a sensor suite (add new sensors,
remove ones that aren’t appropriate for the task), the designer must be able
to remove one sensor and/or its perceptual schema without impacting any
other sensing capability
...
Redundancy
...
4 Proprioceptive Sensors

PHYSICAL
REDUNDANCY

LOGICAL
REDUNDANCY

FAULT TOLERANCE

6
...
Sensors are especially important because a
faulty sensor can cause the robot to “hallucinate
...

There are two types of redundancy
...
Fig
...
4
shows a robot with redundant cameras
...

Logical redundancy means that another sensor using a different sensing modality can produce the same percept or releaser
...
If the surface was ﬂat, a camera would see a ﬂat line, whereas if the surface was rough the line would
appear jagged
...
The laser striping sensor and the
stereo cameras are logically redundant
...
However, logically redundant sensors are not necessarily
equivalent in processing speed or accuracy and resolution
...

Physical redundancy introduces new issues which are the area of active
research investigation
...

Surviving a failure is referred to as fault tolerance
...

Proprioceptive Sensors
Proprioception is dead reckoning, where the robot measures a signal originating within itself
...
In robotics, actuators are generally
motors
...
If the gearing and wheel size is known, then the
number of turns of the motor can be used to compute the number of turns
of the robot’s wheels, and that number can be used to estimate how far the
robot has traveled
...
This is due to the impact of the
environment on the actual movement of the robot
...
The texture of
the different surfaces caused the wheels of HARV mobile to slip to varying
degrees
...

6
...
1
INERTIAL NAVIGATION
SYSTEMS

(INS)

6
...
2
PRECISION
AGRICULTURE

Inertial navigation system (INS)
Aircraft, submarines, and missiles use inertial navigation systems (INS)
...
As
long as the movements are smooth, with no sudden jarring, and the samples are taken frequently, an INS can provide accurate dead reckoning to 0
...
52 However, this technology is unsuitable for
mobile robots for several reasons
...
The cost is due in part to having to stabilize the
accelerometers with gyroscopes, as well as the nature of precision electronics
...

A hard bump or sudden turn can exceed the accelerometers’ measurement
range, introducing errors
...
Sojourner, the Mars rover, carried an INS system
...
Instead, by using exteroception, it got within 4 cm
...
GPS systems work by receiving signals from satellites orbiting
the Earth
...
GPS isn’t a proprioceptive sensor per se since the robot must receive
signals from the satellites, external to the robot
...
Since they tend to be used in place of dead reckoning on
outdoor robots, only GPS will be covered here
...
4 Proprioceptive Sensors

SELECTIVE
AVAILABILITY

DIFFERENTIAL GPS

(DGPS)

URBAN CANYONS

209

tained by the Russian Federation Ministry of Defense
...
Until early in the
year 2000, the U
...
military actually introduced an error in the satellite message as to where the satellite actually is, which could result in triangulation
errors of up to 100 meters
...
S
...
This was intended to prevent a hostile country from putting a GPS
receiver on a guided missile and precisely targeting where the President is
giving a talk
...
S
...

Many inexpensive hand-held receivers sold to hunters and hikers attempt
to improve on localization by averaging or ﬁltering the readings
...
Surveyors and GPS specialty companies such as Trimble and Rockwell have found a way to subtract the error in
the public channel and get performance near the Y-code’s rumored accuracy
of centimeters
...
One remains stationary, while the other is put on the robot
...
The ultimate
ﬁx to the induced error will probably come in a few years due to the commercial sector
...
Teledesic is scheduled to go on-line in 2004
...
First, GPS does not work indoors
in most buildings, especially ofﬁces or factories with large amounts of steelreinforced concrete
...
Likewise, GPS may not work outdoors in major
cities where skyscrapers act as urban canyons and interfere with reception
...
Several web sites now offer free “do-it-yourself” DGPS
code to create a DGPS from two inexpensive receivers
...
5

Proximity Sensors
Proximity sensors measure the relative distance (range) between the sensor
and objects in the environment
...
Most proximity sensors are active
...

6
...
1

TIME OF FLIGHT

Sonar or ultrasonics
Sonar refers to any system for using sound to measure range
...
Ground vehicles commonly use sonars with an ultrasonic frequency,
just at the edge of human hearing
...

Ultrasonics is possibly the most common sensor on commercial robots operating indoors and on research robots
...
The time
of ﬂight (time from emission to bounce back) along with the speed of sound
in that environment (remember, even air changes density with altitude) is
sufﬁcient to compute the range of the object
...
Its evolution paralleled the rise
of the Reactive Paradigm
...
The ring conﬁguration gave a 360
coverage as a polar plot
...
Besides providing direct range measurements, the transducers were cheap, fast, and had
terriﬁc coverage
...
A bigger
version, the Polaroid Lab Grade ultrasonic transducer, costs on the order of
$30 USD and can measure ranges from 1 to 25 feet with inch resolution over
a ﬁeld of view of 30
...
Ultrasonics became the sensor
of choice for behavior-based robots
...
5 Proximity Sensors

211

Figure 6
...
The membrane is the disk
...
6
...
The transducer is about the
size and thickness of a dollar coin, and consists of a thin metallic membrane
...
The sound is audible as a faint clicking noise, like a crab opening and closing its pinchers
...
The reﬂected sound, or echo,
vibrates the membrane which is ampliﬁed and then thresholded on return
signal strength; if too little sound was received, then the sensor assumes the
sound is noise and so ignores it
...

The key to how useful the data is requires understanding how the sound
wave is generated by the transducer
...
Secondary sound waves are
called side lobes
...
The width of
the main lobe is often modeled as being 30 wide at about 5 meters away
...
3

212

SPECULAR REFLECTION

6 Common Sensing Techniques for Reactive Robots

to 3 meter range
...
Ch
...

The strength of the main lobe in the environment determines the maximum range that the sonar can extract reliability
...
So while the upper limit of the range reading depends on the sensor and the environment,
the lower limit does not
...
The decay
time translates into an inability to sense objects within 11 inches because
measurements made during this period are unreliable because the membrane may not have stopped ringing
...
e
...
6
...
Region I is the region associated with the range reading
...
The arc has a width, because there are some resolution and measurement errors; the width of Region I is the tolerance
...
If that area was not empty, the range reading would have been
shorter
...
Region IV is outside of the beam and not of
interest
...
Ultrasonic sensors rely on reﬂection, and so are susceptible to specular reﬂection
...

Ideally all objects would have a ﬂat surface perpendicular to the transducer,
but of course, this rarely happens
...
In that case, the time of ﬂight will not
correspond to the true relative range
...
An exception to this is glass, which
is very common in hospitals and ofﬁces where mail robots operate, but induces serious specular reﬂection
...
5 Proximity Sensors

R

β

III

I
IV

II

Figure 6
...

θ
range returned

b
...

b
...
7 Three problems with sonar range readings: a
...
) specular reﬂection, and c
...

214

6 Common Sensing Techniques for Reactive Robots

FORESHORTENING

SPECULAR REFLECTION

CROSS - TALK

strong to pass the thresholding in the transducer circuit
...
Recall that a sonar has a 30 ﬁeld of view
...
If the surface is
not perpendicular to the transducer, one side of the cone will reach the object
ﬁrst and return a range ﬁrst
...
If it uses the reading (which is really the reading for
15 ), the robot will respond to erroneous data
...

Specular reﬂection is not only by itself a signiﬁcant source of erroneous readings; it can introduce a new type of error in rings of sonars
...
Suppose the sonars ﬁre (emit a sound) at about the same
time
...
The receiving sonar is unable to tell
the difference between sound generated by itself or by its peers
...
Most robot systems stagger the ﬁring of the sonars in a ﬁxed pattern
of four sonars, one from each quadrant of the ring) at a time
...
If the sonar sound
frequency and ﬁring rate can be changed (which is generally not the case),
then sophisticated aliasing techniques can be applied
...

One researcher, Monet Soldo, told a story of when she developed a reactive
mobile robot for IBM’s T
...
Watson Laboratories during the late 1980’s
...
The day came for the big demonstration, which was to be held not
in the hallways of the laboratory but in the front reception area
...
She rebooted, and tried again, but with the same
result
...
The specular reﬂection and cross-talk caused
the robot to hallucinate, although in different ways each time
...
In that environment, the robot was expected to navigate among
ofﬁce cubes, or work areas delimited by partitions
...
Unfortunately, the
cloth also absorbed the sound from the sonars! These stories emphasize the

215

6
...

b
...
8 Maps produced by a mobile robot using sonars in: a
...
) a
hallway
...
)

need to consider the operating environment for the sensor and its impact on
the signal
...
6
...
Some walls are invisible, others are too
close
...

The 30 cone also creates resolution problems
...

The further away from the robot, the larger the object has to be
...

In practice, another problem leads to spurious sonar readings: power
...
If the robot
is operating at low power levels, the correct waveform will not be generated and the return signal worthless
...
9 The ring of IR sensors on a Khepera robot
...

by looking at the sonar returns, which often suggest specular reﬂection or
crosstalk
...

One method of eliminating spurious readings, regardless of cause, is to
take the average of three readings (current plus the last two) from each sensor
...

As will be seen in later chapters, other approaches treat the reading as being uncertain and apply formal evidential reasoning techniques to reﬁne the
reading
...
11
...
5
...
They emit nearinfrared energy and measure whether any signiﬁcant amount of the IR light
is returned
...
IR
sensors have a range of inches to several feet, depending on what frequency
of light is used and the sensitivity of the receiver
...
Fig
...
9 shows the IR emitters and receivers
placed side by side in a single rectangular package on a Khepera robot
...
5 Proximity Sensors

217

ambient lighting or is absorbed by dark materials (i
...
, the environment has
too much noise)
...
This typically ensures that
an object in range doesn’t absorb the light, causing the sensor to miss the
presence of the object
...

6
...
3
TACTILE

Bump and feeler sensors
Another popular class of robotic sensing is tactile, or touch, done with bump
and feeler sensors
...

Bump sensors are usually a protruding ring around the robot consisting of
two layers
...
In theory, the sensitivity of a bump sensor can be adjusted for different contact pressures; some robots may want a “light” touch
to create a signal rather than a “heavier” touch
...
In the “Hors d’Oeuvres, Anyone?” event of
the 1997 AAAI Mobile Robot Competition, humans were served ﬁnger food
by robot “waiters
...
Humans were supposed to communicate to the Colorado
School of Mines’ Nomad 200 robot waiter (seen in Fig
...
10) that they were
done eating by kicking the bump sensor
...

Placement of bump sensors is a very important issue
...
The Denning mobile robot platforms built in the 1980’s
used a bump sensor that looked much like a thick piece of gray tape
...
Unfortunately, in certain turning conﬁgurations, the wheels extend beyond the
cowling
...

Feeler sensors are whiskers or antennae, only not as sensitive as those on
animals
...
Bump and feeler sensors are actually tactile sensors since they
require the robot to be touching something in order to generate a reading
...

b
...
10 CRISbot, the Colorado School of Mines’ entry in the 1997 AAAI Mobile Robot Competition Hors d’Oeuvres, Anyone? event
...
) Interacting with the
audience
...
) Audience members “communicated” by kicking the two protruding
bumpers near the bottom of the robot
...
)

However, they are often interchangeable with IR sensors because IR sensors
often operate over the short range (inches) with less reliability
...
6
C OMPUTER VISION
IMAGE

PIXELS

Computer Vision
Computer vision refers to processing data from any modality which uses the
electromagnetic spectrum which produces an image
...
Unlike sonar, which
returns a single range reading which could correspond to an object anywhere within a 30 cone, an image implies multiple readings placed in a
two-dimensional array or grid
...
The elements in image arrays are called pixels, a contraction

6
...
6
...
” The modality of the device determines what
the image measures
...
g
...
If a thermal camera is used,
then the value is the heat at that region
...

Computer vision includes cameras, which produce images over the same
electromagnetic spectrum that humans see, to more exotic technologies: thermal sensors, X-rays, laser range ﬁnders, and synthetic aperature radar
...

As noted in the Introduction, computer vision is a separate ﬁeld of study
from robotics, and has produced many useful algorithms for ﬁltering out
noise, compensating for illumination problems, enhancing images, ﬁnding
lines, matching lines to models, extracting shapes and building 3D representations
...
Most of the algorithms, especially those that remove noise, require many computations on
each pixel in the image; until recently, the algorithms were too computationally expensive to run in real-time
...
Therefore a robot designed
to follow paths which used vision to extract the path boundary lines in the
current image based on knowledge of the width, then predicted where the
path boundary lines should be in the next image would be on the borderline
of reactivity
...
A video
camera, such as a camcorder, is arranged so that light falls on an array of
closely spaced metal-oxide semiconductor (MOS) capacitors
...
The capacitors form a shift register, and output is
either a line at a time (“line transfer”) or the whole array at one time (“frame
transfer")
...
Consumer digital cameras post an analog signal,
but the update rate is too slow at this time for real-time reactive robot control
...
6
...

A framegrabber is a card which ﬁts inside a computer, accepts analog camera signals and outputs the digitized results
...
Framegrabbers can produce a grayscale or a color digital image
...
Now color framegrabbers can be purchased from
$300 to $500 USD, and TV tuners which can capture a single frame are available for $50 USD
...
This leads to 256 discrete values of
gray, with 0 representing black and 255 representing white
...
255
...
First, there are many different methods of
expressing color
...
Most commercial devices in the U
...
use a NTSC
(television) standard
...
This is simply abbreviated as RGB
...
6
...
The cubic represents all possible colors
...
Black is
(0,0,0) or 0+0+0, or no measurements on any of the three color planes
...
The pure colors of red, green, and blue are represented
by (255,0,0), (0,255,0), and (0,0,255) respectively
...

Notice that the cube dimensions in the ﬁgure are 256 by 256 by 256, where
256 is the range of integers that can be expressed with 8 bits
...
The 8-bit color model is what is used to colorize old black
and white movies
...
The pixel values of a person’s red
lips might be 185, while their dark blue dress is also 185
...
8-bit color is not often used for robots, unless the robot will

221

6
...
11

Green
(0,255,0)

Yellow
(255,255,0)

RGB color cube
...

24-bit color is usually sufﬁcient for robotics
...
Those applications may
use 10 bits for each color plane
...
Companies such as Pixar make special computers for
these applications
...
Interleaved
...
, and it is the more common representation
...
Below is a code fragment where the color
is displayed for a pixel at location row, col
...

red = image[row][col][RED];
green = image[row][col][GREEN];
blue = image[row][col][BLUE];
display_color(red, green, blue);

2
...
Some framegrabbers store an image as three separate two-dimensional arrays, as shown below
...
The equivalent
code fragment is:

int
int
int

image_red[ROW][COLUMN];
image_green[ROW][COLUMN];
image_blue[ROW][COLUMN];

...
Color in RGB is
a function of the wavelength of the light source, how the surface of the object modiﬁes it (surface reﬂectance), and the sensitivity of the sensor
...
RGB is based on the sensitivity
of the three color sensing elements to reﬂected light
...
Fig
...
12 shows the same program and parameters to segment
an orange landmark that is serving as a “ﬂag” for tracking a small robot just
outside of the image
...
6
...
6
...
The only difference is that the ﬂagged robot has moved, thereby
changing the incidence of the light
...
Moreover, CCD devices are notoriously insensitive to red
...

Clearly a device which was sensitive to the absolute wavelength of the
reﬂected light (the hue) would be more advantageous than having to work

223

6
...

b
...
12 Images showing visual erosion of an orange landmark sticking up from
a small robot (not visible): a
...
) original
image and degradation in RGB segmentation as robot moves farther from camera
...
Such a device would work on the HSI (hue,
saturation, intensity) representation of color
...
Saturation is the lack of whiteness in the color; red is saturated, pink is
less saturated
...
So HSV is a very different color scheme than RGB
...
13 HSV space representation
...
6
...
The hue,
or color, is measured in degrees from 0 to 360
...
These are generally scaled to 8-bit numbers
...

HSV space is challenging for roboticists for many reasons
...
This equipment is prohibitively expensive
...
These singularities occur at places where the three colors
for a pixel are the same; the ﬂatness of the red color plane in CCD cameras
increases the likelihood that a singularity will occur
...
140 That color space was designed

225

6
...
5°

Angle A
45°
Angle B

67
...
14

22
...
5°

90°
90°

Spherical coordinate transform
...

b
...

d
...
15 A comparison of color spaces: a
...
)
RGB segementation, c
...
) SCT segmentation
...
It is used in biomedical imaging, but has not been
widely considered for robotics
...
6
...
Initial results indicate it is much more insensitive to
changes in lighting
...
6
...

6
...
3
REGION
SEGMENTATION

BINARY IMAGE
THRESHOLDING

Region segmentation
The most ubiquitous use of computer vision in reactive robotics is to identify
a region in the image with a particular color, a process called region segmentation
...
There are many color region segmentation algorithms available
and Newton Labs sells a Cognachrome board dedicated to the rapid extraction of colored regions
...
The ﬁrst step is to threshold all pixels which share the same color
(thresholding), then group those together and throw out any pixels which
don’t seem to be in the same area as the majority of the pixels (region growing)
...
5 described a robot which used red to signify a red Coca-Cola can for
recycling
...
This partitioning of the world can be achieved by thresholding the
image and creating a binary image
...
6 Computer Vision

227

Note that the output of a thresholded color image is a two-dimensional
array, since there is no need to have more than one value attached at each
pixel
...
However, on many compilers there is no particular beneﬁt to doing a bit
level representation, and it can complicate code reuse
...
The difference between 1
and 0 is not detectable by the human eye
...

Thresholding works better in theory than in practice due to the lack of
color constancy
...
The common solution is to specify a range of high and low values on
each color plane
...
The
color range for the object has to be made even wider to include the set of
color values the object will take
...
Otherwise, if there
are objects which have a color close enough to the object of interest, those
objects may be mistaken for the target
...
Thresholding an image requires a signiﬁcant contrast between the
background and foreground to be successful
...

b
...
16 Segmentation of a red Coca Cola can: a
...
) resulting red regions
...

Fig
...
16 is the output of a threshold on color alone
...
The perceptual schema could
take the weighted centroid of all the red pixels
...

Or, the perceptual schema could attempt to ﬁnd the largest region where red
pixels were adjacent to each other, then take the centroid of that region
...
)
Color regions can also be helpful in cluttered environments
...
6
...

The international orange vest of the workman provides an important cue
...

6
...
4
COLOR
HISTOGRAMMING

Color histogramming
Thresholding works well for objects which consist of one color or one dominant color
...
137 Essentially, color histogramming is a way to match the proportion of colors in a
region
...
6 Computer Vision

a
...

c
...
17 An urban search and rescue scene
...
) a Denning mobile robot searching, b
...
) segmentation for orange
...
A histogram is a bar chart of
data
...
The
size of the bar is the number of data points whose value falls into the range
for that bucket
...
Constructing a color histogram is straightforward for a region in hue
space, as shown in Fig
...
18
...
The grayscale and hue image histograms had only one
axis for buckets, because these images have only one plane that matters
...
As a result, it has
buckets for each color plane or axis
...

a
...
18 a
...
) the image of the children’s toy, Barney
...

The real advantage of a color histogram for reactive robots is that color
histograms can be subtracted from each other to determine if the current
image (or some portion), I , matches a previously constructed histogram, E
...
The number
of mismatched pixels divided by the number of pixels in the image gives a
percentage match
...
1)

Pn min(I ; E )
j
intersection = j=1Pn E j
j =1 j
For example, a robot can “wake up” and imprint the object in front of it by
constructing the color histogram
...
The robot can use the color histogram to determine if a
particular object is of interest or not
...
7 Range from Vision

231

Because the color histogram of a current image can be matched with another image, the technique appears to be model-based, or recognition
...
Is this a contradiction? No; a color histogram is an example of a local, behavior-speciﬁc
representation which can be directly extracted from the environment
...

However, the robot will follow a purple triangle with a green region, because the ratio of colors is the same
...

Note that the intersection can be considered to be a measure of the strength
of the stimulus, which is helpful in reactive robotics
...
It learned the
histogram, then after learning the object (e
...
, ﬁxating on it), it would begin to
move towards it, playing a game of tag as a person moved the poster around
...
The motor schema used the location to compute the
direction to the poster, but the intersection inﬂuenced the magnitude of the
output vector
...

If the match was strong, the robot would speed up
...
Then if the human moved the
poster back to a more favorable position, the robot would resume playing
with no hard feelings
...
7

STEREOPSIS
OPTIC FLOW

Range from Vision
An important topic in computer vision is how to extract range information
...
In most cases, though not always, depth
perception is due to having two eyes and being able to triangulate as shown
in Fig
...
20, also known as stereopsis
...
This section covers three types of vision sensors which are
commonly used to create an image representing a depth map: stereo camera
pairs, light stripers, and laser ranger ﬁnders
...
19 A Denning mobile robot using a color histogram to play tag with a
poster of Sylvester and Tweety
...
7
...
” One way to extract depth is to try to superimpose a camera over each eye as in Fig
...
20a
...
The cameras are known
as the stereo pair
...
The ﬁrst is that it is hard to design and
build a mechanism which can precisely move to verge on the points
...
) The
second challenge is even more fundamental: how does the robot know that it
is looking at the same point in both images? This problem is referred to as the
correspondence problem, since the task is to ﬁnd a point in one image that corresponds to a point in the other image
...
The algorithm that selects interesting pixels is
called an interest operator
...
Therefore, interest operator algorithms usually return a list of interesting pixels, not just one, and a matching algorithm
tries to ﬁnd the best correspondence between all of them
...
7 Range from Vision

θ1

θ2

θ1

θ2

baseline

a
...

Figure 6
...
) vergence of the
cameras to determine the depth of a point, and b
...

RECTIFIED IMAGES

DISPARITY

EPIPOLAR LINES
EPIPOLAR PLANE

points are established, the rest of the pixels have to be labeled with a depth
relative to those points
...
Instead, cameras can be mounted in place with the optic axes parallel to each
other and perpendicular to the mount, producing rectiﬁed images
...
6
...
The space between the axes
of two cameras is called the baseline
...
69 Fig
...
20b shows
the geometry behind a stereo pair
...
If the cameras
are aligned precisely, then one row in the left image will correspond to a row
in the right image
...
Whenever the robot ﬁnds an interesting point in one image,
it only has to consider the pixels along the epipolar line in the other image
...
However, it only works if the

234

6 Common Sensing Techniques for Reactive Robots

Figure 6
...

CAMERA CALIBRATION

RANGE IMAGE
DEPTH MAP

cameras are perfectly matched optically and remain in alignment
...
The alignment can be periodically compensated for in software through a camera calibration process, where the robot
is presented with a standard and then creates a calibration look up table or
function
...
6
...
As a result, many researchers are turning to units which package a
stereo pair in one ﬁxed case, where the alignment cannot be altered
...
6
...

The ﬁrst robot to use stereo vision successfully was Hans Moravec’s Stanford Cart shown in Fig
...
23a Moravec worked on the Cart while at graduate
school at Stanford between 1973 and 1980
...
6
...
Longer baselines tend to be more accurate because a slight measurement error has a smaller impact, but smaller baselines have a smaller
“footprint,” in effect, take up less room
...

Fig
...
24 shows the simpliﬁed ﬂow of operations in extracting range from
a pair of images
...
The left-right
pair can be grayscale or color, but the depth map is a grayscale map, where
intensity is proportional to the distance the pixel is away from the cameras
...
6
...

6
...
22
Moravec
...

235

(Photograph courtesy of Hans

The major drawback with extracting range information from vision is that
the algorithms tend to be computationally complex
...
This means that to process an
image of size 640 by 480 takes on the order of 9 1010 instructions, while
to simply segment a color is on the order of O(nm) or 3 105 instructions
...

6
...
2

Light stripers
Light striping, light stripers or structured light detectors work by projecting a
colored line (or stripe), grid, or pattern of dots on the environment
...
For
example, in Fig
...
26a, the striper projects four lines
...
If
the surface is not ﬂat, as shown in Fig
...
26b, the lines will have breaks or

236

6 Common Sensing Techniques for Reactive Robots

b
...

Figure 6
...
) The Stanford Cart developed in the late 1970’s
...
) b
...
(Image courtesy of the National Aeronautics and Space Administration
...
A vision algorithm can quickly scan each of the designated
rows to see if the projected line is continuous or not
...
The vision
algorithm can also look for where the dislocated line segments appears, since
the distance in image coordinates is proportional to the depth of the object
...
The more
lines or ﬁner-grained grid, the more depth information
...
First, since they are producing a line or pattern to be measured, expensive time-of-ﬂight detectors
are unnecessary
...
Second, producing a
thick line that can be detected by a camera does not require a laser
...
Finally, light stripers pro-

237

6
...
24

A simpliﬁed ﬂow of operations in extracting range from a stereo pair
...
25 A set of stereo images from a Triclops stereo camera and resulting depth
map
...
)

238

6 Common Sensing Techniques for Reactive Robots

a
...

c
...
26 Situations and resulting images of a
...
) an obstacle, and
c
...

duce a fairly coarse pattern; they don’t project a line or grid onto every pixel
in the image
...

Light stripers were popular in the late 1980’s and early 1990’s, and laboratories such as the University of Pennsylvania’s GRASP Lab under the direction of Ruzena Bajcsy produced excellent results in extracting 3D information
...
The results did not transfer particularly
well to mobile robots
...
Also, in the open world, objects were often the same color
as the projected light or near enough to it to confuse the vision system
...

239

6
...

b
...
27 Lidar images from an Odetics LADAR camera: a
...
) intensity
...
)

The Sojourner Mars rover used a light striping system for obstacle avoidance
...
134 This worked
well because Mars has little color or light diversity
...
Instead, the robot would periodically take a stereo pair for reconstruction and map-making on Earth
...
7
...
The same principle can be used with
lasers; a laser beam can emitted and then the reﬂectance measured
...
If the laser is directed in a scan, just like
a raster scan on a CRT, the device can cover a reasonable area and produce
an image, where the image function produces depth values
...
They can generate up to one million range pixels per second, 52 with
a range of 30 meters and an accuracy of a few millimeters
...
28 Sick laser, covering a 180 area
...
A less expensive solution for navigation is to create a planar
laser range ﬁnder
...
Fig
...
27 shows the
images produced by a Odetics laser range (LADAR) camera
...
This corresponds to
how humans perceive the scene The image function for the range image is
depth from the camera
...
A ﬂat ﬂoor usually appears as a radiating set of semicircles going from near to far; trigonometry is then used to compute that the
circles represent a ﬂat surface
...

Lidars have some problems in practice
...
6
...
But as can be seen
from the intensity image the area is actually far away
...
The errors were due to out of range conditions, absorption of the light
(not enough light returned), or to the optical equivalent of specular reﬂection
(light hitting corners gets reﬂected away from the receiver)
...
6
...
The map is essentially a high resolution polar

6
...
Robots such as Nomads and Pioneers originally came with Sick lasers
mounted in parallel to the ﬂoor
...
Also, as with sonars, robots ran
the risk of being decapitated by obstacles such as tables which did not appear
in the ﬁeld of view of the range sensor but could hit a sensor pod or antenna
...
As the robot moves forward,
it gets a different view of upcoming obstacles
...

6
...
4

LIGHTWEIGHT VISION

RADIAL DEPTH MAP

Texture
The variety of sensors and algorithms available to roboticists can actually
distract a designer from the task of designing an elegant sensor suite
...
Ian Horswill designed the software and camera
system of Polly, shown in Fig
...
29, speciﬁcally to explore vision and the
relationship to the environment using subsumption
...

Polly served as an autonomous tour-guide at the MIT AI Laboratory and
Brown University during the early 1990’s
...
The percept for the obstacle
avoidance behavior was based on a clever affordance: texture
...
The “color” of the
carpet in the image tended to change due to lighting, but the overall texture
or “grain” did not
...
3
...
Every sector with the texture
of the carpet was marked empty
...
Although this methodology had some problems—for example, strong shadows on the ﬂoor created “occupied” areas—it was fast and
elegant
...
29 Polly, a visually guided reactive robot using a black and white camera
...
)

6
...
The entry used two cooperative robots
...
Although the robots executed under a hybrid deliberative/reactive style of architecture (covered later in Ch
...
5, and produced a largely behavioral system
...

243

6
...

b
...

d
...
30 USF robots in the “Hors d’Oeuvres, Anyone?” event
...
) Family portrait, where Borg Shark is on the left, Puffer Fish on the right, with b
...
) the SICK laser located behind the
Borg Shark’s teeth (head piece is removed for better view), and d
...

244

6 Common Sensing Techniques for Reactive Robots

Step 1: Describe the task
...
Each robot was scored on covering the area, noticing when the tray needed reﬁlling, interacting with people naturally, having
a distinct personality, and recognizing VIPs
...
6
...
The Borg Shark was the server robot, and navigated through audience following a pre-planned route
...
It used a DEC-talk
synthesizer to broadcast audio ﬁles inviting audience members to remove a
treat from its mouth, but it had no way of hearing and understanding natural language human commands
...
If it
saw a person, it estimated the location in image coordinates of where a VIP’s
colored badge might be, given the location of the face
...
Puffer Fish would be stationary in
sleep mode, inhaling and exhaling through her inﬂatable skirt and turning
her cameras as if avoiding people crowding her
...
She would also look for
Borg Shark’s distinctive blue costume, using both dead reckoning and visual search to move to goal
...
A human would physically swap trays, then kick the bumpers
to signal that the transfer was over
...

Both robots were expected to avoid all obstacles: tables, chairs, people
...
Borg
Shark, who was programmed to be smarmy, would announce that it was
coming through and begin moving
...

Step 2: Describe the robots
...

6
...
The environment was a convention center
arena, with variable lighting and many people causally dressed
...
The VIPs’ badges were marked with distinctively
colored ribbons, so if the robot was sure it was looking in the right area, the
color would afford a VIP
...
If the robot scanned at about chest height for ribbons, it would likely
ﬁnd a shirt of that color in the crowd who would be wear wearing bright
tourist wear
...
Another approach was to use sonars to ﬁnd a person: read a range, point the
camera in that direction and tilt to the angle where the eyes would be for a
person of average height
...
However, it was known from experience that people rarely stood along the acoustic axis of a sonar
...
If multiple people were present, the robot would seem even more dysfunctional, and the
focus-of-attention mechanism would not work correctly
...
Notice that
detection is not the same thing as recognition
...
Recognition would label the face
and be able to recognize it at a later time, which is a deliberative function, not
reactive
...

Once the robot had found a colored region about the size and shape of a
head, it could then more reliably ﬁnd the VIP badges
...
Although Puffer Fish would receive Borg Shark’s coordinates,
it was unlikely that Puffer Fish could reliably navigate to Borg Shark using
only dead reckoning
...
Then Puffer Fish would accumulate dead reckoning error, more so if it had to stop and start and avoid people
...
Borg Shark’s
head piece was deliberately made large and a distinctive blue color to afford
visibility over the crowd and reduce the likelihood of ﬁxating on someone’s
shirt
...
The choice of sensors for other
behaviors, such as treat removal, was inﬂuenced by the physical location of
the sensors
...
6
...
The research platform, nominally
the top of the robot, was at hand height, making it a logical place to attach a
tray for holding food
...
Other teams tried various approaches such as having a
colored tray and counting the amount of that color visible (more color means
fewer treats on the tray, covering the color)
...

An interesting aspect of the robots that impacted the sensor suite indirectly were the costumes
...
The Puffer Fish had an inﬂatable skirt that puffed out when
the robot was crowded or annoyed
...

Fig
...
30c shows the proﬁle of the skirt
...

247

6
...
attraction(waypoint)

SERVING-TIME-OUT,
TRAY-FULL=bumper()
FOOD-DEPLETED=
treat-removal(laser)

ribbon-color
waypoint

look-for-ribbon(VIP-color)
list of waypoints

track-face

center-face(face-centroid)

face-centroid

ﬁnd-face(vision)

The initial behavior table for Puffer Fish was:
Releaser
always on
AT-HOME=
dead-reckoning(encoders)
AWAKE=radio-signal()
AWAKE=radio-signal()
TRAY-FULL=bumper()

Behavior
avoid()
sleep()
move-to-goal()
move-to-goal()
move-to-goal()

Motor Schema
vfh()
turn-camera-head()
cycle-skirt()
pﬁelds
...
attraction(shark)
pﬁelds
...
11
...
While color was a reasonable affordance for a face, the algorithm
often returned false negatives, missing faces unless in bright light
...
The rates were:
logical sensor
Face-Find
Food-Count

False Positives
1
...
7%

False Negatives
27
...
7%

The solution to the find-face performance was to exploit another affordance of a human, one used by mosquitoes: heat
...

Heat would make a good the decision criteria
...
Fortunately, the team
was able to transfer an E 2 T digital thermometer used on another robot to
Borg Shark
...
6
...
31 State diagrams for the Borg Shark, annotated to show sensors being
used
...

Why wasn’t the vision system simply replaced with the thermal sensor?
This gets back to the attributes of sensors and how they ﬁt with the environment
...

Instead, the vision system covered a much wider ﬁeld of view and could generate a small list of candidate regions
...

The problem with the food-count was greatly reduced by a simple AND
function with the sonars
...

6
...
7% 27
...
7% 76
...
5%
0%
6
...
7%

At this point it is helpful to step back and examine the sensing for the
Hors d’Oeuvres, Anyone? entry in terms of the attributes listed in Sec
...
3
...
The ﬁeld of view and
range of the sensors was an issue, as seen by the differences in vision and
thermal sensors for the face-ﬁnding behavior
...
Repeatability was clearly a problem for laser with its high
false positive/false negative rate
...
Each of the
sensors had reasonable responsiveness from a hardware perspective, though
the algorithms may not have been able to take advantage of them
...
Reliability and size of the hardware were
not serious considerations since the hardware was already on the robots
...
Both were deﬁnitely a factor in the design of the perceptual schemas for the reactive behaviors
...
This
could have been used to ﬁnd faces, but given the high computational complexity, even a Pentium class processor could not process the algorithm in
real-time
...
The vision face ﬁnding algorithm
was very unreliable, not because of the camera but because the algorithm
was not well-suited for the environment and picked out extraneous blobs
...
The sensor suite for both Nomad robots can be considered simple and modular in that it consisted of several separate sensors,
mostly commercially available, able to operate independently of each other
...
There was also a large amount of logical redundancy,
which was exploited through the use of behavioral sensor fusion
...
9

Summary
The success of a reactive system depends on the suitability of the robot’s sensor suite
...
Reactive perception may appear limited since it uses only behavior-speciﬁc representations
and does not involve recognition
...
Advances in electronics
have led to a plethora of range sensors and algorithms
...

The design of a perceptual schema or a sensor suite requires a careful analysis
...
Likewise, the entire sensor suite should provide complete coverage
of all perceptual processing required by the robot
...
Outdoor robots may carry GPS, and
this trend is expected to increase as the cost of receivers goes down and inexpensive DGPS systems emerge
...
Proprioception can guide a robot on a path, but exteroception can prevent it from hitting an unmodeled obstacle or falling off a cliff
...
An ultrasonic transducer is an active sensor which returns a single range reading based on the time-of-ﬂight of an acoustic wave
...
Other popular
proximity sensors are IR and laser rangers
...
Computer vision processing operates on images, regardless of the modality which generated it
...
The two

251

6
...
HSV treats color
in absolute terms, but RGB is favored by equipment manufacturers
...
Many reactive robots exploit
color as an affordance
...
A color affordance method which
works well for objects with multiple colors is color histogramming
...
Laser range ﬁnders, particularly the inexpensive planar
rangers, have grown in popularity over the past few years
...

This may stem from the split between computer vision and robotics in the
formative years of the ﬁeld
...
This is no longer
true, in part because of the increased computational power of general purpose chips
...

6
...
1
Deﬁne sensor suite, active/passive sensors, dead reckoning, computer vision
...
2
Compare and contrast the RGB and HSV color representations specifying the advantages and disadvantages of each type
...
3
Ultrasonic sensors have many positive and negative attributes
...

y

Exercise 6
...
5
Describe the three major problems of ultrasonic sensing, and deﬁne a hypothetical
instance in which a robot would encounter each problem (such as a room with a
large amount of glass surfaces)
...
6
Describe with examples the three attributes that should be considered for an entire
sensing suite
...
7
Consider a Lego Mindstorms robot
...

Exercise 6
...

y

Exercise 6
...

y

Exercise 6
...
Turning gives the robot a 360 coverage
...

Exercise 6
...

y

Exercise 6
...
In addition, give priorities for each to determine which you
would consider to be the most and least important of these characteristics for a robot that was going to be designed for the 1994 AUVS Unmanned Ground Robotics
Competition
...
13
Pick a sensor suite that you think would do a good job if you were designing a robot
for the 1995 UGV competition described in an earlier chapter
...

y

Exercise 6
...
The robot is
designed to seek out people in a smoke ﬁlled building
...
Visibility is often very limited due to smoke
...
Heat can be both an attactive force (e
...
human) or repulsive (e
...
open ﬂame)
...
Obstacles may have a wide variety of sound absorbtion (e
...
carpeting or furniture)
...
10 Exercises

Describe the types of sensors that may be needed and how they will be used
...
Extra
credit: Comment on using simple voice recognition software and a microphone to
seek out human voices (e
...
, cries for help)
...
15
How are the concepts of logical sensors in robotics and polymorphism in objectoriented programming similar?
Exercise 6
...
What is the image function for
a
...
the depth map?
Exercise 6
...
18
Consider an obstacle avoidance behavior which consists of a perceptual schema that
provides a polar plot of range and motor schema which directs the robot to the most
open sector
...
Which of these are logically redundant?
Physically redundant?
Exercise 6
...
5 meters high and a planar laser range ﬁnder
...
in a classroom?
b
...
outdoors in unknown terrain?
State any assumptions your design is based on
...
20

[Programming]

Write your own code to give you the threshold of a small interleaved image
...
21

y

[World Wide Web]

Search for Kludge, another robot built by Ian Horswill
...

Exercise 6
...

254

6 Common Sensing Techniques for Reactive Robots

Exercise 6
...
Construct the color histogram, E , for four different brightly colored objects, such as dolls of the South Park or the Simpsons cartoon
characters
...
Does the highest match
correctly identify the character? Why or why not?

6
...

Hobart Everett literally wrote the book, Sensors for Mobile Robots, 52 on robotic sensors, which provides both analytical details of sensors plus practical experience from
Everett’s many years with the Navy work on robots
...
Everett’s laboratory has to be one of the most ideally
situated in the world
...

Hans Moravec
...
He has also become a well-known author, arguing for the inevitability of machine intelligence in controversial books such as Mind
Children 94 and Robot: mere machine to transcendent mind
...
Documentaries will occasionally run edited footage
of the Stanford Cart navigating outdoors, avoiding obstacles
...
Moravec’s ofﬁce mate, Rodney Brooks, helped with the
recording
...

It is interesting to note that the ﬁrst serious analysis of the Polaroid sonars was done
by an undergraduate at MIT, Michael Drumheller
...

Ballard and Brown
...
Both have worked with robots, though Brown more
than Ballard
...

I had just arrived from a harrowing bus ride from the airport and had to go past the
pool to my room in the hopes of recovering from jet lag before the meeting started in

6
...
As I walked past the pool, one of my friends said, “Oh, Chris Brown
is over there!” I immediately turned to look for what I thought would be an older,
digniﬁed author wearing a suit and tie
...
From that day on, I never assumed that
textbook authors were dull and digniﬁed
...

Tom Henderson at the University of Utah was one of the founders of the concept of
logical sensors
...

Stereo with a single camera
...
He used a prism to project two slightly
different veiwpoints onto the lens of a camera, creating an image which had a different image on each side
...

USAR and Picking Up the Trash
...
The two tasks have much in common,
as illustrated by the work of Jake Sprouse, and behaviors and schemas developed
in one domain can be transferred to another
...
The same vision program he wrote for
ﬁnding “red” was used to ﬁnd “international orange
...
108
Run over Barney
...
His use of a
stuffed Barney doll started out from a class project: program a mobile robot to ﬁnd a
Barney doll and run over it
...
The
Barney doll is a distinctive purple, making it easy for the vision system to ﬁnd it
...
Hawkins’ program was the clear winner, running completely over Barney
...

Reactive soccer
...
One amusing aspect is that
many of these behaviors are purely reﬂexive; if the robot sees the ball, it responds,
but if it loses the ball, it stops
...
Because of the angle of the camera, the robot would lose sight of the ball when it was almost touching it
...
The robot would see the ball
and accelerate rapidly to its estimated location to “kick” it
...
The ball would
slowly roll forward, back into the now-stationary robot’s ﬁeld of view
...

Photographs and scanning
...

7

The Hybrid Deliberative/Reactive
Paradigm

Chapter objectives:
Be able to describe the Hybrid Deliberative/Reactive paradigm in terms
of i) sensing, acting, and planning and ii) sensing organization
...

Given a list of responsibilities, be able to say whether it belongs in the
deliberative layer or in the reactive layer
...

Be able to describe the difference between managerial, state hierarchy, and
model-oriented styles of Hybrid architectures
...

7
...
The Reactive Paradigm
allowed robots to operate in real-time using inexpensive, commercially available processors (e
...
, HC6811) with no memory
...
This meant that a robot could not plan optimal trajectories (path planning), make maps, monitor its own performance, or even select the best behaviors to use to accomplish a task (general planning)
...
In order to differentiate these more cognitively oriented functions from path planning, the term deliberative was coined
...
Techniques for sequencing or assembling behaviors to produce a system capable of achieving a series of sub-goals also relied
heavily on the designer
...
The consensus was
that behavioral control was the “correct” way to do low level control, because
of its pragmatic success, and its elegance as a computational theory for both
biological and machine intelligence
...
Many roboticists
looked at adding layers of higher, more cognitive functions to their behavioral systems, emulating the evolution of intelligence
...
Other robot
systems which do not strongly adhere to an architectural style, such as Rhino
and Minerva, will be discussed in later chapters
...
The concept of considering an
intelligent system, or agent, as being situated in its environment, combined
with the existence proof that detailed, Shakey-like world representations are
not always necessary, led to a new style of planning
...
Many researchers who had worked in traditional
AI became involved in robotics
...
21 Architectures stemming from the planning community
roots showed their traditional AI roots
...
2 Attributes of the Hybrid Paradigm

259

chical ﬂavor with global world models, especially Saphira 77 and TCA
...
At ﬁrst, Hybrids were viewed as an artifact
of research, without any real merit for robotic implementations
...
If the task was to be performed in a knowledge-rich environment, easy to model, then the Hierarchical Paradigm was preferable, because
the software could be engineered speciﬁcally for the mission
...

The current thinking in the robotics community is that Hybrids are the
best general architectural solution for several reasons
...
) allow deliberative functions to execute independently of reactive behaviors
...

Second, good software modularity allows subsystems or objects in Hybrid
architectures to be mixed and matched for speciﬁc applications
...

7
...
It is shown in Fig
...
1
...

The robot would ﬁrst plan how to accomplish a mission (using a global world
model) or a task, then instantiate or turn on a set of behaviors (SENSE-ACT)
to execute the plan (or a portion of the plan)
...

The idea of PLAN, then SENSE-ACT evolved from two assumptions of
the Hybrid Paradigm
...
1 P,SA Organization of Hybrid Deliberative/Reactive Paradigm in terms
of Primitives
...
It is good for setting objectives and selecting methods, but not for making ﬁnely grained decisions
...
g
...
g
...
Second, planning and
global modeling algorithms are computationally expensive, so they should
be decoupled from real-time execution just from a standpoint of practicality
because they would slow down the reaction rate
...

Sensing is truly hybrid, as seen in Fig
...
2
...
But planning and deliberation requires global world models
...
The model is constructed by
processes independent of the behavior-speciﬁc sensing
...
Furthermore, the model making processes can share
the percepts created by the perceptual schemas for behaviors (eavesdrop) or
it can have sensors which are dedicated to providing observations which are
useful for world modeling but aren’t used for any active behaviors
...
Although many architectures will have discrete

261

7
...
2 Sensing Organization in Hybrid Paradigm, showing that the global
world model can have its own sensors, can “eavesdrop” on sensors or percepts
used/created by behaviors, and can act as a virtual sensor for use by a behavior
...

7
...
1

SKILL VS
...
That is not entirely true
...
In the Reactive
Paradigm, “behavior” connotes purely reﬂexive behaviors
...
This can be
confusing, and at least one architecture uses the term skill instead of “behavior” to avoid confusion with purely reﬂexive behaviors
...
Because the Hybrid implementations are
interested in more complex emergent behaviors, there is more diversity in
methods for combining the output from concurrent behaviors
...
2
...
3

Connotations of “global”
The term “global” is used almost synonymously with “deliberative” and “local” with “reactive
...

The deliberative portion of a Hybrid architecture contains modules and
functions for things which are not easy to represent in reactive behaviors
...
But other activities
require global knowledge of a different sort
...
This is
global knowledge in that it requires the module to know something outside
of itself, as compared to a reactive behavior which can function without any
knowledge of whether there are other behaviors actively executing
...
Consider writing a program to diagnose whether the robot’s not moving forward is a problem with
the terrain (it’s stuck in the mud) or a sensor (the shaft encoders don’t report
wheel turns correctly)
...
Therefore,
a deliberative function may not need a global world model, but may need
to know about the internal workings of the robot on a global scale, if only
to know what other modules or deliberative capabilities the program should
interact with
...
This deter-

7
...
Likewise, it is important to subdivide the deliberative portion into modules or objects
...
While Hybrid architectures are most noteworthy for how they incorporate deliberation into mobile robotics, they also introduce some changes in
the way reaction is organized
...
Since then at least three other mechanisms have
been introduced: voting (in the DAMN architecture), 121 fuzzy logic (Saphira), 77
and ﬁltering (SFX)
...
This section
attempts to introduce some conceptual organization on Hybrids in two ways
...
Second, it divides Hybrid into three broad categories:
managerial, state hierarchies, and model-oriented
...
3
...
Generally a
Hybrid architecture has the following modules or objects:

S EQUENCER

A Sequencer agent which generates the set of behaviors to use in order to
accomplish a subtask, and determines any sequences and activation conditions
...
Recall from Ch
...

R ESOURCE MANAGER

A Resource manager which allocates resources to behaviors, including selecting from libraries of schemas
...
The behavioral manager would ascertain whether the IR sensors can
detect at a sufﬁcient range, the stereo vision can update fast enough to
match the robot’s desired velocity, and the sonars have enough power to
produce reliable readings
...
g
...

C ARTOGRAPHER

A Cartographer which is responsible for creating, storing, and maintaining map or spatial information, plus methods for accessing the data
...

M ISSION P LANNER

A Mission Planner which interacts with the human, operationalizes the
commands into robot terms, and constructs a mission plan
...
” The mission planner would interpret that command to mean to
ﬁrst physically search for a person, identify him as a sheriff because of
the uniform, attract his attention, and then lead him back to the current
location
...
The plan might ﬁrst be to navigate to the sheriff’s ofﬁce
...
Notice that this requires the robot
to exhibit some type of self-awareness
...
3
...
Managerial
styles focus on subdividing the deliberative portion into layers based on the
scope of control, or managerial responsibility, of each deliberative function
...
State hierarchies
use the knowledge of the robot’s state to distinguish between reactive and
deliberative activities
...
Deliberative functions can
be divided into those that require knowledge about the robot’s Past state
(where it is in a sequence of commands) and about the Future (mission and
path planning)
...
They are characterized by behaviors that have access to portions of a world model, often to the
point that they appear to have returned to the Hierarchical monolithic global
world model
...
4 Managerial Architectures

7
...
4
...
At the top are
agents which do high level planning, then pass off the plan to subordinates,
who reﬁne the plan and gather resources, and pass those down to the lowest
level workers, the reactive behaviors
...
As with subsumption, a layer can only modify the
layer below it
...
Only when an agent cannot
solve its own problem does it ask for help from a superior agent; the agent is
said to fail upwards in this case
...
It was
actually designed and implemented by Arkin at the same time Brooks was
beginning to publish his work with subsumption
...

Two of the subsystems comprise the deliberative portion: the Planner, and
the Cartographer
...

It is subdivided into three components, equivalent to the Nested Hierarchical
Controller 93 discussed in Ch
...
The Cartographer encapsulates all the map
making and reading functions needed for navigation
...
For example, a human operator might load in
a map ﬁle for the Cartographer to use
...

The Planner subsystem is divided into the Mission Planner, Navigator, and
Pilot
...
The Navigator works with the Cartographer to compute a path for the robot and break it into subtasks (go over the
mountain to the water tower, follow the road along the ridge to the camp)
...
The Pilot portion of the Planning subsystem interacts with the Motor Schema Manager in the Motor subsystem, giv-

266

7

The Hybrid Deliberative/Reactive Paradigm

Mission Planner,
Sequencer,
Performance Monitoring Agent
Cartographer

Planner
mission planner

Cartographer

navigator
pilot

Homeostatic
Control

Deliberative Layer
Reactive Layer

Behavioral Manager

Sensor

Motor
motor schema manager

ps1

ps2

ps3

ms1

Σ

ms2

actuators

ms3

sensors
Figure 7
...

ing it the list of behaviors it needs to accomplish the current subsegment
...
The motor schemas represent actions with potential ﬁelds,
and the overall behavior emerges due to vector summation
...
These classes contain libraries of perceptual and motor schemas,

7
...
The schemas themselves can consist of assemblages of primitive schemas, coordinated by ﬁnite state machines
...
Behaviors are not restricted to being purely reﬂexive; behavior speciﬁc knowledge, representations, and memory is permitted
within the schemas
...

The ﬁfth subsystem, Homeostatic Control, falls into a gray area between
deliberation and reaction
...
As an example, consider a planetary rover operating on a rocky planet
...
The return vehicle has a ﬁxed launch date; it will
blast off, returning to Earth on a set day and time no matter what
...
It may stay two meters away from each obstacle,
giving itself a wide margin of error
...
Now consider what happens towards the time when the return vehicle is set to launch
...
The robot
should be willing to perform the equivalent of sacriﬁcing its own existence
for the sake of the mission
...
Many aspects of AuRA are
motivated by biology, and homeostatic control is no exception
...

For example, an animal who needs food becomes increasingly focused on
ﬁnding food
...
Fortunately,
changing the emergent behavior is straightforward in a potential ﬁelds representation of behaviors, since the output vector produced by each behavior
can be scaled by a gain term
...

268

7

The Hybrid Deliberative/Reactive Paradigm

The table below summarizes AuRA in terms of the common components
and style of emergent behavior:
AuRA Summary
Sequencer Agent
Navigator, Pilot
Resource Manager
Motor Schema Manager
Cartographer
Cartographer
Mission Planner
Mission Planner
Performance Monitoring Agent
Pilot, Navigator, Mission Planner
Emergent behavior
Vector summation, spreading activation
of behaviors, homeostatic control

7
...
2

Sensor Fusion Effects (SFX)
Another managerial style of architecture is the Sensor Fusion Effects (SFX)
architecture, which started out as an extension to AuRA by Murphy to incorporate pathways for sensing
...
Over time, SFX has reorganized both the reactive and deliberative
components of AuRA although the two architectures remain philosophically
identical
...
SFX has been used on eight robots for indoor ofﬁce navigation, outdoor
road following, and urban search and rescue
...
7
...
The model suggests that sensory processing is initially
local to each sensor, and may have its own sensor-dependent receptive ﬁeld
...
Sensor processing then appears to branch, with
duplicates going to the superior colliculus (a mid-brain structure responsible
for motor behaviors) and the other to the cerebral cortex (responsible for more
cognitive functions)
...
In SFX, the equivalent superior colliculus functions are implemented in the reactive layer, and cortical activities
in the deliberative layer
...

The deliberative component is divided into modules or object classes, each
of which is actually a software agent, a self-contained software program
which specializes in some area of competence and can interact with other

269

7
...
4 The cognitive model used for the implementation of the Sensor Fusion
Effects Architecture
...
The dominant agent is called the Mission Planner agent
...
The agents in the deliberative layer
attempt to ﬁnd (and maintain) a set of behaviors which can accomplish the
mission while meeting the constraints
...
But the nature of deliberation suggests that they have to negotiate with other peers to ﬁnd a satisfactory set of behaviors to accomplish the
current task
...
The agents in the lower deliberative layer are
middle-management, planning how to organize the workers’ assignments,
monitoring productivity, and adapting assignments if necessary
...
The resource manager functions are divided across managers because the types of knowledge and algorithms are different for managing sensors and actions
...
They are not allowed to relax any constraints
speciﬁed by the Mission Planner, so they essentially know what the robot is
supposed to do, but only the Mission Planner knows why
...

The Sensing Manager in SFX is particularly noteworthy because of its explicit commitment to performance monitoring and problem solving
...
If
a behavior fails or a perceptual schema detects that sensor values are not
consistent or reasonable, the Sensing Manager is alerted
...
Imagine a mobile robot in a convoy of robots hauling
food to refugees
...
Instead, it should immediately switch
to a back-up plan or even begin to smoothly slow down while it identiﬁes
a back-up plan
...
Then working in a background mode, the Sensing Manager can
attempt to diagnose the cause of the problem and correct it
...
4 Managerial Architectures

a
...

b
...

e
...
5 A robot using SFX to ﬁgure out that something is wrong with the camera
...

Shirt covering the lens, as shown in Fig
...
5
...
The Cartographer is responsible for map making and path planning, while the performance agents attempt to observe the
progress of the robot toward its goals and notice when the robot is unsuccessful, using work by Hughes
...
The follow-road behavior is executing correctly; it is
seeing the road and giving actuator commands correctly
...
So some other process with more selfawareness than a reactive behavior is needed to notice that the robot isn’t
moving
...

272

STRATEGIC
TACTICAL BEHAVIORS

7

The Hybrid Deliberative/Reactive Paradigm

The reactive component is also divided into two layers
...
As noted earlier, just about every hybrid architecture has its own method of combining
behaviors in the reactive portion
...
The SFX philosophy is similar to subsumption: there are some fundamental behaviors that should override other
behaviors
...

The idea of a tactical behavior is best seen by example
...
The use of a repulsive ﬁeld was
simple, but could lead to a global minima, where the repulsion could cancel out any other motive ﬁelds such as move-to-goal
...
That vector would lead to a tangential ﬁeld in addition to the
repulsive ﬁeld, resulting in an avoid-obstacle ﬁeld which repulses the robot
towards the direction it was heading in to begin with
...
But the avoid-obstacle ﬁeld was like an infantry
man; it was trying to go in the direction given by the general but not hit
anything
...

Another example of a tactical behavior in SFX is speed-control
...
The emergent speed in AuRA is the magnitude of the vector summed from all the active behaviors
...
The safe velocity of a robot depends on many inﬂuences
...
Likewise, it may need to go slower as it goes up or down
hills
...
But the other behaviors should have some inﬂuence on the speed as
well
...
If the strategic speed is less than the safe speed computed from tactical inﬂuences, then the output speed is the strategic speed
...
Tactical behaviors serve as ﬁlters on strategic commands to

273

7
...
6 Strategic and tactical behaviors for following an outdoor path, used in
the 1995 UGV Unmanned Ground Robot competition
...
The interaction of strategic and tactical behaviors
is still considered emergent behavior
...
This means that the need to combine behaviors does not
occur often, and so the combination mechanism is not particularly important
...
There were many strategic behaviors,
but they were explicitly coordinated and controlled according to behaviorspeciﬁc knowledge
...
5

State-Hierarchy Architectures
State-hierarchy styles of architectures organize activities by the scope of time
knowledge
...
Unfortunately, many other Hybrid architectures have 3 layers but with the layers
meaning something totally different
...
As with managerial styles, the organization is broken down into layers
...
As with managerial styles, a higher layer has
access to the output of lower layers and can operate on the next lower layer
...
5
...
The roots of 3T stem
from continuing work at NASA, where aspects of Slack’s NaT system, 132
Gat’s subsumption-style ATLANTIS architecture, 57 and Firby’s RAPs system 53 were merged at JPL under the initial direction of David Miller, and
reﬁned by Pete Bonasso and Dave Kortenkamp at NASA’s Johnson Space
Center
...
Fig
...
7 show the layers actually running on three different computers at NASA Johnson Space Center controlling the Extra-Vehicular Activity
Helper-Retriever (EVAHR) robot simulator
...

7
...
7 Each of the 3T layers running on a separate computer, controlling the
EVAHR robot
...
)

The top layer of 3T is Planner
...
These goals are passed
to the middle layer, called the Sequencer
...
The Sequencer is responsible for the
sequencer and performance monitoring functions of a generic Hybrid architecture
...
These behaviors form the bottom layer, called the Controller or Skill Manager
...
” Its behaviors have the same broader scope as
AuRA and SFX, permitting sensor fusion and assemblages of primitive behaviors
...
A
skill is often an assemblage of primitive skills; indeed, one of the interesting
aspects of 3T is its foundation as a tool for learning assemblages
...
8 3T architecture
...
In some regards, events are equivalent to the innate releasing mechanisms; both let the world be its own best representation
...
They also represent a philosophy organized by scope of state,
rather than scope of responsibility
...
6 Model-Oriented Architectures

UPDATE RATE
...
The components in the Sequencer layer operate on state information reﬂecting memories about the Past, as well as the Present
...
This adds a great
deal of robustness and supports performance monitoring
...
It can also use information from the Past (what the robot has done or tried) and Present (what the
robot is doing right now)
...

In practice, 3T does not strictly organize its functions into layers by state
(Past, Present, Future); instead it often uses update rate
...
This appears to be a situation where the pragmatic considerations of computation inﬂuenced the design rules; in the early 1990’s behaviors were very fast, reactive planning (especially RAPs and Universal Plans)
were fast, and mission planning was very slow
...

The table below summarizes 3T in terms of the common components and
style of emergent behavior:
3T
Sequencer Agent
Resource Manager
Cartographer
Mission Planner
Performance Monitoring Agent
Emergent behavior

7
...
The designers sought to add more cognitive functionality to the Reactive Paradigm
...
However, a new inﬂux of researchers
from the traditional AI community, particularly Stanford and SRI, has en-

278

7

The Hybrid Deliberative/Reactive Paradigm

tered robotics and made a distinct contribution
...
One
hallmark of these architectures is that they concentrate symbolic manipulation around a global world model
...
In this case, the global world model
serves as a virtual sensor
...
However, there
are four practical differences
...
The world model is often only interested in labeling regions of the
sensed world with symbols such as: hallway, door, my ofﬁce, etc
...
In effect, the “eavesdropping”
on perception for behaviors is an equivalent form of distributed processing
...
This can dramatically improve the performance of the robot
...

Two of the best known model-oriented architectures are the Saphira architecture developed by Kurt Konolige with numerous others at SRI, and the
Task Control Architecture (TCA) by Reid Simmons which has been extended
to do multi-task planning with the Prodigy system
...

7
...
1
S APHIRA

COORDINATION ,
COHERENCE ,
COMMUNICATION

Saphira
The Saphira architecture, shown in Fig
...
9, has been used at SRI on a variety of robots, including Shakey’s direct descendents: Flakey and Erratic
...
77 A robot must coordinate its actuators
and sensors (as has been seen through in the Reactive Paradigm), but it must
also coordinate its goals over a period of time (which is not addressed by the
Reactive Paradigm)
...

279

7
...
9 Simplifed view of Saphira architecture
...
77 Finally, communication is important because robots have to interact with humans and, as will be seen in
Ch
...
This introduces the problem of having common frames
of reference (anchors) for communicating
...
77
PRS-lite is capable of taking natural language voice commands from humans
(“deliver this to John”) and then operationalizing that into navigation tasks
and perceptual recognition routines
...
Much processing is

280

7

VIRTUAL SENSOR

devoted to maintaining an accurate model of the world based on the robot’s
sensors and assigning symbolic labels to regions
...
This provides a high degree
of ﬂexibility
...
In the 1996 AAAI Mobile Robot
Competition, 78 robots running Saphira actually had some of their planning
reside on a local workstation, transmitted through a radio link
...
8
...
The behaviors
extract virtual sensor inputs from the central world model, the Local Perceptual Space
...
Fuzzy logic turns out to be a very
natural way of fusing competing demands, and is less ad hoc than Boolean
logic rules (e
...
, “if x and y but not z, then turn left”)
...
The fuzzy logic
mechanism for combining behaviors produces essentially the same results
as a potential ﬁeld methodology, as described by Konolige and Myers
...
Although this central processing
introduces a computational penalty, the increases in processor power and
clock speeds have made the computational costs acceptable
...
6
...
7
...
It is also the intelligence inside Xavier (shown in Fig
...
11, the
Carnegie Mellon University robot which is accessible over the web)
...
6 Model-Oriented Architectures

Task Scheduling
(PRODIGY)

Mission Planner

Path Planning
Global
World
Models

Cartographer

Navigation
(POMDP)

Sequencer,
Resource Manager

Deliberative Layer

Obstacle Avoidance
(CVM)

Sensors
Figure 7
...

in response to 30,000 requests by web users to navigate to a particular site
...
Also, there are no behaviors per se
...

The basic layout of TCA for indoor navigation is presented in a somewhat
different form than usual 131 in Fig
...
10 to provide consistency with the terminology of this chapter
...
11) which can be thought of as a distributed global
world model
...

The basic task ﬂow is determined by the Task Scheduling Layer, which uses
the Prodigy planner
...
) This layer interacts with the user and
determines the goals and order of execution
...

The Hybrid Deliberative/Reactive Paradigm

b
...
11 Two robots using TCA: a
...
) Dante (photograph courtesy
of NASA Ames Research Center)
...
Once the current task has been established, the Path Planning layer is
engaged
...
As with
the relationship between strategic and tactical behaviors in SFX, the Obstacle Avoidance Layer takes the desired heading and adapts it to the obstacles
extracted from the evidence grid virtual sensor
...

The table below summarizes TCA in terms of the common components
and style of emergent behavior:

Sequencer Agent
Resource Manager
Cartographer
Mission Planner
Performance Monitoring Agent
Emergent behavior

TCA
Navigation Layer
Navigation Layer
Path-Planning Layer
Task Scheduling Layer
Navigation, Path-Planning, Task-Scheduling
Filtering

7
...
7

283

Other Robots in the Hybrid Paradigm
One implicit criterion for evaluating the utility of a Paradigm and derivative architectures is its popularity
...
It is clearly popular
...
Almost every robot has a set of functions that are equivalent to behaviors, though several have followed the lead of 3T in calling those functions
by other names (e
...
, “skills”) to reduce the connotations of purely reﬂexive
behaviors
...

One of the major inﬂuences on the mobile robot community has been the
DARPA UGV Demo II and Demo III projects, which forwarded the state of
the art in outdoor ground vehicle control and navigation
...
It would autonomously plan
a route, drive along roads, then go off roads as needed
...
Demo II even had multiple HMMWVs traveling autonomously in formation
...
The United States has shown some interest through
several iterations of “intelligent vehicle highway systems” programs
...
116
A third inﬂuence on mobile robots has been NASA’s push for autonomous planetary rovers, which culminated in the Mars Pathﬁnder mission
with the Sojourner rover
...
Unfortunately,
from the Viking lander missions, Mars was known to be very rocky
...
One, led by MIT, favored deploying many
small, legged robots, such as Genghis, that would climb over small rocks and
go around others
...
The other approach, characterized by
Carnegie Mellon University’s Ambler robot, proposed a single large robot
with a higher viewpoint and stability
...
In the end, planetary rover researchers
have gravitated towards wheeled vehicles with some type of articulation to
maintain stability, such as seen with Sandia National Laboratories’ Rattler
...
These were built to rappel down steep canyons and volcanoes on
Mars (and Earth)
...
It was dropped
while being lifted out by a helicopter, twisting its frame
...
8

Evaluation of Hybrid Architectures
In some regards, it is difﬁcult to evaluate Hybrid architectures individually
...
Returning to the four criteria set forth in the
overview in Part I, it is interesting to evaluate the architectures as a whole
...

Most are divided into layers, which are then subdivided into modules
...
AuRA and SFX clearly exhibit an organization which lends itself to
object-oriented programming
...

Hybrids tend to have a high degree of niche targetability
...
However, the partitioning between
reaction and deliberation allows the reactive portion to be used alone for
purely reactive applications
...
In the SFX and 3T architecture, modules
within the various deliberative components attempt to monitor the performance of the reactive behaviors and either replace or adapt the conﬁguration
as needed
...
9 Interleaving Deliberation and Reactive Control

285

An evaluation of Hybrid architectures would not be complete without asking the question, “so what’s the difference between what Shakey could do (if
it had had inﬁnite computation power) and what a Hybrid can do?” Speed of
execution aside, it is interesting to ponder whether the only fundamental difference is how Hierarchical and Hybrids achieve the same ends
...
The two paradigms also certainly
reﬂect different attitudes toward world modeling
...
The frame problem either doesn’t exist or is minor because 1) execution is reactive and therefore
well-suited for unstructured environments, and 2) software agents can use
agent-speciﬁc abstractions to exploit the structure of an environment in order to fulﬁll their particular role in deliberation
...
The robot
can think in terms of a closed world, while it acts in an open world
...
Under
STRIPS, Shakey planned every move, down to the lowest level of granularity, and had problems conﬁrming that an action had been accomplished
...

It is not clear whether a SENSE, PLAN, ACT paradigm with a more suitable planner would produce a more intuitively appealing robot architecture
...
The Hybrid Paradigm has its roots in ethology,
and provides a framework for exploring cognitive science
...

7
...
However, interleaving the two functions is often task dependent
...
Another issue is monitoring the progress of behaviors and terminating behaviors correctly
...
Many robot architectures have well-developed
modules for handling navigational tasks
...
However, advances in technology and new domains are causing researchers to rethink
the strict separation
...
These path planning algorithms exhibited several drawbacks
...
This prevented the robot from continuously generating the path
...
That map would represent the best, currently available knowledge about the world: where obstacles were, unpassable terrain, etc
...
So the
robot can generate an optimal path, but it may discover an unmodeled obstacle blocking that path
...

If the robot used reactive behaviors to navigate, there was no guarantee
that it would make it to the goal any faster
...
By letting
the robot solely react to the world, it might make poor choices
...
The end of each path segment is called a waypoint, because
it is a point along the way to the ﬁnal goal
...
When the robot reaches the ﬁrst goal, the sequencer agent can
give the behavioral manager the coordinates or landmarks for the next goal,
and so on
...

As will be seen in Part II, this strategy has its own set of drawbacks
...
However, with the advent of the Intel Pentium and
Sparc microprocessors, the computational demands of path planning algorithms have been mitigated
...
As a result, the mis-

7
...
However, the software engineering reason for the partition remains: things which operate on
symbols and global information should be in the deliberator; things which
operate directly on sensors and actuators should be in the reactor
...
The deliberative layer(s) decompose the mission
into ﬁner and ﬁner steps until it arrives at a set of behaviors capable of accomplishing the ﬁrst subgoal
...
And it is important to note that deliberation can be triggered bottom-up as well
...
In the case of sensing, it might be desirable for the deliberator to make
some of the global models being constructed available to the behaviors
...
Now suppose that it is
looking for the large tree, since it is supposed to turn right at the intersection after the tree
...
The reactive
behaviors of following a road and avoid an obstacle do not need the explicit
representation of a tree
...
One way to do this is to
permit methods on the world model to act as virtual sensors or perceptual
schema
...
This is an example of how the
deliberative layer can aid with selective attention, or ﬁltering of perception for
a context
...

Most of these cases involve failure or exception handling
...
The
schema cannot diagnose itself
...
If the
Sensing Manager cannot ﬁnd a replacement perceptual schema or equivalent
behavior, then it can no longer meet the constraints imposed by the Mission

288

7

COGNIZANT FAILURE

7
...
In the SFX system, the Sensing Manager would trigger the Mission
Planner
...
This
is an example of “failing upwards,” where a lower level module fails and a
higher level, or smarter, module has to take over
...
Erann Gat motivated the need for cognizant failures through the “Wesson Oil” problem
...

Ideally the mother would not only turn off the stove but remove the chicken
from the oil
...

Since she is using Wesson Oil, the chicken isn’t ruined when they return
...
There may be behaviors in the sequence that have to
execute or disaster will occur (turn off the stove) as well behaviors which
must run or lead to merely undesirable side effects (chicken sits in oil if it is
not removed)
...
Deliberation is deﬁnitely not trivial!

Summary
The Hybrid Deliberative-Reactive Paradigm can be thought of as PLAN, then
SENSE-ACT (P,S-A)
...
Planning can be interleaved with execution;
then for the most part, the robot plans a set of behaviors which will execute for a long time
...

Architectures in the Hybrid paradigm usually encapsulate functionality
into modules
...
Hybrid architectures
always have portions which are recognizably deliberative and reactive
...
11 Exercises

289

rule of thumb is that functions which operate on symbolic information go in
the deliberative layer, or deliberator, while functions which transform sensor
data into actuator commands go in the reactive layer, or reactor
...
Behaviors may
be called skills or by other names to eliminate confusion with reﬂexive behaviors
...
The reactive portion tends to make more
use of assemblages of behaviors than in the Reactive Paradigm
...
The layering
may reﬂect scope in managerial responsibilities or in time horizon
...
These agents may
have agent-speciﬁc world models or share a single global world model
...

Managerial styles of architectures favor a bottom-up organization, concentrating on layering planning on top of behaviors and behavior management
...
The overall behavior emerges as a function of the behavioral layer
...
The three states are
Past, Present, Future
...
The overall emergent behavior is more due to the sequencing of behaviors, or skills, rather than concurrency
...
The world model then serves as a virtual sensor to
the behaviors and as a data structure for planning
...
As such, they distribute planning and problem-solving
...
11

Exercises
Exercise 7
...
sensing, acting, and planning, and
b
...

290

7

The Hybrid Deliberative/Reactive Paradigm

Exercise 7
...

Exercise 7
...

a
...
SFX
c
...
Saphira
e
...
4
Why do behavioral management and performance monitoring require global knowledge?
Exercise 7
...
6
List the ﬁve basic components of a Hybrid architecture
...
7
Describe the difference between managerial, state hierarchy, and model-oriented styles
of Hybrid architectures and give one example of each
...
8
Describe the use of state (Past, Present, Future) to deﬁne behaviors and deliberative
responsibilities in a state-hierarchy architecture
...
9

[Advanced Reading]

Search for technical papers on Cog, Rodney Brooks’ controversial humanoid robot
project
...
10

[Advanced Reading]

Look up technical reports on Shakey
...

Now consider the possible impact of the radical increases in processing power since
the 1960’s
...
(This question was
posed by Tom Garvey of SRI
...
12 End Notes

Exercise 7
...
77 They describe the system built for a demonstration for the TV special,
Scientiﬁc American Frontiers
...
How would
the same tasks be accomplished using AuRA, SFX, 3T, or TCA?

7
...

Behavior-Based Robotics by Ron Arkin 10 is the most complete work on AI robotics
...
Well worth reading and
using as a reference
...

So why did RWI/ActivMedia choose the Saphira architecture for their Pioneer line?
Because the Pioneer robot is based on a robot designed by a class taught at Stanford
by Kurt Konolige, one of the architects of Saphira
...

One criticism that applies to many of the Hybrid architectures is that they have
moved away from subsumption or vector summation to more open-ended methods such as voting, rule-based arbitration, fuzzy logic, etc
...
For example, consider what happened at a
1995 demonstration of the UGV Demo II program
...
The Martin team was responsible for integrating various research contracts into a form usable
by the HMMWV and for hosting demos
...
That way, the team could arrange the boulders in challenging positions to test
the Demo II vehicles (versus trying to move real boulders)
...

The demonstration involving Foam-henge started with the audience standing at
the rise of a canyon
...
This was intended to be a test of its ability to detect and
respond to unmodeled obstacles
...
The vehicle approached Foam-henge,
then stopped, clearly detecting the boulder nearest it
...
The wheels then turned hard to the left
...

About “Red” Whitaker
...
Whitaker has had an
impressive career focusing on building the hardware for industrial applications of
mobile robots, including the robots used to clean up Chernobyl (through the company RedZone Robotics) and Three-Mile Island
...

Dante and the media
...
A small, Dante-like robot appears in the movie, Dante’s Peak, as part
of the team of vulcanologists led by Pierce Brosnan
...
In the “Firewalker” episode of the TV series The X-Files, a robot looking
like an Odetics “spider” robot is down in a volcano and broadcasts disturbing images
implying that the vulcanologists have been attacked by a mysterious life form
...

EV
AHR
...
It was envisioned as a jet-pack
with arms to help astronauts construct the space station
...

8

Multi-agents

Chapter Objectives:
Deﬁne the types of control regimes, cooperation strategies, and goals in multiagents
...

Compute the social entropy of a team
...

Describe use of social rules and internal motivation for emergent societal
behavior
...
1

SOCIETIES
MULTI - AGENTS

SWARM ROBOTS

Overview
This chapter explores artiﬁcial intelligence methods for coordinating and
controlling collections of mobile robots working on completing a task
...

Multi-agent teams are desirable for many reasons
...
Like ants and other insects, many cheap robots working together could
replace a single expensive robot, making multi-agents more cost effective
...
Another motivation of multiple robots is
redundancy: if one robot fails or is destroyed, the other robots can continue
and complete the job, though perhaps not as quickly or as efﬁciently
...

Multi-agent teams are becoming quite popular in robot competitions, especially two international robot soccer competitions: RoboCup and MIROSOT
...
The soccer task explicitly requires multiple robots that must
cooperate with each other, yet react as individuals
...
Those
similarities are not accidental; software and physical agents fall into a research area in Artiﬁcial Intelligence often referred to as Distributed Artiﬁcial
Intelligence (DAI)
...
Arkin, 10 Bond and Gasser, 119 Brooks, 26 and Oliveira
et al
...
How does a designer recognize the characteristics
of a problem that make it suitable for multi-agents? How does the designer (or the agents themselves) divide up the task? Are there any tools
to predict and verify the social behavior?

INTERFERENCE

There is a “too many cooks spoil the broth” effect
...

It is hard for a team to recognize when it, or members, are unproductive
...
But this may not be
possible for every type of team or the vagaries of the open world may undermine that engineering
...
This in turn returns to the
issue of communication
...
1 Overview

295

It is not clear when communication is needed between agents, and what to say
...
g
...
Formation control is often done simply by perceiving the proximity to or actions of other agents; for example, schooling ﬁsh try to remain equally close to ﬁsh on either side
...
How can this unparalleled
ability be exploited? What happens if the telecommunications link goes
bad? Cell phones aren’t 100% reliable, even though there is tremendous
consumer pressure on cell phones, so it is safe to assume that robot communications will be less reliable
...
Agents with a high degree of individual autonomy may create more interference with the group goals, even to the point of seeming
“autistic
...

EMERGENT SOCIAL
BEHAVIOR

The ﬁrst question in the above list essentially asks what are the architectures
for multi-agents? The answer to that question at this time is unclear
...
4) or Hybrid Deliberative/Reactive (Ch
...
Recall that under the Reactive Paradigm, the multiple behaviors
acting concurrently in a robot led to an emergent behavior
...
Likewise in multi-agents, the concurrent but independent actions of each
robot leads to an emergent social behavior
...
” As will be seen in this chapter, fairly complex team actions such
as ﬂocking or forming a line to go through a door emerge naturally from reactive robots with little or no communication between each other
...
Complete architectures for designing teams of robots are
still under development; Lynne Parker’s ALLIANCE architecture 114 is possibly the most comprehensive system to date
...
For the purposes of this chapter,
heterogeneity, control, cooperation, and goals will be used as the dimensions
...
2
H ETEROGENEITY
H ETEROGENEOUS
TEAMS
HOMOGENEOUS TEAMS

8
...
1

Heterogeneity
Heterogeneity refers to the degree of similarity between individual robots that
are within a collection
...
Heterogeneous teams have at least two
members with different hardware or software capabilities, while in homogeneous teams the members are all identical
...

Homogeneous teams and swarms
Most multi-agent teams are homogeneous swarms
...
The
biological model for these teams are often ants or other insects which have
large numbers of identical members
...
Insect
swarms have been modeled and mimicked since the 1980’s
...

An example of a successful team of homogeneous robots is Ganymede, Io,
and Callisto ﬁelded by Georgia Tech
...
5
...
The majority
of the entries used a single agent, concentrating on model-based vision for
recognizing trash, cans, and bins and on complex grippers
...
19 The robots are shown in Fig
...
1, and were constructed from an Intel 386 PC motherboard mounted on a radio-controlled toy tracked vehicle
...
The
ﬂapper-style grippers had an IR to indicate when something was in the gripper
...
The robots
were painted ﬂuorescent green
...
2 Heterogeneity

297

Each robot was programmed with a sequence of simple reactive behaviors (renamed here for clarity), following the reactive layer of Arkin’s AuRA
architecture described in Ch
...
The motor schema was a random potential ﬁeld, the perceptual
schema was color blob detection, where trash=“red” and trashcan=“blue
...

The motor schema was an attractive potential ﬁeld, and the perceptual
schema for the trash and trashcan were the same as in the wander-forgoal
...

avoid-other-robots The three robots did not communicate with each other,
instead using only the repulsive ﬁeld created by avoid-other-robots to reduce interference
...
”
grab-trash The robot would move toward the trash until the perceptual schema reported that the IR beam on the gripper was broken; the motor schema
would close the gripper and back up the robot
...

8
...
2

Heterogeneous teams
A new trend in multi-agents is heterogeneous teams
...
That robot serves as the team leader and can direct the
other, less intelligent robots, or it can be used for special situations
...

One interesting combination of vehicle types is autonomous air and ground
vehicles
...
1 Georgia Tech’s winning robot team for the 1994 AAAI Mobile Robot
Competition, Pick Up the Trash event
...
)

miniature helicopter
...

A special case of a cooperative, heterogeneous team of robots has been
dubbed marsupial robots
...
Micro-rovers often have limited battery power, which they
can’t afford to spend just traveling to a site
...
A marsupial team consists of a large robot which carries one or
more smaller robots to the task site, much like a kangaroo mother carries
a joey in her pouch
...
The
mother can protect a delicate mechanism or sensor from collisions while it
navigates through an irregular void
...
It can serve as a proxy workstation, moving to maintain communications
...
The mother will have a better viewpoint and sensors, so in some
circumstances it can communicate advice to the smaller daughter to help it
cope with a “mouse’s eye” view of the world
...
2 Heterogeneity

a
...

Figure 8
...
a
...
” b
...

(Photographs by Tom Wagner
...

At this time, there appear to be only two physically realized implementations of autonomous marsupials: the University of South Florida teams,
one of which is shown in Fig
...
2, and the robots at the US Department of
Energy’s Idaho National Energy and Engineering Laboratory (INEEL)
...
The Mars Pathﬁnder mission is similar to a marsupial robot in that a micro-rover was transported to a mission site and the
transport vehicle served as a support mechanism
...

300

8

8
...
3
SOCIAL ENTROPY

(8
...
2)

Social entropy
The above examples show how different heterogeneous teams can be
...
16 (Entropy is a measure of disorder in a system, especially in the sense of the Third Law of Thermodynamics
...
) The point of social entropy is to assign a numerical value for rating diversity (or disorder) in a team
...
The number should have
the maximum value if all the team members are different
...

To compute social entropy, consider a marsupial team R with a mother
robot and three identical (hardware and software) micro-rovers
...
Therefore c = 2
...
If i = 1 for the mother, and i = 2 for the daughters:

p1 = 1 = 0:25
4
3 = 0:75
p2 = 4
Substituting into Eqn
...
1 (and remembering that log2 n
cial entropy is:

(8
...
3 Control

301

Now consider a case where the daughters are not identical
...
In that case c = 3, where p1 = 4 , p2 = 4 , and p3 = 1
...
8
...
5
...

8
...
In centralized control, the robots communicate with a central computer
...
, to the remote robots
...
In distributed control, each robot makes its own decisions
and acts independently
...

Examples of full and partial centralized control can be found by comparing the RoboCup and MIROSOT robot soccer competitions
...
In the small sized league of RoboCup and MIROSOT, teams of three,
very small self-contained robots (7
...
5cm x 7
...

Each robot had a unique pattern of bright colors to make it visible from the
overhead cameras, and the overhead camera is connected to a central processor
...

In MIROSOT, the central processor commands each robot by supplying the
direction to move
...
Fig
...
3 shows a
view of the small-sized league from the 1998 RoboCup World Cup
...
They represent
the extreme of centralized control, where everything must go through a single computer, much like the battle-droids in the Star Wars movie, The Phantom Menace
...

a
...

Figure 8
...
) Overhead view of the RoboCup playing
ﬁeld and teams for the small-sized league (overhead camera is not shown), b
...
) Legged league using Sony Aibo robots
...
All rights reserved
...
In the RoboCup case, the robots
must have a set of basic tactical behaviors (see Ch
...
This type of control is conceptually equivalent to the Hybrid
Reactive-Deliberative Paradigm, where the reactive layer physically resides
on the robot and the deliberative layer resides on the central workstation
...
An example of distributed control in robot soccer playing is the mid-sized league
in RoboCup
...
Although they may be physically the same, each robot is programmed with a different role, most especially Goalie, Striker, and Defender
...
4 Cooperation

303

Likewise, there are signiﬁcant advantages to planning and learning strategies, two deliberative functions
...

8
...
Robots can show active cooperation by acknowledging one another and
working together
...
For example, in robot soccer, one robot can pass
the ball to another robot as part of an offensive play
...

More often robots are programmed to exhibit non-active cooperation, whereby they individually pursue a goal without acknowledging other robots but
cooperation emerges
...
Active cooperation requires that
robot be able to distinguish its peer robots from other aspects of the environment
...
If the
robots had not been green, they would have been treated as obstacles to be
avoided
...

It is easy to think of cooperation in terms of robots working together on a
task
...
Marsupial robots are
certainly a type of physical cooperation, especially during deployment and
docking
...
One of the ﬁrst such systems was proposed by Toshio Fukuda,
called CEBOT for “celluar robot system
...
Another aspect of reconﬁgurable robots
is cooperative mobility, where one robot might come over and help another
robot in trouble
...
67

304

8

8
...
If all the robots in the collection work on attaining the
same explicit goal, then they are said to share a single goal, versus having
individual goals
...
78 The
ofﬁce navigation event had a robot that was supposed to search a series of
rooms, ﬁnd an empty conference room, and then go to a list of rooms where
people were and tell them that a meeting was going to begin in the empty
conference room
...
62 Each of the three robots ran the Saphira architecture (see Ch
...
While the robots were responsible for autonomous navigation, their goals were set by the central strategy
agent
...

The robots were able to ﬁnd an empty room and inform the attendees in 4
minutes and 30 seconds
...

An example of purely reactive robots working on individual goals is a
problem originally posed by Ron Arkin: 9 a group of robotic space “ants”
foraging for mineral rich asteroids or Near Earth Objects (NEOs)
...
(Notice that a behavior that permits them to notice other robots and
be repulsed will help disperse the robots
...

Emergent cooperation is not the same thing as having a single goal
...
Each robot might have a
set of behaviors: ﬁnd-stationary-asteroid, move-to-asteroid, push-asteroidto-home, and avoid-robots
...
An attractive “asteroidtropic” potential ﬁeld could be used for the move-to-asteroid behavior
...
Avoid-robot could be done with a repulsive ﬁeld
...
6 Emergent Social Behavior

305

no awareness of the goals of the other team members
...
The robot stays there pushing
...
As it is attracted to the “dark
side” of the asteroid, it will come into range of the ﬁrst robot
...
The second robot will also feel a repulsive force and
slow down
...
Together, the interaction between the two robots
should cause them to naturally balance themselves behind the asteroid and
push together
...

8
...
The robot teams often are the result of extensive design efforts,
where the teams aren’t too large to interfere with each other, and are optimally sized for the particular task, etc
...
How do social behaviors emerge in those cases?
And how can social rules or conventions be established to make the team
self-regulating and productive? This section summarizes two approaches:
creating social rules for the robots to follow, and allowing internal motivation to cause the robots to adapt their behavior to problems
...
6
...
She explored the impact of density and the impact of societal rules on overall team
performance
...
She set up a scenario where up to 20
identical robots (now known as “The Nerd Herd”) were given the same location as a goal
...
4 The Nerd Herd
...
)

IGNORANT
COEXISTENCE

INFORMED
COEXISTENCE

INTELLIGENT
COEXISTENCE

time
...

In the ﬁrst set of demonstrations, the robots functioned with ignorant coexistence
...
A robot treated another robot as an obstacle
...
Since robots
were treated as obstacles, once the robots gathered at the opening, they spent
most of their time avoiding each other
...
Worse yet, the larger the
number of robots ﬁelded, the larger the trafﬁc jam, and the longer to get all
the team members through
...
In addition to move-to-goal and avoid-obstacle, a third behavior was created for avoiding robots
...
If the blocking robot was still in the way
after p, the robot would turn left and then resume moving to the goal
...

The real surprise came in the third demonstration, intelligent coexistence
...
6 Emergent Social Behavior

307

Figure 8
...
(Photograph courtesy of Oak Ridge National Laboratories
...
(Each robot broadcast its heading over a radio transmitter to compensate for the inability to
recognize each other by vision or sonar, so that isn’t considered communication
...
Together the two effects created a strong
need to go through the door single ﬁle, even though there was no such explicit direction
...

8
...
2

Motivation
In Mataric’s work, the robots reduced interference through simple social
rules with no communication, but the members of the team could not actively help out failed colleagues or change tasks dynamically
...
The central idea is that members in the team can either observe or

308

8

MOTIVATION

ROBOT IMPATIENCE
ROBOT ACQUIESCENCE

Multi-agents

“hear” the progress of others in teams, as well as their own progress
...
Likewise, if a robot is free and another robot
has been unable to accomplish a task, it should try to complete the unﬁnished
task
...
g
...
These
changes in behaviors are regulated by a simple mechanism: motivation
...
The more frustrated a robot with gets
with another robot’s performance on ti , the higher the impatience associated
with that task ti
...
If the frustration threshold
is exceeded, then the robot either takes over the unﬁnished task or abandons
its current task and changes behavior
...
8
...
(This example isn’t really a sequential series of
tasks in the manner used by ALLIANCE, but this conveys the elegance of
motivation
...
This makes it
a bit different than the “no communication” approach
...
(We assume there is no frustration for the ﬁnd
task
...
Robot 1 has now
taken responsibility for Task 1 (T1), pushing A1 to home
...
Instead, it begins to accrue impatience about T1
...
As with the no-communication example, a single robot cannot
push the asteroid
...
All the while its impatience over T1 is growing
...
Also at time 7, Robot 2’s patience with Robot 1 and
task T1 has been exceeded
...
Meanwhile, at time 9, Robot 1 gives up on T1
...
Finally, at time 10, the frustration over T2
reaches the limit and Robot 1 is free to help Robot 2
...
7 Summary
time
0
1
2
3
4
5
6
7

8

9

10

11
12

Robot 1
ﬁnd-stationary-asteroid
sees A1
move-to-asteroid(A1)
arrives at A1
push-asteroid-to-home(A1)
T1-acquiescence++
push-asteroid-to-home(A1)
T1-acquiescence++
push-asteroid-to-home(A1)
T1-acquiescence++
push-asteroid-to-home(A1)
T1-acquiescence++
T2-impatience++
push-asteroid-to-home(A1)
T1-acquiescence++
T2-impatience++
T1-acquiescence>limit
gives up on T1
ﬁnd-stationary-asteroid
T2-impatience++
T2-impatience>limit
now attempts T2
move-to-asteroid(A2)
push-asteroid-to-home(A2)
T2-acquiescence = 0
arrives at HOME

Robot 2
ﬁnd-stationary-asteroid
sees A1
move-to-asteroid(A1)
resumes ﬁnd-stationary-asteroid
ﬁnd-stationary-asteroid
T1-impatience++
sees A2
T1-impatience++
move-to-asteroid(A2)
T1-impatience++
push-asteroid-to-home(A2)
T1-impatience>limit
put T1 on stack
T2-acquiescence++
push-asteroid-to-home(A2)
A1-impatience++
T2-acquiescence++
push-asteroid-to-home(A2)
T2-acquiescence++

T2-acquiescence++

push-asteroid-to-home(A2)
T2-acquiescence = 0
arrives at HOME

Figure 8
...

8
...
These collections of multiple robots are often referred
to as multi-agents
...
As with an overall behavior emerging

310

8

Multi-agents

on a single reactive agent, societies of reactive agents often exhibit an emergent societal behavior
...
Heterogeneity refers to
whether the member robots are identical in software and hardware
...
A robot society may have
a single, explicitly shared goal or each robot may have its own goal
...

From a practical side, the emphasis in multi-agents has been on how favorable group dynamics emerge from teams of homogeneous, purely reactive robots while operating under distributed control
...
However, more interest is emerging in robots that have either
heterogeneous software or hardware capabilities, such as marsupial and reconﬁgurable robots
...

8
...
1
Give three reasons why multi-agents are desirable
...

Exercise 8
...

b
...

d
...
3
Consider the example of space ants
...
4
Draw a FSA or write a script to coordinate the sequencing of the Pick Up the Trash
behaviors for Io, Ganymede, and Callisto
...
8 Exercises

Exercise 8
...

Exercise 8
...
7

[Programming)]

Implement the space ant example with 3-5 robots capable of phototaxis and dead
reckoning
...
Multi-agent foraging
...
The program will start with an
empty world consisting of a light (you may need to make a "bigger light" by placing lights next to each other)
...
Each will wander through the
world, avoiding obstacles, until it comes to a light
...
If more than one robot is attracted to the same light,
they should center themselves evenly around the light
...
5 which had a single robot forage for lights
...
Cooperating to bring the food home
...
What happens
now?
Exercise 8
...
robocup
...
Which team has performed the best
over the past 3 years? Describe the multi-agent organization in terms of control and
cooperation
...
9

[Advanced Reading]

Read Ed Durfee’s humorous invited paper on DAI, “What Your Computer Really
Needs to Know, You Learned in Kindergarten” (proceedings of the Tenth National
Conference on Artiﬁcial Intelligence, 1992)
...
), describe how this applies to robots
...

Exercise 8
...
14,
no 6
...

Exercise 8
...
Classify their teamwork in terms of heterogeneity, control, cooperation and goals
...
9

Multi-agents

End Notes
For further reading
...
It is well worth reading
...

The references to swarm robots are too numerous to cite here; many papers explore
details of insect behavior and coordination strategies as well as provide simulations
...
As noted
in Behavior-Based Robotics, Craig Reynolds’ work in computer graphic simulation of
ﬂocks of in “Flocks, herds, and schools: A distributed behavior model,” in Computer
Graphics, 1987, showed how ﬂocks emerge from simple, individual interactions
...
”
The term “Fast, Cheap and Out of Control” later became the title of a 1997 awardwinning documentary by Errol Morris on four men, including Rodney Brooks
...
Although the movie is not about robotics
per se, it features interviews with Brooks and contains stunning shots of some of
Brooks’ robots walking over broken glass, shining like diamonds in the bright lights
...

Languages for multi-agents
...

Robot soccer
...
RoboCup was originally proposed by Minoru Asada,
a noted Japanese researcher in visually guided mobile robots, in 1995 for the 1997 IJCAI, giving researchers two years to prepare
...
MIROSOT was started by Jong-Kwan Kim in Korea in 1996, a year
after RoboCup was announced but a year before the ﬁrst ofﬁcial RoboCup game
...

More robots are being named after women these days
...
9 End Notes

313

computer science
...
The robots in my lab at the University of South Florida are
named after women science ﬁction authors
...
)
More robot name trivia
...
Brightly
colored toasters, but toasters nonetheless
...

PART II

Navigation

316

Part II

Contents:
Chapter 9: Topological Path Planning
Chapter 10: Metric Path Planning
Chapter 11: Localization and Map Making
Chapter 12: On the Horizon

Overview

TOPOLOGICAL
NAVIGATION
METRIC NAVIGATION
QUALITATIVE
NAVIGATION
QUANTITATIVE
NAVIGATION

The ﬁrst three chapters in this part focus on navigation, a critical ability for a
robot that claims to be mobile
...
Reactive robots have provided
behaviors for moving about the world without collisions, but navigation is
more purposeful and requires deliberation
...
Two different categories of techniques have emerged: topological navigation and metric navigation, also known
as qualitative navigation and quantitative navigation, respectively
...
6
...

The Four Questions
Part I presented the material in a historical sequence, following the steps
in development of the three AI robotic paradigms
...
The functions of navigation can be
expressed by four questions presented below
...
A planetary rover may be directed to move to a far away crater and
look for a speciﬁc rock in the caldera
...

What’s the best way there? This is the problem of path planning, and is
the area of navigation which has received the most attention
...

Qualitative path planning methods will be covered in Ch
...
10
...

As a robot explores new environments, it may be part of its mission to map
the environment
...
g
...
A new wall may be erected, furniture rearranged, paintings moved or replaced
...

Where am I? In order to follow a path or build a map, the robot has to
know where it is; this is referred to as localization
...
6, a robot is very poor at dead reckoning
...
Localization can be relative to a local environment (e
...
, the robot is in the center of the room), in topological coordinates
(e
...
, in Room 311), or in absolute coordinates (e
...
, latitude, longitude, altitude)
...
11
...
II) provide an
excellent example of how these four questions arise naturally in applications
...
Rhino hosted tours of the Deutsches Museum in Bonn,
Germany, while Minerva was used in the Smithsonian’s National Museum
of History in Washington, DC
...
Minerva also could create a custom tour of requested
exhibits, requiring it to know how to get to each of the exhibits (path planning)
...
This
also means that the robot had to remember where it had been
...
An
important aspect of the map was that it included the time needed to navigate between exhibits, not just the distance
...
For example,

318

Part II

a
...

Figure II
...

(Photographs courtesy of Sebastian Thrun and Wolfram Burgard
...

Criteria for Evaluating Path Planners
Although it should be clear by the end of Part II that navigation is more than
path planning, path planning is the most obvious aspect of navigation from
a programmer’s standpoint
...
As with everything else
robotic, the choice of technique depends on the ecological niche the robot
will operate in
...
Complexity
...
Sufﬁciently represents the terrain
...
Outdoor robots may be operating in rough terrain
with steep inclines or undesirable areas, such as slippery sand or mud
...

3
...
Robots
have physical limitations
...
Recall that holonomic robots can turn in place
...
Likewise, robots may not be round
...
A robot
which is not round introduces more complexity
...

4
...
Path planners are deliberative by definition
...
A technique which simpliﬁes this
transformation is desirable
...
Supports corrections to the map and re-planning
...
Therefore, a robot
may start out with one map, discover it is incorrect, and need to update
the map and re-plan
...

The Impact of Sensor Uncertainty
Since navigation is a fundamental capability of a mobile robot, researchers
have been investigating navigational techniques since the 1960’s
...
As
a result, most researchers in navigation were forced to develop techniques
using simulators and assumptions about how real robots would physically
work
...
First, it was generally assumed that the robot
could localize itself accurately at each update
...
As was seen just with sonars in Ch
...
Sensors are always noisy and have vulnerabilities
...
In the Reactive Paradigm, the way in which the sensors were coupled with the actuators accepted this uncertainty
...
However, the process of moving often eliminated the source of the noisy data,
and soon the robot was back to doing the right thing
...
These methods are mathematical
in nature and are covered in Ch
...

Navigation and the Robotic Paradigms
The questions posed call to mind deliberation
...
Map making and localization imply memory and labeling speciﬁc locations (room, hall, river, canyon); these are symbolic representations and so also ﬁt the notion of deliberation from the Hybrid Para-

Part II

321

digm
...

One important observation is that the four questions of navigation largely
ignore an implicit ﬁfth question: how am I going to get there? Based on Part I,
the obvious answer is “by using reactive behaviors
...
Work addressing this issue of
interleaving planning and execution is presented in Ch
...

Spatial Memory

SPATIAL MEMORY

The answer to what’s the best way there? depends on the representation of
the world that the robot is using
...
63 Spatial memory is the heart of the cartographer
object class (or its equivalent) in a Hybrid architecture, as described in Ch
...

Spatial memory should provide methods and data structures for processing and storing output from current sensory inputs
...
”
Even for the coordination and control of reactive behaviors, the robot needs
to operationalize concepts such as “hall,” “red,” “door” into features to look
for with a perceptual schema
...
It would also be advantageous if the robot sensed a barrier or dead-end and updated its map of
the world
...
Suppose a robot
is directed this time to to “go down the hall to the third door on the right
...
By looking for “red” and “yellow” in
addition to other perceptual features of a door, the robot can more reliably
identify doors, either by focus of attention (the robot only runs door detection on red and yellow areas, not every image) or by sensor fusion (more
sources of data means a more certain percept)
...
Attention
...
Reasoning
...
Path planning
...
Information collection
...
Route representations express space in terms of the connections between landmarks
...
Look for the museum on the right, and turn
left at next trafﬁc light
...
Route representations also tend to supply orientation cues:
“out of the parking lot” (versus being contained in it), “turn left,” “on the
right
...

Layout representations are the opposite of route representations
...
Layout representations are often called metric representations because
most maps have some approximate scale to estimate distances to travel
...
A layout representation is essentially a bird’s-eye view of
the world
...
The layout is orientation and position independent
...
(Consider how easy it is to read a map and give verbal directions
to a driver, versus drawing an accurate map of a road you’ve only been on
once
...
An agent
can use this information to generate alternative routes if the desired route is
blocked
...
The successes of the Reactive Paradigm suggest that for robots “less is more
...
This is a
gray area
...

How accurately and efﬁciently does the robot have to navigate? Is time critical, or can it take a slightly sub-optimal route? Navigational tasks which
require optimality tend to require more dense and complex world representations
...
? What are the properties of the available
sensors in that environment?

End Notes
Invasion of the robot tour guides
...
A group also at the CS department at CMU developed SAGE, a tour guide for the Carnegie Museum in Pittsburgh, under the direction
of Illah Nourbakhsh
...

Given a description of an indoor ofﬁce environment and a set of behaviors, build a relational graph representation labeling the distinctive places
and local control strategies using gateways
...

Given a ﬁgure showing landmarks, create a topological map showing
landmarks, landmark pair boundaries, and orientation regions
...
1
TOPOLOGICAL
NAVIGATION

Overview
Topological, route, or qualitative navigation is often viewed as being more simple and natural for a behavior-based robot
...
” Even without a map
of where everything is, there is enough information for navigation as long as
the robot knows what a “hall,” “dead-end,“ and “room” is
...
Relational
...

326

9 Topological Path Planning

2
...
Relational techniques tend to focus on the graph-like representation of spatial memory
...

Because relational techniques use an explicit representation, they can support path planning
...

9
...
A landmark is
one or more perceptually distinctive features of interest on an object or locale
of interest
...
” A landmark can be a grouping of objects; for example, “McDonald’s” means a tall sign, bright building of a certain shape, and
a parking lot with a lot of activity
...
”
Landmarks are used in most aspects of navigation
...
If the robot plans a path consisting of segments, landmarks are needed so the robot can tell when it has completed a
segment and another should begin
...

Dave Kortenkamp popularized a particularly interesting special case of
landmarks: gateways
...
For example, an intersection of two hallways
is a gateway; the robot can choose to go straight or turn left or right
...

Landmarks may be either artiﬁcial or natural
...
” An artiﬁcial landmark is a set of features added to an existing object or locale in
order to either support recognition of the landmark or some other perceptual
activity
...
It is put there with the purpose of being easy to see (retro-reﬂective)
and the white-on-green font is sized for optimal visibility (perceptual activity is reading the sign)
...
2 Landmarks and Gateways

CRITERIA FOR
LANDMARKS

327

perceptual activity
...
Clearly, the McDonald’s was not built
with the purpose of being a navigational cue to a private home
...

Regardless of whether a landmark is artiﬁcial or natural, it must satisfy
three criteria:
1
...
If the robot can’t ﬁnd the landmark, it isn’t useful
...
Support the task dependent activity
...
Suppose a landmark is intended to provide
position estimation to guide docking the space shuttle to a space station
...

3
...
If the landmark is widely visible, the robot may never ﬁnd it
...
9
...
42 Each robot was supposed to follow a route between any sequence of waypoints in an arena
...
Each waypoint is readily recognizable by the checkerboard pattern
...
Notice that the use of a cylinder guaranteed that
landmarks would be perceivable from any viewpoint in the arena
...
It should be
passive in order to be available despite a power failure
...
It should have
distinctive features, and, if possible, unique features
...
g
...
If the feature occurs only once in
the entire region of operations (e
...
, there is only one McDonald’s in Tampa),
then the feature would be considered globally unique
...
If
the robot needs to position itself to within 0
...

328

9 Topological Path Planning

Figure 9
...

(Photograph courtesy of AAAI
...
3

Relational Methods
Relational methods represent the world as a graph or network of nodes and
edges
...
Edges represent
a navigable path between two nodes, in effect that two nodes have a spatial relationship
...
Paths can be computed between two points using standard graph algorithms, such as Dijkstra’s single source shortest path
algorithm
...
)
One of the earliest investigations of relational graphs for navigation was by
Smith and Cheeseman
...
They

9
...
2 Representation of a ﬂoor plan as a relational graph
...

As would be expected from the section on proprioception in Ch
...

9
...
1
DISTINCTIVE PLACE

Distinctive places
Kuipers and Byun tied relational graphs to sensing in their seminal work
with distinctive places
...
Their work was motivated by research in cognitive science indicating that spatial representation
in the animal kingdom forms a multi-level hierarchy
...
)
The lowest level, or most primitive way of representing space, was by identifying landmarks (doors, hallways) and the procedural knowledge to travel

330

9 Topological Path Planning

landmark 1

landmark 4

landmark 2

landmark 3

Figure 9
...

LOCAL CONTROL
STRATEGY

HILL - CLIMBING
ALGORITHM

between them (follow-hall, move-thru-door)
...
It represented the landmarks and procedural knowledge in a relational
graph, which supported planning and reasoning
...
Higher
layers represented increasing intelligence
...
Each node represents a distinctive place
...
One
example of a distinctive place was a corner
...
) The idea was
that the robot could repeatably move around in the neighborhood the corner
until, for example, 1 meter from each wall
...

An arc or edge in the relational graph was called a local control strategy,
or lcs
...
When the robot senses the landmark it is ﬁlling
in values for a set of features
...
The
hill-climbing algorithm directs the robot around in the neighborhood until
a measurement function (e
...
, how far away the walls are) indicates when
the robot is at a position where the feature values are maximized (e
...
, both

331

9
...
4 Multi-level spatial hierarchy, after Byun and Kuipers
...
The point where the feature values are maximized is the
distinctive place
...
The
idea is that for most hills, if you always choose the next step which is the
highest (you can’t look ahead), then you will get to the top quickly
...

Although developed independently of each other, reactive behaviors map
nicely onto distinctive places and local control strategies, as shown in Fig
...
6
...
The local control strategy is a behavior, such as follow-hall, which operates in conjunction with a releaser, look-for-corner
...
When it is triggered, the robot is in the neighborhood of the distinctive
place, and the released behavior, hillclimb-to-corner(distance=1),
directs the robot to 1 meter from each wall
...
3
...

The robot may drift off-course because the hall is wide, but as soon as it
reaches the neighborhood, it self-corrects and localizes itself
...
5 A robot reacting to a corner and moving to a distinctive place within the
neighborhood
...
6 Behaviors serving as local control strategies, and releasers as means of
signalling the entrance to a neighborhood
...
4 Associative Methods

333

rors could use multiple trips between nodes to build up a reasonable metric
map, since most of the errors would average out
...
As long as the robot
found something distinctive that it could reliably localize itself to, it could be
put on a topological map
...

Returning to the discussion of landmarks, it should be noticed that a landmark must be unique to a node pair
...

The distinctive place approach as originally formulated encountered some
problems when behavior-based roboticists began to apply it to real robots
...
Good distinctive
places are hard to come by; conﬁgurations that seemed useful to humans,
like corners, proved difﬁcult to reliably sense and localize against
...
Another challenge was learning the local control strategy
...
But how did it learn the appropriate local
control strategy? In an indoor environment, the robot might resort to always
using wall following, even though other behaviors would be better suited
...
The issue of indistinguishable locations
has also been tackled to some degree by work with probablistic methods,
which will be covered in Ch
...

9
...
The underlying assumption is that a location or landmark of
interest for navigation usually has two attributes:

PERCEPTUAL STABILITY

1
...
perceptual distinguishability: that views far away should look different
...
7 Derivation of an image signature
...

These principles are implicit in the idea of a neighborhood around a distinctive place
...
The main difference is that associative methods use very coarse
computer vision
...
4
...
An image signature is created by partitioning an image into sections
...
9
...
Next, each section is
examined and a measurement is made of some attribute in the section
...
The image signature reduces the image to n measurements
...

Notice that the image signature in the ﬁgure forms a pattern
...
4 Associative Methods

Figure 9
...

335

Image measurements and the appropriate response from two different

is in the neighborhood of the location, then the image measurements should
be approximately the same pattern as the image signature, only the pattern
may be offset due to the not-quite-at-the-right-place viewpoint
...
The use of image signatures to direct a robot to a speciﬁc
location is called visual homing
...
It is easy to speculate that
baby bees are learning the image signatures of their hive as they execute the
zoom in/out behavior described in Ch
...
In that case, the compound eyes
serve as de facto partitions of what to humans would be a single image
...
4
...
85 At that time, topological navigation using relational graphs appeared promising for indoor environments, but seemed to
resist application to outdoor terrains
...
9 Division of an outdoor area into orientation regions, after Levitt and
Lawton
...
With that challenge went the associated question of what could
the robot use as a landmark
...
The idea was to have a robot vehicle capable
of stealthily scouting an area and seeing if enemy forces were nearby
...

Inside of trying to localize the vehicle to a particular landmark, the idea
behind QualNav 85 (an acronym for “qualitative navigation”) was to localize
the vehicle to a particular orientation region, or a patch of the world
...
A landmark pair
boundary is an imaginary line drawn between two landmarks
...

An orientation region is conceptually similar to a neighborhood
...
If

9
...
If the vehicle is in OR2, the
landmarks will be tree-building-intersection
...
For example, as the robot moves from OR1 to OR2, the
building-tower landmark pair boundary is in front of it, then is on both sides
of the vehicle, and is ﬁnally behind it
...

The use of orientation regions allows the robot to create an outdoor topological map as it explores the world, or to localize itself (coarsely) to a metric
map
...

How the robot navigates inside an orientation region has the ﬂavor of visual homing
...
(A military robot might need to return without being seen
...
But if it has to remember the angles
to each landmark every n minutes, it can move to follow the angles
...

One amusing aspect of the viewframe approach is that it assumed the robot had cameras literally in the back of its head
...
Daryl Lawton tells the story
of trying to convince the union driver of the vehicle to stop every 10 feet
and do a 360 turn so he could record the viewframes
...
The image signature and viewframe concepts do not require
the robot to recognize explicitly what a landmark is, only that it is perceptually stable and distinguishable for the region of interest
...

Of course, this is more problematic for indoor environments than outdoor
ones
...
10 A student testing an unidentiﬁed entry in the competition arena at the
1994 AAAI Mobile Robot Competition “Ofﬁce Delivery” event
...
)

9
...
The 1994 competition had
an ofﬁce navigation event
...

Entrants were given a topological map, but weren’t allowed to measure the
layout of the rooms and halls
...
5) were used to simplify behavioral management, and the lessons
learned
...
5 Case Study of Topological Navigation with a

9
...
1

Hybrid Architecture

339

Path planning
The topological map was entered as an ASCII ﬁle in Backus-Naur Form
...
9
...
The input map consists of three node types:
room (R), hall (H), and foyer (F)
...

Since edges between nodes can be in only four directions, it is convenient
to refer to them as north (N), south (S), east (E), and west (W), where N is set
arbitrarily in the map
...
The topological map is structurally correct, but does not necessarily
represent if a corridor or door is blocked
...
An additional assumption is that the outside of each
door is marked with a landmark such as a room number or room name
...
It takes
as input a gateway type of topological map and a start node and goal node,
and produces a list of nodes representing the best path between the start and
goal
...
The preprocessing step begins by
building a database of the nodes in the input map, reclassifying the corridor
nodes which represent a hall to door connection as Hd
...
A Hd node may be connected to a room which is not
visitable, that is, it is neither the goal room, the start room, or a room with
more than one door
...
A sample input graphical topological representation for a metric map is shown in Fig
...
11
...
9
...
Note that Hd3-R1 and Hd4-R6 were eliminated because they
were not associated with the start or goal rooms and could not be used as a
shortcut since they had only one entrance
...
For
example, if the robot is going down the hallway from H11 to H8 and the path
is blocked, the robot will need to return to a known location in order to reorient itself and replan
...
To solve this dilemma of traveling the same part
of the corridor possibly three times, the cartographer maintains nodes which
represent possible alternative strategies
...

340

9 Topological Path Planning

H20

H8

H9

H10

H7

R5

H11

H13

R7

H12

R4
R6

H1

H2

H3

H4

R1

H5 H6

F1

H14

R2

H19

H15

R3

H18

F2

H17

H16

a
...

H17

H20

H4

Hd5

Hd9

H10

Hd7

H3

H8

R5

Hd13

H6

F1

H14

R2

H16

H19

H11

R4

Hd12

H15

R3*

Hd18

F2

Hd17

H16

c
...
11 a
...
) graphical topological representation, and c
...

An optimal path is computed using Dijkstra’s single source shortest path
algorithm
...
Since the topological representation is not metric, connections between the nodes reﬂect preferences in constructing routes
...
In this
implementation, navigating through foyers was considerably more computationally expensive and unreliable; therefore the edge weighting for any
path subsegment originating from a foyer was set at 3
...
g
...
All other connections were set to 1
...
5 Case Study of Topological Navigation with a

TRANSITION TABLE

Hybrid Architecture

341

The Task Manager in SFX uses the path computed by the cartographer to
select the appropriate abstract navigation behavior (Ch
...
It maintains a pointer to the current node and the
next intended node from the path
...
Also, the table is not necessarily symmetrical
...
The ANB itself uses the information in the database entries for the nodes as parameters
for instantiating the script to the current waypoint pair
...
If the obstacle avoidance behavior posts to the
whiteboard structure that a BLOCKED condition has occurred, the Task Manager terminates the currently active abstract navigation behavior
...
But it triggers a simple move-to-goal behavior, which allows the robot to reorient itself more quickly than reinstantiating the abstract
navigation behavior with new parameters
...
The Cartographer removes the connection between the current and intended nodes, then
recomputes a new path with the current node as the start
...

Fig
...
12 demonstrates the robot moving through the map handed out at
the actual AAAI Mobile Robot Competition
...
The cartographer computes the path as R7-H1-H2-H5-R2, eliminating H3 and H4 because they are not relevant for this task
...
12 Scenario for moving from R7 to R2
...

9
...
The second line shows that the current navigational task is to
move from R7 to H1
...
By examining the database, the door is known to be to the south
...
When the
door is found, the initialization phase of the abstract behavior is over and the
nominal activity of moving through the door is triggered
...

The next navigational task is to move from H1 to H2, again in the direction
south
...
The task of moving from H2
to H5 is interesting because it shows that the termination condition for hall
following is different than for H1 to H2
...

Once the door has been identiﬁed visually, the robot is considered at H5
...
However, in this case, the
ultrasonics has not yet identiﬁed the door opening and so the initialization
phase consists of wall following until the door opening is recognized
...

The second simulation shown in Fig
...
13 uses the sample topological map
provided prior to the competition
...
As shown in the ﬁgure, the robot detects
that the path is blocked and uses move to goal to return to H8
...
The output of the
script below shows the robot conﬁrming the nodes H5 and H6 even though
the plan did not have them enter the associated rooms
...

9
...
2

Navigation scripts
The path planning and execution components are clearly deliberative
...
The transition table takes the role of a high level sequencer
...
13

Scenario for moving from R1 to R0 with blocked path
...
5 Case Study of Topological Navigation with a

Hybrid Architecture

345

are used to specify and carry out the implied details of the plan in a modular
and reusable way
...
The pseudocode for navigate-door is shown below in C ++
style and takes advantage of the indexing properties of the switch statement:

switch(door)
case door-not-found:
//initialization phase
//follow wall until find door
if wall is found
wallfollow to door
else
move-ahead to find a wall
case door-found:
//nominal activity phase
move-thru-door(door-location)

The nominal behavior, move-thru-door, is self-terminating, so there is
no separate termination condition
...

The navigate-hall is used for traveling between corridors, foyers and
corridors, and from corridors to corridor/door nodes
...
One condition is that the robot is in a foyer and detects a
hall
...
The task manager uses the
directional information stored in the database to determine which wall of
the foyer to follow
...
Since
the robot is not guaranteed to be facing the centerline of the hall (ANBs have
no knowledge of what was done prior to their instantiation except via global
variables), the sub-script turns it to line itself up with the desired hall
...
There are three behaviors which look
for gateways
...
These behaviors run concurrently with the nominal
behaviors
...
It assumes that two foyers’ nodes connected are a connvenient representation of a single large foyer with entries from different directions (i
...
,
multiple gateways into the foyer)
...
This gets the robot away from
potentially confusing ultrasonic signatures
...
There is no case statement in the pseudocode since the sequence
of activities is ﬁxed
...
5
...
5 Case Study of Topological Navigation with a

Hybrid Architecture

347

fail
...
The tests
before and after the competition offered several practical lessons
...
The biggest problem with the implementation has not been with
the scripts per se, but rather with the quality of the primitive behaviors
...
Second, contention for sensing resources is an emerging issue for robotic control schemes
...

In one instantiation of the navigate-hall behavior between a H node
and a Hd node, the nominal behavior of hallfollow frequently pointed
Clementine and her camera away from the anticipated location of the door,
interfering with the confirm-door perceptual behavior used to terminate
navigate-hall
...
One alternative is to let the two active behaviors take turns with control of the robot
...

Note that the representation and path planning algorithm support the addition of metric information
...
The distance could be attached to the reference index in the node class, much the same as the nodes connected to the
N, E, S, W are stored
...
If wallfollow takes the robot
around the perimeter of a foyer in order to exit, the distance traveled will not
reﬂect the straight line distance between nodes
...
On
the other hand, if the width of a foyer were known, the robot might possibly
be able to use dead reckoning to move directly across it to the desired exit
...
A short hall
can take a long time to navigate if it is cluttered during one visit, but a short
time during another
...
Metric distances may not be known for all pairs of nodes,
making it difﬁcult to apply Dijkstra’s algorithm
...
Utility theory can
be used to quantitify the impact of these competing concerns
...
6

Summary
Landmarks simplify the where am I? problem by providing orientation cues
...
There are two categories of qualitative navigation methods: relational
and associative
...
The robot can use the graph to plan a path using techniques such as the single source shortest path algorithm
...
When it sees the landmark of the location, it is in the neighborhood, and
then can use another behavior, such as hill-climbing, to localize itself relative to the landmark
...
An image can be compared to a image signature or a viewframe to
generate the next movement for the robot to take
...

9
...
1
List the four questions associated with navigation
...
2
List and describe the criteria for evaluating a path planner
...
3
Describe the difference between route and metric representations of space
...
4
Deﬁne the difference between natural and artiﬁcial landmarks and give one example
of each
...
5
Deﬁne a gateway
...
6
Build a relational graph representation labeling the distinctive places and local control strategies using gateways for a ﬂoor of a building on campus
...
7 Exercises

Exercise 9
...

Exercise 9
...

Exercise 9
...
Do these three levels ﬁt naturally
within a Hybrid architecture? How would they be implemented in a state-hierarchy
style? A model-oriented style?
Exercise 9
...
11

[World Wide Web]

Visit the Minerva site as http://www
...
cmu
...
Write a one-page paper
summarizing the project
...
12

[Advanced Reading]

Read the scientiﬁc papers at the Minerva web site
...
The path planner used and evaluate it using the criteria presented in this overview
...
Discuss the impact of sensor uncertainty on Minerva’s navigation and how it was
addressed
...
List which of the four functions of spatial memory were used
...
13

[Advanced Reading]

Read the article about the two winners of the 1992 AAAI Mobile Robot Competition
in Congdon et al
...

Exercise 9
...
Can they be used
interchangeably?
a
...
hill-climbing
Exercise 9
...

Exercise 9
...

[Programming]

350

9 Topological Path Planning

a
...

b
...
Implement Dijkstra’s single source
shortest path algorithm to compute the shortest path between two points speciﬁed
by a user
...
Implement a minimal spanning tree algorithm to allow the robot to visit all waypoints efﬁciently
...
8

End notes
Of batteries and topological navigation
...
At one point, the team was using the car battery from the school
van parked outside the Seattle Convention Center
...
No security guard or
policeman ever said a word
...

Topological navigation examples
...

10

Metric Path Planning

Chapter objectives:
Deﬁne Cspace, path relaxation, digitization bias, subgoal obsession, termination
condition
...

Represent an indoor environment with a generalized Voronoi graph, a regular grid, or a quadtree, and create a graph suitable for path planning
...

Apply wavefront propagation to a regular grid
...

10
...
As with topological methods, the objective is to determine
a path to a speciﬁed goal
...
Another difference is that
metric paths are usually decompose the path into subgoals consisting of waypoints
...

These locations may have a mathematical meaning, but as will be seen with
meadow maps, they may not have a sensible correspondence to objects or

352

10

COMPONENTS OF
METRIC PATH
PLANNERS

Metric Path Planning

landmarks in the world
...

The terms “optimal” and “best” have serious ramiﬁcations for robotics
...
As will
be seen, some metric methods are able to produce an optimal path because
they consider all possible paths between points
...
Fortunately, some algorithms (especially one named “A*” for reasons that will be discussed later) are more clever about rejecting non-optimal
paths sooner than others
...
The ability to
produce and compare all possible paths also assumes that the planning has
access to a pre-exisiting (or a priori) map of the world
...
As such, metric methods
are compatible with deliberation, while qualitative methods work well with
more reactive systems
...

Metric path planners have two components: the representation (data structure) and the algorithm
...
They use a variety of techniques to represent
the world; no one technique is dominant, although regular grids appear to
be popular
...
Path planning algorithms generally work on almost any conﬁguration space representation,
although as with any algorithm, some methods work better on certain data
structures
...
Regardless of what algorithm is used,
there is always the issue in a Hybrid architecture of when to use it
...

10
...
1

10
...

Conﬁguration Space
The physical space robots and obstacles exist in can be thought of as the
world space
...

A good Cspace representation reduces the number of dimensions that a
planner has to contend with
...
A
person may specify the location of the object as a (x y z ) coordinate in some
frame of reference
...
Three more degrees are needed to represent where the front
of the chair is facing, whether it is tilted or not, or even upside down
...

Six degrees of freedom is more than is needed for a mobile ground robot in
most cases for planning a path
...
However, the z coordinate will be
of interest if the robot is an aerial or underwater vehicle
...
Who cares which way the robot is facing if all
the robot wants to do is to plan a path around it? But the pitch of a planetary
rover or slope of an upcoming hill may be critical to a mission over rocky
terrain
...
10
...
In general, metric path planning algorithms for mobile robots have assumed only two DOF,
including for the robot
...
This implicitly assumes

354

10

HOLONOMIC

10
...
Many research robots are sufﬁciently close to holonomic to meet this assumption
...

Much work is being done in non-holonomic path planning, where the pose
of the robot must be considered (i
...
, can it actually turn that sharply without
colliding?), but there no one algorithm appears to be used in practice
...

Cspace Representations
The number of different Cspace representations is too large to include more
than a coarse sampling here
...
The representations offer different ways of partitioning free space
...
Each partition can be labeled with additional information: “the terrain is rocky,” “this is off-limits
from 9am to 5am,” etc
...
3
...

This map could be digitized in some manner, and then the robot could apply
various algorithms to convert the map to a suitable Cspace representation
...

Meadow maps transform free space into convex polygons
...
The polygon represents a safe region for the robot to traverse
...
Meadow maps are not that common in robotics,
but serve to illustrate the principles of effecting a conﬁguration space and
then planning a path over it
...
First, the planner begins with
a metric layout of the world space
...
10
...
3 Cspace Representations

Figure 10
...

a
...

Figure 10
...
) partitioning of free space into convex polygons, and
b
...

planner to treat the robot as if it were a point, not a 2D object
...

The next step in the algorithm is to construct convex polygons by considering the line segments between pairs of some interesting feature
...
The algorithm can then determine which combination of line segments
partitions the free space into convex polygons
...
Each convex polygon represents a safe passage for
the robot
...
Some of the line segments

356

10

Metric Path Planning

Figure 10
...

PATH RELAXATION

forming the perimeter aren’t connected to another a polygon (i
...
, they are
part of a wall), so they should be off limits to the planning algorithm
...

It would make a difference to the overall path length where the robot cuts
across the polygon
...
So the issue becomes how to specify candidate points on the polygon
...
Note that each of these midpoints becomes a node, and
if edges are drawn between them, an undirected graph emerges
...

One disadvantage of a meadow map, indeed of any Cspace representation, is evident on inspection of Fig
...
3: any path which is chosen will be
somewhat jagged
...

One outcome of the partitioning process is that the free space is divided up
in a way that makes sense geometrically, but not necessarily for a robot to
actually travel
...

Chuck Thorpe at CMU devised a solution to paths generated from any kind
of discretization of free space
...
Now imagine pulling on both ends to tighten the string (the technical name for this is
path relaxation) This would remove most of the kinks from the path without
violating the safe zone property of convex polygons
...
One
problem is that the technique to generate the polygons is computationally

10
...
5 Graph from a generalized Voronoi graph (GVG)
...
)

complex
...
Unless the robot has accurate localization, how does it know it is halfway down a long,
featureless hall and should turn 30 ? The third major disadvantage is that
it is unclear how to update or repair the diagrams as the robot discovers
discrepancies between the a priori map and the real world
...
3
...
Unlike a meadow map, a GVG can be
constructed as the robot enters a new environment, thereby creating a topological map as shown by Howie Choset at CMU
...
As seen in Fig
...
5, this line goes down the middle of
hallways and openings
...
Notice the vertices often have a physical correspondence
to conﬁgurations that can be sensed in the environment
...

If the robot follows the Voronoi edge, it will not collide with any modeled
obstacles, because it is staying “in the middle
...
Edges serve as freeways or major thorough-

358

10

Metric Path Planning

Figure 10
...

fares
...
It is only the length, not the physical
reality, of the edges that make any difference
...
3
...
The regular
grid method superimposes a 2D Cartesian grid on the world space, as shown
in Fig
...
6
...
Hence, regular grids are often referred to as
occupancy grids
...
11
...
The center of each element
in the grid can become a node, leading to a highly connected graph
...

Unfortunately, regular grids are not without problems
...
This leads
to wasted space and leads to very jagged objects
...
This ﬁne granularity means a high storage cost, and a high
number of nodes for a path planning algorithm to consider
...
4 Graph Based Planners

359

Figure 10
...

10
...
4
QUADTREES

10
...

Quadtrees are a recursive grid
...

If an object falls into part of the grid, but not all of it, the Cspace algorithm
divides the element into four (hence the name “quad”) smaller grids, each 16inches square
...
A three dimensional quadtree is called
an octree
...
This means that the path between the initial node and the
goal node can be computed using graph search algorithms
...
However, many of those
algorithms require the program to visit each node on the graph to determine
the shortest path between the initial and goal nodes
...
Therefore, there has

360

10

Metric Path Planning

been a great deal of interest in path planners which do a “branch and bound”
style of search; that is, ones which prune off paths which aren’t optimal
...
It is derived from the A search method
...
Both assume a metric map, where the location of each node is known in
absolute coordinates, and the graph edges represent whether it is possible to
traverse between those nodes
...
It generates the
optimal path incrementally; each update, it considers the nodes that could
be added to the path and picks the best one
...
The heart of the method is the formula (or
evaluation function) for measuring the plausibility of a node:

f (n) = g(n) + h(n)
where:

f (n) measures how good the move to node n is
g(n) measures the cost of getting to node n from the initial node
...
Assume that a
Cspace representation yielded the graph in Fig
...
8
...
There are only two nodes to choose from: B and C
...
The
plausibility of B as the next move is:

f (B ) = g(B ) + h(B ) = 1 + 2:24 = 3:24
where g (B ) is the cost of going from A to B , and h(B ) is the cost to get from
B to the goal E
...
4 Graph Based Planners

361

Figure 10
...

The plausibility of C is:

f (C ) = g(C ) + h(C ) = 1 + 1 = 2:0
where g (C ) is the cost of going from A to C , and h(C ) is the cost of getting
from C to E
...

But this assumes that h(n) was known at every node
...
This leads
to visiting all the nodes
...
A* search takes an interesting approach to
reducing the amount of paths that have to be generated and compared: it
compares the possible paths to the best possible path, even if there isn’t a
path in the real world that goes that way
...
The estimate can then be used to determine which nodes are
the most promising, and which paths have no chance of reaching the goal
better than other candidates and should be pruned from the search
...
9

ADMISSIBILITY
CONDITION
HEURISTIC FUNCTION

Metric Path Planning

An A* example
...
In path planning, g (n)
is the same as g (n): the cost of getting from the initial node to n, which is
known through the incremental build of the path
...

So what is a way to estimate the cost of going from n to the goal? Furthermore, how can we be sure that the estimate will be accurate enough that we
don’t end up choosing a path which isn’t truly optimal? This can be done by
making sure that h (n) will never be smaller than h(n)
...
Since h (n) is an estimate, it
is also called a heuristic function, since it uses a rule of thumb to decide which
is the best node to consider
...
Recall that the
locations of each node are known independently of the edges
...

The straight line distance is always the shortest path between two points,
barring curvature of the earth, etc
...

To see how A* uses this to actually eliminate visiting nodes, consider the
example in Fig
...
9
...

The choices are B and D, which can be thought of as a search tree (see
Fig
...
10a) or as a subset of the original graph (see Fig 10
...
Regardless

10
...
10 a
...
) the original
graph
...
) What A* “sees” from considering a path A-D-?-E, and d
...

364

10

Metric Path Planning

of how it is visualized, each node is evaluated to determine which is the
most plausible move
...
The choices evaluate to:

f (B ) = g (B ) + h (B ) = 1 + 2:24 = 3:24
f (D) = g (D) + h (D) = 1:4 + 1:4 = 2:8
A path going from A ; D;? ; E has the potential to be shorter than a
path going from A ; B ;? ; E
...
Notice
that A* can’t eliminate a path through B because the algorithm can’t “see” a
path that actually goes from D to E and determine if it is indeed as short as
possible
...
10
...

The two options from D are E and F , which are evaluated next:

f (E ) = g (E ) + h (E ) = 2:8 + 0 = 2:8
f (F ) = g (F ) + h (F ) = 2:4 + 1:0 = 3:4
Now the algorithm sees that E is the best choice among the leaves of the
search tree, including the branch through B
...
) It is better than F and B
...

The optimal path is A ; D ; E , and we didn’t have to explicitly consider
A ; B ; F ; E
...
f (F ) didn’t need to be computed if the algorithm looks at its choices
and sees that one of them is the goal
...

Another important insight is that any path between A and E has to go
through D, so the B branch of the search tree could have been pruned
...
It’s easy to imagine in a larger graph
that there might be a case where after several expansions through D, the leaf
at B in the search tree came up the most plausible
...
Since D already occurred in
another branch, with a cheaper g (D), the B branch could be safely pruned
...
5 Wavefront Based Planners

365

This is particularly useful when A* is applied to a graph created from a regular grid, where the resulting graph is highly connected
...
The
major impact that Cspace has on the A* planner is how many computations
it takes to ﬁnd the path
...
For
example, the straight line distance may be over rocky terrain or sand that
poses a risk to the robot
...
In order to factor in the impact
of terrain on the path costs, the heuristic function h (n) has to be changed
...
If the new h just takes the worst case energy cost
sibility condition: h
or safety cost, it will be admissible, but not particularly useful in pruning
paths
...
(Negative weights
pose an interesting problem in that the robot can get into a loop of rolling
down a hill repeatedly because it’s an energy-efﬁcient solution rather than
actually making forward progress! Bellman-Ford types of algorithms deal
with this situation
...
5

Wavefront Based Planners
Wavefront propagation styles of planners are well suited for grid types of
representations
...
Eventually the heat will spread and reach the goal, if there
is a way, as shown in the sequence in Fig
...
11
...
An interesting aspect of wavefront propagation is that
the optimal path from all grid elements to the goal can be computed as a
side effect
...
One of the
many wavefront types of path planners is the Trulla algorithm developed
by Ken Hughes
...

366

10

Metric Path Planning

a
...

c
...

e
...

g
...

Figure 10
...
Elements holding the
current front are shown in gray, older elements are shown in dark gray
...
6 Interleaving Path Planning and Reactive Execution

367

An attractive feature of wavefront propagation is how it treats terrain types
...
0 (no heat can be propagated
through that grid element), while an open area has inﬁnite conductivity
...
This means that heat will travel
slower through undesirable regions
...
A wavefront naturally handles the tradeoffs between a longer path through desirable terrain
versus taking shortcuts through less desirable terrain
...
10
...
Undesirable terrain is
shown on the Trulla display in gray, obstacles in black and open areas in
white
...
In Fig
...
12a, the robot Clementine can move over an extension cord
but would prefer not to, as shown by the gray region
...
In Fig
...
12 this isn’t possible due to
the placement of obstacles
...

10
...
Almost all techniques break a path into segments;
even a wavefront planner actually produces a goal location (waypoint) for
each directional vector
...

The Sequencer can then employ a series of move-to-goal behaviors, deactivating and re-instantiating the behavior as a new subgoal is met
...

Subgoal obsession is when the robot spends too much time and energy trying
to reach the exact subgoal position, or more precisely, when the termination
conditions are set with an unrealistic tolerance
...
Suppose the next
waypoint is at location (35, 50)
...
In practice, it is very hard for
a robot, even a holonomic robot, to reach any location exactly because it is
hard to give the robot precise movement
...
5, 50)
...
5 meters ahead, and move again to attempt
to reach (35, 50)
...
5,50
...

368

10

Metric Path Planning

a
...

Figure 10
...
) Trulla output and photo of Clementine navigating around cord
...
)
Trulla output and photo of Clementine navigating over the cord
...
A resulting see-sawing set of motions
emerges that can go on for minutes
...
The problem is exacerbated by
non-holonomic vehicles which may have to back up in order to turn to reach
a particular location
...

To handle subgoal obsession, many roboticists program into their move-togoal behaviors a tolerance on the terminating condition for reaching a goal
...
So if a cylindrical, holonomic robot with a diameter of 1
meter is given a goal of (35, 50), it will stop when 34
...
5 and 49
...
6 Interleaving Path Planning and Reactive Execution

369

< y < 50
...
There is no common heuristic for non-holonomic robots, because

D* ALGORITHM

the maneuverability of each platform is different
...
For example, consider a subgoal at the opposite end
of a hall from a robot, but the hall is blocked and there is no way around
...
One solution is for the Sequencer to estimate a maximum allowable time for the robot to reach the goal
...
The advantage of the latter solution is that the code can become part of a monitor, leading to some form of
self-awareness
...
Suppose that the robot is heading
for Subgoal 2, when an unmodeled obstacle diverts the robot from its intended path
...
In a classic implementation, the robot would not be looking for
Subgoal 3, so it would continue to move to Subgoal 2 even though it would
be more optimal to head straight for Subgoal 3
...
What happens when the robot discovers it is being sent
through a patch of muddy ground? Trying to reactively navigate around
the mud patch seems unintelligent because choosing left or right may have
serious consequences
...
The issue becomes how
does a robot know it has deviated too far from its intended path and needs
to replan?
Two different types of planners address the problems of subgoal obsession and replanning: the D* algorithm developed by Tony Stentz 136 which is
a variant of the A* replanning algorithm, and an extension to the Trulla algorithm
...
D* does it by executing an A* search
from each possible location to the goal in advance; this converts A* from being a single-source shortest path algorithm into an all-paths algorithm
...
Since Trulla is a wavefront type of planner, it generates the optimal path
between all pairs of points in Cspace as a side effect of computing the path
from the starting location to the goal
...

a
...
13 Layout showing unmodeled obstacle
...
) Gray line shows expected
path, long dashed line the actual path with Trulla, and short dashed line shows purely
reactive path
...
) Clementine opportunistically turning
...
It means that if the robot can localize
itself on the a priori map, it can read the optimal subgoal for move-to-goal
on each update
...
10
...
Note how the metric path becomes a virtual
sensor, guiding the move-to-goal behavior replacing the direct sensor data
...

This approach eliminates subgoal obsession, since the robot can change
“optimal” paths reactively and opportunistically move to a closer waypoint
...
At some
point though, the sheer number of unmodeled obstacles might force the robot to get trapped or wander about, changing subgoals but making no real
progress
...
D*
represents one extreme on the replanning scale: continuous replanning
...
First, it may be too computationally expensive to be practical for a robot with an embedded processor

10
...
7

371

and memory limitations, such as a planetary rover
...
If the robot senses an
unmodeled obstacle at time T1, it computes a new path and makes a large
course correction
...
The result can be a robot which has a very jerky
motion and actually takes longer to reach the goal
...
Trulla uses
the dot-product of the intended path vector and the actual path vector
...
Therefore
the dot product acts as an affordance for triggering replanning: the robot
doesn’t have to know why it is drifting off-course, only that it has drifted
noticeably off-course
...
But it doesn’t handle the
situation where the real world is actually friendlier
...
10
...
The robot could achieve a signiﬁcant savings
in navigation by opportunistically going through the gap
...
A continuous replanner such as D*
has a distinct advantage, since it will automatically notice the change in the
world and respond appropriately, whereas Trulla will not notice the favorable change because it won’t lead to a path deviation
...

Summary
Metric path planning converts the world space to a conﬁguration space, or
Cspace, representation that facilitates path planning
...
These representations can then be converted to
graphs, suitable for an A* search
...
Regular grids work

372

10

Metric Path Planning

goal
start
missing
wall

Figure 10
...
The gray line is the actual path, while
the dashed line represents a more desirable path
...

Metric path planning tends to be expensive, both in computation and in
storage
...
Two problems in interleaving metric path planning with reactive
execution are subgoal obsession and when to replan
...
One solution is to continuously
replan if resources permit and sensor reliability is high; another is eventdriven replanning which uses affordances to detect when to replan
...

A possibly even more serious omission is that popular path planners are
applicable for only holonomic vehicles
...
8

Exercises
Exercise 10
...

10
...
2
Represent your indoor environment as a regular grid with a 10cm scale
...

Exercise 10
...

Exercise 10
...
How many edges will a graph have if the
neighbors are
a
...
8-connected?
Exercise 10
...
the midpoints of the open boundaries, and
b
...

Draw a path from A to B on both graphs
...

Exercise 10
...

Exercise 10
...
8
Apply wavefront propagation to a regular grid
...
9
Subgoal obsession has been described as a problem for metric planners
...
10
Trulla uses a dot product of 0 or less to trigger replanning, which corresponds to
from the desired path
...
11
Describe the difference between continuous and event-driven replanning
...

374

10

Exercise 10
...
Model at
least three different scenarios and see what the path generated is
...
13

[Programming]

Program an A* path planner
...
9
...
9

End Notes
For a roboticist’s bookshelf
...
82
Voronoi diagrams
...
Computational Geometry:
Theory and Applications 44 reports that the principles were ﬁrst uncovered in 1850, although it wasn’t until 1908 that Voronoi wrote about them, thereby getting his name
on the diagram
...

Kevin Gifford and George Morgenthaler have explored some possible formulations
of a heuristic function for path planning over different terrain types
...
60
Trulla
...

Through grants from the CRA Distributed Mentoring for Women program, Eva Noll
and Aliza Marzilli implemented it in software
...
Dave Hershberger
helped write the display routines used in this book
...

Be able to update an occupancy grid using either Bayesian, DempsterShafer, or HIMM methods
...

11
...
Both are closely related, because a robot cannot
create an accurate map if it does not know where it is
...
11
...
The hallway makes a complete circuit around
the center of the building
...
The mobile robot senses, updates the map with the portions of the
hallway that have come into view, then moves, updates, and so on
...

As can been seen from the ﬁgure, as well as discussions in Ch
...
Worse yet, the inaccuracies are highly
dependent on surfaces
...
Developing an error model to estimate the
slippage is often unrealistically difﬁcult
...

376

11

Localization and Map Making

Figure 11
...
The ground truth is in black
...
The signal is often unobtainable indoors, in tunnels, or in cities with large buildings (sometimes referred
to as urban canyons)
...

Researchers have attempted to solve the localization problem in a number
of ways
...
While
this had the advantage of being simple, it eliminated the use of global path
planning methods
...
Another
approach was to use topological maps, which have some symbolic information for localization at certain points such as gateways, but don’t require continuous localization
...
9, it is hard
to have unique gateways
...

More sophisticated systems either identiﬁed natural landmarks which had
noticeable geometric properties or added artiﬁcial landmarks
...
(This in-

11
...
) Other techniques attempted to match the raw sensor data to
an a priori map using interpretation trees or similar structures
...
Consider attempting to match
noisy sonar data to the layout of a room
...
The use of small, local maps for localization brings the process back
full circle to the need for good map-making methods
...
Iconic algorithms appear to be the more popular in practice, in part
because they usually use an occupancy grid
...
Fusion is done
either following an algorithm provided by a formal theory of evidence, either Bayesian or Dempster-Shafer, or by a popular quasi-evidential method
known as HIMM
...
Many Hybrid architectures also use
the occupancy grid as a virtual sensor for obstacle avoidance
...
Since sonars are a popular range sensor for mapping and obstacle avoidance, the chapter next covers sonar sensor models
and the three methods for using sensor models to update a grid: Bayesian,
Dempster-Shafer, and HIMM
...
The comparison
of the three methods discusses practical considerations such as performance
and ease in tuning the method for a new environment
...
It is useful for metric map building and generally uses
an occupancy grid-like structure
...
Feature-based methods have become popular with the advent of partially ordered Markov decision process (POMDP) methods to simplify reasoning about them; POMDPs
are outside the scope of this book but the basic localization strategy is presented
...

378

11

11
...
Models of sensor uncertainty can be generated in a number of ways
...
The frequency of a correct reading leads to
a belief in an observation; the set of beliefs from all possible observations
form the model
...
Subjective methods
rely on a designer’s experience, which are often an unconscious expression
of empirical testing
...
This chapter will use Polaroid sonars as an example; however, the principles of scoring and fusing belief apply to any sensor
...
11
...
6
...
This ﬁeld of view can be projected onto a regular grid
...
As shown in Fig
...
2, the ﬁeld of view can be divided into three
regions:
Region I: where the affected elements are probably occupied (drawn as a
“hill”),
Region II: where the affected elements are probably empty (drawn as a “valley”), and
Region III: where the condition of the affected elements is unknown (drawn
as a ﬂat surface)
...
Regardless of empty or occupied, the readings
are more likely to be correct along the acoustic axis than towards the edges
...

While the sensor model in Fig 11
...
2 Sonar Sensor Model

a
...

Figure 11
...
) three dimensional representation and
b
...

380

11

Localization and Map Making

Figure 11
...
(Photograph courtesy of Hans Moravec
...
Each of the three methods covered in the following sections does
the translation slightly differently
...
3

Bayesian
The most popular evidential method for fusing evidence is to translate sensor readings into probabilities and to combine probabilities using Bayes’ rule
...
Later Moravec turned to a form of Bayes’
Rule which uses probabilities expressed as likelihoods and odds
...
The likelihood/odds formulation is equivalent to the traditional

11
...
In a Bayesian approach, the sensor model generates conditional probabilities of the form P (sjH )
...
Two probabilities, either from two different sensors sensing at the same time or from two different times, can be fused using
Bayes’ rule
...
3
...
In the case of updating an occupancy grid with sonar readings, the experiment is sending the acoustic wave out and measuring the
time of ﬂight, and the outcome is the range reading reporting whether the
region being sensed is Occupied or Empty
...
This can be written H = fH :H g or H = fOccupied Empty g
...
This is expressed by:

1 ; P (H ) = P (:H )
UNCONDITIONAL
PROBABILITIES

CONDITIONAL
PROBABILITIES

As a result, if P (H ) is known, P (:H ) can be easily computed
...
An example of an unconditional probability is a robot programmed to
explore an area on Mars where 75% of the area is covered with rocks (obstacles)
...

Unconditional probabilities are not particularly interesting because they
only provide a priori information
...
It is more useful to a robot to have a function that computes the probability that a region grid i] j ] is either Occupied
or Empty given a particular sensor reading s
...
P (H js) is the probability that H has really occurred given a particular sensor reading s (the “j” denotes “given”)
...

382

11

Localization and Map Making

In an occupancy grid, P (Occupiedjs) and P (Empty js) are computed for
each element, grid i] j ], that is covered by a sensor scan
...
A tuple can
be implemented as a C struct
typedef struct {
double occupied;
double empty;
} P;
P occupancy_grid[ROWS][COLUMNS];

Probabilities provide a representation for expressing the certainty about a
region grid i] j ]
...
11
...
One set of functions which quantify this model into probabilities is given below
...
1)

TOLERANCE

;

)

Maxoccupied
P (Occupied) = R 2
P (Empty) = 1:0 ; P (Occupied)
where r and are the distance and angle to the grid element, respectively
...
11
...
Likewise, the nearer the grid ele;
ment is to the origin of the sonar beam, the higher the belief (the RR r term)
...
A Maxoccupied = 0:98 means that a grid element can
never have a probability of being occupied greater than 0
...

It is important to note that Region I in Fig
...
2 has a ﬁnite thickness
...
87 meters might actually be
between 0
...
92 meters, or 0
...
05 meters
...
It has the impact of making Region I wider, thereby covering
more grid elements
...
2)

P (Empty) =

;
( RR r ) + (
2

;

)

383

11
...
5
α=0

Figure 11
...

Note that unlike an element in Region I, an element in Region II can have
a probability of being empty of 1
...

To see how these formulas would be applied, consider the example in
Fig
...
4
...
0 feet with a tolerance
of 0:5 feet
...
98
...
The element of interest grid i] j ]
is shown in black, and is at a distance r = 3:5 feet and an angle of = 0
from the robot
...

The ﬁrst step is to determine which region covers the element
...
Therefore, the correct formulas to
apply are those in Eqn
...
2:
R;r

;

P (Empty) = ( R )+( )
= ( 10 )+(
2
2
P (Occupied) = 1:0 ; P (Empty) = 1 ; 0:83
10;3:5

15;0 )
15

= 0:83
= 0:17

The example in Fig
...
5 shows an element in Region I
...

384

11

Localization and Map Making

R=10

β=15

s=6

r=6
α=5

Figure 11
...

R;r

;

P (Occupied) = ( R )+( ) Maxoccupied = ( 10 )+(
2
2
P (Empty) = 1:0 ; P (Occupied)
= 1 ; 0:52
11
...
2

Conditional probabilities for

10;6

15;5 )
15

0:98 = 0:52
= 0:48

P (H s)
j

The sensor model represents P (sjH ): the probability that the sensor would
return the value being considered given it was really occupied
...
The laws
of probability don’t permit us to use the two conditionals interchangeably
...
3)

s H) (
P (H js) = P (sjH )P (P () j+ PP(sH ) )P (:H )
H
j:H

Substituting in Occupied for H , Eqn
...
3 becomes:

(11
...
The
other terms, P (Occupied) and P (Empty ), are the unconditional probabiliP (Occupiedjs) =

ties, or prior probabilities sometimes called priors
...
3 Bayesian

Eqn
...
4 in boxes
...

In some cases, such as for a planetary rover, there may be some knowledge
that produces the prior probabilities
...
In those cases, it is assumed that P (Occupied) = P (Empty ) =
0:5
...
11
...

For grid i] j ]:

P (s = 6jOccupied)
P (s = 6jEmpty)
P (Occupied)
P (Empty)

=
=
=
=

0:62
0:38
0:5
0:5

Substituting into Eqn
...
4 yields:

(0:62)(0:
P (Occupiedjs = 6) = (0:62)(0:5) + (05) :5) = 0:62
:38)(0
(0:38)(0:5)
P (Emptyjs = 6) = (0:38)(0:5) + (0:62)(0:5) = 0:38
The use of 0
...

11
...
3

Updating with Bayes’ rule
Now that there is a method for computing conditional probabilities of the
correct form, the question becomes how to fuse it with other readings
...
Each element in the occupancy grid is initialized with
the a priori probability of being occupied or empty
...
If the a priori
probabilities are not known, it is assumed P (H ) = P (:H ) = 0:5
...

But what about the second observation? Or an observation made from
another sonar at the same time? It turns out that in both cases, Bayes’ rule

386

11

Localization and Map Making

can be used iteratively where the probability at time tn;1 becomes the prior
and is combined with the current observation (tn )
...
For n multiple observations,
s1 s2 : : : sn , Bayes’ rule becomes:
(11
...
Ideally, this
requires a sonar model of getting occupied and empty values for all grid i] j ]
with n combinations of sensor readings
...
Now, the program only has to remember all previous n ;1 readings
...
The occupancy grid goes from being
a two dimensional array with a single two ﬁeld structure to being a two dimensional array with each element a potentially very long linked list
...
11
...
The computational overhead begins to be considerable since
an element in a hallway may have over 100 observations
...
6)

) ( n
P (H jsn ) = P (s jH )P (HPs(sn jH+PP H jsj:;1 ))P (:H js )
j n;1 ) (sn H
n
n;1
So at each time a new observation is made, Eqn
...
6 can be employed and
the result stored at grid i] j ]
...

11
...
It is a much newer theory, originating in the work of A
...
Dempster, a mathematician at Harvard, during
the 1960’s with extensions by Glen Shafer in 1987
...
Possibilistic means that the function represents partial evidence
...
4 Dempster-Shafer Theory

387

may provide direct evidence for an event H, but, due to occlusions, it may
not be perceiving the entire object
...
The possibilistic belief functions, also called Shafer belief functions, are combined used Dempster’s rule
of combination
...
Unlike Bayes’ rule, Dempster’s rule
has a term which indicates when multiple observations disagree
...

11
...
1

Shafer belief functions
Belief is represented by Shafer belief functions in Dempster-Shafer theory
...
Instead of measuring the probability of a proposition, belief functions measure the belief
mass, m
...
0, but can distribute that
mass to any combination of propositions
...

A probability function quantiﬁes the evidence for a set of outcomes, H =
fH :H g
...
The frame of discernment is either abbreviated by FOD or represented
by capital theta,
...
A belief function can represent that the sensor
had an ambiguous reading, that it literally doesn’t know what is out there
...

The number of all possible subsets that the belief mass can be distributed to
by a belief function is 2 or 2 raised to the power of the number of elements
in the set
...
Belief that
an area is fOccupied Empty g means that it is either Occupied or Empty
...
The term dontknow will be used instead

388

11

Localization and Map Making

of

to reduce confusion when belief mass is assigned to the proposition
fOccupied Emptyg
...

2
...
” A sensor may return a totally ambiguous reading, but it did
make an observation
...

Bel( ) = 1: This speciﬁes the quantum of belief to be 1
...

3
...

To summarize, a belief function representing the belief that an area grid i] j ]
is expressed as a tuple with three members (unlike the two in probabilities),
occupied, empty, dontknow
...
One possible implementation is:
typedef struct {
double occupied;
double empty;
double dontknow;
} BEL;
BEL occupancy_grid[ROWS][COLUMNS];

389

11
...
4
...
This means the observer is completely uncertain, and the belief function is humorously called
the vacuous belief function
...
It is also used to
initialize the occupancy grid if there is no a priori belief
...
11
...
7)

;

)

;

)

m(Occupied) = R 2
Maxoccupied
m(Empty) = 0:0
m(dontknow) = 1:00 ; m(Occupied)
For Region II:

m(Occupied) = 0:0
(11
...
”
Returning to the examples in Figs
...
4 and 11
...
For Example 1:

m(Occupied) = 0:0
;
R;r
10;3:5 15;0
m(Empty) = ( R )+( )
= ( 10 )+( 15 ) = 0:83
2
2
m(dontknow) = 1:0 ; m(Empty) = 1 ; 0:83
= 0:17
Resulting in:

Bel = m(Occupied) = 0:0 m(Empty) = 0:83 m(dontknow) = 0:17

390

11

Localization and Map Making

For Example 2:

m(Occupied) =

=
m(Empty) =
m(dontknow) =
=

;
( RR r ) + ( ; )
Maxoccupied
2
10;6 ) + ( 15;5 )
( 10
15
0:98 = 0:52
2
0: 0
1:0 ; m(Occupied)
1 ; 0:52 = 0:48

Resulting in:

Bel = m(Occupied) = 0:52 m(Empty) = 0:0 m(dontknow) = 0:48
The belief function produced by the sonar model is now ready to be combined with any other observations
...
Recall that
the sonar model generated the probability of P (sjH ) and Bayes’ rule had to
be applied with an assumption of P (H ) = P (:H ) to convert it P (H js)
...
4
...
The
most popular is the original rule, called Dempster’s rule of combination or the
orthogonal sum
...
Dempster’s rule is
very similar to Bayes’ rule in that it can be applied to any two belief functions
as long as they are independent
...

Dempster’s rule of combination is notationally dense, so a graphical example will be given ﬁrst, followed by the formal mathematical expression
...
Both observations believe that grid i] j ] is in Region I
...
11
...
0 corresponding to the one quanta of belief mass
...
4 Dempster-Shafer Theory

don’t know

1
...
4

U

0
...
0

don’t know
= don’t know

Bel 1

don’t know

0
...
0
don’t know

a
...
6

0
...
0
occupied

don’t know
0
...
0

1
...

U

don’t know

1
...
6 X 0
...
36

0
...
4 = 0
...
4 X 0
...
24

0
...
0

0
...
4

don’t
know

0
...
4 = 0
...

0
...
6

0
...
0

Bel 2

c
...
6 Graphical description of Dempster’s rule of combination
...
) Two belief
functions as numberlines, b
...
) associated
belief mass, and d
...

numberline can be divided up into the mass associated with each focal element; note that since the masses all add up to 1
...

In Dempster’s rule, the two numberlines form orthogonal axes, forming
a square of unit area 1
...
The interior of the square can be divided into
subregions representing the belief in the focal element produced by the set
intersection from the two axes
...
11
...
Note that there are four sub-

392

11

Localization and Map Making

regions: three for Occupied and one for dontknow
...
The subregions of a square of unit 1
...
This means that the “area” of belief mass created
by the intersections of the two belief functions forms a third belief function!
The belief mass for Occupied is taken by summing the areas of each of the
three subregions, as seen in Fig
...
6c
...
11
...
Since these are mutually exclusive propositions,
the resulting set intersection is the empty set
...
Recall from the deﬁnition of a Shafer
belief function that no belief mass can be assigned to
...
24
associated with is simply left out, the resulting combined belief function
will not equal 1
...

Dempster’s rule solves this problem by normalizing the belief function; the
area for is distributed equally to each of the non-empty areas
...
0
...
4 Dempster-Shafer Theory

U

don’t know

1
...
6 X 0
...
36

0
...
4 = 0
...
4

occupied

U

Occupied
=

φ

U

don’t know
Occupied

Empty

= Occupied
0
...
4 = 0
...
4 X 0
...
24
0
...
0

don’t know
0
...
0

Bel 2

Figure 11
...

belief
...
0, so the
belief for a set was computed as:

P
m
m(C ) = k (1:0(Ck )

If a portion of that area belongs to
total area is 1:0 ; m( ):

(or there are

p areas for

), then the

P
P (C
m(C ) = 1 ; k (mm(k ) )
p
p

Normalizing leads to:

0
m(Occupied) = 1;:01624 = 0:21
:
0:36 = 0:47
m(Empty) = 1;0:24
0
m(dontknow) = 1;:02424 = 0:32
:
This idea of summing up the areas for a particular proposition and normalizing can be captured with a mathematical formula known as Dempster’s

394

11

Localization and Map Making

rule
...
The rule is:

P

(11
...
Dempster’s rule reduces to:

P

m(Ai )m(Bj )
B
m(Occupied) = 1Ai \Pj =Occupied (A )m(B )
; Ai \Bj = m i
j
P
m(Ai )m(Bj )
A \B
m(Empty) = 1 ;iP j =Empty m(A )m(B )
i
j
Ai \Bj =
P
m(Ai )m(Bj )
i Bj
m(don0 tknow) = 1A;\P=dontknow (A )m(B )
j
Ai \Bj = m i
11
...
4

Weight of conﬂict metric
Normalizing contradictory evidence may produce a justiﬁable measure of
evidence, but a robot needs to be aware of such discordances
...
The larger the area assigned to , the more disagreement between the two beliefs about the FOD
...
10)

1
Con(Bel1 Bel2 ) = log( 1 ; ) where =

X

Ai \Bj =

m1 (Ai )m2 (Bj )

Con takes a value between 0 and 1; as ! 0:0, Con ! 0:0, and as ! 1:0,
Con ! 1
...
5 HIMM

395

more than two belief functions can be measured
...
Con can be stored as a fourth ﬁeld in the BEL structure,
but most Dempster-Shafer implementations of occupancy grids do not
...
5

HIMM
The Histogrammic in Motion Mapping (HIMM) algorithm developed by Borenstein and Koren 23 at the University of Michigan provides a different approach to scoring whether a particular element in an occupancy grid is occupied or empty
...
In order to safely run their Cybermotion robot,
CARMEL, at its top speed of 0
...
The processor was a 20MHz 80386, fast for
1992, but still quite slow
...
HIMM was developed as a
quasi-evidential technique where it scores certainty in a highly specialized
way suitable for sonars
...
11
...
CARMEL resoundingly won ﬁrst place that year in the task of navigating between a series of
waypoints marked by landmarks on posts
...
After that, HIMM
and occupancy grids became standard on many platforms
...
5
...
11
...
The model has two
striking features in contrast to the sonar model in Fig
...
2
...
This eliminates up to 90% of
the grid elements that are updated by Bayesian and Dempster-Shafer methods, thereby signiﬁcantly reducing the order complexity
...
Second, the uncertainty
score is expressed as an integer from 0 to 15, which means each element

396

11

Localization and Map Making

Figure 11
...
(Photograph
courtesy of David Kortenkamp and AAAI
...

The sonar model also shows the updating rule
...

If an l is empty, the increment I is -1
...

This implies that HIMM believes a region is occupied more than it believes
the intervening regions are empty; this is the opposite of evidential methods
...
The

397

11
...
9 HIMM sonar model
...
11)

where 0

I=

grid i] j ] 15
I + if occupied
I ; if empty

While it should be clear that HIMM executes much more quickly than true
evidential methods, it may not be clear why it would produce more reliable
occupancy grids for navigation
...
11
...
This means a wall (shown in
gray dots) would be appear on the grid as a series of one or more small,
isolated “posts
...
If the robot moves,
the gaps get ﬁlled in with subsequent readings
...
If the velocity and grid update rates are
well matched, the gaps in the wall will be minimized as seen in Fig
...
10a
...
11
...
HIMM actually suffers in performance when the robot moves
slowly because it sees too little of the world
...
11
...
11
...
In the example, the HIMM sonar
model is shown as a rectangle with darker outlines
...
10 HIMM updating example for a wall a
...
) when update rate is slower than velocity, leading to holes
...
Also, the robot’s
motions are such that the sonar is always facing “up” and aligned with the
grid
...

11
...
2

Growth rate operator
A disadvantage of HIMM is that the scattered sampling means that a particular element grid i] j ] may get only one or two updates
...
First, it means the values in the grid tend to be low; an element that is occupied and only has two updates has a score of 6, which
is less than half of the maximum score of 15 for occupancy
...
One way to handle this is to change
the scoring increments
...

Another approach is to consider nearby grid elements
...
Note that this
is handled in evidential methods by having a sonar model with a large
...
5 HIMM

reading 1

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

reading 2

0
0
0
0
0

0
0
0
0
0

0
0
3
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
3
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
3
5
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
3
5
0
0

0
3
5
0
0

0
3
0
0
0

0
0
0
0
0

0
0
0
0
0

reading 3

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
0
0
0

0
3
5
0
0

0
0
0
0
0

MASK

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
3
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
3
5
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
3
5
0
0

0
3
5
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
3
5
0
0

0
3
5
0
0

3
2
0
0
0

0
0
0
0
0

0
0
0
0
0

reading 8

Figure 11
...

HIMM instead uses a companion mechanism called a Growth Rate Operator
or GRO to capture this heuristic
...
12 shows the GRO and how it works
...
The
GRO uses a mask, W , to deﬁne “neighbors
...
The mask is literally a small array overlaid on the oc-

400

11

1
...
Apply I=+3

Localization and Map Making

0

2

3

0

2

3

3

0

7

3

0

7

15

9

1

15

9

1

0

2

3

0

2

3

3

0

7

3

0

7

15

9

1

15

9

1

0
...
Apply the mask & compute:
(0
...
5x2)+(0
...
5x3)+(1
...
5x7)+
(0
...
5x9)+(0
...
5

0
...
5

1
...
5

0
...
5

0
...
Replace the element with the new value

2

3

9

1

15

9

1

0

2

3

0

2

3

3

15

7

3

0

7

15

9

1

15

9

1

0

2

3

0

2

3

3

0

7

3

0

7

15

9

1

15

9

1

Figure 11
...
Only the effects of incrementing an occupied
element are shown
...
5 HIMM

cupancy grid
...
In this case,
the mask is a 3x3 mask; it has 3 columns and 3 rows
...
Masks of larger
sizes can be applied using the same method; larger sizes will grow the score
of clusters of occupied regions much quicker since it considers the inﬂuence
of more neighbors
...
For example, the fact
that l has an occupied neighbor should increase the score, but if the score of
the neighbor is low, grid i] j ]’s score shouldn’t increase very much
...
In Fig
...
12, the mask
W has a center with 1
...
5
...
The second step is
to apply the weighting speciﬁed by the mask, W
...
This is telling the
program to start with the element 1 row “up” and 1 column “to the left” of
the grid element
...
11
...
11
...
The resulting occupancy grid now
shows that there is deﬁnitely an obstacle present
...

402

11

reading 1

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

reading 2

0
0
0
0
0

0
0
0
0
0

0
0
3
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0 0
12 0
5 12
0 0
0 0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0 0 0
1215 15
5 11 0
0 0 0
0 0 0

0
0
0
0
0

0
0
0
0
0

reading 3

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
12
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
6
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
6
0
0

0
0
12
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0 0
1215
5 11
0 0
0 0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0
0
0
0
0

0 0 15
1215 14
5 11 0
0 0 0
0 0 0

0
0
0
0
0

0
0
0
0
0

reading 6

reading 7

0 0
1215
5 11
0 0
0 0

0
0
3
0
0

reading 4

reading 5

0
0
6
0
0

Localization and Map Making

0 0
12 0
5 12
0 0
0 0

0
0
0
0
0

reading 8

Figure 11
...
11
...

The advantage of GRO is that it is very sensitive to potential obstacles
...
Obstacles tend to be bigger because sonar errors and
errors in localization make the readings around a small object much larger
...
As long as the robot is not
trying to navigate in cluttered, narrow areas, this is not a signiﬁcant problem
...
6 Comparison of Methods

11
...

Bayesian and Dempster-Shafer theory are formal theories, and other readings from other sensor modalities, such as range from stereo or a laser, can be
easily fused as long as there is a sensor model
...
As seen in Fig
...
14, all three
produce similar occupancy grids, with a slight advantage going to Bayesian
and Dempster-Shafer grids
...

11
...
1

Example computations
The similarities and differences between the three methods is best seen by
an example
...

Step 1: Initialize the Occupancy Grid
...
The occupancy grid shown
in Fig
...
15 covers an area of 12 units by 10 units
...
The grid starts in
an initial unsensed state
...
The
value of each ﬁeld depends on the unconditional probability that the area
represented by the grid is occupied or empty
...
This translates to P (Occupied) = P (Empty ) = 0:5
...
5, 0
...
In a Dempster-Shafer implementation, each element in the grid would be a structure Bel with three
ﬁelds: m(Occupied) m(Empty ) and m(dontknow)
...
Every element in the grid would start with (0
...
0,
1
...
Every element in a HIMM occupancy grid would be a single 8-bit integer, and would be initialized to 0
...
11
...
At time t1 , the sonar

404

11

a
...

c
...
14 Comparison of occupancy grids generated from the same set of readings in a favorable hallway by a
...
) Dempster-Shafer, and c
...
The
Bayesian and Dempster-Shafer maps show the path of the robot, while the HIMM
map does not
...

405

11
...
5
Max_occupied = 0
...
15 Example occupancy grid size 24 x 21 elements, showing the robot’s
position and range reading for three updates
...
)

reading returns 9
...
The robot is at grid 21] 10], so grid 3] 10] is 9 units away
from the robot (r = 9) with an of 0
...
For Bayesian and Dempster-Shafer uncertainty models, the grid element must satisfy two conditions:

j j j j, the grid element is within the angular ﬁeld of view of the sonar
model, and

r s + tolerance, the grid element is within the upper bound of the range
reading
...
Since HIMM
considers only grid elements on the acoustic axis, the key tests are:

= 0, the grid element is on the acoustic axis of the sonar model, and
r s + tolerance, the grid element is within the upper bound of the range
reading
...

so it is also affected by the sonar model in

406

11

Localization and Map Making

Step 2: Compute the uncertainty of the observation
...
This computation can be done in a series of sub-steps
...
Region I,
Occupied, extends for s tolerance
...
In this case, s = 9, tolerance = 0:5, and r = 9, and the
substitution results in 9 ; 0 9 9 + 0:5 being true
...

At this point, the three methods diverge in computing the “score” from
the reading at t1
...
This is done using Eqn
...
1:

P (sjOccupied) =

=
P (sjEmpty) =
=

;
( RR r ) + ( ; )
Maxoccupied
2
;
;
( 1010 9 ) + ( 1515 0 ) 0:98 = 0:54
2
1:0 ; P (sjOccupied)
1:0 ; 0:54 = 0:46

Dempster-Shafer theory uses Eqn
...
8, which produces essentially the
same score for the sensor reading as with the Bayesian method:

m(Occupied) =

=
m(Empty) =
m(dontknow) =
=

;
( RR r ) + ( ; )
Maxoccupied
2
;
;
( 1010 9 ) + ( 1515 0 ) 0:98 = 0:54
2
0: 0
1:00 ; m(Occupied)
1:0 ; 0:54 = 0:46

The HIMM score is the I term in Eqn
...
11
...

407

11
...

The third step in the updating process is to combine the score for the grid
element from the current reading with the previous score stored there
...
11
...
12)

) ( n
P (H jsn ) = P (s jH )P (HPs(sn jH+PP H jsj:;1 ))P (:H js )
j n;1 ) (sn H
n
n;1
where sn;1 terms are from the reading stored at grid 3] 10] and the sn terms
are the current reading
...
This can be written
recursively as:
(11
...
The method uses Dempster’s rule of
combination given in Eqn
...
9 and reprinted below:

P

(11
...
0

don’t know

U

U

Bel’ t

= Occupied

= don’t know

0
...
0

don’t know

Bel’ t

Localization and Map Making

Occupied
don’t know

= Occupied

= Occupied

0
...
0
occupied

occupied

don’t know
0
...
0

Bel t

don’t know
0
...
0

1
...
0

2

b
...

don’t
know

U

don’t know
Empty

U

1
...
86

U

Occupied
Empty

2

U

Occupied
don’t know

=φ

= Occupied

occupied

Bel’ t

0
...
50

0
...
0

3

c
...
16

Applying Dempster’s rule after three updates for the example
...
Although this can be done analyti-

cally, it is easier to sketch the unit square out, as shown in Fig
...
16a
...
There
is no belief mass for
...
6 Comparison of Methods

(11
...
11
...
Since grid 3] 10] is
in the HIMM occupied region, the increment term I is I + = 3
...

At t2 , the robot has translated forward
...
0
...
0 units away with an = 0
...
The application of Eqn
...
1 produces a
probability of P (sjOccupied) = 0:69 P (sjEmpty ) = 0:31, a belief function of
m(Occupied) = 0:69 m(Empty) = 0:0 m(dontknow) = 0:31, and a HIMM
increment of +3
...
The Bayesian
updating is:

(t
( j )
P (Ojst2 ) = P (s jO)PPOss2 jO)PPOsst1E )P (E js )
( j t1 ) + ( t2 j
t2
t1
:54)
= (0:69)(0(0:69)(0(0:31)(0:46)
:54) +
= 0:72
P (E jst2 ) = 1 ; P (Ojst2 ) = 0:28
The Dempster-Shafer updating generates a higher occupancy score, which
can also be seen from Fig
...
16b
...
5 units
...
7 units from the grid element with an of 5
...

The probability score for the Bayesian model is computed using Eqn
...
2
instead of Eqn
...
1:

;
( RR r ) + ( ; )
P (sjEmpty) =
2
10;6:7 ) + ( 15;5 )
= ( 10 2 15 = 0:50
P (sjOccupied) = 1:0 ; P (sjEmpty) = 1:0 ; 0:50 = 0:50
The result happens to be an almost even probability that grid 3] 10] is occupied
...
11
...
11
...
The difference between the probability and belief function is that the :Empty score
was assigned to P (sjOccupied) in the Bayesian method and to m(dontknow)
in the Dempster-Shafer
...
11
...
6 Comparison of Methods

m(dontknow) = 1:0(0:14)(0:5) :5) = 0:12
; (0:86)(0
m(Empty) = 1:0(0:14)(0:5) :5) = 0:12
; (0:86)(0

Since grid 3] 10] is not affected by the HIMM sonar model for the reading
at t3 , there is no update
...
The score for
grid 3] 10] at each observation is:
sonar
certainty:

Bayesian

Dempster-Shafer
m(dontknow)
0
...
00
0
...
69
0
...
31
0
...
50
0
...
54
0
...
50

0
...
31
0
...
The
numbers are the same, but where those numbers go is quite different
...
69
...
31, while Dempster-Shafer doesn’t commit to the area being empty; rather it can’t tell if it is empty or occupied
...

The updated value of grid 3] 10] after each observation, that is, the combination of the current score with the previous score, is:
after
update:

t1
t2
t3

Bayesian

P (Ojs) P (E js)
0
...
72
0
...
46
0
...
28

Dempster-Shafer
m(O) m(E ) m(dontknow)
0
...
00
0
...
86
0
...
14
0
...
12
0
...
In
the HIMM, the value of grid 3] 10] after t3 is 6 because nothing is done to it
after t2 ; it is neither incremented nor decremented
...
6
...
11
...
Performance scores are easy to compute
...
0 or 15 values and empty grid
elements 0
...
16)

score =

X
ij

jtruth i] j ] ; grid i] j ]j

Low scores mean there was less difference between the perfect map and the
sensed map
...
This is not surprising since they both use the same model of uncertainty for sonars
...

HIMM tends to be less accurate than Bayesian or Dempster-Shafer theory,
as would be expected from a method that only updates along the acoustic
axis
...

Two solutions are available to improve Bayesian and Dempster-Shafer performance
...
The
area of coverage can be dynamically adapted as a function of the robot speed
...
At that point, it becomes reasonable to update only along
the acoustic axis
...
In this case, the term in the sensor model changes as a function of
speed
...
103

11
...
3

Errors due to observations from stationary robot
All three methods produce incorrect results if the robot is stationary and repeatedly returns the same range reading
...

Due to updating on the acoustic axis, only a very small part of the world is
updated after each observation
...
If the wall is far enough away, the gaps between “poles” can
be quite large, causing the robot to attempt to head through them and then
have to avoid as subsequent updates prove the previous map wrong
...
6 Comparison of Methods

413

or crosstalk, obstacles might be missed or appear at the wrong distance
...

Bayesian and Dempster-Shafer theory also suffer from the same defect
...
But problems with phantom readings still cause incorrect maps
...

Once P (H jS ) or m(H ) reach 1
...
HIMM can
revise belief because it subtracts strictly based on the current reading
...

The reason Bayesian and Dempster-Shafer methods degenerate when the
robot is stationary and receives multiple, identical readings is because the
assumption that the observations are independent has been violated
...
Since the robot hasn’t moved, the observations cannot be considered to be taken from two different experiments
or by two different observers
...

11
...
4

3 WAYS TO TUNE

Tuning
Fig
...
14 shows the performance of the three updating methods for a hallway with signiﬁcant specular reﬂection
...
This would be a serious problem for navigation and obstacle avoidance
...
In many cases, a large amount of sensor noise can be
eliminated by tuning the model and updating algorithms
...
For example, in environments which
< 8 is often
provoke a high degree of specular reﬂection in sonars, a
used to reduce the registration of noise in the occupancy grid
...

Occupancy grids can be tuned for a task environment in at least three
ways
...
For example, the time of ﬂight of sound

414

11

TUNING

B AYESIAN
MODEL

TUNING

DS MODEL

TUNING

HIMM

Localization and Map Making

depends on the density of air, so if a robot is going from sea level to high
mountains, an adjustment factor can be added to the raw readings
...
Lowering the threshold makes the interpretation of the occupancy map more conservative; more occupied regions that may be phantoms are treated as if they were real
...
Increasing the threshold can make the robot less sensitive to
small occupied regions which may not get many readings
...

Other possibilities for tuning the performance include changing the sonar
model and the update rules
...
In difﬁcult environments, the range R accepted as valid is
often shortened
...
The rationale is that the likelihood that long readings are accurate is
small and the robot is more interested in obstacles nearby
...

Likewise, the for the ﬁeld of view is often adjusted
...
11
...
However, this isn’t
necessarily true
...
Consider a robot operating in a narrow hallway
...
Most of
the ﬁeld of view is likely to be occupied, which may argue for a P (H )
P (:H )
...
However, this requires
the robot or designer to gather data in advance of the robot being able to use
the data
...

Dempster-Shafer theoretic methods have less to tune
...
If there is prior information, the appropriate expectations can be placed into the grid
...
Tuning
with Dempster-Shafer consists primarily of changing the ﬁeld of view parameters, and R
...
7 Localization

415

often be a disadvantage in the ﬁeld as tweaking more than one can have
conﬂicting side effects
...

Less frequently, the size of the mask W and the individual weights Wp q are
changed
...
7

ICONIC
FEATURE - BASED
LOCALIZATION

Localization
Fig
...
1 shows a metric map built up from sensor data using shaft encoders
to localize
...

Localization can either use raw sensor data directly (iconic) or use features extracted from the sensor data (feature-based)
...
Feature-based localization
might ﬁrst extract a corner from the sonar data or occupancy grid, then on
the next data acquisition, the robot would extract the corner and compute the
true change in position
...

Current metric map-making methods rely heavily on iconic localization,
and many methods use some form of continuous localization and mapping
...
Map matching is made more complex by the uncertainties in the occupancy grid itself: what the robot thought it was seeing at
time tn;1 may have been wrong and the observations at tn are better
...

There is rising interest in feature-based methods for topological map-making because gateways are of interest for maps and can be readily perceived
...

Shaffer et al
...
127 They concluded that iconic methods were more accurate for localization than featurebased methods with fewer data points
...
However, feature-based algorithms
were often faster because there was less data to match during the localization

416

11

a
...

Figure 11
...
) a global map constructed from previous readings and
b
...
Shaded elements in a
...

process
...
Feature-based
algorithms were also better able to handle poor initial location estimates
...

An important point to remember about localization is that no technique
handles a dynamic environment
...
If the robot is localizing itself to an
a priori map, it cannot tolerate a large number of discrepancies between the
map and the current state of the real world
...

Likewise, a hallway in a hospital which is usually clear but suddenly cluttered with gurneys and equipment presents a challenge
...
7
...
Exteroceptive
methods involve the robot matching its current perception of the world with
its past observations
...
Once
the true position of the robot is known with respect to the map, the current
perception is then added to the map in a process often called registration
...
11
...
The robot has moved from a to b according to

11
...
It has created a map in the form of an occupancy grid,
shown in Fig
...
17a
...
As a
result, based on the current, or local, observation, the robot could actually be
in any one of the shaded elements
...
The shaft encoders provide a set of possible locations,
not a single location, and orientations; each possible (x y ) will be called a
pose
...
If the robot has not moved
very far, then a large portion of the grid should match what is already in
the global map
...
11
...
The
possibilities get higher if the uncertainty in the local and global grids are
considered
...

Localization methods, therefore, attempt to balance competing interests
...
The less the robot has
moved, the smaller the set of possible poses to be considered
...
But if the robot localizes itself every
time it gets a sensor reading, the more likely it is that the current observation
will have noise and that the method will produce the wrong match
...
Usually the choice of n is
done by trial and error to determine which value works well and can execute
on the particular hardware (processor, available memory)
...
10
...
11
...
In
order to ﬁlter sensor noise, the robot constructs a local occupancy grid from the
past n readings
...
Matching is done k times, one for every possible (x y )
pose generated from the shaft encoder data
...
In
theory, the possible number of poses, k , is actually inﬁnite
...
18 Flowchart showing the continuous localization and mapping process
...

POSE GENERATION
FUNCTION

and +10 , because the robot has a tendency to drift to the right and because
it initially might not be pointed straight
...
The function might only consider errors in increments
of 5
...
Of
course, the pose generation function has to discretize the errors in the x and
y degrees of freedom too
...
In order to cut down on the computations while the robot is navigating, a pose generation function might use a
lookup table which stores possible poses for a particular motion computed
in advance
...
To do this, it essentially overlays the center of the local
grid at the k pose onto the global grid
...
The algorithm scores how well the values in the
local grid match the values in the overlapping global occupancy grid
...
There are a variety of possible measures;
Eqn
...
16 can be adapted as an example of a simple one:

score =

K
X
k=i j

jglobal i] j ] ; local i] j ]j

11
...
This (x y ) is declared the correct current
position of the robot on the global grid and is used to update the robot’s
odometry
...

A good example is the continuous localization component of the ARIEL
system 125 developed under the direction of Alan Schultz at the US Naval
Research Laboratory
...

The localization process is illustrated in Fig
...
19c
...

Fig
...
19a shows an example of a short-term perception map generated by
the sonars in a large room
...

A second, global occupancy grid, the Long-Term Perception Map, represents the robot’s larger environment
...
11
...
In terms of updating, it serves as an a priori
map
...
More frequent localization reduces the amount
of motion between each attempt, which in turn reduces the odometric drift,
and fewer poses are needed to adequately cover the uncertainty in position,
which in turn decreases the computational effort
...
Every two feet (which corresponds to about 800 individual sensor readings), the matching function estimates the possible choices k, and for each
pose compares the match between the mature Short-Term Map and the LongTerm Map
...

Fig
...
20 shows the results of using the ARIEL continuous localization and
mapping process to map a previously unknown environment (a 70-foot-long
hallway)
...

But what if the environment is dynamic? After the Long-Term Map is
obtained, if a hallway is obstructed or a door is closed, the robot needs to

420

11

a
...

c
...
19 Continuous localization in ARIEL: a
...

Height of pixels indicates the certainty, b
...
) registration of short-term perception map with long-term map
...
)

11
...
To allow this,
ARIEL has a map adaptation mechanism (the dashed lines in Fig
...
19c)
...
Changes in the environment sensed by the robot and put into the Short-Term Perception Map
therefore make their way into the Long-Term Map, where they can be used
for improved localization (e
...
when a box shows up in a previously featureless hallway) and planning (e
...
planning a path to a goal via another route)
...

Although continuous localization can be computationally intensive, the
fusion of sensor data over each element in the Short-Term Map would be
done for obstacle avoidance anyway
...
The map adaptation mechanism has a small cost, as it reuses
the fused data in the Short-Term Map, rather than resensing and fusing the
individual sensor data into the Long-Term Map
...

11
...
2

Feature-based localization
Feature-based localization has two ﬂavors
...

The feature acts as a local, natural landmark
...
9, the
use of natural landmarks can be challenging for navigation in general, especially since a “good” set of features to extract may not be known in advance
...

The second ﬂavor is really an extension of topological navigation
...
It may
not know how long a hallway is, but it knows what the end of the hallway
looks like and important information (e
...
, the hallway terminates in a tjunction and has 3 doors on the left and 3 on the right)
...

An interesting question that has been considered by many researchers is
what happens when a robot is given a topological map (such as in Fig
...
21),
told it was at Position A, and was really at Position B? How would the robot ever notice it was misinformed, rather than assume its sensors were re-

422

11

Localization and Map Making

Figure 11
...
(Figure courtesy of the Navy Center for Applied Research in Artiﬁcial
Intelligence, Naval Research Laboratory
...

Sensor uncertainty plays a large role in localization, even in a topological
map
...
They wrote algorithms to detect walls, closed doors, open doors, and
hallways
...
7 Localization

3

2-3

2

A

1

3-4
4
4-5
5

5-6

6

6-7

7

8

B
5-9
9
9-10
10

N
W

E
S

Figure 11
...

a wall had a 30% chance of being seen as a closed door
...
POMDPs are beyond the scope of this book, but the
basic idea is illustrated by the quasi-evidential state-set progression methodology developed by Illah Nourbakhsh
...
11
...
The
gateways are labeled with a single number and represent a state where the
robot can localize itself relative to the map
...
)
Notice that the hallways are not gateways
...
g
...

Since hallways can be considered as the state of the robot which is between
gateways, the hallway are labeled with two numbers, gateway ; gateway ,
representing the pair of gateways the robot is between
...
So it is currently in one of
three possible states
...
For example, it the robot starts moving
east, it will encounter different gateways based on where it really is
...

The above method works very well for indoor environments with orthogonal intersections
...
This
would lead to a belief that the robot was more likely to be at {3} rather than
at {6}
...
0
...
8

Exploration
Exploration attempts to answer the question of where haven’t I been? versus
where am I? A central concern in exploration is how to cover an unknown
area efﬁciently
...
After some
(often very large) period of time, statistics indicate it should have covered
the entire area
...
Then the robot is repulsed by areas that
have been recently visited
...
This “avoid past” behavior 17 when combined with the random
potential ﬁeld drives the robot towards new areas
...
As the robot explores a new area, many cells on the grid
will be unknown, either P (Occupied) = P (Empty ) in a Bayesian system or
m(dontknow) = 1 in a Dempster-Shafer implementation
...

As it moves to the goal (in conjunction with avoid-obstacle), new sensor

11
...
Hughes and Murphy 105
showed that this move-to-unknown-area behavior was suitable for indoor
exploration and even localization
...
Suppose a robot
has encountered a hallway intersection; how does it choose which area to
explore?
Two basic styles of exploration methods have emerged which rank unexplored areas and make rational choices: frontier-based and generalized Voronoi
graph methods
...
Both use behaviors for navigation, but
are different in how they set the navigational goals
...

11
...
1

FRONTIER

Frontier-based exploration
Frontier-based exploration was pioneered by Brian Yamauchi
...
As shown in Fig
...
22, when a robot enters
a new area, there is a boundary between each area that has been sensed and
is open and the area that has not been sensed
...
) There are two such
boundaries in Fig
...
22; each of these lines form a frontier that should be
explored
...
A simple strategy is to explore the nearest frontier ﬁrst
...
Since the world is unknown, the robot has
no way of knowing if upon reaching a big frontier it will discover a wall just
a meter away
...
In practice, this doesn’t happen
that often with indoor environments
...
Every
cell in the occupancy grid that the boundary runs through is considered an
edge
...
In order to eliminate the effects of

426

11

Robot Current Position

Localization and Map Making

Open Space

Occupied Space

Unknown Space

Centroid

Frontier

Figure 11
...

(Figure courtesy of the Navy Center for Applied Research in Artiﬁcial Intelligence,
Naval Research Laboratory
...

Once the robot has chosen a frontier to investigate, it must decide precisely
where to head
...
x and y is the location of the
centroid, either in map coordinates or the i,j indices in the grid
...

The computation for the centroid in pseudocode is:
x_c = y_c = count = 0
for every cell on the frontier line with a location of (x, y)
x_c = x_c + x
y_c = y_c + y
count++

11
...
Regardless, once the robot is at the frontier, the map is updated and new frontiers
may be discovered
...

11
...
2

Generalized Voronoi graph methods
Another method of deciding how to explore a space is to have the robot build
a reduced generalized Voronoi graph (GVG) as it moves through the world
...
35;34
To review Ch
...
Essentially, the robot tries to
move ahead but stay in the “middle” or at a tangent to surrounding objects
...
Generating and following the path is
straightforward to do with a behavior
...
As shown in Fig
...
23, dead ends produce
two GVG edges
...
The robot can
then backtrack along the path it had been on, either to the start or to another
branch
...

Fig
...
23 shows how the robot would explore the an area
...
The robot would begin by sensing the walls on
each side (without any recognition that it was in a hall)
...
It would attempt to center itself between the walls while
moving perpendicular to the line of intersection
...
This creates two edges, neither of which appears to terminate at
an object
...
It does this
repeatedly, as seen in Fig
...
23b
...
The robot then backtracks using the same edge behavior (stay
in the middle)
...
11
...
The robot
continues to explore and backtrack until the entire area is covered, as shown
in Fig
...
24b
...
9

Summary
Map-making converts a local, or robot-centered, set of sensor observations
into a global map, independent of robot position
...
An occupancy grid is a two dimensional array regular grid which represents a ﬁxed
area in an absolute coordinate system
...

Greater accuracy is obtained in occupancy grids by fusing together multiple uncertain sensor readings
...
Bayesian methods use a probabilistic sensor model, representing uncertainty as probabilities
and updating with Bayes’ rule
...

HIMM uses an ad hoc sensor model and update rule
...
Because
of the improvement due to sensor fusion, occupancy grids are often used for
obstacle avoidance, serving as a virtual sensor for reactive behaviors
...
In general, the more often the robot is able to localize itself, the more
accurate the map
...
Raw sensor data, especially odometry, is imperfect and confounds the localization
and map-making process
...

The two categories of localization methods are iconic and feature-based
...
They ﬁt raw observations into the map directly
...

Feature-based methods perform less well for metric map-making, but work
satisfactorily for topological map-making
...
9 Summary

a
...

Figure 11
...
a
...
) explores the center
...
(Figures courtesy of Howie Choset
...

b
...
24 Example of a robot exploring an indoor area using a GVG method,
continued
...
) reaches a dead end and backtracks, and b
...
(Figures courtesy of Howie Choset
...
10 Exercises

431

sor readings
...
The choice
of features gives rise to two problems: the certainty with which a feature
can be detected (how likely is the robot to miss a hallway on its left) and
reasoning about indistinguishable locations (one hallway intersection looks
like another)
...

All existing map-making methods tend to break in the presence of changing
environments
...
Frontier-based methods look as
the perimeter of the of sensed region on an occupancy grid and then rank
areas to explore
...

11
...
1
An important property of an occupancy grid is that it supports sensor fusion
...

Exercise 11
...
3
What is the difference between iconic and feature-based localization?
Exercise 11
...
Discuss why or why not each of the following might make a good landmark for feature-based localization:
a
...
Intersection of hallways
c
...
5
Name three applications where metric map-making might be useful
...
6
Why is localization important in map-making?

432

11

Localization and Map Making

Exercise 11
...
Therefore
if the algorithm operated over a 10 10 grid, approximately 100 operations would be
executed for localization
...
100

100?

b
...
5000

2000?

Exercise 11
...

Exercise 11
...
The sonar model is: range is 12, it’s
= 15 , the
Maxoccupancy = 0:98, and the tolerance is pm1:0
...
0
...
5 units away from the robot with an
of 0
...
At t2, the robot has translated sideways
...
0
...
5 units away with an = 14:0
...
5 units
...
0 units from the
grid element with an of 5
...
What is the initial value of every element in the occupancy grid for
i) Bayesian
ii) Dempster-Shafer
iii) HIMM
b
...
Fill in the values for the grid element after every update:
after
update:
t1
t2
t3

Bayesian

Dempster-Shafer
m(dontknow)

P (Ojs) P (E js) m(O) m(E )

HIMM

Exercise 11
...
11
...

11
...
11
Consider how the robot moves from one frontier to another in frontier-based exploration
...
12
Compare frontier-based exploration with GVG exploration
...
13
Consider how both exploration strategies (frontier and GVG) often have to choose
between possible areas to explore and then save the choices not taken for further
exploration
...
14

[Programming]

Write a program that displays a sonar model on a 26 by 17 grid
...
Assume a sensor reading of s = 10 with a
tolerance of 2 units
...
Create a simple ASCII display
...
For elements outside of the ﬁeld of view,
the label is 4
...
Create a simple Bayesian display
...

j

c
...
Each element should be labeled with the
correct m(Occupied) value (Dempster-Shafer)
...
Save the data from parts b
...
and put in a spreadsheet or plotting program
...

Exercise 11
...
Test it
for a variety of room conﬁgurations, and describe where it stops in each case
...
16

[Advanced Reading]

Read “Dervish: An Ofﬁce-Navigating Robot,” by Illah Nourbakhsh, and “Xavier: A
Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models,” by Sven Koenig and Reid Simmons in Artiﬁcial Intelligence and Mobile
Robots: Case Studies of Succesful Robot System, ed
...
Describe how these approaches would work for the example in Fig
...
21
...

434

11

11
...

CARMEL stands for Computer Aided Robotics for Maintenance, Emergency, and
Life Support
...

Uncertainty and mapping
...
Alan
Schultz and Bill Adams also provided the description of ARIEL
...

Be familiar with trends in AI robotics, both in research and in society
...
1

Overview
At this point in time, AI robotics remains largely a research or specialized scientiﬁc endeavor
...
” This chapter attempts to identify new trends
on the horizon
...
1 offered a timeline of the historical forks in the development of robotics
...
12
...
In retrospect, the most
unfortunate fork was not the split between industrial and AI robots in the
1960’s, but rather the split between vision and robotics communities
...
4, the success of a robot is based largely on its perceptual capabilities
...

Now there are signs of the two communities reuniting
...
Early forays by the vision
community into robotics could be characterized as being “disembodied” or
separating vision from the rest of the robot
...
One of the ﬁrst
computer vision scientists to take a more principled approach to merging

436

12

On the Horizon

AI Robotics

planetary rovers

vision

telesystems

telemanipulators

Industrial
Manipulators

manufacturing

1960

Figure 12
...

vision and robotics was Avi Kak at Purdue University
...
80 In the early
1990’s, with slow hardware, the robot could go 8 to 9 meters per second
...

Another positive inﬂuence to reunite robotics and vision has been the various DARPA and ESPIRIT projects in mobility
...
The size of the projects require hundreds of researchers from
many universities to collaborate, providing the opportunity to cross-fertilize
the ﬁelds
...
The prestigious AAAI
Mobile Robot Competitions have changed the events and the rules each year
to reward researchers who use computer vision in creative ways, instead of
relying on the standard sonars
...

There is no other way to play soccer than to see the ball and the players in
real-time
...

The ground and aerial robot competitions sponsored by the Association for
Unmanned Vehicle Systems also promote the integration of computer vision
with robotics, but winning those competitions still largely depends on hardware, not software algorithms
...
1 Overview

437

The central theme of the ﬁrst part of this book has been the paradigms
of robotics
...
It is reasonable to expect that a new paradigm is on the horizon
...
Right now, robots have emergent behavior from manually
coded schemas and deliberative functions
...
The lack of a computational theory of learning and of
bridge-builders such as Arbib, Brooks, and Arkin is possibly delaying this
next step
...
Connell and Mahadevan published an inﬂuential book, Robot Learning,
in 1993, and researchers such as Patty Maes have demonstrated multi-legged
insect robots waking up and learning to walk within minutes
...
The ﬁeld of robotic learning is growing rapidly and it is too large to
cover in this book, but techniques such as neural networks and genetic algorithms are making increasing contributions and roboticists should keep an
eye on developments in those communities
...
What
trends can be gleaned from the current state of robotics? One assessment
is that mobile robots are now truly mobile; they have demonstrated that
they can reliably navigate
...

In terms of hardware, mobile robot design has been largely static for the
past four or ﬁve years, centered about three main types or “species
...
Another robot species is that of
the somewhat fragile insectoids
...
New developments have led
to polymorphic or shape-shifting robots, in addition to continued progress in reconﬁgurable platforms
...

Software advances, particularly in agency, are also likely to have a profound inﬂuence on mobile robotics
...
1, the Reactive movement contributed signiﬁcantly to the rise of software agents, especially webots and know-bots
...
Another software trend is the push for software
toolkits and better development environments for robots
...
92
For whatever reason, robots are being considered for a broader set of applications than ever before
...
In the 1970’s
and 1980’s, robots became identiﬁed with manufacturing
...

A subtle, but noticeable, change has been societal expectations of robots
...
This view changed
a bit in the research community with the appearance of robot societies
...
Or the human was present, but actively
supporting the robot
...
The end of the century has already seen robot pets and
robot tour guides
...

12
...
Snakelike robots constitute an important class of shape-shifters
...

Fig
...
2 shows a snake at CMU exploring a search and rescue site
...
Perhaps the best known is the IS Robotics’ Urban robot shown in
Fig
...
3
...
It can use the ﬂippers to help
navigate over rough terrain, raise the sensor platform for a better view, and
to right itself should it fall over
...
12
...

12
...
2 A snake robot exploring a search and rescue site
...
)

LEGGED LOCOMOTION

Shape-shifting almost certainly necessitates an ecological approach to programming
...
This type of deformation is a direct response to
the environment; it doesn’t matter what type of material is making the tunnel smaller, the robot must get smaller to squeeze in
...
For example, cockroaches and rats both exhibit a thigmotropic behavior: they prefer
to be touching something
...
In order to accomplish this, roaches and rats use feedback from
cilia and hair distributed throughout their bodies to determine proximity
...

Another major hardware advance is legged locomotion
...

440

12

On the Horizon

Figure 12
...
(Photographs courtesy of IS Robotics
...
2 Shape-Shifting and Legged Platforms

b
...

Figure 12
...

c
...
) full upright position, b
...
)

Early work in the 1980’s by Ken Waldron with the Ohio State Hexapod was
disappointing from an AI standpoint
...
In the end, the robot had to be teleoperated
...

He created a series of one-, two-, and four-legged robots for his PhD thesis, based in part on studies of gaits in animals
...
Although the
control was biologically motivated, the hopping mechanisms were built like
pogo-sticks
...

442

12

12
...
Many of the applications being pursued by AI
robotics researchers are fulﬁlling that potential
...
One area is demining, where
land mines endangering civilians are safely removed and disposed of
...
The Oklahoma City bombing in the US, and earthquakes in Japan, Mexico, Taiwan, and Turkey have
offered painful lessons
...

Robots can be used for more commonplace tasks
...
This introduces a new challenge:
robots’ close proximity with humans
...
The Sony Aibo dog (Fig
...
5) is perhaps the best known of
the robot pets to enter the commercial market
...
For example, people communicate in non-verbal
ways such as facial expressions
...
Kismet, shown in Fig
...
6 is one example
...
Data collected during her stay at the Smithsonian indicated that the
audience preferred the expressive face
...
One target audience is the
elderly
...
Much of the robotic technology developed for autonomous vehicles
can be adapted for use by the disabled
...
12
...
The wheelchair uses sonars to simplify the task of going through doorways and areas of clutter
...
Although the robotic wheelchair can go

12
...
5 The Sony Aibo robot pet
...
)

Figure 12
...
(Photograph courtesy of the MIT Artiﬁcial Intelligence Laboratory
...
7 Wheelesley, a robotic wheelchair being guided by a person using the
Boston University EagleEyes system
...
)

ADJUSTABLE
AUTONOMY

through a door or navigate through an obstacle course by itself, the human
can take control over as much of the wheelchair as desired
...
The human is allowed to dynamically adjust the level of autonomy of the robotic agent
...
1 with the autonomy
community
...
Everything the robot interacts with is designed for human ergonomics and unlikely to be reengineered
...
Due to hardware and control

12
...
Now the pendulum is swinging towards humanoid robots, fueled in part by the need for robots to act in human spaces and by advances
such as with legged locomotion
...
Human-like functionality seems to favor an implementation under the Hybrid Paradigm
...
12
...

Most observers report a feeling that there is a person inside a space suit,
just crouched a bit lower than usual
...
The motivation for the robot stemmed from the company setting a long-term research challenge
...
” The most advanced form of intelligent mobility is an autonomous mobile robot
...
Ergo,
the research goal became a humanoid robot
...
Cog is currently without legs, and much of the effort has been
directed at vision and learning
...
12
...

NASA Johnson Space Center has moved from the EVAHR concept discussed in Ch
...
12
...
Robonaut
is a human-sized robot complete with a stereo vision head, a torso, two arms
and two ﬁve-ﬁngered hands
...
4

Summary
The ﬁeld of AI robotics has created a large class of robots with basic physical
and navigational competencies
...
Whereas robots were initially developed for dirty, dull, and dangerous applications, they are now being considered for personal assistants
...

12
...
1
Deﬁne polymorphism
...

446

12

On the Horizon

b
...

c
...
8 Humanoid robots: a
...
) Robonaut, an anthropomorphic robot under development
at NASA’s Johnson Space Center (photograph courtesy of NASA), and c
...
(Photograph courtesy of the MIT Artiﬁcial Intelligence
Laboratory
...
6 End Notes

447

Exercise 12
...
What are the advantages and disadvantages of polymorphism? What are the advantages and disadvantages of a humanoid platform?
Exercise 12
...
What teleoperation scheme does it most closely resemble?
Is there a real difference between it and teleoperation?
Exercise 12
...
Specify the
Paradigm and architecture you would use
...

a
...

b
...

Exercise 12
...
How are they being used for robotics? What
makes them different from other approaches? Who are the researchers using neural
networks?
Exercise 12
...
How are they being used for robotics? What
makes them different from other approaches? Who are the researchers using genetic
algorithms?
Exercise 12
...

What is the motivation for each pet? What are the difﬁcult issues being addressed?
Exercise 12
...
Compare and contrast the motivation and objectives of each
...
6

End Notes
Robots that run
...

Marc Raibert’s thesis was published as Legged Robots that Balance in 1986
...
The video contained a blooper reel, showing memorable events such as when the one-legged hopper jumped on an obstacle
instead of over it
...

The one-legged hopper appears for a few seconds in the movie “Rising Sun
...
It
has been reported that it took several days to get the hopper hopping long enough,
coordinated with the actors, for the ﬁlm
...

Bob Savely provided the photo of Robonaut
...

Cog has a very ambitious set of goals, announced in a talk
...
Although Heinlein had not written
the stories yet, the timeline itself was published by John Campbell
...
Progress with Cog has been relatively slow, despite frequent coverage by science magazines, leading many in the community to criticize the project as premature
...
” As
with all other aspects of AI robotics, time will tell
...
, and Proctor, F
...
, “A Reference Model Architecture for Intelligent Hybrid
Control Systems,” proceedings of the International Federation of Automatic Control,
San Francisco, CA, June 30–July 5, 1996
...
A
...
, Transducers: Theory and Application, Prentice-Hall, 1984
...
O
...
D
...
S
...
1750–1753
...
L
...
, “Animal Behavior as a Paradigm for Developing
Robot Autonomy,” Robotics and Autonomous Systems, vol
...
145–186
...
, “Perceptual Structures and Distributed Motor Control,” Handbook of
Physiology—The Nervous System II, ed
...
1449–1465, 1981
...
, “Schema Theory,” The Handbook of Brain Theory and Neural Networks,
Michael A
...
, MIT Press, pp
...

[7] Arbib, M
...
-S
...

Rosenschein, ed
...
53–80
...
C
...

[9] Arkin, R
...

[10] Arkin, R
...

[11] Arkin, R
...
, "Temporal Coordination of Perceptual Algorithms
for Mobile Robot Navigation," IEEE Transactions on Robotics and Automation, vol
...
3, June, 1994, pp
...

[12] Arkin, R
...
and Murphy, R
...
, “Autonomous Navigation in a Manufacturing Environment,” IEEE Transactions on Robotics and Automation, vol
...
4, August, 1990,
pp
...

450

Bibliography

[13] Arkin, R
...
, Riseman, E
...
, and Hansen, A
...
417–413
...
Kortenkamp, P
...
Murphy, editors, MIT Press, 1998
...
, I, Robot, Bantam Books, 1994 (reprint)
...
, “Social Entropy: a New Metric for Learning Multi-Robot Teams,” proc
...

[17] Balch, T
...
and Arkin R
...
, “Avoiding the Past: A Simple but Effective Strategy
for Reactive Navigation,” IEEE International Conference on Robotics and Automation,
1993
...
, and Arkin, R
...
, “Behavior-Based Formation Control for Multirobot
Teams,” IEEE Transactions on Robotics and Automation, vol
...
6, 1999, pp
...

[19] Balch, T
...
, Collins, T
...
, MacKenzie, D
...
, “Io,
Ganymede, and Callisto: A Multiagent Robot Trash-Collecting Team,” AI Magazine, vol
...
2, 1995, pp
...

[20] Barlow, J
...
, “General Principles: The Senses Considered as Physical Instruments,”
The Senses, ed
...
B
...
B
...

1–34
...
P
...
, Gat, E
...
, Miller, D
...
, “A Proven
Three-tiered Architecture for Programming Autonomous Robots,” Journal of Experimental and Theoretical Artiﬁcial Intelligence, vol
...
2, 1997
...
, and Koren, Y
...
7, no
...
278–288
...
, and Koren, Y
...
7, no
...
535–539
...
G
...
, “The Evolution of Sensory Systems,” Perception: Essays in Honor of
James J
...
by R
...
MacLeod and H
...
Pick, Jr
...
141–153
...
, Vehicles: Experiments in Synthetic Psychology, MIT Press, 1984
...
, “Challenges for Complete Creature Architectures,” proceedings of First
Intenational Conference on Simulation of Adaptive Behavior, pp
...

[27] Brooks, R
...
, "A Robust Layered Control System for a Mobile Robot," IEEE Journal
of Robotics and Automation (now IEEE Transactions on Robotics and Automation), vol
...
1, 1986, pp
...

Bibliography

451

[28] Brooks, R
...

[29] Brooks, R
...
, Irie, R
...
, Marjanovic, M
...
,
Williamson, M
...
961–968
...
, and Flynn, A
...

[31] Cai, A
...
; Arai, F
...
3, pp
...

[32] Cattaneo, M
...
G
...
L
...
155–60
...
, “Visuomotor Coordination in Frogs and Toads,” The Handbook
of Brain Theory and Neural Networks, Michael A
...
, MIT Press, pp
...

[34] Choset, H
...
, “Sensor Based Motion Planning: Incremental Construction of the Hierarchical Generalized Voronoi Graph,” To appear in International
Journal of Robotics Research, 2000
...
and Burdick, J
...

[36] Choset, H
...
, “Sensor Based Motion Planning: The Hierarchical Generalized Voronoi Graph,” to appear in the International Journal of Robotics Research
...

Dorf, editor-in-chief, John Wiley and Sons, Inc
...

[38] Congdon, C
...
, Kortenkamp, D
...
, Myers, K
...
, and
Ruspini, E
...
Flakey: A Comparison of Two Winners,” AI Magazine, vol
...
1, Spring 1993, pp
...

[39] Connell, J
...

[40] Connolly, C
...
, and Grupen, R
...
, “The applications of harmonic functions to robotics,” Journal of Robotic Systems, vol
...
931–946
...
, Allen, J
...
, Artiﬁcial Intelligence: Theory and Practice, Benjamin/Cummings Publishing, 1995
...
, Bonasso, R
...
, “1992 AAAI Robot Exhibition and Competition,” AI Magazine, vol
...
1, Spring 1993, pp
...

[43] Dean, T
...
, Planning and Control, Morgan Kaufmann Publishers,
San Mateo, CA, 1991
...
, vanKreveld, M
...
, Schwarzkoph, O
...

[45] Dickmanns, E
...
and Graefe, V
...
J
...
1, 1988, pp
...

[46] daaper
...
, Collins, J
...
Hanson and E
...
“The Schema System,” International Journal of Computer Vision, vol
...
209–250
...
, “Mobile Robot Localization Using Sonar”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol
...
2, Mar 1987, pp
...

[48] Duchon, A
...
, and Warren, W
...
, “Robot Navigation form a Gibsonian Viewpoint,”
1994 IEEE International conference on Systems, Man, and Cybernetics, vol
...
2272–
7
...
R
...
H
...
L
...
L
...
520–526
...
P
...
V
...
of
Computer Science, March, 1992
...
, "Supervised Space Robots are Needed in Space Exploration," Conference
on Intelligent Robotics in Field, Factory, Service, and Space, NASA Conference Publication 3251, vol
...
705–716
...
R
...
K
...

[53] Firby, R
...
, Kahn, R
...
, Prokopwicz, P
...
, and Swain, M
...
, “An Architecture for
Vision and Action,” proceedings of 1995 International Joint Conference on Artiﬁcial
Intelligence, Morgan Kaufmann Pub
...
1, 1995, pp
...

[54] Firby, R
...
,Prokopwicz, P
...
, Swain, M
...
,Kahn, R
...
, and Franklin, D
...

71–81
...
, Scot, K
...
, UML Distilled, 2nd ed
...

[56] Gardner, H
...
, New York, 1987
...
, “Reliable Goal-directed Reactive Control for Real-World Autonomous Mobile Robots,” Ph
...
dissertation, Dept
...

[58] Gat, E
...
Kortenkamp, R
...
Murphy, editors, MIT Press, 1998
...
J
...

Bibliography

453

[60] Gifford, K
...
, and Morgenthaler, G
...
, “Optimal Path Coverage Strategies for Planetary Rover Vehicles,” 46th International Astronautical Federation Congress, Oslo,
Norway, October, 1995
...
and Kuhnert, K
...
, “Toward a Vision-Based Robot with a Driver’s License,” Proceedings, IEEE International Workshop on Intelligent Robots and Systems,
(IROS’88), Tokyo, 1988, pp
...

[62] Guzzoni, D
...
, Julia, L
...
, “Many Robots Make Short
Work: Report of the SRI International Mobile Robot Team,” AI Magazine, Spring,
vol
...
1, 1997, pp
...

[63] Haber, R
...
, and Haber, L
...
1383, 1990, pp
...

[64] The Handbook of Artiﬁcial Intelligence, A
...
Feigenbaum, editors, William
Kaufmann, Inc
...

[65] Henderson, T
...
, “Logical Sensor Systems,” Journal of Robotics Systems, vol
...
2, 1984, pp
...

[66] Hinkle, D
...
, “The 1995 Robot Competition and Exhibition,” AI Magazine, vol
...
1, Spring 1996, pp
...

[67] Hirose, S
...

[68] Honderd, G, Jongkind, W, Klomp, C
...
, and Paliwoda, R
...
7, no
...
165–79
...
K
...
, Robot Vision, MIT Press, 1986
...
, “Polly: A Vision-Based Artiﬁcial Agent,” proceedings AAAI-93, July
1993, pp
...

[71] Hughes, K
...
D
...

[72] Hughes, K, and Murphy, R
...

[73] Humanitarian Demining: Developmental Technologies 1998, CECOM NVESD, DoD Humanitarian Demining R&D Program, Fort Belvoir, VA 22060-5606
...
, Powell, M
...
R
...

[75] Iberall, T
...
, “Towards Perceptual Robotics,” proceedings of 1984 IEEE
International Conference on Systems, Man and Cybernetics, Halifax, NS, Canada, Oct
...
147–157
...
, Flynn, A
...
, Mobile Robots: Inspiration to Implementation, second
ed
...
K
...

[77] Konolige, K
...
, “The Saphira Architecture for Autonomous Mobile
Robots,” Artiﬁcial Intelligence and Mobile Robots, D
...
Bonasson, R
...

[78] Kortenkamp, D, Nourbakhsh, I
...
, “The 1996 AAAI Mobile Robot
Competition and Exhibition,” AI Magazine, Spring, vol
...
1, 1997, pp
...

[79] Kortenkamp, D
...
, “Topological Mapping for Mobile Robots using
a Combination of Sonar and Vision Sensing,” proceedings AAAI-94, 1994, pp
...

[80] Kosaka, A
...
C
...

271–329
...
T
...
8, 1991, pp
...

[82] Latombe, J
...
, Robot Motion Planning, Kluwer Academic, 1990
...
, “The Functions of Vision,” Modes of Perceiving and Processing Information, ed
...
L
...
and E
...
159–170
...
, and Luo, R
...
, “LOLA: Object Manipulation in an Unstructured Environment,” AI Magazine, vol
...
1, 1996, pp
...

[85] Levitt, T
...
, and Lawton, D
...
, “Qualitative Navigation for Mobile Robots,” AI Journal, vol
...
3, Aug 1990, pp
...

[86] Lewis, H
...
, Papadimitriou, C
...
, Elements of the Theory of Computation, PrenticeHall, Inc
...

[87] Lim, W
...
, “Plan-Behavior Interaction in Autonomous Navigation,”
SPIE Mobile Robots V, Boston, MA, Nov
...
464–475
...
, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, W
...
Freeman and Co
...

[89] Mataric, M
...

[90] Mataric, M
...
830–835
...
, and Elfes, A
...
1003, 1988, pp
...

Bibliography

455

[92] McKenzie, D
...
; Arkin, R
...
; Cameron, J
...
, “ Multiagent mission speciﬁcation and
execution,” Autonomous Robots, vol
...
1, 1997, p
...

[93] Meystel, A
...
2, Ed
...
Saridis, JAI Press, Greenwich, CT, 1990, pp
...

[94] Moravec, H
...

[95] Moravec, H
...
, “Sensor Fusion in Certainty Grids for Mobile Robots,” AI Magazine,
vol
...
2, Summer, 1988, pp
...

[96] Moravec, H
...

[97] Murphy, R
...
, “A Strategy for the Fine Positioning of a Mobile Robot using Texture,” proceedings of SPIE Mobile Robots IV, Philadephia, PA, Nov
...

267–279
...
R
...
1995, Vancouver, BC
...
1723–1728
...
R
...
26, No
...

42–51
...
R, “Use of Scripts for Coordinating Perception and Action,” IROS-96,
Nov
...
156–161
...
, “1997 International Ground Robotics Vehicle Competition,” Robotics
Competition Corner, Robostics and Autonomous Systems, Elsevier, vol
...

[102] Murphy, R
...
14, no
...

[103] Murphy, R
...
, Gomes, K
...
, “Ultrasonic Data Fusion as a Function of Robot Velocity,” 1996 SPIE Sensor Fusion and Distributed Robotic Agents, Nov
...
, pp
...

[104] Murphy, R
...
, Hawkins, D
...
, and Schoppers, M
...
, “Reactive Combination of
Belief Over Time Using Direct Perception,” IJCAI-97, Nagoya, Japan, Aug
...
1353–1358
...
R
...
, “Ultrasonic Robot Localization using DempsterShafer Theory,” SPIE Stochastic Methods in Signal Processing, Image Processing, and
Computer Vision, invited session on Applications for Vision and Robotics, San
Diego, CA, July 19-24, 1992
...
, Hughes, K
...
, and Marzilli, A
...
27, 1999, pp
...

456

Bibliography

[107] Murphy, R
...
, “Lessons Learned in Integrating Sensing into Autonomous Mobile Robot Architectures,” Journal of Experimental and Theoretical Artiﬁcial
Intelligence special issue on Software Architectures for Hardware Agents, vol
...
2,
1997, pp
...

[108] Murphy, R
...
, “Strategies for Searching an Area with SemiAutonomous Mobile Robots,” 2nd Conference on Robotics for Challenging Environments, June 1-6, 1996, Albuquerque, NM
...
15–21
...
, Cognition and Reality, W
...
Freeman, 1976
...
, “Direct Perception and Recognition as Distinct Perceptual Systems,”
address presented to the Cognitive Science Society, August, 1991
...
C
...
65, no
...
281–291
...
, Powers, R
...
, “DERVISH: an ofﬁce-navigating robot,”
AI Magazine, vol
...
2, pp
...

[113] Oliveira, E
...
, and Stephankova, O
...
27, 1999, pp
...

[114] Parker, L
...
, “ALLIANCE: An Architecture for Fault Tolerant Multirobot Cooperation,” IEEE Transactions on Robotics and Automation, vol
...
2, pp
...

[115] Penrose, Roger, The Emperor’s New Mind: Concerning Computers, Minds, and the Laws
of Physics, Oxford University Press, Oxford, 1989
...
, and Jochem, T
...
, 1996
...
, and Yuta, S
...

[118] Raibert, M
...

[119] Readings in Distributed Artiﬁcial Intelligence, A
...
Gasser, editors, Morgan
Kaufmann, 1988
...
, and Knight, K
...

[121] Rosenblatt, J
...
E
...
5-9, 1995, pp
...

[122] Schach, S, Classical and Object-Oriented Software Engineering, 3rd edition, Richard
D
...

[123] Schank, R
...
, and Riesbeck, C
...
, Inside Computer Understanding: Five Programs
Plus Miniatures, Lawrence Erlbaum Associates, Hillsdale, NJ, 1981
...
, Spatial Orientation: The Spatial Control of Behavior in Animals and Man,
translated by C
...

[125] Schultz, A
...
, and Yamauchi, B
...
6 no
...
293–308
...
, A Mathematical Theory of Evidence, Princeton University Press, 1976
...
, Gonzalez, J
...
, “Comparison of two range-based pose estimators for a mobile robot,” SPIE Mobile Robots VI, Boston, MA, USA; Nov
...

661–7
...
, “Space Teleoperation Through Time Delay: Review and Prognosis,”
IEEE Transactions on Robotics and Automation, vol
...
5, Oct
...
592–605
...
, AI Magazine, Summer, vol
...
2, 1995, pp
...

[130] Simmons, R
...
, Goodwin, R
...
, O’Sullivan, J
...

[131] Simmons, R
...
, Haigh, K
...
, and O’Sullivan, J
...
245–252
...
, “Navigation templates: mediating qualitative guidance and quantitative
control in mobile robots,” IEEE Transactions on Systems, Man and Cybernetics, vol
...
2, 1993, p
...

[133] Smith, R
...
, “On the Representation of and Estimation of Spatial
Uncertainty,” International Journal of Robotics Research, vol
...
56–68
...
8,
1997
...
, and Bowyer, K
...

[136] Stentz, A
...
1995,
pp
...

[137] Swain, M
...
, “Color indexing,” International Journal of Computer Vision, vol
...
11–32
...
E
...
318–321
...
, Bennewitz, M
...
, Dellaert, F
...
, Haehnel, C
...
, Roy, N
...
, and Schulz, D
...
3, pp
...

458

Bibliography

[140] Umbaugh, Scott E
...
Computer Vision and Image Processing, Englewood Cliffs,
NJ:Prentice-Hall, 1998
...
R
...
124–129
...
, “Teleoperators, Supervisory Control,” Concise International Encyclopedia of Robotics: Applications and Automation, R
...
Dorf, editor-in-chief, John Wiley
and Sons, Inc
...
997
...
H
...

Index

3T, 258, 274–277, 283, 284, 290, 291
4-connected neighbors, 358
8-connected neighbors, 358
a priori, 43
abstract behavior, 158, 184
action-oriented sensor fusion, 200
action-perception cycle, 83
active cooperation, 303
active sensing, 196
active sensor, 196
Adams, B
...
, 54, 57, 59, 65, 283
algorithm deﬁnition, 44
ALLIANCE, 295, 307
allothetic, 75
Ambler, 280, 284
Angle, C
...
, 67, 69, 91, 94, 96, 102, 437
architecture deﬁnition, 10
ARIEL, 419–421, 434

Arkin, R
...
, 312
Asimov, I
...
, 238
Balch, T
...
, 254
ballistic control, 22
Bayes rule, 380, 381, 384–387, 390, 407,
410, 421, 428
behavior, 73, 99, 108
behavior table, 169
behavior, innate, 75

460

Index

behavior, innate with memory, 76
behavior, learned, 76
behavior, sequence of innate, 75
Behavorial management, 262
binary image, 226
black factory, 25
Blitch, J
...
P
...
, 395
Bowyer, K
...
, 68, 102
Brigante, A
...
, 106, 109, 113, 115, 120, 152,
191, 200, 254, 265, 290, 294, 312,
437, 445, 448
Brown, C
...
, 318
Byun, Y
...
, 329–331, 349
Callisto, 296, 310
Calvin, S
...
, 2, 3
CARMEL, 395, 396, 434
Cartographer, 264
castes, 300
causal chain, 185
CEBOT, 303
centralized control, 301
Chaplin, C
...
, 328
CHIP, 146
Choset, H
...
, 192, 437
conscious behavior, 73
continuous replanning, 370
control trading, 34
Cooperation, 303
cooperative mobility, 303
coordinated control program, 157
correspondence, 232
cross-talk, 214
curvature-velocity method, 282
D* algorithm, 369
Dante, 280, 282, 284, 292
Darkstar, 31
DARPA, 27, 42, 283, 335, 436
data coupling, 135
Dean, T
...
, 312
depth map, 234
Devore, T
...
, 64, 283
difference, 44
difference evaluator, 44
difference table, 45

461

Index

differential GPS (DGPS), 209
digitization bias, 358
Dijkstra single source shortest path algorithm, 328, 340, 347, 349, 350,
374
direct perception, 68, 90
disparity, 233
display, 28
distinctive place, 329
Distributed Artiﬁcial Intelligence (DAI),
294
distributed control, 301
dominance, 83
Draper, B
...
, 254
EagleEyes, 444
echo, 211
ecological approach, 85
ecological robotics, 111
ego-centric, 111
Elfes, A
...
, 334
Engleberger, J
...
, 17
Erratic, 65, 278
ethology, 68
EVAHR, 274
event-driven replanning, 371
events, 276
Everett, H
...
, 258, 274
Flakey, 65, 278
focus-of-attention, 186
foreground, 227
foreshortening, 214
frame problem, 53, 69
framegrabber, 220
frontier, 425
Fukuda, T
...
, 152, 274, 288
gateways, 326
General Problem Solver (GPS), 44
Generalized Voronoi Graphs, 357
Genghis, 71, 72, 152, 153, 283
Gibson, E
...
, 103
Gibson, J
...
, 68, 83, 85, 103
Gifford, K
...
, 412
Graefe, V
...
, 192, 255, 256
Hazbot, 39
HelpMate, 3, 19, 25, 39

462

Index

Henning, F
...
, 206, 374, 412
Heterogeneity, 296
Heterogeneous teams, 296
heuristic, 31
heuristic function, 362
Hierarchical Paradigm, 6
high cohesion, 112
hill-climbing algorithm, 330, 348, 349
Hillis, D
...
, 303
holonomic, 354
homogeneous teams, 296
Honda humanoid robot, 446
horizontal decomposition, 106
horizontal FOV, 204
Horswill, I
...
, 271, 374, 425
Hybrid Deliberative/Reactive Paradigm,
9
Iberall, T
...
, 255
Kaebling, L
...
, 436
Kambhamettu, C
...
, 312
Kismet, 152, 442, 443
Kitano, H
...
, 433
Konolige, K
...
, 395
Kortenkamp, D
...
, 36
Kuipers, B
...
C
...
, 335–337
layers of competence, 114
legged locomotion, 439
Lego Mindstorms, 101, 153, 155, 190,
252
LeGrand, R
...
, 336
lidar, 239
lightweight vision, 241
linear drop off, 128
local control strategy, 330
local occupancy grid, 417
locomotion load, 205

463

Index

logical equivalence, 197
logical redundancy, 207
logical sensors, 197
logically redundant, 199
LOLA, 173, 174
Lorenz, K
...
, 91
Maes, P
...
, 437
Managerial styles, 264
Marr, D
...
, 374
mask, 399
Mataric, M
...
, 54
Micire, M
...
, 152, 274
Minerva, 258, 317, 318, 323, 349, 442
minimal spanning tree algorithm, 350
Minsky, M
...
, 256
MIROSOT, 226, 255, 294, 301, 312
Mission Planner, 54, 264
modality, 197
Model-oriented styles, 264
modularity, 11
Moravec, H
...
, 374
Morris, E
...
, 91, 200, 202, 268, 395, 412,
425, 433
Myers, K
...
, 68, 83, 90, 91, 111
Nelson, R
...
, 374
Nomad, 170, 192, 199, 204, 217, 244,
249, 256, 319, 376, 419
non-active cooperation, 303
Nourbakhsh, I
...
, 295, 307, 312
passive sensor, 196
path planning, 321, 351
path relaxation, 356
Payton, D
...
, 16
percept, 157
perception, 162
perceptual distinguishability, 333
perceptual schema, 92

464

Index

perceptual stability, 333
performance monitoring, 262
Performance Monitoring and Problem
Solving, 264
perpendicular ﬁeld, 125
physical cooperation, 303
physical redundancy, 199, 207
Pick, A
...
, 103
Pilot, 54
Pioneer, 178, 180, 278, 291
pixels, 218
polar plot, 115
Polly, 241, 242
polymorphism, 438
Pomerleau, D
...
, 256
precision agriculture, 208
preconditions, 45
Predator, 31
predicates, 47
predictive displays, 31
primitive behavior, 158
probability function, 381
procedural cohesion, 136
proprioception, 202
PRS-lite, 279
quadtrees, 359
qualitative navigation, 316
QualNav, 336
quantitative navigation, 316, 351
Quinn, R
...
, 441, 447, 448
rana computatrix, 94
range image, 234

range segmentation, 240
reactive behavior, 73
Reactive Paradigm, 8
reactive planning, 258
reactor, 260
Real-time Control System (RCS), 57
reasoning, 321
recognition, 90
reconﬁgurable robots, 303
rectiﬁed images, 233
redundant, 199
reﬂex, 126
reﬂexive behavior, 73
reﬂexive behaviors, ﬁxed-action patterns, 74
reﬂexive behaviors, reﬂexes, 74
reﬂexive behaviors, taxes, 74
region segmentation, 226
registration, 416
releaser, 77
resolution, 204
Resource manager, 263
Reynolds, C
...
, 14, 30, 38
Schultz, A
...
, 278, 280, 282, 283, 292,
433
simulator sickness, 31
situated agent, 111
skill, 261
skills, 173, 184, 275
Skinner, B
...
, 9
Slack, M
...
, 328
snake robot, 439

465
social entropy, 300
societies, 293
software agents, 36
Sojourner, 15, 30, 34, 207, 208, 239, 283
Soldo, M
...
, 192, 255, 256
SRI, 41, 63, 277, 278, 290, 304
Stanford Cart, 110, 234, 254
Stark, L
...
, 312
Stentz, A
...
, 303
strategic, 272
Strips, 44
structural models, 88
sub-script, 185
subgoal obsession, 367
supervisory control, 33
suppression, 119
Swain, M
...
, 356
thresholding, 226
Thrun, S
...
, 77
time of ﬂight, 210
time to contact, 87
Tinbergen, N
...
, 277
Uranus, 235
Urban, 440
urban canyons, 209, 376
urban search and rescue, 442
vacuous belief function, 389
value, 223
vector summation, 123
vectors, 123
Veloso, M
...
, 441

Walter, G
...
, 10
Wheelesley, 444
Whitaker, W
...
, 350
Wiener, N
...
, 425
Yanco, H
...
, 26

Title: Robotics
Description: Robotic

Buy These Notes Preview

Notesale: Turn your study into money

Already a Member? >

Search for notes by fellow students, in your own course and all over the country.

My Basket

Document Preview