Intelligent Agent Perception

::
Intelligent
Agent
Perception
::
Molly Maymar
Columbia University
July 2015

What is Intelligent Agent Perception?
Perception is the process by which intelligent agents
sense, perceive, and interpret their external worlds [8],
called domains [4]. Agents use built-in sensors in order
to obtain sensory data [2].
Intelligent Agent Perception | Molly Maymar | 7.10.2015

During the perceptual process, an intelligent agent
assigns meaning to incoming visual, auditory,
olfactory, and/or tactile sensory data [2]. The
assignment of meaning can be bottoms-up, by
which data significance is drawn directly from
sensation, or top-down, by which previously
perceived or programmed meanings are recalled
and assigned to current data [2]. Perception can be
top-down within a single cycle, in that the agent
relies on recent recollections of the same data—in
the same domain and within the same problem
space—to inform data significance in real-time [2].
I. What is Intelligent Agent Perception?

Perception entails the filtering of sensory information
[2]. Through a process called attention—by which
the agent determines how to direct its perceptual
resources [4] and select the most relevant, urgent,
or important information in its world—the agent
ignores some sensory data while processing others
[2].
The output of the perceptual process is a percept—
a categorical judgment or concept that the agent
has created as a result of the perceptual process
[11,2].

• “Perception tends to be ambiguous and noisy.” [9]
• Noisy sensors interfere with agents’ abilities to
perceive their worlds completely and accurately [4]
• Dynamic worlds [4] mean that agents cannot rely on
pre-programmed models or sensory information [5]
and must undertake more complex data filtering
processes [2]
Impediments to the Perceptual Process

The agent combines perceptual data; recognition
and categorization of familiar patterns to build a
model of its domain and/or the dynamics of that
domain, in order that the agent can best sense and
act [4,9,2]
An example is an agent using vision to detect
features of its world and then using these features to
determine its position within the world, as well as the
obstacles it will encounter [9]
Situation Assessment

Intelligent agents sense through simple, inexpensive modalities
like temperature measurement and/or complex, expensive
modalities [4] like stereo vision—seeing in 3D by using two
cameras as “eyes” [5]. The input from these modalities can be
combined and output as a single assessment [4,11].
How Do Intelligent Agents Perceive?

The most common method for acquiring sensory
data is through sensors.
Sensor data comes in two varieties [10]:
• Data that characterizes the momentary situation
EX: camera images, laser range scans
• Data related to a change in the situation
EX: controls, odometer readings
II. How Do Intelligent Agents Perceive?

Model-based robotics rely on complete and accurate,
pre-programmed models of the robot and its
environment in order for the robot to perceive and act.
This method presents challenges in dynamic or noisy
environments, which do not accurately reflect the
models [10].
Traditional Approaches
to Robotic Perception

Behavior-based robotics rely on immediate sensor
feedback for determining a robot’s action. This method
places physical limits on perceptual process and poses
boundaries on the types of task that can be handled
using this approach [10].
II. How Do Intelligent Agents Perceive? :: Traditional Approaches

Bayes’ Theorem
Bayes’ Theorem is a rule that is used to determine
how an agent filters new data, based on previous
knowledge [9,8]. Bayes’ rule is an equation for
showing the relationship between a conditional
probability (the probability of event1, given that
event2 has already occurred) and its reverse form
(the probability of event2, given event1) [9].
How is Sensor Data Interpreted?

Bayes Theorem is as follows [8]:
P(A|B) = P(B|A) * P(A)/P(B)
P(A) is the probability of A, independent of B
P(B) is the probability of B, independent of A
P(A|B) is the probability of A, given that B is true
P(B|A) is the probability of B, given that A is true
II. How Do Intelligent Agents Perceive? :: Sensor Data :: Bayes’ Rule

Molly’s Example:
An agent needs to know if an object is a cat
P(cat) is the probability of an object being a cat, whether or
not it has pointy ears
P(ears) is the probability of an object having pointy ears,
whether or not it is a cat
P(cat|ears) is the probability the object is a cat, given it has
pointy ears
P(ears|cat) is the probability an object has pointy ears, given
that it is a cat

Molly’s Example:
The agent would determine the probability of whether or not
the object is a cat using the following formula:
P(cat|ears) = P(ears|cat) * P(cat)/P(ears)

Probabilistic Algorithms
Probabilistic algorithms, also called randomized
algorithms, account for the inherent uncertainty in
intelligent agent perception [10].
Agents use sensors to input data from their
environments. As a result of sensor limitations, noise,
and environmental dynamism and unpredictability, their
input is uncertain. Probabilistic algorithms process
sensor data in a way that computes a probability
distribution over what might be the case in the agent’s
world overall, versus generating a single best guess [10].
II. How Do Intelligent Agents Perceive? :: Sensor Data

Probabilistic algorithms have proven successful with
mobile robot localization—the problem of finding out the
robot’s coordinates relative to its environment—and
mapping—the problem of generating maps from sensor
measurements [10].
II. How Do Intelligent Agents Perceive? :: Sensor Data :: Probabilistic Algorithms

Probabilistic algorithms are preferable to traditional
approaches to robotic perception, like methods
associated with model- and behavior-based
robotics, in that they are more robust in the face
of sensor limitations, noise, and environmental
dynamics. They are computationally inefficient in
that they consider entire probability densities, so
they generate approximate outputs [10].
II. How Do Intelligent Agents Perceive? :: Sensor Data :: Probabilistic Algorithms

Artificial Neural Networks
An artificial neural network (ANN) is an information
processing technique inspired by the way that the
human nervous system processes information. In
human brains, neurons receive signals—electric
impulses transmitted by chemical processes—from
other neurons, modify the signals, and forward them
to their connections. An artificial neuron is a
computational unit with many inputs and one output. It
forwards signals, or “fires”, based on the input pattern
received [5].
II. How Do Intelligent Agents Perceive? :: Sensor Data

ANNs are comprised of interconnected firing
neurons capable of being trained. ANNs consist of
three layers: an input layer, a hidden layer, and an
output layer. Each sensory input arrives in an input
node in the input layer. This input is then provided to
every node in the hidden layer, where each
connection is assigned a weight. The output value of
every hidden node is then provided to the output
node, where connections are again weighted and
then output as a percept [5,11,2].
II. How Do Intelligent Agents Perceive? :: Sensor Data :: Artificial Neural Networks

Machine learning is the science of pattern
recognition and computational learning theory in
artificial intelligence. Machine learning focuses on
the construction and study of algorithms that learn
from and make predictions on data [6].
In order to ensure more robust robot perception in
light of unpredictable, dynamic, and noisy real-
world scenarios, scientists pair machine learning
approaches with biologically-inspired sensor
systems [5].
Sensors + Machine Learning

Some Examples:
Visual Sensors + ANN
Researchers working with the iCub humanoid robot
sought to show that artificial neural networks can be
trained to allow humanoid robot spatial perception in
Cartesian (3D) coordinates. Spatial perception is the
detection and localization of objects in the robot’s
world [5].
II. How Do Intelligent Agents Perceive? :: Sensors + Machine Learning

The iCub, moving autonomously in a world, obtained visual
information from two cameras. The ANN provided the robot
with the ability to estimate the position of camera-identified
objects relative to itself in this 3D space. The ANN was
comprised of three separate neural networks—one trained
to predict object position and robot poses along the X axis,
one along the Y, and one along the Z. Visual input regarding
image coordinates and robot poses arrived in each input
node and each output layer provided an estimate of the
object and robot’s position along one Cartesian axis [5].
The iCub showed that by training ANNs, artificial agents can
learn the skill of estimating object locations based on visual
perception [5].
II. How Do Intelligent Agents Perceive? :: Sensors + Machine Learning :: Visual + ANN

Haptic Sensors + Classification Algorithms
Researchers working with the ALoF robot
(autonomous legged robot on four legs) used a
classification algorithm—an algorithm that selects
the best possible hypothesis based on observed
data [1]—to identify the geometric texture shape
and surface properties of a robot’s selected
footholds based on haptic information, or tactile
feedback [3].

Researchers bisected each of ALoF’s legs and placed
force sensor resisters at the bisection in order to
capture haptic feedback from ALoF’s knee joint
movement when it was trodding on different surfaces.
The resistors registered the normal force (the force
supporting the weight of ALoF’s “femur” on its “tibia”, or
tibia on the floor) and moments (the force necessary to
rotate the tibia on the knee/axis) in the joint. ALoF
“walked” and “scratched” on artificial terrain textured
separately in four geometrical shapes, ranging from a
Teflon-like surface to a rocky-feeling one [3].
II. How Do Intelligent Agents Perceive? :: Sensors + Machine Learning :: Haptic + Classification

When trained in real-world walking force and contact
force measurement data samples, a special classification
algorithm, called a boosting algorithm—which combines
a weighted ensemble of weak classifiers into a stronger
one, and is slightly more accurate than a probabilistic
algorithm—was able to identify surface types based on
the haptic feedback [3].

ALoF showed that robots can use haptics, or tactile
sensory information, to identify local surface properties.
This will help robots characterize their environment
while walking through it, especially when detailed
environmental models have not been supplied in
advance [3].

Pressure Sensors + ANN + Offline Planning
Through the integration of sensor-based
techniques, offline planning, and an ANN,
researchers sought to inform the development of a
new breed of safe and intelligent robots that could
share a common workspace with humans [7].

Conventional offline planning techniques—
preprogrammed automation tasks reliant on
complete knowledge of the static environment—
require complete knowledge about the static
environment. They do not account for human
motion, which is unpredictable, and unstructured or
time-varying interactive environments [7].
II. How Do Intelligent Agents Perceive? :: Sensors + Machine Learning :: Sensors + ANN + Offline

In this study, humans stood on mats consisting of
pressure-activated nodes. Data collected by the
nodes was clustered into subsets corresponding to
each foot using a self-organizing map (SOM)—a
special class of ANN that discovers salient features
in time-varying data sets without any prior training.
Using these subsets, each human’s body
orientation and location were derived and paired
with average human body dimensions in order to
obtain a model of each person [7].

Researchers will use online information sensed by
the mats and interpreted by the SOM to modify
robots’ predefined motion paths, in order that the
robots can reach their intended goals, or preform
their automated functions, while avoiding obstacles.
This will allow robots to collaborate safely and
amicably with humans [7].

References
[1] Castelli, V. (2005). Lecture 2. Retrieved from: http://www.ee.columbia.edu/~vittorio/Lecture-2.pdf
[2] Goertzel, B., & Wang, P. (2007). A foundational architecture for artificial
general intelligence. Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms, 6, 36.
[3] Höpflinger, M., Remy, C. D., Hutter, M., Spinello, L., & Siegwart, R. (2010, May). Haptic terrain classification for
legged robots. In Robotics and Automation (ICRA), 2010 IEEE International Conference on (pp. 2828-2833). IEEE.
[4] Langley, P., Laird, J. E., & Rogers, S. (2009). Cognitive architectures: Research issues and
challenges. Cognitive Systems Research, 10(2), 141-160
[5] Leitner, J., Harding, S., Frank, M., Forster, A., & Schmidhuber, J. (n.d.). Artificial neural networks for spatial
perception: Towards visual object localisation in humanoid robots. The 2013 International Joint Conference on
Neural Networks (IJCNN).
[6] Machine Learning (2015, July 3). Wikipedia. Retrieved from:
https://en.wikipedia.org/wiki/Machine_learning
[7] Najmaei, N., & Kermani, M. R. (2011). Applications of artificial intelligence in safe human–robot
interactions. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 41(2) 448-459.
[8] Nilsson, N. J. (2009). The quest for artificial intelligence. Cambridge University Press.
[9] Poole, D. L., & Mackworth, A. K. (2010). Artificial Intelligence: foundations of computational agents. Cambridge
University Press.
[10] Thrun, S. (2000). Probabilistic algorithms in robotics. Ai Magazine, 21(4), 93.
[11] Wang, P. (2014). Perception. Introduction to artificial intelligence [online lecture notes]. Retrieved from:
http://www.cis.temple.edu/~pwang/3203-AI/Lecture/IO-2.htm

::
Thank You
::
All Images obtained
from http://sid766.tumblr.com

Intelligent Agent Perception

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Intelligent Agent Perception

Ähnlich wie Intelligent Agent Perception (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Intelligent Agent Perception