2. where E is the low-bounded energy stored within the sys- III. A PPLICATION OF PTDPC TO B ILATERAL
tem and Pdiss is the (non-negative) energy dissipated from T ELEOPERATION
the system by friction or other damping mechanism. This
means that all energy entering the system is either stored or Passivity schemes can be applied to any network where the
dissipated [3]. Integrating Equation (1) gives us: transmission variables can be in terms of power or energy. In
t t [5], the author lays out groundwork for applying PTDPC to
P dτ = E(t) − E(0) + Pdiss dτ ≥ −E(0), (2) a bilateral teleoperated system. The commanded master and
0 0 slave variables are simply delayed output signals from the
and if assume the system starts from rest and substitute 0 other side of the communication port. Fig.2 further describes
for E(0), using (1), Equation (2) becomes the variable transmitted through the communication port with
t different channel delays Td1 and Td2 . Hence we have
F vdτ ≥ 0. (3)
0
Equation (3) simply states that the energy entering a &
xm = vm &
xsc = vsc
Td1
passive system must be greater than the energy leaving. The
idea behind PTDPC is to monitor this power value in real
time and ensure that it never becomes negative by selectively Fmc Fs
engaging a passivity controller. The metric to determine if Td 2
a system has become active is termed “Passivity Observer”
(PO): Fig. 2. Communications Channel
dE
Pobsv = F v − , (4)
dt
dE
where Pobsv is the passivity observer. The E in the xsc (t) = xm (t − Td1 ); Fmc (t) = Fs (t − Td2 ),
˙ ˙ (6)
dt
term is the port-side stored energy, which we assume can
be modeled. As with the traditional TDPC, if Pobsv falls where xm (t) is the master side position signal and xsc (t) the
below zero we can determine that the monitored channel has signal to the slave side, and Fmc (t) is the transmitted force
become active, and the passivity controller (PC) is engaged. from the slave side (e.g. Fs (t)).
If P O ≥ 0 at any given time, the system connected From [6], a positive constant b is introduced to relate the
to the monitored port is dissipating energy and can be different units of force and velocity. Using (6), the power
considered passive. If the observer falls to a negative value, flow between systems is then determined by:
the PC introduces an adjustable element α which engages
to dissipate the excess energy observed (as in Fig.1). The P = xm (t)Fmc (t) − xsc (t)Fs (t)
˙ ˙
PO monitors exactly how much energy is generated by the 1 2 b 2 1
unstable port, and this dissipative element removes only the = Fmc (t) + xm (t) − (Fmc − bxm )2 (t)
˙ ˙
2b 2 2b
excess amount of energy and is therefore lossless. The PO 1 2 b 1
may also incorporate the environmental energy dissipation if + Fs (t) + x2 (t) − (Fs + bxsc )2 (t)
˙ ˙
2b 2 sc 2b
the model is known; more information regarding this may 1 2 1
be found in [4]. = Fmc (t) − (Fmc − bxm )2 (t) + bx2 (t)
˙ ˙ sc
b 2b
The PC is designed such that Pobsv + Pctr = 0 when 1 1 2 1 2
− (Fs + bxsc )2 (t) + Fs (t) − Fmc (t)
˙
Pobsv < 0 where Pctr is the power dissipated by the PC. 2b 2b 2b
The dissipation element α is computed as: b b
+ x2 (t) − x2 (t),
˙m ˙
1) where v1 (n) = v2 (n) is an input; 2 2 sc
2) F2 (n) = FE where FE is the environmental force; 1 2 1
= Fmc (t) − (Fmc − bxm )2 (t) + bx2 (t)
˙ ˙ sc
3) b 2b
t
1 d 1
−Pobsv (n)/v2 (n)2 if Pobsv (n) < 0 − (Fs + bxsc )2 (t) +
˙ Fs (τ )2 dτ
α(n) = ; (5) 2b dt t−Td2 2b
0 if Pobsv (n) ≥ 0
d t b 2
4) + x (τ )dτ.
˙
dt t−Td1 2 m
F1 (n) = F2 + FP C = F2 + αv2 (n) 1 2 1
= Fmc (t) − (Fmc − bxm )2 (t)
˙
= F2 (n) − Pobsv /v2 (n). b 2b
1 dE
FP C is the output of the PC. Similarly, α can be computed + bx2 (t) − (Fs + bxsc )2 (t) +
˙ sc ˙ ,
for the admittance causality case. The combination of the 2b dt
virtual environment and the passivity controller is passive where
and stable. Note that the passivity controller must be disabled
t t
when v2 = 0 to avoid a singularity, in this case F1 (n) = b 2 1 2
E= x (τ ) dτ +
˙ F (τ ) dτ, (7)
F2 (n). t−Td1 2 m t−Td2 2b s
2838
3. by subtracting Eq.(1) from Eq.(7), we see that the dissipated A. Human Operator Input Model
power in the communication channel is:
The purpose of bilateral teleoperation is to expand the
1 2 1 capabilities of a human operator. Every teleoperation scheme
Pdiss = Fmc (t) − (Fmc − bvm )2 (t) + bvsc (t)
2
must have some sort of human “driver” directing the op-
b 2b (8)
1 eration of the overall system. A human input model was
− (Fs + bvsc )2 (t).
2b introduce to produce more repeatable test conditions. Fig.4
illustrates the validity of the human input model by com-
For a passive communication system, we require Pdiss to
paring it to actual human steering input. A PID controller
be positive, and since we cannot communicate the power
levels from each side in real time, we must use two suf- Human input model performance
0.3
ficiency conditions (master and slave) that depend only on 0.2
Desired Position
Real Human Operator
Position (rads)
0.1
local information at each side: i) master side as, 0
−0.1
−0.2
m 1 2 1 0 2 4 6 8 10
Time (s)
12 14 16 18 20
Pobsv = Fmc (t) − (Fmc − bvm )2 (t) ≥ 0, (9) 0.3
b 2b 0.2
Desired Position
Human Input Model
Position (rads)
0.1
0
and ii) slave side as, −0.1
−0.2
0 2 4 6 8 10 12 14 16 18 20
Time (s)
s 2 1
Pobsv = bvsc (t) − (Fs + bvsc )2 (t) ≥ 0. (10)
2b Fig. 4. Performance of a PID Controller vs. an Actual Human Operator
Thus the right hand side of equations (9) and (10) forms
the basis of two passivity observers, one each on the master was developed and tuned such that it could respond to road
and slave sides. It is when these values exceed zero that forces, and would control the steering wheel in a method
passivity is breached and some method of removing energy similar to an actual operator. The modeled human force, Fh ,
from the communication environment is required. Note that represents an additive force input sent to the motor.
these POs do not take into account any modelled power flow t
from the other side of the communication channel; any active Fh =KP (xhd − xm ) + KI (xhd − xm )dt
behavior is regarded as undesirable and will be addressed by 0
passivity controllers, e.g. two PCs attached at each port are + KD (xhd − xm )dt,
m s
activated when Pobsv and Pobsv are less than zero. That is
m m s s m s where xhd is the desired human trajectory. For the steering
Pobsv + Pctr = 0 or Pobsv + Pctr = 0 where Pctr and Pctr
experiments xhd took the form of a sawtooth wave which
are the power dissipation of the PCs.
varied between 0.25 and -0.25 radians. The controller gains
PTDPC offers a novel solution to theinstabilities due to
were set as follows: KP = 5, KD = 0.9 and KI = 1.
delay, in that it allows direct limits to the power transmitted
to the user. The lack of integration in PTDPC is an additional
B. Master Hardware: Haptic Steering Wheel Interface
safety feature, as it is not subject to the same issues related
to data loss or reset as traditional TDPC. The apparatus used in these experiments was constructed
in-house by the lead author, as show in Fig. 5. A haptic
IV. B ILATERALLY T ELEOPERATED ROBOTIC V EHICLE steering wheel interface was constructed with two axial
P LATFORM bearings to isolate a single moment on the steering wheel.
The rotary moment of inertia was calculated to be 0.0286 kg
In an effort to fully explore the utility of PTDPC in a
m2 .
real application, a robotic vehicle was outfitted for bilateral
teleoperation. As shown in Fig.3, the five components are: 1) Quanser Q8
Terminal Board
1) the driver or the human operator; 2) the steering wheel 1 2
2) Motor Drive
3) Shaft Encoder
4) Brushless DC
interface; 3) the communication port; 4) the vehicle steering Motor
5) Steering Wheel
mechanism; 5) the road surface. To improve repeatability and
5
to prevent experimental bias, the human operator is replaced
by a human operator input model. 3
4
Steering Desired
Steering Steering Tire Forces
Wheel
Input Angle
Angle
Torque
Road Fig. 5. 1-DOF Haptic Steering Wheel Use for Master Interface
Steering Comm. Robu
Driver Surface
Wheel Port Car
Forces
Steering Steering Steering Road
Torque
Feedback
Colum
Torque
Colum
Torque
Reaction A feedback force on the steering wheel is produced
Force
through a direct coupling to a MCG IB34005 brushless DC
Fig. 3. Five Corresponding Components of a Bilaterally Controlled Robotic servo motor with a peak torque of 5.2 Nm. The motor is
Vehicle Teleoperation Scheme powered by a BMC12L servo drive, and the command signal
is provided by a Quanser Q8 Data Acquisition (DAQ) card .
2839
4. C. Slave Device: Robotic Vehicle in a SIMULINK/WINCON environment. This SIMULINK
The RobuCAR, a programmable autonomous vehicle de- environment contains a number of different control architec-
veloped by Robosoft, was used as the slave vehicle. The tures, including the human operator input model (Fig.4), the
vehicle has four independently driving wheels and indepen- master side passivity observer, as in Eq.(9), and the passivity
dent front and rear steering. For these experiments, the rear controller. The second computer serves as an intermediary
steering was disabled and the forward speed for each wheel step in the communications signals between the master and
was equal. slave hardware. The laptop is connected via serial cable to the
The vehicle measures 2.1 m in length and 1.2 m in master PC to connect to the SIMULINK environment housed
width; the total vehicle weight is approximately 500 kg. The in the master side computer, and communicates via WiFi to
maximum turning angle is 0.3 radians and the maximum for- the robotic vehicle. The third computer is a component of
ward velocity is 5 m/s (18 km/hour). An on-board computer the robotic vehicle. This computer commands the movements
provides the processing and command center for the vehicle. of the robot and processes signals from its sensors and data
Like an automobile, the RobuCAR has tie rods which acquisition card, including the analog input signal from the
transmit force to each wheel to control the steering. The wheel force sensor.
tie rod in the front left wheel was replaced with a rod that V. B ILATERAL T ELEOPERATION OF THE ROBOTIC
had similar geometry but also contained a TS-2501 load cell. V EHICLE WITH MRDS S OFTWARE S ERVICES
This load cell measures the axial forces transmitted between
the steering servo cam and the wheel. As the tie rod is a A. MRDS Software Programming Realization
rigid body with negligible inertia, the forces on the tie rod The software programs that operate the RobuCAR are
can be considered proportional to the road contact forces; collectively referred to as RobuBOX. RobuBOX is a series
which in this case is equal to the actuator force. of connecting robotic functions based on MRDS [7]. MRDS
is specialized for robotics applications in two aspects: 1)
D. Communication Channel
that it is designed to perform as a series of discrete smaller
Typically, the communication channel is the element re- programs, referred to as services; 2) that these services are
sponsible for most of the overall system delay and also for designed to all run concurrently, as opposed to sequentially.
the instability in a bilateral teleoperation scheme. In these
experiments the communication channel is actually a series B. Teleoperation Control with MRDS Software Services
of devices and systems through which the information is The RobuCAR uses pre-existing operating services (e.g.
relayed (Fig.6). Some devices, such as the Quanser DAQ Cardrive) that handle basic low level functions such as sensor
card, are very robust and fast; any delays or data loss signal processing, obstacle detection and safety control. For
can be disregarded. Others, such as Microsoft Robotics these experiments four new services were developed to en-
Developer Studio (MRDS) service to service communication able the bilateral teleoperation architecture. Using the MRDS
over the WiFi network, are subject to significant delay and communication protocols, commands from these services are
occasionally lose packets of information. transmitted to the cardrive service, which implements the
velocity and steering commands. The following is a brief
Digital
Serial
Comm.
Serial
description of the function of each service:
WINCON / Port ACM LAB
Encoder/
Q8 Data
Simulink Comm.
Service
Data
Collector
1) Serial Communication Service: The sole purpose of
Aquitition
Card Service
this service is to provide serial communication to another
Steering Steering Wheel Robu PC. This service executes the serial transmit and receive
Wheel Angle CAR
functions every 100 ms. It transmits one double2 variable
Steering Colum
and received eight separate bytes which are assembled to
Brushless
Q8 Data
Torque
Load Cell on create a returning double variable.
Motor Driver Steering
Command
Signal
Aquitition
Card /
Strain
Amplifier /
Colum 2) PVS Data Collector Service: The PVS (Position-
Analog RobuCAR Transmitter
Output WINCON /
Simulink
Transmission
Serial
Data
Acquisition
Velocity-State) Data Collector service collects data about
over WIFI
Comm.
Network
using MRDS
Service
Card
the state of the vehicle. It receives updates from the IO
card and the cardrive service with information about vehicle
Fig. 6. 15 Steps Process to Send a Position Signal from Master to Slave telemetry and the state of the IO card. It merges this data
and Return a Force Signal into one object called PVSDataState and makes this object
available to the DCD Supervisor Service (DCD is just a
The control architecture of this teleoperation scheme name we used). This service executes every 200ms, which is
takes place on three computers; one each is associated the maximum execution rate as recommended by the robot
with the master and slave hardware and one with the manufacturer.
communication channel. The master-side computer com- 3) DCD Supervisor Service: The DCD supervisor exe-
municates with a Q8 Quanser DAQ card. This computer cutes higher level control regarding robot movement. It
processes the signals coming from the master hardware
2 double precision floating point variable which occupies 8 bytes of
1 TotalComp TS Series S-TYPE Load cell with 250 lb capacity computer memory
2840
5. Td1 where v2 (n) = vm (n).
vm vsc
+ + + The slave side passivity controller outputs an adjusting
vPC +
Slave velocity value, VP C :
Human Steering Passivity Passivity Passivity Passivity Environ
Controller Observer Observer Controller
Hard
Operator Wheel ment
ware s s
−Pobsv (n)/F2 (n) if Pobsv < 0
+ FPC + Fe VP C = (12)
Fh + + + Fmc Fs - s
0 if Pobsv ≥ 0,
Td2
where F2 (n) = Fs (n).
Fig. 7. Implementation of Bilateral Teleoperation System with PTDPC
D. Experimental Gains
In the conducted experiments, a gain of 1/270 lbs of
receives and processes information from the PVS Data Col- vehicle wheel force per Newton of haptic feedback was
lector Service, implements control methods and protocols, chosen. It was found that higher values produced oscillations
and commands the vehicle’s velocity and steering angle. The in the steering position, regardless of whether the PTDPC
PTDPC algorithm for the slave device is implemented in this was applied.
service. The DCD Supervisor is associated with a graphic For this experiment, the ‘b’ value for the slave side
user interface which allows the operator to set the velocity passivity observer was set at 5 and the ‘b’ value for the
speed and enable or disable the PTDPC. This service also master side passivity observer was set at a value of 80.
records the slave side robotic telemetry, including vehicle These values were set by taking an set of experimental
velocity, steering position, commanded position, PO value, bilateral teleoperation data and mathematically calculating
wheel force, Vpc value and a time-stamp. The DCD Super- the expected effects of the passivity observer and controller
visor executes whenever data is available from the PVS Data on the system. The values for ‘b’ were chosen for their levels
Collector Service. of activity on the system.
4) DCD Dispatcher Service: The DCD Dispatcher is a To make the steering gain more in line with a commercial
communication service that passes commands from the vehicle, the ratio of steering wheel angle to vehicle steering
DCD Supervisor Service to low level steering and velocity angle was reduced to 2/3. Additionally, to achieve more
controllers (motor drivers). The DCD Dispatcher Service effect from the slave side passivity controller, the Vpc term
executes when new velocity and steering commands are was increased by a factor of two. It was found that this
made available from the DCD Supervisor Service. The DCD additional contribution from the PC improved the slave side
Dispatcher Service was constructed separately from the DCD tracking without contributing to instability.
supervisor so that different robots may be able to use
VI. E XPERIMENTAL R ESULTS
the DCD Supervisor Service simply by changing the DCD
Dispatcher Service. This experiment is designed to represent a teleoperated
robotic vehicle performing normal turns with ideal road
C. Implementation of PTDPCs to the Platform conditions. The test took place on a flat, clean floor. The
As outlined in the Section III, the master PTDPC con- RobuCAR was commanded to move forward with a velocity
troller is built in the same WINCON/SIMULINK environ- of 0.2 m/s, and a steering which swerved from left to right.
ment that interacts with the haptic steering wheel. The slave The ideal steering signal was a sawtooth wave that varied
PTDPC controller is constructed in the DCD supervisor ser- between ±.25 radians with a 10 second period. Fig.8 to Fig.9
vice environment that processes the signals from the steering show the test results with no passivity control.
wheel. Fig.7 shows how the passivity controllers augment the (a) Master Position
incoming signals on either side of the communications port. 0.2 Master
Position (rads)
Desired
0.1
The passivity observer on the master side interacts with 0
−0.1
impedance causality; it monitors the local velocity signal and −0.2
0 2 4 6 8 10 12 14 16 18 20
the force signal transmitted from the slave side. The master Time (s)
(b) Slave Position
side PC adjusts the force feedback signal to make the system 0.2 Slave Commanded
Position (rads)
Slave Actual
0.1
Desired
interaction with the master hardware passive. 0
−0.1
The slave side interacts with admittance causality and the −0.2
0 2 4 6 8 10 12 14 16 18 20
passivity controller adjusts the velocity signal that is sent Time (s)
to the vehicle steering controller. The passivity controller
Fig. 8. Turns at Speed without PC: (a) Master steering input and ideal
should correct for some of the undesirable feedback caused steering path; (b) Slave commanded position, actual position and ideal
by delay. steering path (from master side)
The master side passivity controller outputs an adjusting
force value, FP C : We can see immediately from Fig.8 that with bilateral
teleoperation the master steering input is not as smooth as
m m
−Pobsv (n)/v2 (n) if Pobsv (n) < 0 with open loop control (Fig.4). The feedback forces cause
FP C = m
(11)
0 if Pobsv (n) ≥ 0, the human operator to veer away from the ideal desired
2841
6. trajectory. We also see the effects of delay both in the out the overall velocity signal. These tests show that there is
transmission of the commanded velocity signal from master performance improvement besides ensuring stability of the
to slave, and in the actual response of the robotic vehicle. whole system by using PTDPC.
The total integrated human effort during the test period was
3.45 Nms. 60
40
(a) Master PO
0.2
(b) Slave PO
20 0.1
Power
Power
(a) Absolute Error 0 0
0.25 −20 −0.1
|X − X |
0.2 a d −40
−0.2
Error (rads)
|(Xscwithout PC) − Xd|
0.15 −60
0 5 10 15 20 0 5 10 15 20
0.1 Time (s) Time (s)
(c) Master FPc (d) Slave VPc
0.05 60
Velocity (rads/second)
0.2
40
0
0 2 4 6 8 10 12 14 16 18 20 0.1
Force (lbs)
20
Time (s)
(b) Integrated Absolute Error 0 0
2.5 −20 −0.1
|X − X |
2 a d −40
Integrated Error
|(Xscwithout PC) − Xd| −0.2
1.5 −60
0 5 10 15 20 0 5 10 15 20
1 Time (s) Time (s)
0.5
0
0 2 4 6 8 10 12 14 16 18 20
Time (s) Fig. 11. Turns at Speed with PC: Master and Slave POs and PCs
Fig. 9. Turns at Speed without PC: Absolute and integrated absolute error
(a) Absolute Error
0.25
|Xa − Xd|
Fig.9 deals with the performance metrics for the teleop- 0.2
|(X without PC) − X |
Error (rads)
sc d
0.15
|(X with PC) − X |
sc d
eration system. Fig.9(a) indicates the difference between the 0.05
0.1
slave command signal and the ideal slave position, where xsc 0
0 2 4 6 8 10 12 14 16 18 20
Time (s)
is the slave commanded signal, xd is the ideal slave position 2.5
|X − X |
(b) Integrated Absolute Error
a d
and xa is the actual slave position. This graph also displays 2
Integrated Error
|(X without PC) − X |
sc d
1.5
|(Xscwith PC) − Xd|
the difference between the actual robotic vehicle steering 1
0.5
position and the ideal trajectory. The periodic reduction of 0
0 2 4 6 8 10
Time (s)
12 14 16 18 20
error corresponds to the reversal of commanded steering
angle; at some point the ideal and actual commanded steering Fig. 12. Turns at Speed with PC: Absolute and integrated absolute error
values must be identical. The total integrated error between
the command signal and the ideal signal in this test is
0.04975 radians seconds. VII. C ONCLUSIONS
The work successfully applied the PTDPC approach to the
(a) Master Position
0.2 Master
teleoperated robotic vehicle in the authors’ host lab. During
Position (rads)
Desired
0.1
0
the left-right turn test, the vehicle steering signal tracking
−0.1
−0.2
was slightly improved by the application of PTDPC, with
0 2 4 6 8 10
Time (s)
12 14 16 18 20
the overall human effort decreased by 0.49 Nms. The setup
(b) Slave Position
0.2 Slave Commanded offers a new test platform for future research in the area
Position (rads)
Slave Actual
0.1
0
Desired
of teleoperation systems where the system dynamics are not
−0.1
−0.2
required to be known.
0 2 4 6 8 10 12 14 16 18 20
Time (s)
R EFERENCES
Fig. 10. Turns at Speed with PC: (a) Master steering input and ideal steering [1] C. Melchiorri, “Robotic telemanipulation systems: An overview on
path; (b) Slave commanded position, actual position and ideal steering path control aspects”, Proceedings of the 7th IFAC Symposium on Robot
(from master side) Control, Wroclaw, Poland, pp. 707–717, 2003.
[2] W.R. Ferrell, “Remote Manipulation with Transmission Delay”, IEEE
Transactions on Human Factors in Electronics, Vol.6, no.1, pp. 24–32,
Fig.10 through Fig.12 show this same test with passivity 1965
controllers engaged. The total integrated human effort in this [3] B. Hannaford and J.-H. Ryu, “Time Domain Passivity Control of Haptic
test is 2.96 Nms, which is a reduction from the 3.45 Nms Interfaces”, IEEE Transactions on Robotics and Automation, Vol. 18,
no.1, pp. 1–10, 2002.
value recorded during the test without passivity control. An- [4] J.H. Ryu, C. Preusche, B. Hannaford and G. Hirzinger, “Time
other difference is apparent when looking at Fig. 12, which Domain Passivity Control with Reference Energy Following”, IEEE
shows the error with and without the passivity controller. Transactions on Control Systems Technology, Vol. 13, no.5, pp. 737–
742, 2005.
The difference between the slave command signal and the [5] Y. Ye, Y.J. Pan and Y. Gupta, “Time Domain Passivity Control of
ideal steering signal is compared to the slave command signal Teleoperation Systems with Random Asymmetric Time Delays”, 48th
without passivity control; The total absolute integrated error IEEE Conference on Decision and Control, Shanghai, China, December
2009, pp.7533–7538.
for the test period is 0.0417 radian seconds with the passivity [6] G. Niemeyer and J.E. Slotine, “Stable Adaptive Teleoperation”, IEEE
controller. If we subtract the contribution of the passivity Journal of Oceanic Engineering, Vol. 16, no.1, pp. 152–162, 1991.
controller to the commanded position, the total absolute [7] K. Johns and T.Taylor “Professional Microsoft Robotics Developer
Studio”, Wrox, 1st edition, May 2008.
integrated error for the test period is 0.04655. This value is is [8] K. Walker, Y.J. Pan and J. Gu, “Bilateral teleoperations over networks
an improvement over the period recorded without passivity based on stochastic switching approach”, IEEE/ASME Transactions on
control, because the effects of the master side PC also smooth Mechatronics, Vol. 14, No.5, pp. 539-554, 2009.
2842