A Multiplexing Scheme for Multimodal Teleoperation

Burak Çizmeci
Ph.D. Dissertation Defense
Technische Universität München
Fakultät für Elektrotechnik und Informationstechnik
Lehrstuhl für Medientechnik
München, 14. September 2017
A Multiplexing Scheme for Multimodal Teleoperation

2A Multiplexing Scheme for Multimodal Teleoperation
Overview
 Introduction
 Teleoperation system architecture
 Problem definition
Contribution I
 Rate control for low-delay video communication
Contribution II
 Multiplexing algorithm
 Channel adaptive streaming for teleoperation
 Experimental validation
 Conclusion

Teleoperation - Telemanipulation
Operator (OP) Teleoperator (TOP)
Network
position/velocity position/velocity
audio-video-forceaudio-video-force
[1] Ferrel and Sheridan, “Supervisory Control of Remote Manipulation„ IEEE Spectrum, 1967
Network delay
 Queueing delay
 Channel delay
 Propagation delay
delay due to transmission rate
Haptic data rate: 1000 packets/sec in both directions

Perceptual haptic data reduction
[2] Hinterseer et al. “Perception-based data reduction and transmission of haptic data in TPTA systems„ IEEE Trans. on Signal Proc., 2008
Force samples captured at TOP side Force samples displayed at OP side
IkΔI Sample based data reduction using the Weber's law:
ΔI
I

Control architecture
[3] Ryu et al., “A passive bilateral control scheme for a teleoperator with time-varying communication delay”, Mechatronics, 2010
[4] Xu and Cizmeci et al., “Haptic data reduction for time-delayed teleoperation using the time domain passivity approach”, In IEEE WHC,
2015.

LMT Teleoperation testbed

Problem definition
 First come first serve (FCFS) scheduling discipline
CBR: constant bitrate

Problem definition

Preemptive and resume scheduling
 A little delay on video is tolerable
 Challenging to apply on a real communication medium

Related work
References Transmission
rate estimation
Video
rate control
Multiplexing
Transport
Layer
Ping et al.  x x
Cen et al.  x x
ApplicationLayer
Cha et al. x x 
Kaede et al. x x x
Eid et al. x x 
Yamamoto et al.  x x
Cizmeci et al.   

Contribution I
Rate control for low-delay video communication

Macroblock (MB) level ρ-domain rate control algorithm
)1()(  R
[5] Gao and Cizmeci et al., “MB level rate control for low-delay H.264/AVC based video communication”, Picture Coding Symposium 2015.
Accelerated with a
prediction model
Improved with fair
rate assignement

Rate control and delay performance
 Video resolution:1280x720 @ 25 fps
 Link transmission rate: 1.2 Mbps, video bitrate: 1 Mbps

Contribution II
Multiplexing algorithm &
Channel adaptive streaming for teleoperation

Proposed multiplexing scheme
[6] Cizmeci et al., “A multiplexing scheme for multimodal teleoperation”, ACM Trans. on Multimedia Comp., Comm. & Apps. (TOMM) , 2017.

Modelling transmission rate inspired by ATM cells
A: audio data
V: video data
F: force data
ATM: asynchronous transfer mode

Multiplexing algorithm
A: audio data
V: video data
F: force data

Transmission rate estimation
[7] Capone et al., “Bandwidth estimation schemes for TCP over wireless networks”, IEEE Trans. on Mobile Computing, 2014.
(i)
(i)
(i)
SampTime
PacLen
Tr
avg
avg
avg

1)(iT
T
(i)T
T
1(i) TreTreTr avg
(i)
avg
(i)
est const
est
const
est
















 


Video bitrate adaptation
  framerateConstrVideoDelayS(i)TrβVideoBr est 
[8] Demircin et al., “Delay-constrained and R-D optimized transrating for HD video streaming over WLANs”, IEEE Trans. on Multimedia, 2008.
CBR: constant bitrate

Multiplexing buffer adaptation





 

c)Tr(bits/se
8(bits)MTUSize
roundBuferSize
StepSize
StepSize
BuferSize
ceilferSizeAppliedBuf 






5StepSize
MTU size: Maximum transmission unit size
bytes1500MTUSize

Congestion detection and control

Experimental validation

TOP over CBR links
TR
(kbps)
TRest performance Packet rate
μ
(kbps)
σ
(kbps)
RMSE
(kbps)
μ
(pac/s)
σ
(pac/s)
1000 989.2 2.8 11.1 139.6 6.7
2000 1964.5 4.1 35.6 192.2 4.1
3000 2938.2 9.1 62.4 251.1 7.8

TOP over CBR links
TR
(kbps)
Force delay (ms) Video delay (ms) Audio delay (ms)
μ σ min max μ σ min max μ σ min max
1000 10.1 5.1 1 15 31.3 1.6 23.3 52.7 12.2 5.6 2.0 29.0
2000 6.4 2.8 1 10 33.0 1.1 31.4 50.5 9.8 3.5 1.6 20.2
3000 3.8 1.0 1 5 33.8 1.5 31.4 46.6 8.1 2.1 1.4 13.3

A Multiplexing Scheme for Multimodal Teleoperation

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Empfohlen

Empfohlen (20)

A Multiplexing Scheme for Multimodal Teleoperation