Transport methods in 3DTV--A Survey

Transport Methods
in 3DTV—A Survey
Tang kai
April, 24th, 2011

1

Index
• Introduction
• 3DTV Broadcast
• 3DTV Over IP Networks
• Discussion and Conclusion

2

Introduction
• Ultimate goal
• dynamic holography
• Most systems available today
• via stereoscopy

• Actually, 3DTV systems can be designed to support
• fixed-view stereoscopy: only two views
• free-view stereoscopy: multiple views

3

Introduction
• History of 3-D movie
• 1903, first stereoscopic 3-D movie was created
• 1922, the first full length stereoscopic movie was shown
• in the 1950s, Hollywood started 3-D movie production in big
numbers

• Consensus: a lasting success
• backwards compatible
• supports different numbers of users
• with affordable 3-D display technologies
• requires low additional transport/transmission overhead
• perceived quality and viewing comfort is better 4

3DTV Broadcast
• Analog Transmission
• US
• April 29th, 1953: a trial live broadcast of the series SPACE
PATROL was run in Los Angeles
• viewers with a pair of special polarization lenses

• December 19th, 1980: The first “nonexperimental” 3DTV “Miss
Sadie Thompson,” and Three Stooges
• 3-D Video Corporation developed a system: anaglyph format

• April 10th, 1981: musical classic, “Kiss Me Kate.”
• 3-D Video Corporation: perfect in color

5

3DTV Broadcast
• Analog Transmission
• European
• 1982: Netherlands two popular-scientific 3-D series
• a simple red/green anaglyph format
• H.-J. Herbst (Hamburg, Germany) and Philips Research Lab

• More than 40 million red/green viewing spectacles were sold
• “the TV of the future” was disillusioned

• 1983 at the International Audio and Video Fair in Berlin
• based on a standard PAL channel chain in two-channel mode
• For display, two projectors with orthogonal polarization filters were
used

• so successful that were continued at IAVF in 1985 and 1987
• Unfortunately, transmission system required custom receiver.
6
LIMITED

3DTV Broadcast
• Digital Transmission
• Background: ongoing transition from analogue to digital TV
services

• MPEG developed a new compression technology as part of
MPEG-2
• The MPEG-2 multiview profile (MVP)

• Left-eye view --- MPEG-2 main profile --- backwards-
compatibility
• Right-eye enhancement layer using the scalable coding tools

• MVP, unfortunately, has not found use in commercially
services 7

3DTV Broadcast
• Some promising attempts

• 1998 Nagano Winter Games in Japan
• right-eye and left-eye HDTV images @ 45Mbps
• projected onto a large screen. Impressing and Powerful

• 2002 FIFA World Cup in Korea/Japan
• the right-eye and left-eye HDTV images were compressed in
side-by-side format using the MPEG-2 Main Profile

8

3DTV Broadcast
• Fixed-view -> flexible 3-D visual data representation
formats

• Australian DDD company : “video-plus-depth”
representation
• combination of monoscopic color video and associated per-
pixel depth maps
• encodes the depth data low bit rate format
• transmitted in the “user data” of an MPEG-2 Transport Stream
• receiver : rendered by using depth-image-based rendering
(DIBR)
9

3DTV Broadcast
• European IST project ATTEST
• “video-plus-depth” representation
• Standard MPEG technologies: H.264/AVC
• depth data: 200–300 kbps
• overhead for the 3-D visual information is only 10% CMP 2-D

10

3DTV Broadcast
• European IST project ATTEST
• First demonstration based on ATTEST
• Diagram as follows

1st demo of a 3DTV service 3-D programs,the “video-plus-depth” 3-D
TS Contained two
on each contains
basedvideo stream
• an MPEG-2 coded color
data representation formatcoded depth-image DVB-T transmission.
• an H.264/AVC using a real sequence.

DTV-Recorder-Generator PC with a PCI DVB-T card
11

real-time replay of an offline- Received MPEG-2 TS was demultiplexed in software
generated MPEG-2 TS video bit streams were decoded in real-time

3DTV Broadcast
• “video-plus-depth” representation has been standardized
within MPEG as a result of work initiated by Philips and
Fraunhofer HHI.

• The new standard has been published in two parts:
• Specification of the depth format itself is called ISO/IEC
23002-3 (MPEG-C)
• a method for transmitting “video-plus-depth” within a
conventional MPEG-2 TS has become an amendment (Amd. 2)
to ISO/IEC 13818-1 (MPEG-2 Systems).

12

3DTV Over IP Networks
• Background
• IP is proving to be flexible in accommodating
communication services
• Classical telephone -> VOIP

• Transmission of video over Internet is active in R & D
• VoD
• 2.5G and 3G offer wireless video service

• The IP itself leaves many aspects of the transmission to be
defined by other layers of the protocol stack and,
• thus, offers flexibility in designing the optimal
communications system for various 3-D data 13
representations and encoding schemes.

• General Outline
• 3DTV streaming architectures

• Server Unicasting
• Server Multicasting
• P2P Unicasting
• P2P Multicasting

• Protocol
• Current state of the art: RTP/UDP/IP 14
• Next generation: RTP/DCCP/IP

• Streaming Protocols
• Most widely used : RTP over UDP
• does not contain any congestion control mechanism
• lead to congestion collapse when large volumes of video are
delivered

• Datagram congestion control protocol (DCCP) is designed as
a replacement for UDP for media delivery

• TCP minus reliability and in-order packet delivery
• UDP plus congestion control, connection setup, and
acknowledgements 15

• DCCP is a transport protocol that implements bi-directional
unicast connections of congestion-controlled, unreliable
datagrams.

• Despite of the unreliable datagram flow
• Reliable handshakes for connection setup/teardown and
reliable negotiation of options

16

• DCCP also accommodates two congestion control
mechanisms.
• TCP-like Congestion Control
• TCP-Friendly Rate Control(TFRC)

• TCP-like Congestion Control
• identified by CongestionCCID2 in DCCP
• behaves similar to TCP’s AIMD congestion control

• halving the congestion window in response to a packet drop

• respond quickly to changes in available bandwidth 17
• must tolerate the abrupt changes in the congestion window size

• identified by CCID3

• a form of equation-based flow control that minimizes abrupt
changes in the sending rate while maintaining longer-term
fairness with TCP

• Appropriate for applications that would prefer a rather smooth
sending-rate with a small or moderate receiver buffer
• streaming media applications

18

• Operation: CCID3/TFRC calculates TFRC rate
• using the TCP throughput equation

• Request gives feedback to sender application
• Sender may use this rate information to adjust rate to get better
results

19

• (exp)RFC for TCP-Friendly Multicast Congestion Control
(TFMC)
• compute the TFRC rate in a multicast scenario
• each receiver computes own TFRC rate as a function of RTT
loss rate
• server then selects the minimum of these rates
• (limited number clients to prevent feedback explosion)

• DCCP is the same way doing this.

• Hence, future 3DTV over IP services is expected to employ
the DCCP protocol with effective video rate adaptation to 20
match the TFRC rate.

• Multiview Video Encoding and Rate
Allocation/Adaptation
• Multiview 3-D video can be represented and encoded
• Implicitly: “video-plus-depth” representation (discussed)
• Explicitly: in raw form
• a trade-off between
• random access
• ease of rate adaptation
• compression efficiency

• simulcast coding
• scalable simulcast coding
• multiview coding
• scalable multiview coding 21

• Multiview Video Encoding and Rate Allocation/Adaptation
• The rate adaptation differs, since rate allocation between views
offers new flexibilities.

• According to the suppression theory of human visual
perception
• if the right and left views are transmitted and displayed with
unequal spatial, temporal and/or quality resolutions, the overall 3-
D video quality is determined by the view with the better resolution

• Therefore, rate adaptation may be achieved by
• adaptation of the spatial, temporal and/or signal-to-noise (SNR)
resolution of one of the views
• while encoding/transmitting the other view at full rate. 22

• Several open loop and closed loop rate adaptation strategies

• closed loop strategies
• client estimates some function of the received signal and feeds
it back to the transmitter
• The transmitter determines an optimized rate

• open loop strategies
• transmitter does not use feedback from the receiver

23

• open-loop rate adaptation strategies
• First paper: content-adaptive video scaling
• Rate adaptation has been achieved by
• 1) spatial subsampling;
• 2) temporal subsampling;
• 3) scaling the quantization step-size;
• 4) content-adaptive scaling

• content-adaptive video scaling approach
• Four categories: high/low temporal spatial detail.
• Scaling their resolutions
24
• Experiments show that better compression with better quality

• open-loop rate adaptation strategies
• Second paper: adaptive selection of temporal levels and quality
layers

• video is encoded offline with a predetermined number of spatial,
temporal and SNR scalability layers.
• Content-aware bit allocation among the views is performed during
bit stream extraction by adaptive selection scalability layers

• The required bit rate reduction is only applied to one of the views.

• Experiments shows that better rate-distortion performance
compared to static cases.
25

• closed-loop rate adaptation strategies
• rate adaptation is done at the server side by feedback from the user.

• First paper:
• The user’s head position is tracked and predicted
• The system allocates more bandwidth to the selected views in order
to render the current viewing angle.
• Make use of MVC and SVC

• Second paper:
• Each view is streamed to a different IP-multicast address
• A viewer’s client joins appropriate multicast groups to only receive
the 3-D information relevant to its current viewpoint 26

• Error Correction and Concealment
• Sources: packet losses in the wired or wireless IP links
• Wired Internet: Congestion -> packet losses
• Wireless Internet: capacity limited by bandwidth of radio
spectrum
• Noise, interference and fading, error bursts(from mobility)
• Joint source and channel coding techniques

• Error concealment methods (at the decoder) to limit temporal
error propagation

27

• Common error correction approaches for reliable
transmission
• ARQ
• ACK
• Delay, not desirable
• FEC

• In broadcast and multicast services, channel coding
techniques have been widely applied

28

• First paper:
• Macroblock classification into unequally important slice groups
• Using FMO tool of H.264/AVC

• LT codes are used for error protection for low complexity and
advanced performace

29

• Second paper:
• Stereoscopic video streaming using FEC techniques

• Frames are classified according to their contribution to overall
quality
• three layers used for UEP
• I-frame
• P-frame
• Left
• Right

• To find optimum packetization and UEP strategies
• Comparative analysis and simulation of Reed–Solomon (RS) and 30
systematic Luby transform (LT) codes

• Error concealment algorithm for monoscopic not applicable
for stereoscopic.
• Based on interpolation -> is not sufficient for not depth info is
preserved.
• Human perception of errors in 3-D video is different
• A small degradation -> significant perceptual distortion

• Third paper: an error concealment algorithm
• Make full use of characteristic of stereoscopic video
• Based on the relativity of prediction mode of right frames ->
prediction mode of macroblock
• restore the lost macroblock according to the estimated motion 31
vector or disparity vector.

• capabilities of error concealment

• To increase the quality of the reconstructed block

• a stereoscopic movie: the two views are highly
correlated(why)

• information about the corresponding region is highly useful
for the reconstruction of the lost block.
• corresponding pixel pairs identified using feature matching and
principles of epipolar geometry
• robust estimation of the transformation parameters is used to 32
educe the negative effect of outliers

• 3D Video Streaming Demonstrations
• end-to-end prototype system for point-to-point streaming of
stereoscopic video over UDP
supports the
1.over a LAN autostereoscopic
Sharp 3-D laptop
with no
packet losses
supports a monocular
display to demonstrate
2.employs the backwards
protocol stack compatibility
RTP/UDP/IP
supports an in-house
polarized 3-D
projection display
system
that uses a pair
projector

33

Discussion and conclusion
• A comprehensive survey of the state-of-the art in
transport techniques
• While the transport solutions must address backwards
compatibility issues with the existing digital TV
standards and infrastructure
• 3DTV flexible

• Current and future research issues for 3-D TV
transmission
• joint transport and coding
• Why
• determination of the best rate adaptation method
• error resilient video encoding and streaming strategies 34

Transport methods in 3DTV--A Survey

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Transport methods in 3DTV--A Survey

Ähnlich wie Transport methods in 3DTV--A Survey (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Transport methods in 3DTV--A Survey