The increasing popularity of head-mounted devices and 360° video cameras allows content providers to offer virtual reality video streaming over the Internet, using a relevant representation of the immersive content combined with traditional streaming techniques. While this approach allows the user to freely move her head, her location is fixed by the camera’s position within the scene. Recently, an increased interest has been shown for free movement within immersive scenes, referred to as six degrees of freedom. One way to realize this is by capturing objects through a number of cameras positioned in different angles, and creating a point cloud which consists of the location and RGB color of a significant number of points in the three-dimensional space. Although the concept of point clouds has been around for over two decades, it recently received increased attention by ISO/IEC MPEG, issuing a call for proposals for point cloud compression. As a result, dynamic point cloud objects can now be compressed to bit rates in the order of 3 to 55 Mb/s, allowing feasible delivery over today’s mobile networks. In this paper, we propose PCC-DASH, a standards-compliant means for HTTP adaptive streaming of scenes comprising multiple, dynamic point cloud objects. We present a number of rate adaptation heuristics which use information on the user’s position and focus, the available bandwidth, and the client’s buffer status to decide upon the most appropriate quality representation of each object. Through an extensive evaluation, we discuss the advantages and drawbacks of each solution. We argue that the optimal solution depends on the considered scene and camera path, which opens interesting possibilities for future work.

1. P5C-01: Towards 6DoF HTTP Adaptive Streaming
Through Point Cloud Compression
Jeroen van der Hooft, Tim Wauters, Filip De Turck (Ghent University – imec)
Christian Timmerer, Hermann Hellwagner (AAU Klagenfurt)

2. 2
Star Wars, anyone?

3. Holographic-type communication with six degrees of freedom
3

4. x
y
z
What are point clouds?
4
P(1, 2, 1)P(1, 2, 1, 52, 129, 67)

5. Multiple cameras can be used to capture an object
5
8i Labs, 2017

6. Static is boring, so let’s focus on dynamic!
6
4.1 Gb/s 3.8 Gb/s 5.7 Gb/s 5.6 Gb/s

7. Streaming this scene would require 19.2 Gb/s!
7

8. We can solve this by using MPEG’s reference encoder
8
4.5 Mb/s 40.4 Mb/s 5.7 Gb/s

9. HTTP Adaptive Streaming is the de-facto standard
9
2
3
Time
Bit rate
3
2
1
3
2
1
3
2
1
3
2
1
3
2
111
Time
Bit rate
1
Bandwidth

10. A new spatial dimension can be introduced
10
3
2
3
2
1
3
2
1
3
2
1
3
2
1
2
11112
33
2
1
3
2
1
3
2
1
3
2
1
3
2
11112
33
2
1
3
2
1
3
2
1
3
2
1
3
2
1111 Time
Bit rate
2
33
2
1
3
2
1
3
2
1
3
2
1
3
2
1111Object Time
Bit rate
Bandwidth
1
1
1
1
1
1
2
2
3
3
3
2
2
1
2 2
2
1
2
2

11. 1. How should we prioritize the point cloud objects?
11

12. Distance to the user?
12
6 m 5 m
8 m
3 m
1 243

13. Size of the object?
13
1 243

14. Visual area within the viewport?
14
1 342

15. Once prioritized, the available bandwidth can be allocated
15
1 423

16. 16
1 423
2. What is the impact of the buffer size on the visual quality?

17. We generated different scenes of two minutes each
17

18. We implemented an experimental emulation setup
18
Java Jetty
Server
Headless
Python Client
HTTP/1.1

19. We launched 6750 experiments on the Virtual Wall
19

20. The importance of objects is scene-dependent
20
Visual area
Object size
Distance
Scene 1

21. The importance of objects is scene-dependent
21
Object size
Distance
Scene 3
Visual area

22. A smaller buffer results in higher accuracy and quality…
22
Scene 1
4 seconds
2 seconds
1 second

23. … but is more susceptible to bandwidth variations
23
Scene 1

24. Ideally, of course, we have perfect prediction
24
Scene 1
Most recent
Perfect prediction

25. Lessons learned
There is no one-size-fits-all rate adaptation heuristic
A trade-off between interactivity and buffer resilience exists
Accurate prediction of the user’s movement is required
Future work on subjective experimentation
25

26. Visit https://acmmmsys.org today!
8 – 11 June 2020