- The document describes a new technique called content-adaptive parallax barriers for automultiscopic 3D displays. - Content-adaptive barriers optimize the parallax barrier patterns for each image/video by applying non-negative matrix factorization to increase brightness and refresh rate compared to traditional time-multiplexed barriers. - An initial prototype was constructed using off-the-shelf LCD panels to demonstrate the concept. The content-adaptive barriers allowed the system to emit higher rank light fields compared to conventional barriers.

1. Content-Adaptive Parallax Barriers for Automultiscopic 3D Display Douglas Lanman Matthew Hirsch Yunhee Kim Ramesh Raskar MIT Media Lab Imaging Hardware Session Friday, 17 December | 4:15 PM – 6:00 PM | Room E1-E4

3. RealD Cinema, Dolby 3D, XpanD 3D, MasterImage 3D, etc.

4. NVIDIA 3D Vision

5. Nintendo 3DSExpansion of 3D to our living rooms, mobile phones, advertising, etc.

7. perspective and occlusion

12. Trades decreased spatial resolution for increased angular resolutionFrederic Ives. Parallax Stereogram and Process of Making the Same. 1903. Gabriel Lippmann. Épreuves Réversibles Donnant la Sensation du Relief. 1908.

14. Attenuating masks using parallax barriers [Ives, 1903]

15. Refracting lenslet arrays [Lippmann, 1908]

16. Barriers cause severe attenuation  dim displays

17. Lenslets impose fixed trade-off between spatial and angular resolution

18. Masks can be optimized to increase optical efficiency

20. Target 4D Light Field viewer moves right viewer moves up

21. Parallax Barrier Front Mask (1 of 9)

22. Parallax Barrier Rear Mask (1 of 9)

24. Content-Adaptive Parallax Barriers

25. Implementation and Results

26. Discussion and Future Work

28. Time-Multiplexing using Shifted Pinholes L[i,k] ` k g[k] i f[i] light box Ken Perlinet al. An Autosteroscopic Display. 2000. Yunhee Kim et al. Electrically Movable Pinhole Arrays. 2007.

29. Content-Adaptive Parallax Barriers L[i,k] ` k g[k] i f[i] light box

30. Content-Adaptive Parallax Barriers ` =

31. Content-Adaptive Parallax Barriers ` =

32. Content-Adaptive Parallax Barriers ` =

33. Content-Adaptive Parallax Barriers ` =

34. Target 4D Light Field viewer moves right viewer moves up

35. Optimization: Iteration 1 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

36. Optimization: Iteration 10 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

37. Optimization: Iteration 20 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

38. Optimization: Iteration 30 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

39. Optimization: Iteration 40 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

40. Optimization: Iteration 50 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

41. Optimization: Iteration 60 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

42. Optimization: Iteration 70 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

43. Optimization: Iteration 80 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

44. Optimization: Iteration 90 rear mask: f1[i,j] front mask: g1[k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

45. Content-Adaptive Front Mask (1 of 9)

46. Content-Adaptive Rear Mask (1 of 9)

47. Emitted 4D Light Field

48. Benefits of Content-Adaptation 1) Increasing brightness: 2) Increasing refresh rate:

50. Content-Adaptive Parallax Barriers

51. Implementation and Results

52. Discussion and Future Work

56. Increasing Brightness and Refresh Rate 1) Increasing brightness: 2) Increasing refresh rate:

57. Increasing Brightness and Refresh Rate 1) Increasing brightness: 2) Increasing refresh rate:

58. Increasing Brightness Time-Shifted Barriers (central view) Content-Adaptive Barriers (central view)

59. Increasing Refresh Rate Time-Shifted Barriers (central view) Content-Adaptive Barriers (central view)

61. Content-Adaptive Parallax Barriers

62. Implementation and Results

63. Discussion and Future Work

65. Related to aperture problem(i.e., motion of windowed grating)

66. NMF converges to local stationary point, not global minimum

67. Emergent structure predicted by angular gradient:David Marr and Shimon Ullman. Directional Selectivity and Its Use in Early Visual Processing. 1981.

68. Future Work: Local Parallax Barriers viewer moves right (b) viewer moves up (a) Light Field Streamlines of the Angular Gradient* Rear Mask Front Mask *Brian Cabral and Leith Casey Leedom. Imaging Vector Fields using Line Integral Convolution. 1993.

70. Content-Adaptive Parallax Barriers

71. Implementation and Results

72. Discussion and Future Work

74. With a fixed pair of masks, emitted light field is rank-1

75. Achieved higher-rank approximation using temporal multiplexing

76. With T time-multiplexed masks, emitted light field is rank-T

77. Constructed a prototype using off-the-shelf panels

78. Demonstrated light field display is a matrix approximation problem

79. Introduced content-adaptive parallax barriers

80. Applied weighted NMF to optimize weighted Euclidean distance to targetAdaptation increases brightness and refresh rate of dual-stacked LCDs

81. Questions and Answers Content-Adaptive Parallax Barriers “Can we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality? It appears that yes, we can request from Photography infinitely more than from the human hand.” Gabriel Lippmann, 1908