2. Slow Glass: Time Shifted Display
Light of Other Days by Bob Shaw
http://www.fantasticfiction.co.uk/s/bob-shaw/other-days-other-eyes.htm http://baens-universe.com/articles/otherdays
3. Shift Glass
Space Shifting
Angle Shifting
Time Shifting
Illumination Shifting
4D 4D t 4D 4D
4. Motivation: Glasses Free 3D Displays
Eliminating Eyewear Expanding FoV and DoF for Glasses-free Displays
Eliminating Moving Parts Thin Holographic Displays
(Preserving Depth Cues) (Developing or Avoiding High-Res SLM)
Favarola et al Michael Bove et al
7. Opportunities: New Hardware + New Math
Emerging Displays
Multilayer High Frame Rate Directional Backlighting
Compression and Embedded Processing
Non-negative Matrix Factorization (NMF) Non-negative Tensor Factorization (NTF)
8. Camera Culture
Creating new ways to capture and share visual information
MIT Media Lab
Ramesh Raskar
http://cameraculture.info
Facebook.com/cameraculture
Computational Photography Femtosecond Imaging 3D Displays
1. Looking around corners 1. Tensor Display
1.Light-Field Camera A family of compressive light field
A new camera design exploiting the Using short laser pulses and fast detector, we aim to build
a device that can look around corners with no imaging displays comprising all architectures
fundamental dictionary of light-fields
device in the line of sight using time resolved transient employing a stack of time-multiplexed,
for a single-capture capture of light-
imaging. light-attenuating layers illuminated by
fields with full-resolution refocusing
uniform or directional backlighting
effects.
2. Layered 3D
2. Color Primaries Tomographic techniques for image
A new camera design with
synthesis on displays composed of
switchable color filter arrays for compact volumes of light-attenuating
optimal color fidelity and picture material. Such volumetric attenuators
quality on scene geometry, color recreate a 4D light field or high-contrast
and illumination. 2D image when illuminated by a uniform
2. Reflectance Recovery backlight.
We demonstrate a new technique that allows a camera to
3. Flutter-Shutter rapidly acquire reflectance properties of objects 'in the wild' 3. Glasses-free 3D HDTV
A camera that codes the exposure from a single viewpoint, over relatively long distances and Light field displays with increased
time with a binary pseudo-sequence without encircling equipment. brightness and refresh rate by stacking a
to de-convolve and remove motion
pair of modified LCD panels, exploiting
blur in textured backgrounds and rank and constraint of 3D displays
partial occluders. 3. Trillion Frames per Second Imaging
A camera fast enough to capture light pulses moving
through objects. We can use such a camera to understand 4. BIDI Screen
reflectance, absorption and scattering properties of A thin, depth-sensing LCD for 3D
4. Compressive Capture materials. interaction using light fields which
We analyze the gamut of visual
supports both 2D multi-touch and
signals from low-dimensional
unencumbered 3D gestures.
images to light-fields and propose
non-adaptive projections for
efficient sparsity exploiting 5. Living Windows 6D Display
reconstruction. A completely passive display that
responds to changes in viewpoint and
changes in incident light conditions.
May 2012
9. Health & Wellness Human Computer Interaction Visual Social Computing
1. Retinal Imaging 1. Bokode 1. Photocloud
With simplified optics and cleaver Low-cost, passive optical design so A near real-time system for
illumination we visualize images of the that bar codes can be shrunk to fewer interactively exploring a collectively
retina in a standalone device easily than 3mm and read by ordinary captured moment without explicit 3D
operated by the end user. cameras several meters away. reconstruction.
2. NETRA/CATRA
Low-cost cell-phone attachments that 2. Specklesense 2. Vision Blocks
measures eye-glass prescription and Set of motion-sensing configurations On-demand, in-browser,
cataract information from the eye. based on laser speckle sensing . The customizable, computer-vision
3. Cellphone Microscopy underlying principles allow application-building platform for the
A platform for computational microscopy interactions to be fast, precise, masses. Without any prior
and remote healthcare extremely compact, and low cost. programming experience, users can
create and share computer vision
4. High-speed Tomography 3. Sound Around applications.
A compact, fast CAT scan machine using
Soundaround is a multi-viewer
no mechanical moving parts or
interactive audio system, designed to
synchronization.
be integrated into multi-view displays
3. Lenschat
presenting localized audio/video LensChat allows users to share
5. Shield Fields channels with no need for glasses or mutual photos with friends or borrow
3D reconstruction of objects from a single headphones. the perspective and abilities of many
shot photo using spatial heterodyning. cameras.
6. Second Skin
Using 3D motion tracking with real-time
Visit us online at
vibrotactile feedback aids the correct of
movement and position errors to improve
motor learning. Cameraculture.info
fb.com/cameraculture
Light Propagation Theory and Fourier Optics
1. Augmented Light Fields 2. Hologram v Parallax Barrier
Expands light field representations to Defines connections between parallax 3. Ray–Based Diffraction Model
describe phase and diffraction effects by barrier displays and holographic displays by
Simplified capture of diffraction model for
using the Wigner Distribution Function analyzing their operations and limitations in
computer graphics applications.
phase space
Post-Doctorial Researchers: Doug Lanman, Gordon Wetzstein, Alex Olwal, Christopher Barsi
Research Assistants: Matthew Hirsch, Otkrist Gupta, Nikhil Naik, Jason Boggess, Everett Lawson, Aydın Arpa, Kshitij Marwah
Visiting Researchers & Students: Di Wu, Daryl Lim
17. MIT media lab camera culture EyeNetra.com
NETRA: Refractive Error on Mobile Phone
Siggraph 2010
Vitor Pamplona Ankit Mohan Manuel Oliveira Ramesh Raskar
18
19. Camera Culture: Compressive Displays Team
Gordon Wetzstein Matthew Hirsch Douglas Lanman
Postdoctoral Associate Graduate Student Postdoctoral Associate
Wolfgang Heidrich, Professor, University of British Columbia
Yunhee Kim, Postdoctoral Fellow, MIT Media Lab
20. 3D Display: Light and Rank Deficient
Parallax
barrier
Front
Back
LCD display
31. Is a hologram just another ray-based light field?
Can a hologram create any intensity distribution in 3D?
Why does a hologram create a “wavefront”, but parallax barrier does not?
Why does a hologram create accommodation cues?
What are the effective resolution and depth of field for holograms vs. barriers?
33. Augmented Light Field
wave optics based
rigorous but cumbersome
Wigner
WDF
Distribution
Function Augmented LF
Traditional Traditional
Light Field Light Field
ray optics based
Interference & Diffraction
simple and powerful Interaction w/ optical elements
Oh, Raskar, Barbastathis 2009: Augmented Light Field
34
37. Generalizing Parallax Barriers: Rank 1
mask K
…
mask 3
mask 2 mask 2 mask 2
mask 1 mask 1 mask 1
light box light box light box
Conventional Parallax Barrier High-Rank 3D (HR3D) Layered 3D and Polarization Fields
Parallax barriers use heuristic design: front mask with slits/pinholes, rear mask with interlaced views
High-Rank 3D (HR3D) considers dual-layer design with arbitrary opacity and temporal multiplexing
Layered 3D and Polarization Fields considers multi-layer design without temporal multiplexing
38. Layered 3D: Multi-Layer Automultiscopic Displays
mask K
…
mask 3
mask 2
mask 1
light box
Layered 3D
39. Tomographic Light Field Synthesis
virtual plane
Image formation model:
attenuator
ò
- m (r )dr
L(x, q ) = I 0 e C
æ L(x, q ) ö
L(x, q ) = ln ç ÷ = - ò m (r)dr
è I0 ø C
backlight l = -Pa
Tomographic synthesis:
2
arg min l + Pa , for a ³ 0
a
2D Light Field
40. Tomographic Light Field Synthesis
virtual plane
Image formation model:
attenuator
ò
- m (r )dr
L(x, q ) = I 0 e C
æ L(x, q ) ö
L(x, q ) = ln ç ÷ = - ò m (r)dr
è I0 ø C
backlight l = -Pa
Tomographic synthesis:
2
arg min l + Pa , for a ³ 0
a
2D Light Field
41. Multi-Layer Light Field Decomposition
Reconstructed Views
Target 4D Light Field
Multi-Layer Decomposition
42. Prototype Layered 3D Display
Transparency stack with acrylic spacers Prototype in front of LCD (backlight source)
43.
44. Polarization Fields
Four Stacked Liquid Crystal Panels
Two Crossed Polarizers
61. Camera Culture: Compressive Displays Team
Gordon Wetzstein Matthew Hirsch Douglas Lanman
Postdoctoral Associate Graduate Student Postdoctoral Associate
Wolfgang Heidrich, Professor, University of British Columbia
Yunhee Kim, Postdoctoral Fellow, MIT Media Lab
62. Raskar, Lanman, Wetzstein, Hirsch MIT Media Lab http://cameraculture.info
Shift Glass
Capture Display
5D: Looking Compressive Displays
around corners 6D: View and Lighting Aware
4D: Rank Deficient, multilayer
4D: Netra for Optometry
WDF
Analyze G
Augmented
Light LF
~
`
L
= F
Field
4D, 6D, 8D: Augmented Light Field
63. Raskar, Lanman, Wetzstein, Hirsch MIT Media Lab http://cameraculture.info
Layered 3D Polarization Fields High-Rank 3D (HR3D)
www.layered3d.info tinyurl.com/polarization-fields www.hr3d.info
Slow Display 6D Display BiDi Screen
tinyurl.com/slow-display tinyurl.com/6d-display www.bidiscreen.com
64. Compressive Display Research in Camera Culture
Ramesh Raskar, Douglas Lanman, Gordon Wetzstein, Matthew Hirsch
http://cameraculture.media.mit.edu/compressivedisplays