This document describes a multi-camera time-of-flight imaging system that uses multiple synchronized cameras and light sources. It allows control over modulation signals to capture dynamic scenes and extract depth, velocity and non-line-of-sight motion information. The system architecture uses a direct digital synthesis chip to generate programmable modulation signals, synchronized signal conditioning circuits and a real-time controller to coordinate image capture across multiple time-of-flight cameras. It aims to enable applications like phased array imaging, simultaneous Doppler velocity capture and detecting motion behind scattering media.

1. Computational Imaging with
Multi-Camera Time-of-Flight Systems
Felix Heide*1,2Shikhar Shrestha*1 Wolfgang Heidrich2,3 Gordon Wetzstein1
2University of British Columbia 3KAUST1Stanford University
www.computationalimaging.org
* joint first authors

2. Time-of-Flight Imaging
Kinect for Xbox One Google Tango Phone
Direct Sampling Transient Imaging
[Velten et al. 2013]
Google Self-Driving Car
Facial capture in VR
[Li et al. 2015]
Correlation Transient Imaging
[Heide et al. 2013]

3. Multi-Camera ToF System
• Multiple Camera/Sources
• Phased Array
• Control over modulation
• Extensible
• Reproducible

4. light source
camera
Time-of-Flight ImagingTime-of-Flight Imaging

5. Image formation

6. Electron
Field
Photon
Charge
storage
bin 1
Charge
storage
bin 2
Image formation

7. Image formation
1,2,3,4

8. Image formation
1,2,3,4
Static frame
Captured with TI OPT8241
Slow-Motion Video
Captured with TI OPT8241

9. Time-of-Flight ImagingTime-of-Flight Imaging – Dynamic Scene
Doppler shift:
Orthogonal
Frequencies:

10. Doppler Shift

11. Image formation – Multicam

12. Image formation – Frequency and Phase Synchronization

13. Image formation – Frequency and Phase Synchronization

14. Multi-Camera ToF System
• Exposure Synchronization
• Individual Sensor/Source
Modulation Signals
• High Frequency Accuracy
• Compatible with Recent
ToF Cameras
• Extensibility

15. Multi-Camera ToF System – High-Level System Architecture

16. Multi-Camera ToF System – Key System Components
Direct Digital Synthesis (DDS)
Ch 0
Ch 1
Ch 2
Ch 3
- Programmable Frequency/Phase
- Resolution : 0.12 Hz/0.02°
- Range : 0-200 MHz/0-360°
- Control over SPI
• Custom Board for End-to-End
Signal Conditioning
• Combines a High Speed
Comparator with Exposure
Gating IC
• Interfaces Directly with TI ToF
Camera
Signal Conditioning Circuitry
ToF Daughter BoardAD9959
Real-Time Controller
STMF4 Discovery Board
• Controller responsible for
triggering frames, controlling
waveform generation and
synchronization
• Runs low-level firmware based
on an interrupt driven state
machine architecture

17. Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
MCU Host
Real-Time Controller
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
PC
ToF Cameras

18. Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 500 MHz)
Configure Modulation Parameters
ToF Cameras

19. Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Configure Camera Registers

20. Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Initiate a Frame

21. Multi-Camera ToF System – System Workflow
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Start of Micro-Exposure/Quad

22. Multi-Camera ToF System – System Workflow
Enable Modulation

23. Multi-Camera ToF System – System Workflow
Modulation Signals to Sensor/Illum

24. Multi-Camera ToF System – System Workflow
Raw Capture Data

25. Multi-Camera ToF System – System Workflow

26. Multi-Camera ToF System – Demo Unit Design
DC-DC Voltage Regulators
Host MCUDDS Board
ToF Daughter Board
ToF Camera

27. Demo

28. Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media

29. Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media

30. Phased Camera Depth Imaging

31. Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media

32. Simultaneous Doppler Velocity Imaging

33. Simultaneous Doppler Velocity Imaging

34. Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media

35. Non-Line-of-Sight Motion Detection

36. Non-Line-of-Sight Motion Detection

37. Non-Line-of-Sight Motion Detection

38. Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media

39. Motion in Scattering Media

40. Motion in Scattering Media

41. Multi-Camera ToF System
Open-Source Hardware Platform:
www.computationalimaging.org