Slide of the oral presentation for "Photometric Stereo in Participating Media Considering Shape-Dependent Forward Scatter" in CVPR 2018

1. Photometric Stereo in Participating
Media Considering Shape-Dependent
Forward Scatter
CVPR 2018, Salt Lake City
Yuki Fujimura1 Masaaki Iiyama1
Atsushi Hashimoto1,2 Michihiko Minoh1
1Kyoto University, Japan 2OMRON SINIC X Corp., Japan

2. Goal: 3D reconstruction in Participating Media
2
3D reconstruction
Image degraded
by scattered light
Our method can reconstruct
a 3D shape from degrade images.
Previous method
Our method

3. Light scattering in participating media
3
Light is scattered by suspended particles in participating media.
Four components observed at a camera:
• Direct component
• Backscatter component
• Surface-camera forward scatter component
• Source-surface forward scatter component

4. 44
Scattered light depends on the shape
of an object.
=> We propose an iterative algorithm.
Backscatter
removal [1]
Surface-camera
forward scatter
removal
Photometric stereo considering
source-surface forward scatter
Initial shape
Overview of our method
[1] C. Tsiotsios et al., “Backscatter Compensated Photometric Stereo with 3 Sources,” CVPR, 2014

5. 1. How to compute
scattering components?
2. How to remove
scattering components?
5

6. How to compute scattering components
6
ex. Backscatter
A scattering component is the sum of scattered light on the line of sight:
Integral between the surface and camera

7. How to compute scattering components
7
ex. Backscatter
[2] B. Sun et al., “A Practical Analytic Single Scattering Model for Real Time Rendering,” TOG, 2005.
Lookup table [2]

8. How to compute scattering components
8
ex. Surface-camera forward scatter
When we observe surface point p ,
the scattered light from point q
is observed:
Kpq is the effect of surface-camera forward scatter and it also can be
computed using a lookup table.
We consider all discrete points observed at a camera:
Ls(q) : reflected light at q
: the number of pixels.

9. 1. How to compute
scattering components?
2. How to remove
scattering components?
9

10. How to remove scattering components
10
Theoretically
[3] Z. Murez et al., “Photometric Stereo in a Scattering Medium,” PAMI, 2017.
A previous method [3] approximated a
scene as a plane under orthogonal
projection, and this made K a spatially-
invariant PSF.
Previous method [3]
Our method
On the other hand, we
consider spatially-
variant kernels because
of the shape-dependent
forward scatter.
Thus, it is difficult to solve this
linear system directly.

11. How to remove scattering components
11
The effect of the forward scatter from a distant point is small,
thus we approximate K as the effect from near points
( K : sparse matrix) and a constant term .

12. Experiments
12

13. Experiments with synthesized data
13
Synthesized
image
w/o medium Direct
+ Source-surface
FW scatter
+ Surface-camera
FW scatter
+ Backscatter
Ground truth Iteration 1 2 3 4 5
Error (deg.) 5.20 4.65 1.43 1.29 1.29
Normals
Angular
error
Shape

14. Experiments with real data
14
We put a target object in the tank and pour diluted milk as
a participating medium.
Experimental environment
60-cm cubic tank

15. Experiments with real data
15
Observed
image
After
backscatter
removal
[1] C. Tsiotsios et al., “Backscatter Compensated Photometric Stereo with 3 Sources,” CVPR, 2014
Ground truth Iteration 1 2 3 4
Error (deg.) 19.48 5.96 4.38 3.62 3.66
Backscatter
only [1]

16. Experiments with real data
16
Backscatter
only[1]
Ourmethod
Concentration
Water
highlow
[1] C. Tsiotsios et al., “Backscatter Compensated Photometric Stereo with 3 Sources,” CVPR, 2014

17. Conclusion
• We have proposed photometric stereo in participating
media.
• Our method models shape-dependent forward scatter.
﹣To compute the forward scatter, we use an
analytical form using a lookup table.
﹣The sparse matrix approximation enable its
removal.
• Experiments with synthesized and real data
demonstrated that our method can improve 3D
reconstruction in participating media.
17