High dynamic images between devices and vision limits

Le immagini ad alta dinamica
tra i limiti dei dispositivi e quelli
della visione
Alessandro Rizzi
Dipartimento di Informatica e Comunicazione
Università degli Studi di Milano

Friday, June 10, 2011

Outline

HDR imaging

HDR in practice: measuring the limits

Using HDR


The dynamic range


Deﬁne HDR ?
do we need a
threshold number ?


Deﬁne HDR ?
do we need a
threshold number ?

NO


Deﬁne HDR

A rendition of a scene with
greater dynamic range than
the reproduction media


That is ?


Annibale
Carracci

(1560-‐1609)

Paesaggio

Photo: C. Oleari

Source/lamp Average Luminance cd/
Light Xenon
short
arc
m2
200
000
÷
5
000
000
000

levels Sun
Metal
halide
1
600
000
000
10
000
000
÷
60
000
000
Incandescent 20
000
000
÷
26
000
000
compact
Fluorescent
20
000
÷
70
000
Fluorescent 5
000
÷
30
000
Sunlit
clouds 10
000
Candle 7
500
blue
sky 5
000
Preferred
values
for
50
÷
500
indoor
lighIng
White
paper
at
sun 10
000
White
paper
at
500
lx 100
White
paper
at
5
lx 1
Courtesy: C. Oleari

Dynamic ranges


Dynamic ranges

?

Range limits and quantization:
the ‘salame’ metaphor


Range compression
from incorrect pixel perspective


Range compression
from incorrect pixel perspective

Very wide range obtained with isolated stimuli
impossible to obtain in an image

The “salame” metaphor

Dynamic range Quantization


The “salame” metaphor

Dynamic range Quantization

More bits do not mean wider range
Less bits do not mean shorter range

28=256
8 bit
2-3 log unit

Scene Sensor
DR DR

216=65536
16 bit
4-5 log unit


28=256
8 bit
2-3 log unit

Scene
DR
Sensor
DR NO
216=65536
16 bit
4-5 log unit


8 bit 16 bit

2-3 log unit
Scene Sensor Scene Sensor
DR DR DR DR

8 bit 16 bit

4-5 log unit

Scene Sensor Scene Sensor
DR DR DR DR

8 bit 16 bit


The HDR idea

http://www.adolfo.trinca.name/public/2010/11/
ahdrdiagram.jpg

The HDR idea
How ?
general solution ?
rendering intent ?

ahdrdiagram.jpg

http://www.digitalcameratracker.com/how-to-create-high-
deﬁnition-range-hdr-photos/

Two sides of the coin

• Objective data: recording/displaying
physical light colorimetric distribution

• Subjective data: reproducing appearance
(or different rendering intent)


Mapping the world:
the characteristic curve


H&D
curve


Olympus E-3

http://www.dpreview.com/reviews/olympuse3/page21.asp


Exposure problem


History of HDR imaging


HDR 1858
H.P. Robinson “Fading Away


“The Fundamentals of Photography”

Mees (1920) 2 negative print

Ansel
Adams


Ansel Adams - Zone System

ISCC 11/05-McCann

Jones and Condit, 1941
Measurements of dynamic range of real scenes

REFLECTANCE RANGE OF PRINTS

SCENE RANGE OF WORLD
Minimum

Average of 126 outdoor scenes

Maximum

0.0 1.5 3.0
log range


L.A.Jones & H.R.Condit, JOSA,1941


Retinex starting idea
digit ~ luminance 119 119

Green record

55 146
88 230
ratio = ratio =
0.62 0.62

Ratios are constant in sun and shade

Retinex camera

Capturing and reproducing
the scene


Sensors dynamic range

Limited !


Is HDR a technological
problem ?


Expanding sensors dynamic range
• Sensors that compress their response to light due to their
logarithmic transfer function;
• Multimode sensors that have a linear and a logarithmic
response at dark and bright illumination levels, (switches
between linear and logarithmic modes of operation);
• Sensors with a capacity well adjustment method;
• Frequency-based sensors, sensor output is converted into
pulse frequency;
• Time-to-saturation [(TTS); time-to-first spike] sensors,
signal is the time the to saturated pixel;
• Sensors with global control over the integration time;
• Sensors with autonomous control over the integration time,
where each pixel has control over its own exposure.
Spivak A, Belenky A, Fish A & Yadid-Pecht O (2009) Wide dynamic-range CMOS image sensors:
A comparative performance analysis, IEEE Trans. on Electron Devices, 56, 2446-2461.

The HDR idea
How ?

ahdrdiagram.jpg

Multiple image
acquisition


CameraDigit = (radiance * time)

• Multiple Exposures
• Use Multiple Times
• Recover scene radiances at all pixels
from camera digits

New goal:
Accurately measure radiances


Multiple Exposures

Flux = Luminance * time

Scene Luminance = Flux / time

Scene Luminance = Camera Digit / time


Multiple Exposures

One Spot (ScaleD)
250

200

1/8 sec
1/4 sec
1/2 sec
Camera Digit

150

Camera 1 sec
2 sec
4 sec

Digit 100 8 sec
16 sec
32 sec
64 sec
50 FIT

0
0.0001 0.0010 0.0100 0.1000 1.0000 10.0000 100.0000 1000.0000
Exposure Flux [(cd/m2) * sec]

Flux = Luminance * time

HDR file formats

Source: Reinhard et al., High Dynamic Range Imaging: Acquisition, Display, and
Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics)

Acquisition limits


The glare problem


Effect of illumination
1.0 reﬂ * 1.0 illum = 1.0 cd/m2

0.2 reﬂ *0.01 illum = 0.002 cd/m2

Assumes 0.0 glare

Glare is image dependent
1.0 reﬂ * 1.0 illum = 1.0 cd/m2

0.002 cd/m2 *0.001 = 0.000002

0.001

1.0 cd/m2 *0.001 = 0.001
0.001

0.2 reﬂ *0.01 illum = 0.002 cd/m2

Assumes 0.001 glare

Ratio Signal/Glare
1.0 cd/m2)/(0.000002) = 5*10^5

( 0.002 cd/m2)) / (0.001) = 2

Assumes 0.001 glare

Sowerby, “Dictionary of Photography”, 1956


Parasitic Images


Camera limits
• Glare
• Unwanted scattered light in camera
• air - glass reflections
• lens (number of elements)
• aperture
• angle off optical axis
• camera wall reflections
• sensor surface reflections
• We must measure actual veiling glare limit

Measuring overall camera glare


HDR Test Setup


digit 255 = 2094.2 cd/m2

digit 0 = 0.11 cd/m2

Synthetic HDR
(High-Dynamic Range)
Images
Text
18,619:1

Goal Image
2094.2 cd/m2
= 18,619
0.11 cd/m2


20:1

18,619:1

Targets

16 sec exposure - Target 1scaleBlack

16 sec exposure - Target 4scaleBlack

Target 1B
Text

Target 4B
Text

Target 4W

16 sec exposure

Constant Luminance - Variable Surround

Minimum Glare


Mild Glare


Maximum Glare


4.3 log10 scene ----> 3.0 log10 image

Scene In-camera Maximum
Scene Dynamic Accurate Error
Range Range (% radiance)

1scaleB 20:1 20:1 0

4scaleB 18,619:1 3,000:1
1 300% Min

4scaleW 18,619:1 100:1 10,000% Max

Measure In-camera Accuracy

Side Dupe Film

Slide Dupe Film

One Negative Capture
4scale Black - Single Negative

2.50 3.5 Log10 units

2.30

2.10
Log digit

1.90

1.70

1.50
-1.00 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50
Log Cd/m2


Dynamic Range (OD)

HDR from cameras
• Range of usable captured information
• Range of accurate luminance information
(much smaller)

• Scene dependent


Courtesy: M. Fairchild

Glare insertion

Gregory Ward Larson, Holly Rushmeier, and Christine Piatko, “A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes”, IEEE
Trans on VISUALIZATION AND COMPUTER GRAPHICS, VOL. 3, NO. 4, oct-dec 1997


Display:
measuring the human limits


Magnitude estimates (100-1)

• Luminance does not correlate uniquely
with appearance

• No global tone scale can render the
appearance


Magnitude Estimation of Appearance
Change Surrounds

100
90
Magnitude Estimation

80
70
60
50
40
30
20
10
0
0.10 1.00 10.00 100.00 1000.00 10000.00
Log Luminance (cd/m2)

Min [0 cd/m2] Max [2094 cd/m2]


•White surround
•adds glare
•changes surround
(simultaneous contrast)

We need a new range target
•Vary dynamic range with
•constant glare
•contrast surround

Center/Surround
Basic Unit

Gray test areas 12%
(small differences)

Fixed contrast surround 88%


90o rotation


Testing different glares
% of white surround

100%
50%
0%


Single & Double Density Transparencies

Single =
2.7 log10 range

Double =
(superimposed)

5.4 log10 range


5.4 & 2.7 log10 Ranges
Constant Glare & Surround


White[100] = 0.0 rOD - Black [1] = 2.89 rOD

100.0
90.0
80.0
70.0
magnitude estimation

60.0
50.0
40.0
30.0
50% white
surround
20.0
10.0
0.0
6 5 4 3 2 1 0
relative optical density

50% Single Density

100

90

80

70
2.3 log10 units

60

50

40
50% white
30 surround
20

10

0
6 5 4 3 2 1 0

50% Double Density 50% Single Density

100

90
2.0 log10 units
80

70

60

50

40 100% white
surround
30

20

10

0
6 5 4 3 2 1 0

White Double Density White Single Density

100

90
5.0 log10 units
80

70

60

50

40 0% white
30 surround
20

10

0
6 5 4 3 2 1 0
Black Double Density Black Single Density

100

90
5.0 log10 units
80
Over 20
70
not big improvement

60

50

40 0% white
30 surround
20

10

0
6 5 4 3 2 1 0
Black Double Density Black Single Density

Measurements of apparent
range
(depends on area of white)

•100% = 2.0 log units
10

• 50% = 2.3 log units
10

• 8% = 2.9 log units
10


DD DD DD DD

Test summary

• Double transmission contrast
• Double dynamic range
• very small change in appearance range
• Visual limit ~ area of white surround
• area of white controls glare


What is on the retina:
calculated retinal luminance


What comes to the retina is
different from the image

High glare Low glare


Veiling glare increases
gray luminance

Contrast
offsets
glare

Contrast decreases
gray appearance

Glare vs. Contrast

Discussion

• Glare lowers the physical contrast
• Spatial comparisons increase the
contrast of appearance.

• The two act in opposition.
• Change with distance are different and
the cancellation is far from exact.


Glare Spread Function
1Vos, J.J. and van den Berg, T.J.T.P,
CIE Research note 135/1, “Disability Glare”, ISBN 3900734976 (1999).

PIGMENT
Blue eyed Caucasian 1.21
Blue green Caucasian 1.02
Mean over all Caucasian 1.00
Brown eyed Caucasian 0.50
Non Caucasian with pigmented skin and dark brown eyes 0.00


Glare Spread Function

Plotted in log scale


Dynamic Range = 5.4 OD
or 251,189:1

False-color LookUpTable (LUT)

Same LUT applied to SD & DD

Visualize HDR targets

Retinal image


Visualize Retinal Images


Change LUT for Retinal Images

Scene Retina Appearance
1,000,000:1 100:1 1,000:1

Spatial Spatial
Glare Contrast
Two scene-dependent spatial mechanisms:
glare and contrast
Glare masks the strength of spatial
contrast

Ranges


Tone-rendering problem and
spatial comparisons


Choosing a rendering intent


Land experiment


Land experiment

Projector

Land experiment

ES=100 EM=100 EL=100

Projector

Land experiment

ES=100 EM=100 EL=100

Projector Colorimeter


Land experiment

ES=100 EM=100 EL=100 LS=255 LM=115 LL=255



Land experiment

Observer

ES=100 EM=100 EL=100 LS=255 LM=115 LL=255



Land experiment
PINK

Observer

ES=100 EM=100 EL=100 LS=255 LM=115 LL=255



Land experiment

Observer



Land experiment

Observer

ES=50 EM=111 EL=50



Land experiment

Observer

ES=50 EM=111 EL=50 LS=128 LM=128 LL=128



Land experiment
PINK

Observer

ES=50 EM=111 EL=50 LS=128 LM=128 LL=128



Land experiment
GRAY
PINK

Observer

ES=50 EM=111 EL=50 LS=128 LM=128 LL=128



visual sensation


HVS:
local compression of range


Tone mapping vs Tone rendering

No tone mapping operator (global)
can mimic vision

We need an image dependent
tone renderer operator (local)


Black and White Mondrian


HP 945 Images without “Frames of Reference”

Some examples


Bob Sobol, HP

R. Sobol, “ Improving the Retinex algorithm
for rendering
wide dynamic range photographs”, in
Human Vision and Electronic
Imaging VII, B. E. Rogowitz and T. N.
Pappas, ed., Proc. SPIE 4662-41, 341-348,
2002.


ACE
Original ACE
Original ACE


STRESS
Tone Rendering


Judging the results


Beauty contest

C. Gatta, A. Rizzi, D. Marini, “Perceptually inspired HDR images tone mapping with color correction”, Journal of Imaging Systems and Technology, Volume 17 Issue 5, pp. 285-294 (2007).


HDR is in the middle
Glare Post-LUT
Sensor Spatial
graphics
Pre-LUT Algorithm
card

Image Spatial
Scene in CPU Image Display
memory in CPU


Summary


• To understand HDR we need a new perspective!
1.Veiling glare limits the range on the retina
2. Neural processing (spatial) determines appearance
3. Neural is stronger than it appears
[neural cancels glare]
4. General Solution requires spatial process
[mimic vision]
5. Tone-Scale is limited, we need Tone-rendering
[scene dependent]


Take home points
• HDR limits are not (only) technological
• Glare limits both acquisition and vision
• Glare is scene dependent
• Human vision use spatial comparison to
overcome this limit
• Tone renderer operator can use the same
approach


Take home points

HDR works very well
• because preserves image
information
• not because are more accurate
(not possible)


References
• J. J. McCann, A. Rizzi, “Camera and visual veiling glare in HDR images”
Journal of the Society for Information Display 15/9, 721–730 (2007).

• J. J. McCann, “Art, Science and Appearance in HDR” Journal of the Society
for Information Display 15/9, 709–719 (2007).

• A. Rizzi, J. J. McCann, “Glare-limited Appearances in HDR Images”, Journal
of the Society for Information Display, 17/1, pp. 3-12, (2009).

• J. J. McCann, A. Rizzi, “Retinal HDR Images: Intraocular Glare and Object
Size” Journal of the Society for Information Display, 17/11, pp. 913-920,
(2009).


The art and science of HDR imaging
J.J. McCann, A. Rizzi

(expected publication date autumn 2011)


Thank you

alessandro.rizzi@unimi.it


High dynamic images between devices and vision limits

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Mehr von Media Integration and Communication Center

Mehr von Media Integration and Communication Center (15)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

High dynamic images between devices and vision limits