We present HeatWave, a system that uses digital thermal imaging cameras to detect, track, and support user interaction on arbitrary surfaces. Thermal sensing has had limited examination in the HCI research community and is generally under-explored outside of law enforcement and energy auditing applications. We examine the role of thermal imaging as a new sensing solution for enhancing user surface interaction. In particular, we demonstrate how thermal imaging in combination with existing computer vision techniques can make segmentation and detection of routine interaction techniques possible in real-time, and can be used to complement or simplify algorithms for traditional RGB and depth cameras. Example interactions include (1) distinguishing hovering above a surface from touch events, (2) shape-based gestures similar to ink strokes, (3) pressure based gestures, and (4) multi-finger gestures. We close by discussing the practicality of thermal sensing for naturalistic user interaction and opportunities for future work.
1. Heatwave:
Thermal Imaging for
Surface Interaction
Eric Larson, Gabe Cohn, Sidhant Gupta
Xiaofeng Ren, Dieter Fox, Beverly Harrison
Shwetak Patel
UbiComp Lab
Design Use Build Laboratory of Ubiquitous Computing
University of Washington University of Washington Intel Research
1
36. M arble Ti
Pap er le
Rubb
Salt y er
C la
ite
Bran
Plast
r
G
Po ic
lym k
Wate ers
er
r ass
exi -gl
Pl
Foam Plastic Wood Skin Glass Aluminum
36
37. Touch Release Cooling/Warming
Down Touch period
50ms 200ms 100s
34
Temperature (C)
33
32
31
30 Derivative
29
28
C Absolute C
27
26
25
1ms 10ms 100ms 1s 10s 100s
Time (log scale)
x x >
37
50. Heatwave:
Thermal Imaging for
Surface Interaction
eric.cooper.larson@gmail.com
Eric Larson
ubicomplab.cs.washington.edu
Gabe Cohn, Sidhant Gupta
Xiaofeng Ren, Dieter Fox, Beverly Harrison *Ryder Ziola
Shwetak Patel
UbiComp Lab
Design Use Build Laboratory of Ubiquitous Computing
University of Washington University of Washington Intel Research
50
55. Figure 15 Thermal reflections of a person from a wooden wall.
left: visual image; right: close up view with LW IR camera.
Figure 16. Suppression of thermal reflections from varnished) wood as observed through an IR
polarizer oriented perpendicular or parallel to the plane of incidence.
InfraMation 2004, Proceedings Volume 5, 287-298 (2004) 9
55