Computer vision handbook of computer vision and applications volume 2 - signal processing and pattern recognition

1 2 1
2 4 2
1 2 1

HANDBOOK OF
COMPUTER VISION AND
APPLICATIONS
Volume 2
Signal Processing and
Pattern Recognition

Bernd Jähne
Horst Haußecker
Peter Geißler

ACADEMIC
PRESS

Handbook of
Computer Vision
and Applications
Volume 2
Pattern Recognition

Handbook of
Computer Vision
and Applications
Volume 2
Pattern Recognition
Editors
Bernd Jähne
Interdisciplinary Center for Scientiﬁc Computing
University of Heidelberg, Heidelberg, Germany
and
Scripps Institution of Oceanography
University of California, San Diego

Horst Haußecker
Peter Geißler
University of Heidelberg, Heidelberg, Germany

ACADEMIC PRESS
San Diego London Boston
New York Sydney Tokyo Toronto

This book is printed on acid-free paper.
Copyright © 1999 by Academic Press.

All rights reserved.
No part of this publication may be reproduced or transmitted in any form or
by any means, electronic or mechanical, including photocopy, recording, or
any information storage and retrieval system, without permission in writing
from the publisher.

The appearance of code at the bottom of the ﬁrst page of a chapter in this book
indicates the Publisher’s consent that copies of the chapter may be made for
personal or internal use of speciﬁc clients. This consent is given on the con-
dition, however, that the copier pay the stated per-copy fee through the Copy-
right Clearance Center, Inc. (222 Rosewood Drive, Danvers, Massachusetts
01923), for copying beyond that permitted by Sections 107 or 108 of the U.S.
Copyright Law. This consent does not extend to other kinds of copying, such
as copying for general distribution, for advertising or promotional purposes,
for creating new collective works, or for resale. Copy fees for pre-1999 chap-
ters are as shown on the title pages; if no fee code appears on the title page,
the copy fee is the same as for current chapters. ISBN 0-12-379770-5/$30.00

ACADEMIC PRESS
A Division of Harcourt Brace & Company
525 B Street, Suite 1900, San Diego, CA 92101-4495
http://www.apnet.com

ACADEMIC PRESS
24-28 Oval Road, London NW1 7DX, UK
http://www.hbuk.co.uk/ap/

Library of Congress Cataloging-In-Publication Data
Handbook of computer vision and applications / edited by Bernd Jähne,
Horst Haussecker, Peter Geissler.
p. cm.
Includes bibliographical references and indexes.
Contents: v. 1. Sensors and imaging — v. 2. Signal processing and
pattern recognition — v. 3. Systems and applications.
ISBN 0–12–379770–5 (set). — ISBN 0–12–379771-3 (v. 1)
ISBN 0–12–379772–1 (v. 2). — ISBN 0–12–379773-X (v. 3)
1. Computer vision — Handbooks, manuals. etc. I. Jähne, Bernd
1953– . II. Haussecker, Horst, 1968– . III. Geissler, Peter, 1966– .
TA1634.H36 1999
006.3 7 — dc21 98–42541
CIP

Printed in the United States of America
99 00 01 02 03 DS 9 8 7 6 5 4 3 2 1

Contents

Preface xi

Contributors xiii
1 Introduction 1
B. Jähne
1.1 Signal processing for computer vision . . . . . . . . . . . . . . . 2
1.2 Pattern recognition for computer vision . . . . . . . . . . . . . . 3
1.3 Computational complexity and fast algorithms . . . . . . . . . 4
1.4 Performance evaluation of algorithms . . . . . . . . . . . . . . . 5
1.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

I Signal Representation
2 Continuous and Digital Signals 9
B. Jähne
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Continuous signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Discrete signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Relation between continuous and discrete signals . . . . . . . 23
2.5 Quantization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3 Spatial and Fourier Domain 35
B. Jähne
3.1 Vector spaces and unitary transforms . . . . . . . . . . . . . . . 35
3.2 Continuous Fourier transform (FT) . . . . . . . . . . . . . . . . . 41
3.3 The discrete Fourier transform (DFT) . . . . . . . . . . . . . . . . 51
3.4 Fast Fourier transform algorithms (FFT) . . . . . . . . . . . . . . 57
3.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4 Multiresolutional Signal Representation 67
B. Jähne
4.1 Scale in signal processing . . . . . . . . . . . . . . . . . . . . . . . 67
4.2 Scale ﬁlters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.3 Scale space and diﬀusion . . . . . . . . . . . . . . . . . . . . . . . . 76
4.4 Multigrid representations . . . . . . . . . . . . . . . . . . . . . . . 84
4.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

v

vi Contents

II Elementary Spatial Processing
5 Neighborhood Operators 93
B. Jähne
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.2 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.3 Linear shift-invariant filters . . . . . . . . . . . . . . . . . . . . . . 98
5.4 Recursive filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.5 Classes of nonlinear filters . . . . . . . . . . . . . . . . . . . . . . . 113
5.6 Efficient neighborhood operations . . . . . . . . . . . . . . . . . . 116
5.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6 Principles of Filter Design 125
B. Jähne, H. Scharr, and S. Körkel
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
6.2 Filter design criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6.3 Windowing techniques . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.4 Filter cascading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
6.5 Filter design as an optimization problem . . . . . . . . . . . . . 133
6.6 Design of steerable filters and filter families . . . . . . . . . . . 143
6.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
7 Local Averaging 153
B. Jähne
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
7.2 Basic features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
7.3 Box filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
7.4 Binomial filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
7.5 Cascaded averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
7.6 Weighted averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
7.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
8 Interpolation 175
B. Jähne
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.2 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
8.3 Interpolation in Fourier space . . . . . . . . . . . . . . . . . . . . . 180
8.4 Polynomial interpolation . . . . . . . . . . . . . . . . . . . . . . . . 182
8.5 Spline-based interpolation . . . . . . . . . . . . . . . . . . . . . . . 187
8.6 Optimized interpolation . . . . . . . . . . . . . . . . . . . . . . . . 190
8.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
9 Image Warping 193
B. Jähne
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
9.2 Forward and inverse mapping . . . . . . . . . . . . . . . . . . . . 194
9.3 Basic geometric transforms . . . . . . . . . . . . . . . . . . . . . . 195
9.4 Fast algorithms for geometric transforms . . . . . . . . . . . . . 199
9.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Contents vii

III Feature Estimation
10 Local Structure 209
B. Jähne
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
10.2 Properties of simple neighborhoods . . . . . . . . . . . . . . . . 210
10.3 Edge detection by first-order derivatives . . . . . . . . . . . . . . 213
10.4 Edge detection by zero crossings . . . . . . . . . . . . . . . . . . 223
10.5 Edges in multichannel images . . . . . . . . . . . . . . . . . . . . . 226
10.6 First-order tensor representation . . . . . . . . . . . . . . . . . . 227
10.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
11 Principles for Automatic Scale Selection 239
T. Lindeberg
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
11.2 Multiscale differential image geometry . . . . . . . . . . . . . . . 240
11.3 A general scale-selection principle . . . . . . . . . . . . . . . . . . 247
11.4 Feature detection with automatic scale selection . . . . . . . . 251
11.5 Feature localization with automatic scale selection . . . . . . . 262
11.6 Stereo matching with automatic scale selection . . . . . . . . . 265
11.7 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . 269
11.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
12 Texture Analysis 275
T. Wagner
12.1 Importance of texture . . . . . . . . . . . . . . . . . . . . . . . . . . 276
12.2 Feature sets for texture analysis . . . . . . . . . . . . . . . . . . . 278
12.3 Assessment of textural features . . . . . . . . . . . . . . . . . . . 299
12.4 Automatic design of texture analysis systems . . . . . . . . . . 306
12.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
13 Motion 309
H. Haußecker and H. Spies
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
13.2 Basics: flow and correspondence . . . . . . . . . . . . . . . . . . . 312
13.3 Optical flow-based motion estimation . . . . . . . . . . . . . . . 321
13.4 Quadrature filter techniques . . . . . . . . . . . . . . . . . . . . . 345
13.5 Correlation and matching . . . . . . . . . . . . . . . . . . . . . . . 353
13.6 Modeling of flow fields . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.7 Confidence measures and error propagation . . . . . . . . . . . 369
13.8 Comparative analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 373
13.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
14 Bayesian Multiscale Differential Optical Flow 397
E. P. Simoncelli
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
14.2 Differential formulation . . . . . . . . . . . . . . . . . . . . . . . . 398
14.3 Uncertainty model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
14.4 Coarse-to-fine estimation . . . . . . . . . . . . . . . . . . . . . . . . 404
14.5 Implementation issues . . . . . . . . . . . . . . . . . . . . . . . . . 410
14.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
14.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
14.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420

viii Contents

15 Nonlinear Diffusion Filtering 423
J. Weickert
15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
15.2 Filter design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
15.3 Continuous theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
15.4 Algorithmic details . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
15.5 Discrete theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
15.6 Parameter selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
15.7 Generalizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
15.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
15.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
16 Variational Methods 451
C. Schnörr
16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
16.2 Processing of two- and three-dimensional images . . . . . . . . 455
16.3 Processing of vector-valued images . . . . . . . . . . . . . . . . . 471
16.4 Processing of image sequences . . . . . . . . . . . . . . . . . . . . 476
16.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
17 Stereopsis - Geometrical and Global Aspects 485
H. A. Mallot
17.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
17.2 Stereo geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
17.3 Global stereopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
17.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502
18 Stereo Terrain Reconstruction by Dynamic Programming 505
G. Gimel’farb
18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
18.2 Statistical decisions in terrain reconstruction . . . . . . . . . . 509
18.3 Probability models of epipolar profiles . . . . . . . . . . . . . . . 514
18.4 Dynamic programming reconstruction . . . . . . . . . . . . . . . 520
18.5 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
18.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
19 Reflectance-Based Shape Recovery 531
R. Klette, R. Kozera, and K. Schlüns
19.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532
19.2 Reflection and gradients . . . . . . . . . . . . . . . . . . . . . . . . 539
19.3 Three light sources . . . . . . . . . . . . . . . . . . . . . . . . . . . 552
19.4 Two light sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559
19.5 Theoretical framework for shape from shading . . . . . . . . . 571
19.6 Shape from shading . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
19.7 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
19.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587
20 Depth-from-Focus 591
P. Geißler and T. Dierig
20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
20.2 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593
20.3 Principles of depth-from-focus algorithms . . . . . . . . . . . . 595

Contents ix

20.4 Multiple-view depth-from-focus . . . . . . . . . . . . . . . . . . . 596
20.5 Dual-view depth-from-focus . . . . . . . . . . . . . . . . . . . . . . 601
20.6 Single-view depth-from-focus . . . . . . . . . . . . . . . . . . . . . 608
20.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622

IV Object Analysis, Classification, Modeling, Visualization
21 Morphological Operators 627
P. Soille
21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628
21.2 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629
21.3 Morphological operators . . . . . . . . . . . . . . . . . . . . . . . . 637
21.4 Efficient computation of morphological operators . . . . . . . 659
21.5 Morphological image processing . . . . . . . . . . . . . . . . . . . 664
21.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678
22 Fuzzy Image Processing 683
H. Haußecker and H. R. Tizhoosh
22.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684
22.2 Why fuzzy image processing? . . . . . . . . . . . . . . . . . . . . . 691
22.3 Fuzzy image understanding . . . . . . . . . . . . . . . . . . . . . . 692
22.4 Fuzzy image processing systems . . . . . . . . . . . . . . . . . . . 699
22.5 Theoretical components of fuzzy image processing . . . . . . 702
22.6 Selected application examples . . . . . . . . . . . . . . . . . . . . 714
22.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 721
22.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722
23 Neural Net Computing for Image Processing 729
A. Meyer-Bäse
23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 729
23.2 Multilayer perceptron (MLP) . . . . . . . . . . . . . . . . . . . . . . 730
23.3 Self-organizing neural networks . . . . . . . . . . . . . . . . . . . 736
23.4 Radial-basis neural networks (RBNN) . . . . . . . . . . . . . . . . 740
23.5 Transformation radial-basis networks (TRBNN) . . . . . . . . . 743
23.6 Hopfield neural networks . . . . . . . . . . . . . . . . . . . . . . . 747
23.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
24 Graph Theoretical Concepts for Computer Vision 753
D. Willersinn et al.
24.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754
24.2 Basic definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754
24.3 Graph representation of two-dimensional digital images . . . 760
24.4 Voronoi diagrams and Delaunay graphs . . . . . . . . . . . . . . 762
24.5 Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775
24.6 Graph grammars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780
24.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786
25 Shape Reconstruction from Volumetric Data 791
R. Eils and K. Sätzler
25.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
25.2 Incremental approach . . . . . . . . . . . . . . . . . . . . . . . . . . 794

x Contents

25.3 Three-dimensional shape reconstruction from contour lines . 797
25.4 Volumetric shape reconstruction . . . . . . . . . . . . . . . . . . 802
25.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 811
25.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813
26 Probabilistic Modeling in Computer Vision 817
J. Hornegger, D. Paulus, and H. Niemann
26.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 817
26.2 Why probabilistic models? . . . . . . . . . . . . . . . . . . . . . . . 819
26.3 Object recognition: classification and regression . . . . . . . . 821
26.4 Parametric families of model densities . . . . . . . . . . . . . . . 826
26.5 Automatic model generation . . . . . . . . . . . . . . . . . . . . . 844
26.6 Practical issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 850
26.7 Summary, conclusions, and discussion . . . . . . . . . . . . . . . 852
26.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 852
27 Knowledge-Based Interpretation of Images 855
H. Niemann
27.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855
27.2 Model of the task domain . . . . . . . . . . . . . . . . . . . . . . . 859
27.3 Interpretation by optimization . . . . . . . . . . . . . . . . . . . . 864
27.4 Control by graph search . . . . . . . . . . . . . . . . . . . . . . . . 865
27.5 Control by combinatorial optimization . . . . . . . . . . . . . . . 868
27.6 Judgment function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 870
27.7 Extensions and remarks . . . . . . . . . . . . . . . . . . . . . . . . 872
27.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 872
28 Visualization of Volume Data 875
J. Hesser and C. Poliwoda
28.1 Selected visualization techniques . . . . . . . . . . . . . . . . . . 876
28.2 Basic concepts and notation for visualization . . . . . . . . . . 880
28.3 Surface rendering algorithms and OpenGL . . . . . . . . . . . . 881
28.4 Volume rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884
28.5 The graphics library VGL . . . . . . . . . . . . . . . . . . . . . . . . 890
28.6 How to use volume rendering . . . . . . . . . . . . . . . . . . . . . 898
28.7 Volume rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 901
28.8 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905
28.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905
29 Databases for Microscopes and Microscopical Images 907
N. Salmon, S. Lindek, and E. H. K. Stelzer
29.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908
29.2 Towards a better system for information management . . . . 909
29.3 From flat files to database systems . . . . . . . . . . . . . . . . . 911
29.4 Database structure and content . . . . . . . . . . . . . . . . . . . 912
29.5 Database system requirements . . . . . . . . . . . . . . . . . . . . 917
29.6 Data flow—how it looks in practice . . . . . . . . . . . . . . . . . 918
29.7 Future prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 921
29.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 925

Index 927

Preface

What this handbook is about

This handbook offers a fresh approach to computer vision. The whole
vision process from image formation to measuring, recognition, or re-
acting is regarded as an integral process. Computer vision is under-
stood as the host of techniques to acquire, process, analyze, and un-
derstand complex higher-dimensional data from our environment for
scientific and technical exploration.
In this sense the handbook takes into account the interdisciplinary
nature of computer vision with its links to virtually all natural sciences
and attempts to bridge two important gaps. The first is between mod-
ern physical sciences and the many novel techniques to acquire images.
The second is between basic research and applications. When a reader
with a background in one of the fields related to computer vision feels
he has learned something from one of the many other facets of com-
puter vision, the handbook will have fulfilled its purpose.
The handbook comprises three volumes. The first volume, Sensors
and Imaging, covers image formation and acquisition. The second vol-
ume, Signal Processing and Pattern Recognition, focuses on processing
of the spatial and spatiotemporal signal acquired by imaging sensors.
The third volume, Systems and Applications, describes how computer
vision is integrated into systems and applications.

Prerequisites

It is assumed that the reader is familiar with elementary mathematical
concepts commonly used in computer vision and in many other areas
of natural sciences and technical disciplines. This includes the basics
of set theory, matrix algebra, differential and integral equations, com-
plex numbers, Fourier transform, probability, random variables, and
graphing. Wherever possible, mathematical topics are described intu-
itively. In this respect it is very helpful that complex mathematical
relations can often be visualized intuitively by images. For a more for-

xi

xii Preface

mal treatment of the corresponding subject including proofs, suitable
references are given.

How to use this handbook

The handbook has been designed to cover the different needs of its
readership. First, it is suitable for sequential reading. In this way the
reader gets an up-to-date account of the state of computer vision. It is
presented in a way that makes it accessible for readers with different
backgrounds. Second, the reader can look up specific topics of inter-
est. The individual chapters are written in a self-consistent way with
extensive cross-referencing to other chapters of the handbook and ex-
ternal references. The CD that accompanies each volume of the hand-
book contains the complete text of the handbook in the Adobe Acrobat
portable document file format (PDF). This format can be read on all
major platforms. Free Acrobat reader version 3.01 for all major com-
puting platforms is included on the CDs. The texts are hyperlinked in
multiple ways. Thus the reader can collect the information of interest
with ease. Third, the reader can delve more deeply into a subject with
the material on the CDs. They contain additional reference material,
interactive software components, code examples, image material, and
references to sources on the Internet. For more details see the readme
file on the CDs.

Acknowledgments

Writing a handbook on computer vision with this breadth of topics is
a major undertaking that can succeed only in a coordinated effort that
involves many co-workers. Thus the editors would like to thank first
all contributors who were willing to participate in this effort. Their
cooperation with the constrained time schedule made it possible that
the three-volume handbook could be published in such a short period
following the call for contributions in December 1997. The editors are
deeply grateful for the dedicated and professional work of the staff at
AEON Verlag & Studio who did most of the editorial work. We also
express our sincere thanks to Academic Press for the opportunity to
write this handbook and for all professional advice.
Last but not least, we encourage the reader to send us any hints
on errors, omissions, typing errors, or any other shortcomings of the
handbook. Actual information about the handbook can be found at the
editors homepage http://klimt.iwr.uni-heidelberg.de.

Heidelberg, Germany and La Jolla, California, December 1998
Bernd Jähne, Horst Haußecker, Peter Geißler

Contributors

Etienne Bertin received the PhD degree in mathematics
from Université Joseph Fourier in 1994. From 1990 to
1995 he worked on various topics in image analysis and
computational geometry. Since 1995, he has been an as-
sistant professor at the Université Pierre Mendès France
in the Laboratoire de statistique et d’analyses de don-
nées; he works on stochastic geometry.
Dr. Etienne Bertin
Laboratoire de Statistique et d’analyse de donnés
Université Pierre Mendès, Grenoble, France
bertin@labsad.upmf-grenoble.fr
Anke Meyer-Bäse received her M. S. and the PhD in elec-
trical engineering from the Darmstadt Institute of Tech-
nology in 1990 and 1995, respectively. From 1995 to
1996 she was a postdoctoral fellow with the Federal Insti-
tute of Neurobiology, Magdeburg, Germany. Since 1996
she was a visiting assistant professor with the Dept. of
Electrical Engineering, University of Florida, Gainesville,
USA. She received the Max-Kade award in Neuroengineer-
ing in 1996 and the Lise-Meitner prize in 1997. Her re-
search interests include neural networks, image process-
ing, biomedicine, speech recognition, and theory of non-
linear systems.
Dr. Anke Meyer-Bäse, Dept. of Electrical Engineering and Computer Science,
University of Florida, 454 New Engineering Building 33, Center Drive
PO Box 116130, Gainesville, FL 32611-6130, U.S., anke@alpha.ee.uﬂ.edu
Tobias Dierig graduated in 1997 from the University of
Heidelberg with a master degree in physics and is now
pursuing his PhD at the Interdisciplinary Center for Sci-
entiﬁc Computing at Heidelberg university. He is con-
cerned mainly with depth from focus algorithms, image
fusion, and industrial applications of computer vision
within the OpenEye project.
Tobias Dierig, Forschungsgruppe Bildverarbeitung, IWR
Universität Heidelberg, Im Neuenheimer Feld 368
D-69120 Heidelberg, Germany
Tobias.Dierig@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de/˜tdierig

xiii

xiv Contributors

Roland Eils studied mathematics and computer science
in Aachen, where he received his diploma in 1990. After
a two year stay in Indonesia for language studies he joint
the Graduiertenkolleg “Modeling and Scientific Comput-
ing in Mathematics and Sciences” at the Interdisciplinary
Center for Scientific Computing (IWR), University of Hei-
delberg, where he received his doctoral degree in 1995.
Since 1996 he has been leading the biocomputing group,
S tructures in Molecular Biology. His research interests
include computer vision, in particular computational ge-
ometry, and application of image processing techniques
in science and biotechnology.
Dr. Roland Eils, Biocomputing-Gruppe, IWR, Universität Heidelberg
Im Neuenheimer Feld 368, D-69120 Heidelberg, Germany
eils@iwr.uni-heidelberg.de
http://www.iwr.uni-heidelberg.de/iwr/bioinf
Peter Geißler studied physics in Heidelberg. He received
his diploma and doctoral degree from Heidelberg Uni-
versity in 1994 and 1998, respectively. His research in-
terests include computer vision, especially depth-from-
focus, adaptive filtering, and flow visualization as well as
the application of image processing in physical sciences
and oceanography.
Dr. Peter Geißler
Forschungsgruppe Bildverarbeitung, IWR
D-69120 Heidelberg, Germany
Peter.Geissler@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de
Georgy Gimel’farb received his PhD degree from the
Ukrainian Academy of Sciences in 1969 and his Doctor of
Science (the habilitation) degree from the Higher Certify-
ing Commission of the USSR in 1991. In 1962, he began
working in the Pattern Recognition, Robotics, and Image
Recognition Departments of the Institute of Cybernetics
(Ukraine). In 1994–1997 he was an invited researcher in
Hungary, the USA, Germany, and France. Since 1997, he
has been a senior lecturer in computer vision and digital
TV at the University of Auckland, New Zealand. His re-
search interests include analysis of multiband space and
aerial images, computational stereo, and image texture analysis.
Dr. Georgy Gimel’farb, Centre for Image Technology and Robotics,
Department of Computer Science, Tamaki Campus
The University of Auckland, Private Bag 92019, Auckland 1, New Zealand
g.gimelfarb@auckland.ac.nz, http://www.tcs.auckland.ac.nz/˜georgy

Contributors xv

Horst Haußecker studied physics in Heidelberg. He re-
ceived his diploma in physics and his doctoral degree
from Heidelberg University in 1994 and 1996, respec-
tively. He was visiting scientist at the Scripps Institution
of Oceanography in 1994. Currently he is conducting
research in the image processing research group at the
Interdisciplinary Center for Scientiﬁc Computing (IWR),
where he also lectures on optical ﬂow computation. His
research interests include computer vision, especially
image sequence analysis, infrared thermography, and
fuzzy-image processing, as well as the application of im-
age processing in physical sciences and oceanography.
Dr. Horst Haußecker, Forschungsgruppe Bildverarbeitung, IWR
Universität Heidelberg, Im Neuenheimer Feld 368, D-69120 Heidelberg
Horst.Haussecker@iwr.uni-heidelberg.de
Jürgen Hesser is assistant professor at the Lehrstuhl für
Informatik V, University of Mannheim, Germany. He
heads the groups on computer graphics, bioinformat-
ics, and optimization. His research interests are real-
time volume rendering, computer architectures, compu-
tational chemistry, and evolutionary algorithms. In addi-
tion, he is co-founder of Volume Graphics GmbH, Heidel-
berg. Hesser received his PhD and his diploma in physics
at the University of Heidelberg, Germany.
Jürgen Hesser, Lehrstuhl für Informatik V
Universität Mannheim
B6, 26, D-68131 Mannheim, Germany
jhesser@rumms.uni-mannheim.de,
Joachim Hornegger graduated in 1992 and received his
PhD degree in computer science in 1996 from the Uni-
versität Erlangen-Nürnberg, Germany, for his work on
statistical object recognition. Joachim Hornegger was
research and teaching associate at Universität Erlangen-
Nürnberg, a visiting scientist at the Technion, Israel, and
at the Massachusetts Institute of Technology, U.S. He
is currently a research scholar and teaching associate
at Stanford University, U.S. Joachim Hornegger is the
author of 30 technical papers in computer vision and
speech processing and three books. His research inter-
ests include 3-D computer vision, 3-D object recognition, and statistical meth-
ods applied to image analysis problems.
Dr. Joachim Hornegger, Stanford University, Robotics Laboratory
Gates Building 1A, Stanford, CA 94305-9010, U.S.
jh@robotics.stanford.edu, http://www.robotics.stanford.edu/˜jh

xvi Contributors

Bernd Jähne studied physics in Saarbrücken and Hei-
delberg. He received his diploma, doctoral degree, and
habilitation degree from Heidelberg University in 1977,
1980, and 1985, respectively, and a habilitation de-
gree in applied computer science from the University of
Hamburg-Harburg in 1992. Since 1988 he has been a Ma-
rine Research Physicist at Scripps Institution of Oceanog-
raphy, University of California, and, since 1994, he has
been professor of physics at the Interdisciplinary Center
of Scientific Computing. He leads the research group on
image processing. His research interests include com-
puter vision, especially filter design and image sequence
analysis, the application of image processing techniques
in science and industry, and small-scale air-sea interaction processes.
Prof. Dr. Bernd Jähne, Forschungsgruppe Bildverarbeitung, IWR
Universität Heidelberg, Im Neuenheimer Feld 368, D-69120 Heidelberg
Bernd.Jaehne@iwr.uni-heidelberg.de
Reinhard Klette studied mathematics at Halle University,
received his master degree and doctor of natural science
degree in mathematics at Jena University, became a do-
cent in computer science, and was a professor of com-
puter vision at Berlin Technical University. Since June
1996 he has been professor of information technology
in the Department of Computer Science at the University
of Auckland. His research interests include theoretical
and applied topics in image processing, pattern recogni-
tion, image analysis, and image understanding. He has
published books about image processing and shape reconstruction and was
chairman of several international conferences and workshops on computer
vision. Recently, his research interests have been directed at 3-D biomedical
image analysis with digital geometry and computational geometry as major
subjects.
Prof. Dr. Reinhard Klette, Centre for Image Technology and Robotics,
Computer Science Department, Tamaki Campus
The Auckland University, Private Bag 92019, Auckland, New Zealand
r.klette@auckland.ac.nz, http://citr.auckland.ac.nz/˜rklette
Christoph Klauck received his diploma in computer sci-
ence and mathematics from the University of Kaiser-
slautern, Germany, in 1990. From 1990 to 1994 he
worked as research scientist at the German Research
Center for Artificial Intelligence Inc. (DFKI GmbH) at
Kaiserslautern. In 1994 he finished his dissertation in
computer science. Since then he has been involved in
the IRIS project at the University of Bremen (Artificial
Intelligence Group). His primary research interests in-
clude graph grammars and rewriting systems in general,
knowledge representation, and ontologies.

Contributors xvii

Prof. Dr. Christoph Klauck, Dep. of Electrical Eng. and Computer Science
University of Hamburg (FH), Berliner Tor 3, D-20099 Hamburg, Germany
cklauck@t-online.de, http://fbi010.informatik.fh-hamburg.de/˜klauck

Stefan Körkel is member of the research groups for nu-
merics and optimization of Prof. Bock and Prof. Reinelt
at the Interdisciplinary Center for Scientific Computing
at the University of Heidelberg, Germany. He studied
mathematics in Heidelberg. Currently he is pursuing his
PhD in nonlinear and mixed integer optimization meth-
ods. His research interests include filter optimization as
well as nonlinear optimum experimental design.
Stefan Körkel
Im Neuenheimer Feld 368, 69120 Heidelberg
Stefan.Koerkel@IWR.Uni-Heidelberg.de
http://www.iwr.uni-heidelberg.de/˜Stefan.Koerkel/
Ryszard Kozera received his M.Sc. degree in pure mathe-
matics in 1985 from Warsaw University, Poland, his PhD
degree in computer science in 1991 from Flinders Uni-
versity, Australia, and finally his PhD degree in mathe-
matics in 1992 from Warsaw University, Poland. He is
currently employed as a senior lecturer at the University
of Western Australia. Between July 1995 and February
1997, Dr. Kozera was at the Technical University of Berlin
and at Warsaw University as an Alexander von Humboldt
Foundation research fellow. His current research inter-
ests include applied mathematics with special emphasis
on partial differential equations, computer vision, and
numerical analysis.
Dr. Ryszard Kozera, Department of Computer Science, The University of West-
ern Australia, Nedlands, WA 6907, Australia, ryszard@cs.uwa.edu.au
http://www.cs.uwa.edu.au/people/info/ryszard.html
Tony Lindeberg received his M.Sc. degree in engineer-
ing physics and applied mathematics from KTH (Royal
Institute of Technology), Stockholm, Sweden in 1987,
and his PhD degree in computing science in 1991. He
is currently an associate professor at the Department
of Numerical Analysis and Computing Science at KTH.
His main research interests are in computer vision and
relate to multiscale representations, focus-of-attention,
and shape. He has contributed to the foundations of
continuous and discrete scale-space theory, as well as
to the application of these theories to computer vision
problems. Specifically, he has developed principles for
automatic scale selection, methodologies for extracting salient image struc-
tures, and theories for multiscale shape estimation. He is author of the book
“Scale-Space Theory in Computer Vision.”

xviii Contributors

Tony Lindeberg, Department of Numerical Analysis and Computing Science
KTH, S-100 44 Stockholm, Sweden.
tony@nada.kth.se, http://www.nada.kth.se/˜tony
Steffen Lindek studied physics at the RWTH Aachen, Ger-
many, the EPF Lausanne, Switzerland, and the Univer-
sity of Heidelberg, Germany. He did his diploma and
PhD theses in the Light Microscopy Group at the Euro-
pean Molecular Biology Laboratory (EMBL), Heidelberg,
Germany, developing high-resolution light-microscopy
techniques. Since December 1996 he has been a post-
doctoral fellow with the BioImage project at EMBL. He
currently works on the design and implementation of the
image database, and he is responsible for the administra-
tion of EMBL’s contribution to the project.
Dr. Steffen Lindek, European Molecular Biology Laboratory (EMBL)
Postfach 10 22 09, D-69120 Heidelberg, Germany
lindek@EMBL-Heidelberg.de
Hanspeter A. Mallot studied biology and mathematics at
the University of Mainz where he also received his doc-
toral degree in 1986. He was a postdoctoral fellow at
the Massachusetts Institute of Technology in 1986/87
and held research positions at Mainz University and the
Ruhr-Universität-Bochum. In 1993, he joined the Max-
Planck-Institut für biologische Kybernetik in Tübingen.
In 1996/97, he was a fellow at the Institute of Advanced
Studies in Berlin. His research interests include the per-
ception of shape and space in humans and machines,
cognitive maps, as well as neural network models of the
cerebral cortex.
Dr. Hanspeter A. Mallot, Max-Planck-Institut für biologische Kybernetik
Spemannstr. 38, 72076 Tübingen, Germany
Hanspeter.Mallot@tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de/bu/
Heinrich Niemann obtained the degree of Dipl.-Ing. in
electrical engineering and Dr.-Ing. at Technical Univer-
sity Hannover in 1966 and 1969, respectively. From
1967 to 1972 he was with Fraunhofer Institut für In-
formationsverarbeitung in Technik und Biologie, Karls-
ruhe. Since 1975 he has been professor of computer sci-
ence at the University of Erlangen-Nürnberg and since
1988 he has also served as head of the research group,
Knowledge Processing, at the Bavarian Research Institute
for Knowledge-Based Systems (FORWISS). His fields of
research are speech and image understanding and the
application of artificial intelligence techniques in these
fields. He is the author or co-author of 6 books and approximately 250 jour-
nal and conference contributions.

Contributors xix

Prof. Dr.-Ing. H. Niemann, Lehrstuhl für Mustererkennung (Informatik 5)
Universität Erlangen-Nürnberg, Martensstraße 3, 91058 Erlangen, Germany
niemann@informatik.uni-erlangen.de
http://www5.informatik.uni-erlangen.de
Dietrich Paulus received a bachelor degree in computer
science at the University of Western Ontario, London,
Canada (1983). He graduated (1987) and received his
PhD degree (1991) from the University of Erlangen-
Nürnberg, Germany. He is currently a senior researcher
(Akademischer Rat) in the ﬁeld of image pattern recog-
nition and teaches courses in computer vision and ap-
plied programming for image processing. Together with
J. Hornegger, he has recently written a book on pattern
recognition and image processing in C++.
Dr. Dietrich Paulus, Lehrstuhl für Mustererkennung
Universität Erlangen-Nürnberg, Martensstr. 3, 91058 Erlangen, Germany
paulus@informatik.uni-erlangen.de
http://www5.informatik.uni-erlangen.de
Christoph Poliwoda is PhD student at the Lehrstuhl für
Informatik V, University of Mannheim, and leader of the
development section of Volume Graphics GmbH. His re-
search interests are real-time volume and polygon ray-
tracing, 3-D image processing, 3-D segmentation, com-
puter architectures and parallel computing. Poliwoda
received his diploma in physics at the University of Hei-
delberg, Germany.
Christoph Poliwoda
Lehrstuhl für Informatik V
Universität Mannheim
B6, 26, D-68131 Mannheim, Germany
poliwoda@mp-sun1.informatik.uni-mannheim.de
Nicholas J. Salmon received the master of engineering
degree from the Department of Electrical and Electronic
Engineering at Bath University, England, in 1990. Then
he worked as a software development engineer for Mar-
coni Radar Systems Ltd., England, helping to create a
vastly parallel signal-processing machine for radar appli-
cations. Since 1992 he has worked as software engineer
in the Light Microscopy Group at the European Molecu-
lar Biology Laboratory, Germany, where he is concerned
with creating innovative software systems for the con-
trol of confocal microscopes, and image processing.
Nicholas J. Salmon, Light Microscopy Group,
European Molecular Biology Laboratory (EMBL)
Postfach 10 22 09, D-69120 Heidelberg, Germany
salmon@EMBL-Heidelberg.de,

xx Contributors

Kurt Sätzler studied physics at the University of Hei-
delberg, where he received his diploma in 1995. Since
then he has been working as a PhD student at the Max-
Planck-Institute of Medical Research in Heidelberg. His
research interests are mainly computational geometry
applied to problems in biomedicine, architecture and
computer graphics, image processing and tilted view mi-
croscopy.
Kurt Sätzler, IWR, Universität Heidelberg
Im Neuenheimer Feld 368, D-69120 Heidelberg
or
Max-Planck-Institute for Medical Research, Department of Cell Physiology
Jahnstr. 29, D-69120 Heidelberg, Germany
Kurt.Saetzler@iwr.uni-heidelberg.de
Hanno Scharr studied physics at the University of Hei-
delberg, Germany and did his diploma thesis on tex-
ture analysis at the Interdisciplinary Center for Scien-
tiﬁc Computing in Heidelberg. Currently, he is pursu-
ing his PhD on motion estimation. His research interests
include ﬁlter optimization and motion estimation in dis-
crete time series of n-D images.
Hanno Scharr
Im Neuenheimer Feld 368, 69120 Heidelberg, Germany
Hanno.Scharr@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de/˜hscharr/
Karsten Schlüns studied computer science in Berlin. He
received his diploma and doctoral degree from the Tech-
nical University of Berlin in 1991 and 1996. From 1991 to
1996 he was research assistant in the Computer Vision
Group, Technical University of Berlin, and from 1997
to 1998 he was a postdoctoral research fellow in com-
puting and information technology, University of Auck-
land. Since 1998 he has been a scientist in the image
processing group at the Institute of Pathology, Univer-
sity Hospital Charité in Berlin. His research interests
include pattern recognition and computer vision, espe-
cially three-dimensional shape recovery, performance analysis of reconstruc-
tion algorithms, and teaching of computer vision.
Dr. Karsten Schlüns, Institute of Pathology,
University Hospital Charité, Schumannstr. 20/21, D-10098 Berlin, Germany
Karsten.Schluens@charite.de, http://amba.charite.de/˜ksch

Contributors xxi

Christoph Schnörr received the master degree in electri-
cal engineering in 1987, the doctoral degree in computer
science in 1991, both from the University of Karlsruhe
(TH), and the habilitation degree in Computer Science in
1998 from the University of Hamburg, Germany. From
1987–1992, he worked at the Fraunhofer Institute for In-
formation and Data Processing (IITB) in Karlsruhe in the
ﬁeld of image sequence analysis. In 1992 he joined the
Cognitive Systems group, Department of Computer Sci-
ence, University of Hamburg, where he became an assis-
tant professor in 1995. He received an award for his work
on image segmentation from the German Association for
Pattern Recognition (DAGM) in 1996. Since October 1998, he has been a full
professor at the University of Mannheim, Germany, where he heads the Com-
puter Vision, Graphics, and Pattern Recognition Group. His research interests
include pattern recognition, machine vision, and related aspects of computer
graphics, machine learning, and applied mathematics.
Prof. Dr. Christoph Schnörr, University of Mannheim
Dept. of Math. & Computer Science, D-68131 Mannheim, Germany
schnoerr@ti.uni-mannheim.de, http://www.ti.uni-mannheim.de
Eero Simoncelli started his higher education with a bach-
elor’s degree in physics from Harvard University, went
to Cambridge University on a fellowship to study mathe-
matics for a year and a half, and then returned to the USA
to pursue a doctorate in Electrical Engineering and Com-
puter Science at MIT. He received his PhD in 1993, and
joined the faculty of the Computer and Information Sci-
ence Department at the University of Pennsylvania that
same year. In September of 1996, he joined the faculty
of the Center for Neural Science and the Courant Insti-
tute of Mathematical Sciences at New York University. He
received an NSF Faculty Early Career Development (CA-
REER) grant in September 1996, for teaching and research in “Visual Informa-
tion Processing”, and a Sloan Research Fellowship in February 1998.
Dr. Eero Simoncelli, 4 Washington Place, RM 809, New York, NY 10003-6603
eero.simoncelli@nyu.edu, http://www.cns.nyu.edu/˜eero
Pierre Soille received the engineering degree from the
Université catholique de Louvain, Belgium, in 1988. He
gained the doctorate degree in 1992 at the same univer-
sity and in collaboration with the Centre de Morphologie
Mathématique of the Ecole des Mines de Paris. He then
pursued research on image analysis at the CSIRO Math-
ematical and Information Sciences Division, Sydney, the
Centre de Morphologie Mathématique of the Ecole des
Mines de Paris, and the Abteilung Mustererkennung of
the Fraunhofer-Institut IPK, Berlin. During the period
1995-1998 he was lecturer and research scientist at the
Ecole des Mines d’Alès and EERIE, Nîmes, France. Now he is a senior research
scientist at the Silsoe Research Institute, England. He worked on many ap-

xxii Contributors

plied projects, taught tutorials during international conferences, co-organized
the second International Symposium on Mathematical Morphology, wrote and
edited three books, and contributed to over 50 scientific publications.
Prof. Pierre Soille, Silsoe Research Institute, Wrest Park
Silsoe, Bedfordshire, MK45 4HS, United Kingdom
Pierre.Soille@bbsrc.ac.uk, http://www.bbsrc.ac.uk
Hagen Spies graduated in January 1998 from the Univer-
sity of Heidelberg with a master degree in physics. He
also received an MS in computing and information tech-
nology from the University of Dundee, Scotland in 1995.
In 1998/1999 he spent one year as a visiting scientist at
the University of Western Ontario, Canada. Currently he
works as a researcher at the Interdisciplinary Center for
Scientific Computing at the University of Heidelberg. His
interests concern the measurement of optical and range
flow and their use in scientific applications.
Hagen Spies, Forschungsgruppe Bildverarbeitung, IWR
D-69120 Heidelberg, Germany, Hagen.Spies@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de/˜hspies
E. H. K. Stelzer studied physics in Frankfurt am Main and
in Heidelberg, Germany. During his Diploma thesis at
the Max-Planck-Institut für Biophysik he worked on the
physical chemistry of phospholipid vesicles, which he
characterized by photon correlation spectroscopy. Since
1983 he has worked at the European Molecular Biol-
ogy Laboratory (EMBL). He has contributed extensively
to the development of confocal fluorescence microscopy
and its application in life sciences. His group works
on the development and application of high-resolution
techniques in light microscopy, video microscopy, con-
focal microscopy, optical tweezers, single particle analy-
sis, and the documentation of relevant parameters with biological data.
Prof. Dr. E. H. K. Stelzer, Light Microscopy Group,
European Molecular Biology Laboratory (EMBL), Postfach 10 22 09
D-69120 Heidelberg, Germany, stelzer@EMBL-Heidelberg.de,
Hamid R. Tizhoosh received the M.S. degree in electrical
engineering from University of Technology, Aachen, Ger-
many, in 1995. From 1993 to 1996, he worked at Man-
agement of Intelligent Technologies Ltd. (MIT GmbH),
Aachen, Germany, in the area of industrial image pro-
cessing. He is currently a PhD candidate, Dept. of Tech-
nical Computer Science of Otto-von-Guericke-University,
Magdeburg, Germany. His research encompasses fuzzy
logic and computer vision. His recent research efforts
include medical and fuzzy image processing. He is cur-
rently involved in the European Union project INFOCUS,
and is researching enhancement of medical images in radiation therapy.
H. R. Tizhoosh, University of Magdeburg (IPE)

Contributors xxiii

P.O. Box 4120, D-39016 Magdeburg, Germany
tizhoosh@ipe.et.uni-magdeburg.de
http://pmt05.et.uni-magdeburg.de/˜hamid/
Thomas Wagner received a diploma degree in physics in
1991 from the University of Erlangen, Germany. In 1995,
he finished his PhD in computer science with an applied
image processing topic at the Fraunhofer Institute for In-
tegrated Circuits in Erlangen. Since 1992, Dr. Wagner has
been working on industrial image processing problems
at the Fraunhofer Institute, from 1994 to 1997 as group
manager of the intelligent systems group. Projects in
his research team belong to the fields of object recogni-
tion, surface inspection, and access control. In 1996, he
received the “Hans-Zehetmair-Habilitationsförderpreis.”
He is now working on automatic solutions for the design
of industrial image processing systems.
Dr.-Ing. Thomas Wagner, Fraunhofer Institut für Intregrierte Schaltungen
Am Weichselgarten 3, D-91058 Erlangen, Germany
wag@iis.fhg.de, http://www.iis.fhg.de
Joachim Weickert obtained a M.Sc. in industrial math-
ematics in 1991 and a PhD in mathematics in 1996,
both from Kaiserslautern University, Germany. After re-
ceiving the PhD degree, he worked as post-doctoral re-
searcher at the Image Sciences Institute of Utrecht Uni-
versity, The Netherlands. In April 1997 he joined the
computer vision group of the Department of Computer
Science at Copenhagen University. His current research
interests include all aspects of partial differential equa-
tions and scale-space theory in image analysis. He was
awarded the Wacker Memorial Prize and authored the
book “Anisotropic Diffusion in Image Processing.”
Dr. Joachim Weickert, Department of Computer Science, University of Copen-
hagen, Universitetsparken 1, DK-2100 Copenhagen, Denmark
joachim@diku.dk, http://www.diku.dk/users/joachim/
Dieter Willersinn received his diploma in electrical en-
gineering from Technical University Darmstadt in 1988.
From 1988 to 1992 he was with Vitronic Image Process-
ing Systems in Wiesbaden, working on industrial appli-
cations of robot vision and quality control. He then took
a research position at the Technical University in Vienna,
Austria, from which he received his PhD degree in 1995.
In 1995, he joined the Fraunhofer Institute for Informa-
tion and Data Processing (IITB) in Karlsruhe, where he
initially worked on obstacle detection for driver assis-
tance applications. Since 1997, Dr. Willersinn has been
the head of the group, Assessment of Computer Vision
Systems, Department for Recognition and Diagnosis Systems.
Dr. Dieter Willersinn, Fraunhofer Institut IITB, Fraunhoferstr. 1
D-76131 Karlsruhe, Germany, wil@iitb.fhg.de

1 Introduction
Bernd Jähne
Interdisziplinäres Zentrum für Wissenschaftliches Rechnen (IWR)
Universität Heidelberg, Germany

1.1 Signal processing for computer vision . . . . . . . . . . . . . . . 2
1.2 Pattern recognition for computer vision . . . . . . . . . . . . . . 3
1.3 Computational complexity and fast algorithms . . . . . . . . . 4
1.4 Performance evaluation of algorithms . . . . . . . . . . . . . . . 5
1.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

The second volume of the Handbook on Computer Vision and Ap-
plications deals with signal processing and pattern recognition. The
signals processed in computer vision originate from the radiance of an
object that is collected by an optical system (Volume 1, Chapter 5). The
irradiance received by a single photosensor or a 2-D array of photosen-
sors through the optical system is converted into an electrical signal
and finally into arrays of digital numbers (Volume 2, Chapter 2). The
whole chain of image formation from the illumination and interaction
of radiation with the object of interest up to the arrays of digital num-
bers stored in the computer is the topic of Volume 1 of this handbook
(subtitled Sensors and Imaging).
This volume deals with the processing of the signals generated by
imaging sensors and this introduction covers four general topics. Sec-
tion 1.1 discusses in which aspects the processing of higher-dimension-
al signals differs from the processing of 1-D time series. We also elab-
orate on the task of signal processing for computer vision. Pattern
recognition (Section 1.2) plays a central role in computer vision because
it uses the features extracted by lowlevel signal processing to classify
and recognize objects.
Given the vast amount of data generated by imaging sensors the
question of the computational complexity and of efficient algorithms is
of utmost importance (Section 1.3). Finally, the performance evaluation
of computer vision algorithms (Section 1.4) is a subject that has been
neglected in the past. Consequently, a vast number of algorithms exist
for which the performance characteristics are not sufficiently known.

1
Handbook of Computer Vision and Applications Copyright © 1999 by Academic Press
Volume 2 All rights of reproduction in any form reserved.
Signal Processing and Pattern Recognition ISBN 0–12–379772-1/$30.00

2 1 Introduction

This constitutes a major obstacle for progress of applications using
computer vision techniques.

1.1 Signal processing for computer vision

One-dimensional linear signal processing and system theory is a stan-
dard topic in electrical engineering and is covered by many standard
textbooks, for example, [1, 2]. There is a clear trend that the classical
signal processing community is moving into multidimensional signals,
as indicated, for example, by the new annual international IEEE confer-
ence on image processing (ICIP). This can also be seen from some re-
cently published handbooks on this subject. The digital signal process-
ing handbook by Madisetti and Williams [3] includes several chapters
that deal with image processing. Likewise the transforms and applica-
tions handbook by Poularikas [4] is not restricted to one-dimensional
transforms.
There are, however, only a few monographs that treat signal pro-
cessing specifically for computer vision and image processing. The
monograph of Lim [5] deals with 2-D signal and image processing and
tries to transfer the classical techniques for the analysis of time series
to 2-D spatial data. Granlund and Knutsson [6] were the first to publish
a monograph on signal processing for computer vision and elaborate on
a number of novel ideas such as tensorial image processing and nor-
malized convolution that did not have their origin in classical signal
processing.
Time series are 1-D, signals in computer vision are of higher di-
mension. They are not restricted to digital images, that is, 2-D spatial
signals (Chapter 2). Volumetric sampling, image sequences and hyper-
spectral imaging all result in 3-D signals, a combination of any of these
techniques in even higher-dimensional signals.
How much more complex does signal processing become with in-
creasing dimension? First, there is the explosion in the number of data
points. Already a medium resolution volumetric image with 5123 vox-
els requires 128 MB if one voxel carries just one byte. Storage of even
higher-dimensional data at comparable resolution is thus beyond the
capabilities of today’s computers. Moreover, many applications require
the handling of a huge number of images. This is also why appropriate
databases including images are of importance. An example is discussed
in Chapter 29.
Higher dimensional signals pose another problem. While we do not
have difficulty in grasping 2-D data, it is already significantly more de-
manding to visualize 3-D data because the human visual system is built
only to see surfaces in 3-D but not volumetric 3-D data. The more di-
mensions are processed, the more important it is that computer graph-

1.2 Pattern recognition for computer vision 3

ics and computer vision come closer together. This is why this volume
includes a contribution on visualization of volume data (Chapter 28).
The elementary framework for lowlevel signal processing for com-
puter vision is worked out in part II of this volume. Of central impor-
tance are neighborhood operations (Chapter 5). Chapter 6 focuses on
the design of filters optimized for a certain purpose. Other subjects of
elementary spatial processing include fast algorithms for local averag-
ing (Chapter 7), accurate and fast interpolation (Chapter 8), and image
warping (Chapter 9) for subpixel-accurate signal processing.
The basic goal of signal processing in computer vision is the extrac-
tion of “suitable features” for subsequent processing to recognize and
classify objects. But what is a suitable feature? This is still less well de-
fined than in other applications of signal processing. Certainly a math-
ematically well-defined description of local structure as discussed in
Chapter 10 is an important basis. The selection of the proper scale for
image processing has recently come into the focus of attention (Chap-
ter 11). As signals processed in computer vision come from dynam-
ical 3-D scenes, important features also include motion (Chapters 13
and 14) and various techniques to infer the depth in scenes includ-
ing stereo (Chapters 17 and 18), shape from shading and photometric
stereo (Chapter 19), and depth from focus (Chapter 20).
There is little doubt that nonlinear techniques are crucial for fea-
ture extraction in computer vision. However, compared to linear filter
techniques, these techniques are still in their infancy. There is also no
single nonlinear technique but there are a host of such techniques often
specifically adapted to a certain purpose [7]. In this volume, a rather
general class of nonlinear filters by combination of linear convolution
and nonlinear point operations (Chapter 10), and nonlinear diffusion
filtering (Chapter 15) are discussed.

1.2 Pattern recognition for computer vision

In principle, pattern classification is nothing complex. Take some ap-
propriate features and partition the feature space into classes. Why is
it then so difficult for a computer vision system to recognize objects?
The basic trouble is related to the fact that the dimensionality of the in-
put space is so large. In principle, it would be possible to use the image
itself as the input for a classification task, but no real-world classifi-
cation technique—be it statistical, neuronal, or fuzzy—would be able
to handle such high-dimensional feature spaces. Therefore, the need
arises to extract features and to use them for classification.
Unfortunately, techniques for feature selection have widely been ne-
glected in computer vision. They have not been developed to the same
degree of sophistication as classification where it is meanwhile well un-

4 1 Introduction

derstood that the different techniques, especially statistical and neural
techniques, can been considered under a unified view [8].
Thus part IV of this volume focuses in part on some more advanced
feature-extraction techniques. An important role in this aspect is played
by morphological operators (Chapter 21) because they manipulate the
shape of objects in images. Fuzzy image processing (Chapter 22) con-
tributes a tool to handle vague data and information.
The remainder of part IV focuses on another major area in com-
puter vision. Object recognition can be performed only if it is possible
to represent the knowledge in an appropriate way. In simple cases the
knowledge can just be rested in simple models. Probabilistic model-
ing in computer vision is discussed in Chapter 26. In more complex
cases this is not sufficient. The graph theoretical concepts presented
in Chapter 24 are one of the bases for knowledge-based interpretation
of images as presented in Chapter 27.

1.3 Computational complexity and fast algorithms

The processing of huge amounts of data in computer vision becomes a
serious challenge if the number of computations increases more than
linear with the number of data points, M = N D (D is the dimension
of the signal). Already an algorithm that is of order O(M 2 ) may be
prohibitively slow. Thus it is an important goal to achieve O(M) or at
least O(M ld M) performance of all pixel-based algorithms in computer
vision. Much effort has been devoted to the design of fast algorithms,
that is, performance of a given task with a given computer system in a
minimum amount of time. This does not mean merely minimizing the
number of computations. Often it is equally or even more important
to minimize the number of memory accesses.
Point operations are of linear order and take cM operations. Thus
they do not pose a problem. Neighborhood operations are still of lin-
ear order in the number of pixels but the constant c may become quite
large, especially for signals with high dimensions. This is why there is
already a need to develop fast neighborhood operations. Brute force
implementations of global transforms such as the Fourier transform re-
quire cM 2 operations and can thus only be used at all if fast algorithms
are available. Such algorithms are discussed in Section 3.4. Many other
algorithms in computer vision, such as correlation, correspondence
analysis, and graph search algorithms are also of polynomial order,
some of them even of exponential order.
A general breakthrough in the performance of more complex al-
gorithms in computer vision was the introduction of multiresolutional
data structures that are discussed in Chapters 4 and 14. All chapters

1.4 Performance evaluation of algorithms 5

about elementary techniques for processing of spatial data (Chapters 5–
10) also deal with efficient algorithms.

1.4 Performance evaluation of algorithms

A systematic evaluation of the algorithms for computer vision has been
widely neglected. For a newcomer to computer vision with an engi-
neering background or a general education in natural sciences this is a
strange experience. It appears to him as if one would present results
of measurements without giving error bars or even thinking about pos-
sible statistical and systematic errors.
What is the cause of this situation? On the one side, it is certainly
true that some problems in computer vision are very hard and that it
is even harder to perform a sophisticated error analysis. On the other
hand, the computer vision community has ignored the fact to a large
extent that any algorithm is only as good as its objective and solid
evaluation and verification.
Fortunately, this misconception has been recognized in the mean-
time and there are serious efforts underway to establish generally ac-
cepted rules for the performance analysis of computer vision algorithms.
We give here just a brief summary and refer for details to Haralick et al.
[9] and for a practical example to Volume 3, Chapter 7. The three major
criteria for the performance of computer vision algorithms are:
Successful solution of task. Any practitioner gives this a top priority.
But also the designer of an algorithm should define precisely for
which task it is suitable and what the limits are.
Accuracy. This includes an analysis of the statistical and systematic
errors under carefully defined conditions (such as given signal-to-
noise ratio (SNR), etc.).
Speed. Again this is an important criterion for the applicability of an
algorithm.
There are different ways to evaluate algorithms according to the fore-
mentioned criteria. Ideally this should include three classes of studies:
Analytical studies. This is the mathematically most rigorous way to
verify algorithms, check error propagation, and predict catastrophic
failures.
Performance tests with computer generated images. These tests are
useful as they can be carried out under carefully controlled condi-
tions.
Performance tests with real-world images. This is the final test for
practical applications.

6 1 Introduction

Much of the material presented in this volume is written in the spirit
of a careful and mathematically well-founded analysis of the methods
that are described although the performance evaluation techniques are
certainly more advanced in some areas than in others.

1.5 References
[1] Oppenheim, A. V. and Schafer, R. W., (1989). Discrete-time Signal Process-
ing. Prentice-Hall Signal Processing Series. Englewood Cliffs, NJ: Prentice-
Hall.
[2] Proakis, J. G. and Manolakis, D. G., (1992). Digital Signal Processing. Prin-
ciples, Algorithms, and Applications. New York: McMillan.
[3] Madisetti, V. K. and Williams, D. B. (eds.), (1997). The Digital Signal Pro-
cessing Handbook. Boca Raton, FL: CRC Press.
[4] Poularikas, A. D. (ed.), (1996). The Transforms and Applications Handbook.
Boca Raton, FL: CRC Press.
[5] Lim, J. S., (1990). Two-dimensional Signal and Image Processing. Englewood
Cliffs, NJ: Prentice-Hall.
[6] Granlund, G. H. and Knutsson, H., (1995). Signal Processing for Computer
Vision. Norwell, MA: Kluwer Academic Publishers.
[7] Pitas, I. and Venetsanopoulos, A. N., (1990). Nonlinear Digital Filters. Prin-
ciples and Applications. Norwell, MA: Kluwer Academic Publishers.
[8] Schürmann, J., (1996). Pattern Classification, a Unified View of Statistical
and Neural Approaches. New York: John Wiley & Sons.
[9] Haralick, R. M., Klette, R., Stiehl, H.-S., and Viergever, M. (eds.), (1999). Eval-
uation and Validation of Computer Vision Algorithms. Boston: Kluwer.

2 Continuous and Digital Signals
Bernd Jähne
Interdisziplinäres Zentrum für Wissenschaftliches Rechnen (IWR)
Universität Heidelberg, Germany

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Continuous signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Types of signals . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.2 Uniﬁed description . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Multichannel signals . . . . . . . . . . . . . . . . . . . . . 12
2.3 Discrete signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 Regular two-dimensional lattices . . . . . . . . . . . . . 13
2.3.2 Regular higher-dimensional lattices . . . . . . . . . . . . 16
2.3.3 Irregular lattices . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3.4 Metric in digital images . . . . . . . . . . . . . . . . . . . . 17
2.3.5 Neighborhood relations . . . . . . . . . . . . . . . . . . . 19
2.3.6 Errors in object position and geometry . . . . . . . . . 20
2.4 Relation between continuous and discrete signals . . . . . . . 23
2.4.1 Image formation . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.2 Sampling theorem . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.3 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.4 Reconstruction from samples . . . . . . . . . . . . . . . 28
2.5 Quantization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.5.1 Equidistant quantization . . . . . . . . . . . . . . . . . . . 30
2.5.2 Unsigned or signed representation . . . . . . . . . . . . 31
2.5.3 Nonequidistant quantization . . . . . . . . . . . . . . . . 32
2.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

9
Handbook of Computer Vision and Applications Copyright © 1999 by Academic Press
Volume 2 All rights of reproduction in any form reserved.
Signal Processing and Pattern Recognition ISBN 0–12–379772-1/$30.00

10 2 Continuous and Digital Signals

2.1 Introduction

Images are signals with two spatial dimensions. This chapter deals
with signals of arbitrary dimensions. This generalization is very useful
because computer vision is not restricted solely to 2-D signals. On the
one hand, higher-dimensional signals are encountered. Dynamic scenes
require the analysis of image sequences; the exploration of 3-D space
requires the acquisition of volumetric images. Scientific exploration of
complex phenomena is significantly enhanced if images not only of a
single parameter but of many parameters are acquired. On the other
hand, signals of lower dimensionality are also of importance when a
computer vision system is integrated into a larger system and image
data are fused with time series from point measuring sensors.
Thus this chapter deals with continuous (Section 2.2) and discrete
(Section 2.3) representations of signals with arbitrary dimensions. While
the continuous representation is very useful for a solid mathematical
foundation of signal processing, real-world sensors deliver and digital
computers handle only discrete data. Given the two representations,
the relation between them is of major importance. Section 2.4 dis-
cusses the spatial and temporal sampling on signals while Section 2.5
treats quantization, the conversion of a continuous signal into digital
numbers.

2.2 Continuous signals

2.2.1 Types of signals

An important characteristic of a signal is its dimension. A zero-dimen-
sional signal results from the measurement of a single quantity at a
single point in space and time. Such a single value can also be averaged
over a certain time period and area. There are several ways to extend
a zero-dimensional signal into a 1-D signal (Table 2.1). A time series
records the temporal course of a signal in time, while a profile does the
same in a spatial direction or along a certain path.
A 1-D signal is also obtained if certain experimental parameters of
the measurement are continuously changed and the measured parame-
ter is recorded as a function of some control parameters. With respect
to optics, the most obvious parameter is the wavelength of the electro-
magnetic radiation received by a radiation detector. When radiation is
recorded as a function of the wavelength, a spectrum is obtained. The
wavelength is only one of the many parameters that could be consid-
ered. Others could be temperature, pressure, humidity, concentration
of a chemical species, and any other properties that may influence the
measured quantity.

2.2 Continuous signals 11

Table 2.1: Some types of signals g depending on D parameters

D Type of signal Function

0 Measurement at a single point in space and time g
1 Time series g(t)
1 Profile g(x)
1 Spectrum g(λ)
2 Image g(x, y)
2 Time series of profiles g(x, t)
2 Time series of spectra g(λ, t)
3 Volumetric image g(x, y, z)
3 Image sequence g(x, y, t)
3 Hyperspectral image g(x, y, λ)
4 Volumetric image sequence g(x, y, z, t)
4 Hyperspectral image sequence g(x, y, λ, t)
5 Hyperspectral volumetric image sequence g(x, y, z, λ, t)

With this general approach to multidimensional signal processing,
it is obvious that an image is only one of the many possibilities of a
2-D signal. Other 2-D signals are, for example, time series of profiles or
spectra. With increasing dimension, more types of signals are possible
as summarized in Table 2.1. A 5-D signal is constituted by a hyperspec-
tral volumetric image sequence.

2.2.2 Unified description

Mathematically all these different types of multidimensional signals can
be described in a unified way as continuous scalar functions of multiple
parameters or generalized coordinates qd as

g(q) = g(q1 , q2 , . . . , qD ) with q = [q1 , q2 , . . . , qD ]T (2.1)

that can be summarized in a D-dimensional parameter vector or gen-
eralized coordinate vector q. An element of the vector can be a spatial
direction, the time, or any other parameter.
As the signal g represents physical quantities, we can generally as-
sume some properties that make the mathematical handling of the sig-
nals much easier.

Continuity. Real signals do not show any abrupt changes or discon-
tinuities. Mathematically this means that signals can generally be re-
garded as arbitrarily often differentiable.