Slides for a course on signal and image processing.

1. Approximation and Coding
with Orthogonal
Decompositions
Gabriel Peyré
http://www.ceremade.dauphine.fr/~peyre/numerical-tour/

2. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons

3. Sparse Approximation in a Basis

4. Sparse Approximation in a Basis

5. Sparse Approximation in a Basis

6. Hard Thresholding

7. Approximation Speed
Approximation error decay:
(usually polynomial)

8. Approximation Speed
Approximation error decay:
(usually polynomial)
fM ||)
log10 (||f
Log/Log plot: approx. a ne curve
log10 (M/N )

9. Efficiency of Transforms
fM ||)
log10 (||f
Fourier DCT
log10 (M/N )
Local DCT Wavelets

10. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons

11. Compression by Transform-coding
forward
f
transform
a[m] = ⇥f, m⇤ R
Image f Zoom on f

12. Compression by Transform-coding
forward
f
transform
a[m] = ⇥f, m⇤ R q[m] Z
bin T
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Image f Zoom on f Quantized q[m]

13. Compression by Transform-coding
forward codi
f
transform
a[m] = ⇥f, m⇤ R q[m] Z ng
bin T
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Entropic coding: use statistical redundancy (many 0’s).
Image f Zoom on f Quantized q[m]

14. Compression by Transform-coding
forward codi
f
transform
a[m] = ⇥f, m⇤ R q[m] Z ng
bin T
ing
decod
dequantization
a[m]
˜ q[m] Z
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Entropic coding: use statistical redundancy (many 0’s).
Dequantization:
Image f Zoom on f Quantized q[m]

15. Compression by Transform-coding
forward codi
f
transform
a[m] = ⇥f, m⇤ R q[m] Z ng
bin T
ing
decod
backward dequantization
fR = a[m]
˜ m a[m]
˜ q[m] Z
transform
m IT
⇥ 2T T T 2T a[m
|a[m]|
Quantization: q[m] = sign(a[m]) Z
T a[m]
˜
Entropic coding: use statistical redundancy (many 0’s).
Dequantization:
Image f Zoom on f Quantized q[m] fR , R =0.2 bit/pixel

16. Thresholding vs. Quantizing

17. Non-linear Approximation and Compression

18. Non-linear Approximation and Compression
2T T T 2T a[m]
a[m]
˜
⇥
|a[m]|
Quantization: q[m] = sign(a[m]) Z T
T =⇥ |a[m] a[m]|
˜
⇥
1 2
Dequantization: a[m] = sign(q[m]) |q[m]| +
˜ T
2

19. Non-linear Approximation and Compression
2T T T 2T a[m]
a[m]
˜
⇥
|a[m]|
Quantization: q[m] = sign(a[m]) Z T
T =⇥ |a[m] a[m]|
˜
⇥
1 2
Dequantization: a[m] = sign(q[m]) |q[m]| +
˜ T
2
⇤ ⇤ ⇤ ⇥2
T
||f fR || =
2
(a[m] a[m])
˜ 2
|a[m]| + 2
m
2
|a[m]|<T |a[m]| T
||f fM ||2 #=M
Theorem: ||f fR ||2 ||f fM ||2 + M T 2 /4
where M = # {m a[m] = 0}.
˜

20. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .

21. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .

22. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
Simple coding strategy: R = Rval + Rind
⇥
N
= Rind log2 = O(M log2 (N/M )) = O(M log2 (M )).
M

23. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
Simple coding strategy: R = Rval + Rind
⇥
N
= Rind log2 = O(M log2 (N/M )) = O(M log2 (M )).
M
= Rval = O(M | log2 (T )|) = O(M log2 (M ))

24. A Naive Support Coding Approach
Coding: relate # bits R to # coe cients M .
+1
(H1 ) Ordered coe cients | f, m ⇥| decays like m 2 .
=⇤ ||f fM ||2 ⇥ M .
f RN sampled from f0 , error: ||f f0 ||2 ⇥ N
(H2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2 : M ⇥ N ⇥/ .
Simple coding strategy: R = Rval + Rind
⇥
N
= Rind log2 = O(M log2 (N/M )) = O(M log2 (M )).
M
= Rval = O(M | log2 (T )|) = O(M log2 (M ))
Theorem: Under hypotheses (H1 ) and (H2 ), ||f fR ||2 = O(R log (R)).

25. Entropic Coders

26. Entropic Coders

27. Entropic Coders

28. JPEG-2000 Overview

29. JPEG-2000 Overview

30. JPEG-2000 Overview

31. JPEG-2000 Overview

32. JPEG-2000 Overview

33. Contextual Coding
3 × 3 context window
code block width

34. JPEG-2000 vs. JPEG, 0.2bit/pixel

35. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons

36. 1D Fourier Approximation

37. 1D Fourier Approximation

38. Sobolev and Fourier

39. Singularities and Fourier
1
0.8
0.6
0.4
0.2
0
1
0.8
0.6
0.4
0.2
0
1
0.8
0.6
0.4
0.2
0
1
0.8
0.6
0.4
0.2
0

40. Sobolev for Images

41. Sobolev for Images

42. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons

43. Magnitude of Wavelet Coefficients
2
p=2
Vanishing moments: k < p, (x)x dx = 0
k
1
0
−1
−2
−1 0 1 2
2
p=3
1
0
−1
−2 −1 0 1 2
1.5
p=4
1
0.5
f (x) 0
−0.5
−1
−2 0 2 4

44. Magnitude of Wavelet Coefficients
2
p=2
Vanishing moments: k < p, (x)x dx = 0
k
1
x 2j n
If f is C on supp(⇥j,n ), p : t= 0
2j
⇤ ⇥ ⇤
1 x 2 n j
d
−1
⇥f, j,n ⇤ = d f (x) dx = 2j 2 R(2j t) (t)dt
2j 2 2j −2
−1 0 1 2
2
f (x) = P (x 2 n) + R(x
j
2 n) = P (2 t) + R(2 t)
j j j
p=3
1
0
| f, j,n ⇥| Cf || ||1 2j( +d/2)
−1
−2 −1 0 1 2
1.5
p=4
1
0.5
f (x) 0
−0.5
−1
−2 0 2 4

45. Magnitude of Wavelet Coefficients
2
p=2
Vanishing moments: k < p, (x)x dx = 0
k
1
x 2j n
If f is C on supp(⇥j,n ), p : t= 0
2j
⇤ ⇥ ⇤
1 x 2 n j
d
−1
⇥f, j,n ⇤ = d f (x) dx = 2j 2 R(2j t) (t)dt
2j 2 2j −2
−1 0 1 2
2
f (x) = P (x 2 n) + R(x
j
2 n) = P (2 t) + R(2 t)
j j j
p=3
1
0
d
| f, j,n ⇥| Cf || ||1 2j( +d/2)
| f, j,n ⇥| ||f || || ||1 2j 2
−1
−2 −1 0 1 2
1.5
p=4
1
0.5
f (x) 0
−0.5
−1
−2 0 2 4

46. 1D Wavelet Coefficient Behavior
1
0.8
0.6
0.4
0.2
0
0.2
0.1
0
−0.1
−0.2
0.2
0.1
0
−0.1
−0.2
0.5
0
−0.5
0.5
0
−0.5

47. 1D Wavelet Coefficient Behavior
1
0.8
0.6
0.4
0.2
0
0.2
0.1
0
−0.1
−0.2
0.2
If f is C in supp( j,n ), then 0.1
0
2 j( +1/2) −0.1
| f, j,n ⇥| ||f ||C || ||1 −0.2
0.5
0
−0.5
0.5
0
−0.5

48. 1D Wavelet Coefficient Behavior
1
0.8
0.6
0.4
0.2
0
0.2
0.1
0
−0.1
−0.2
0.2
If f is C in supp( j,n ), then 0.1
0
2 j( +1/2) −0.1
| f, j,n ⇥| ||f ||C || ||1 −0.2
0.5
If f is bounded (e.g. around 0
a singularity), then −0.5
0.5
| f, j,n ⇥| 2 j/2
||f || || ||1
0
−0.5

49. Piecewise Regular Functions in 1D
Theorem: If f is C outside a finite set of discontinuities:
O(M 1
) (Fourier),
n
[M ] = ||f fM ||2
n
=
O(M 2
) (wavelets).
For Fourier, linear non-linear, sub-optimal.
For wavelets, linear non-linear, optimal.

50. Examples of 1D Approximations
−1
−1.5
−2
−2.5
−3
−3.5
1
0.8 −4
0.6
−4.5
0.4
−5
0.2
0 −5.5
−6
−2 −1.8 −1.6 −1.4 −1.2 −1 −0.8
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
1 1
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0

51. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant

52. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1/2)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j/2

53. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1/2)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j/2
1
Singular: 2j1
= (T /C) +1/2
Cut-o scales (depends on T ):
Regular: 2j2 = (T /C)2

54. Localizing the Singular Support
S = {x1 , x2 } Small coe cient
f (x) | f, jn ⇥| < T
x1 x2
Large coe cient
j2
j1 Singular support:
|Cj | K|S| = constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1/2)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j/2
1
Singular: 2j1
= (T /C) +1/2
Cut-o scales (depends on T ):
Regular: 2j2 = (T /C)2
Hand-made approximate: ˜
fM = f, j,n ⇥ j,n + f, j,n ⇥ j,n
j j2 n Cj c
j j1 n Cj

55. Computing Error and #Coefficients
f (x) |Cj | K|S| = constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1/2)
j2
j1 n Cj : |⇥f, j,n ⇤| C2j/2
1
2 j1
= (T /C) +1/2
2j2 = (T /C)2
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
(K|S|) C 2 2j + 2 j
C 2 2j(2 +1)
Singulatities
j<j2 j<j1
= O(2 + 2j2 2 j1
) = O(T + T
2 2
+1/2 ) = O(T
2
+1/2 ) regular part

56. Computing Error and #Coefficients
f (x) |Cj | K|S| = constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1/2)
j2
j1 n Cj : |⇥f, j,n ⇤| C2j/2
1
2 j1
= (T /C) +1/2
2j2 = (T /C)2
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
(K|S|) C 2 2j + 2 j
C 2 2j(2 +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T2 2
+1/2 ) = O(T
2
+1/2 ) regular part
M |Cj | + c
|Cj | K|S| + 2 j
j j2 j j1 j j2 j j1
1 1
= O(| log(T )| + T +1/2 ) = O(T +1/2 )

57. Computing Error and #Coefficients
f (x) |Cj | K|S| = constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1/2)
j2
j1 n Cj : |⇥f, j,n ⇤| C2j/2
1
2 j1
= (T /C) +1/2
2j2 = (T /C)2
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
(K|S|) C 2 2j + 2 j
C 2 2j(2 +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T2 2
+1/2 ) = O(T
2
+1/2 ) regular part
M |Cj | + c
|Cj | K|S| + 2 j
j j2 j j1 j j2 j j1
1 1
= O(| log(T )| + T +1/2 ) = O(T +1/2 ) ||f fM ||2 = O(M 2
)

58. 2D Wavelet Approximation
If f is C in supp( j,n ), then
| f, j,n ⇥| 2j( +1)
||f ||C || ||1
If f is bounded (e.g. around a singularity), then
| f, j,n ⇥| 2j ||f || || ||1

59. Piecewise Regular Functions in 2D
Theorem: If f is C outside a set of finite length edge curves,
O(M 1/2
) (Fourier),
n
[M ] = ||f fM ||2
n
=
O(M 1
) (wavelets).
Fourier Wavelet, both sub-optimal.
Wavelets: same result for BV functions (optimal).

60. Example of 2D Approximations

61. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant

62. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j

63. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j
1
Singular: 2j1
= (T /C) +1
Cut-o scales (depends on T ):
Regular: 2j2 = T /C

64. Localizing the Singular Support in 2D
f (x, y) Length(S) = L
j2
j1
|Cj | LK2 j
= constant
Regular, n Cj :
c
|⇥f, j,n ⇤| C2j( +1)
Coe cient behavior:
Singular, n Cj : |⇥f, j,n ⇤| C2j
1
Singular: 2 j1
= (T /C) +1
Cut-o scales (depends on T ):
Regular: 2j2 = T /C
Hand-made approximate: ˜
fM = f, j,n ⇥ j,n + f, j,n ⇥ j,n
j j2 n Cj c
j j1 n Cj

65. Computing Error and #Coefficients
f (x, y) |Cj | LK2 j
= constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1)
j2
n Cj : |⇥f, j,n ⇤| C2j
j1
1
2 j1
= (T /C) +1
2j2 = T /C
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
LK2 j
C 2 22j + 2 2j
C 2 22j( +1)
Singulatities
j<j2 j<j1
= O(2 + 2
j2 2 j1
) = O(T + T
2
+1 ) = O(T ) regular part

66. Computing Error and #Coefficients
f (x, y) |Cj | LK2 j
= constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1)
j2
n Cj : |⇥f, j,n ⇤| C2j
j1
1
2 j1
= (T /C) +1
2j2 = T /C
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
LK2 j
C 2 22j + 2 2j
C 2 22j( +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T
2
+1 ) = O(T ) regular part
M |Cj | + c
|Cj | LK2 j
+ 2 2j
j j2 j j1 j j2 j j1
1
= O(T 1
+T +1 ) = O(T 1
)

67. Computing Error and #Coefficients
f (x, y) |Cj | LK2 j
= constant
n Cj : |⇥f,
c
j,n ⇤| C2j( +1)
j2
n Cj : |⇥f, j,n ⇤| C2j
j1
1
2 j1
= (T /C) +1
2j2 = T /C
||f fM ||2 ||f ˜
fM ||2 |⇥f, j,n ⇤|
2
+ |⇥f, j,n ⇤|
2
j<j2 ,n Cj c
j<j1 ,n Cj
LK2 j
C 2 22j + 2 2j
C 2 22j( +1)
Singulatities
j<j2 j<j1
= O(2 + 2 j2 2 j1
) = O(T + T
2
+1 ) = O(T ) regular part
M |Cj | + c
|Cj | LK2 j
+ 2 2j
j j2 j j1 j j2 j j1
1
= O(T 1
+T +1 ) = O(T 1
) ||f fM ||2 = O(M 1
)

68. Overview
• Approximation and Compression
• Decay of Approximation Error
• Fourier for Smooth Functions
• Wavelet for Piecewise Smooth Functions
• Curvelets and Finite Elements for Cartoons

69. Geometrically Regular Images
Geometric image model: f is C outside a set of C edge curves.
BV image: level sets have finite lengths.
Geometric image: level sets are regular.
| f|
Geometry = cartoon image Sharp edges Smoothed edges

70. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges.
˜
Piecewise linear approximation on M triangles: fM .

71. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges. Regular areas:
M/2 equilateral triangles.
˜
Piecewise linear approximation on M triangles: fM .
1/2
M
1/2
M

72. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges. Regular areas:
M/2 equilateral triangles.
˜
Piecewise linear approximation on M triangles: fM .
1/2
M
1/2
M
Singular areas:
M/2 anisotropic triangles.

73. Geometic Construction : Finite Elements
Approximation of f , C2 outside C2 edges. Regular areas:
M/2 equilateral triangles.
˜
Piecewise linear approximation on M triangles: fM .
1/2
M
1/2
M
Singular areas:
M/2 anisotropic triangles.
Theorem: If f is C2 outside a set of C2 contours, then one has for an adapted
˜
triangulation ||f fM ||2 = O(M 2 ).
Di culties to build e cient approximations.
No optimal strategies (greedy solutions).

74. Greedy Triangulation Optimization
Bougleux, Peyr´, Cohen, ECCV’08
e

75. Curvelet Atoms
[Candes, Donoho] [Candes, Demanet, Ying, Donoho]
Parabolic dyadic scaling:
Rotation:
“width length2 ”

76. Curvelet Tight Frame
Angular sampling:
Spacial sampling:
Tight frame of L2 (R2 ):

77. Curvelet Approximation
M -term curvelet approximation:
Theorem: If f is C2 outside a set of C2 edges, ||f fM ||2 = O(M 2
(log M )3 ).
Discrete curvelets: O(N log(N )) algorithm.
Redundancy 5 =⇥ not e cient for compression.

78. Curvelets Denoising
Works on elongated edges.
Works also on locally parallel textures !

79. Conclusion

80. Conclusion

81. Conclusion
1
0.8
0.6
0.4
0.2
0

82. Conclusion
1
0.8
0.6
0.4
0.2
0