Separable bilateral filtering for fast video preprocessing
1. ICME 2005
IEEE International Conference on Multimedia & Expo
July 6-8, 2005, Amsterdam, The Netherlands
Separable bilateral filtering
for fast video preprocessing
Tuan Q. Pham & Lucas J. van Vliet
Delft University of Technology
The Netherlands 1
Quantitative Imaging Group
Faculty of Applied Sciences
2. Contents
1. Bilateral filtering:
Edge-preserving filter
High computational complexity
2. Separable implementation:
Good approximation of the original filter
Linear computational complexity
3. Application to video preprocessing:
Noise reduction
Better compressed video
3. Gaussian filtering revisited
Gaussian filtering: weights depend on distance to the center pixel
* =
Noisy step edge Gaussian weights Gaussian filtered result
Adaptive filtering: avoid filtering across edges
. =
Noisy step edge Edge-preserving weights Edge-presered filtered result
4. Gaussian vs bilateral filtering
Gaussian filtering: weights depend on distance to the center pixel
* =
Noisy step edge Gaussian weights Gaussian filtered result
Bilateral filtering: weights depend on both spatial closeness and
photometric similarity
. =
Noisy step edge Edge-preserving weights Edge-presered filtered result
5. How bilateral filtering works?
Every sample is a weighted average of its neighbors:
1
O ( s0 ) = ∑ w ( s, s0 ) . I ( s )
K s
Local modes
The weight w = w s . wt is product of two Gaussian weights: after
bilateral
− ( s − s0 ) 2 filtering
Spatial proximity: w s = exp
2σ s2
− ( I ( s ) − I ( s0 ) ) 2
Tonal similarity: wt = exp
2σ t 2
8. Computational complexity
Bilateral filtering kernel is space-variant → complexity is:
O ( Nm d )
where N: number of pixels in the image
m: size of filtering kernel (m ≈ 7 is good enough)
d: image dimensionality
Previous attempt for fast bilateral filtering:
Piecewise-linear: approximate bilateral filtering with M Gaussian
filtering - Durand & Dorsey (SIGGRAPH 2002) → complexity is:
O ( N log( N ) . M ) M ≈ 17 for 8-bit images
Our approach: separable bilateral filtering
O ( Nmd )
9. Is bilateral filter separable?
Gaussian filter is space-invariant and separable:
g(x)
g(y)
∑ w ( s − s ) . I ( s)
s 0
O( s ) = s
* =
∑ w (s − s )
0
s 0
s
y
x
w s ( s ) = gauss( x ) . gauss( y )
Kernel center
Bilateral filter is NOT separable: I(s0)
Space-variant kernel due to local I(s)
intensity dependency
However, even a highly non-linear filter like median filter is
approximately separable (Narendra – PAMI 1981)
10. Separable bilateral filtering result
Separable bilateral filtering is a good approximation of full kernel filtering:
noisy Erika σnoise = 10 bilateral filtered in x-dimension followed by y-dimension filtering
11. Separable bilateral filtering result
Separable bilateral filtering is a good approximation of full kernel filtering:
noisy Erika σnoise = 10 bilateral filtered in x-dimension followed by y-dimension filtering
Image size Brute-force Durrand 2002 Separable Aniso. diffusion
256x256 4.46 0.37 0.21 2.76
512x512 17.88 1.59 0.89 26.02
61x61x61 5.29 4.20 0.45 5.25
256x256x212 56 min Out of memory 50.3 Out of memory
12. How separable bilateral filtering works?
Pros: extremely fast (fixed spatial weight + LUT for tonal weight)
Cons: effective filtering kernel is a slightly distorted
13. Performance of separable bilateral
filtering
Very good approximation of Almost linear execution time
the full-kernel filter per pixel
14. MPEG-1 Foreman with bilateral preprocessing
16 without preprocessing
with 3x3x3 full-kernel
with 9x9x5 separable
15
RMSE
14
13
0 400 800 1200
bit-rate (K bits/s)
Better RMSE is achieved with separable bilateral filtering compared to
full-kernel bilateral filtering with the same computation requirement
16. Conclusions
Separable implementation of bilateral filtering:
Very good approximation of the original filter
Much faster than the original or other approximations
Applications in video preprocessing:
Improved quality of compressed video
Reduced processing time → real-time possibility
17. Literature
C. Tomasi and R. Manduchi, Bilateral fitering for gray and color
images, Proc. of ICCV, USA, 1998, 839-846.
T.Q. Pham and L.J. van Vliet, Separable bilateral filtering for fast
video preprocessing, Proc. of ICME’05.
F. Durrand and J. Dorsey, Fast bilateral filtering for the display of
high dynamic range images, Proc. of SIGGRAPH’02, 2002, 844-847.
P. Perona and J. Malik, Scale-space filtering and edge detection
using anisotropic diffusion, PAMI, vol. 12, no. 7, 1990, 629-639.
R. v.d. Boomgaard and J. v.d. Weijer. On the equivalence of local-
mode finding, robust estimation and mean-shift analysis as used in
early vision tasks. In Proc. of ICPR, pages 927-930, 2002.