1. Background
Subtraction
Shashank Dhariwal
Manmeet Singh Kapoor
Adham Beyki
1

2. Agenda

Introduction

Challenges

Various Techniques &Comparison

Mixture of Gaussians

Advantages & Disadvantages

Implementation & Results

References
2

3. Introduction
A
computational vision process
 Identifies
frame
A
moving objects from the video
class of techniques for segmentation[1]
3

4. The Problem

Aim: Given a frame sequence from a fixed
camera, detecting all the foreground objects.

Approach: detecting foreground as
difference between the current frame & an
image of the scene’s static background.
| framei – backgroundi| > Th
4

5. Challenges
Illumination
Changes
Motion
Changes
Changes in
Background
Geometry
• Gradual
• Sudden
• Camera Oscillations
• High Frequency Objects
• Parked cars…
5

6. Various Techniques
 Running
Gaussian Average
 Temporal Median Filter
 Mixture of Gaussians(MoG)
 Kernel Density Estimation(KDE)
 Sequential Kernel Density Approximation(SKDA)
 Co-occurrence of Image Variance
 Eigenbackgrounds
6

7. Comparison
Method
Speed
Memory
Accuracy
I
I
Low-Medium
Temporal Median Filter
ns
ns
Low-Medium
Mixture of Gaussians
m
m
High
Kernel Density Estimation
n
n
High
Sequential KD Approximation
m+1
m
Medium-High
Co-occurrence of Image
Variance
8n/N2
nK/N2
Medium
M
n
Medium
Running Gaussian Average
Eigenbackgrounds
Table 1 – Comparison [2]
7

8. Mixture of Gaussian
Deals with
• Repeated motion
• Background clutter
• Long term scene change
8

9. Mixture of Gaussian

Problems with other techniques [3]
1. Averaging images over time
-Not robust to scenes with slow moving objects
-Single Threshold for entire scene
2. Modelling each pixel using Kalman filter
-Not robust to backgrounds with repetitive change
-Takes significant time to re-establish the background
3. Modelling each pixel using single Gaussian
-Good indoor performance
- Not good enough out-door scenes for repetitive
change
9

10. Mixture of Gaussian
 Hypothesis
:
If we model each pixel as a mixture of Gaussians to
determine whether or not a pixel is part of the background, then
we will arrive at an effective approach to separate the
background and foreground, which can be used for real-time
tracking.[ 3]

Used in tracking of moving objects:
1. Separating Foreground from Background (our agenda)
2. Tracking Objects in Foreground (not in the scope)
10

11. Mixture of Gaussian
Fig 1 . [ 3 ]
11

12. Background Subtraction -MoG

Pixel process : It’s the history of the pixel ‘X’’s value
from frame 1 to ‘t’. [4]

Part 1 : For above pixel process of ‘X’ the
probability of observing a ‘X’ at frame ‘t’ is as
follows,
K
= no. of Gaussian in the mixture (generally 3-5). [4]
weight(i,t)
= estimate of the weight of Gaussian in the mixture
means (i,t)
= mean value of the Gaussian at time ‘t’
Covariance(i,t) = Covariance matrix of Gaussian in the matrix
= Variance * Identity matrix
12

13. Background Subtraction -MoG

Decision making : For each Gaussian in the mixture
of pixel X

if pixel X <= 2.5 (probability of .996) standard deviation form
the mean then the Gaussian is said to be ‘matched’
- Increase the weight
- Adjust the mean closer to X(t)
- Decrease the Variance

Else the Gaussian is ‘unmatched’
- Decrease the weight

If all the Gaussians in the mixture for pixel X(t) are unmatched
- Mark X(t) as foreground pixel
- Find the least probable Gaussian in the mixture and replace it with a new
Gaussian with the following parameters:
- Mean = X(t) i.e present value of X
- Variance as a high value
- Weight as a low value
13

14. Background Subtraction -MoG

Part 2 : Updating of parameters and using a suitable
heuristic for distributions that represent background
pixels
Based on the decision made , change the following parameters using the
equations given below:
Where
- α (alpha) is the learning parameter.
- M (i,t) value is set to 1 for model that matched and 0 for rest
- µ(mean) and σ (Std Deviation) for unmatched remain same
and changes for the matched distributions
- ρ is the updating parameter
14

15. Background Subtraction -MoG

Advantages :



Robust against movement that are part of
background, e.g moving branches of a tree
Robust against rain , snow, etc….
Disadvantages:




Not a good subtraction when shadows are there
Difficulty with objects overlapping
Fast lighting changes were also an issue.
Gives false positives
15

16. Implementation
Outdoor Scene
Moving cars
Stopped cars
Pedestrians
Luminosity changes
Shadows
Leaves and branches of trees
16

17. Implementation
Frame Difference
as the easiest method
o Objects with uniformly
distributed intensity
o Objects must be moving all the
time!
 Computationally cheap
 Highly adaptive background
model
 Tuning threshold value (=25 for
our example)
17

18. Implementation
MoG
a complex method
o Parametric model
o Mixture of Gaussian
components
o Comparing pixel value with
tracking Gaussian components
 Very good at separating objects
 Suppressing background noise
 Parameter optimisation
 Not quickly enough adaptive
background model
18

19. Implementation
19

20. References
1.
2.
3.
4.
5.
6.
7.
Tarun Baloch, MSc Thesis ‘Background Subtraction in Highly Illuminated Indoor
Environment’ Indian Institute of Technology, 2010
M. Piccardi. Background Subtraction techniques : A Review . In IEEE International
Conference on Systems, Man and Cybernetics, 2004, Volume 4, pages 3099–
3104, 2005. URL http://dx.doi.org/10.1109/ICSMC.2004.1400815.
Eric Thul , ECSE-626 Final Project: An evaluation of Chris Stauffer and W. E. L.
Grimson’s
method
for
background
subtraction,
2008,
www.cs.mcgill.ca/~ethul/pub/course/ecse626/project-report.pdf
C. Stauffer and W. E. L. Grimson. Adaptive background mixture models for realtime tracking. IEEE Computer Society Conference on Computer Vision and
Pattern
Recognition,
1999,
2:252,
1999.
URL
http://doi.ieeecomputersociety.org/10.1109/CVPR.1999.784637.
A A Mazeed, Mark Nixon and Steve Gunn, Classifiers Combination for Improved
Motion Segmentation, 2004,eprints.ecs.soton.ac.uk › ECS › Research ›
Publications
Omar Javed, Khurram Shafique and Mubarak Shah, A hierarchical approach to
robust background subtraction using colour and gradient information,2002,
visionnas2.cs.ucf.edu/papers/javed_wmvc_2002.pdf
Sen-Ching S.Chung and Chandrika Kamath, Robust techniques for background
subtraction in urban traffic video, 2004,
www.llnl.gov/casc/sapphire/pubs/UCRL-CONF-200706.pdf
20