Post on 07-Feb-2018
7/22/2019 BS Presentation1
1/20
In The Name Of God,The Most Beneficent, The Most Merciful
Dynamic Background Generation
Presented By: Bijan Shoushtarian
Computer Engineering DepartmentThe University of Isfahan
1392/6/23
7/22/2019 BS Presentation1
2/20
Common Approaches ToMoving Target Detection
1. Optical Flow
2. Temporal Differencing
3. Background Subtraction
7/22/2019 BS Presentation1
3/20
Optical FlowIf an observer (a camera or a human eye) moves in a 3D scene, the patternof motion of objects, surfaces, and edges in a visual scene caused by therelative motion between the observer and the scene is called optical flow .
(a) A Rubik's cube on a rotating turntable;(b) Flow vectors calculated from comparingthe two images of a Rubik's cube (Russelland Norvig, 1995)
(a)
(b)
7/22/2019 BS Presentation1
4/20
Optical Flow Advantages andDisadvantages
Target detection and tracking can beperformed using optical flow even thoughthere is no prior knowledge about thebackground or when the camera is moving.
Optical flow methods need very complexcomputations which cannot be coded inreal-time algorithms without specialisedhardware.
Optical flow methods are sensitive to noise.Thus, many researchers usually do not prefer usingoptical flow methods for implementing real-timebackground generation techniques.
7/22/2019 BS Presentation1
5/20
Temporal Differencing In temporal differencing method, the arithmetic difference ofcorresponding pixels in the same physical locations in two (consecutiveor non-consecutive) frames of an image sequence is obtained. Thedifference image contains non-zero values whenever objects have movedto another location but will be black when no moving object is detected.
The problems with consecutive frames are:
1. A pre-selected threshold is necessary to obtain the thresholdeddifference image. As a result, this method will be dependent on thevideo sequence and selected threshold.
2. If an object moves too slowly so that it is stationary in twoconsecutive frames, the object will not appear in the difference frame.
3. Extending the method to use more than two frames is difficult.
7/22/2019 BS Presentation1
6/20
Background Subtraction
The most important approach to identify and segment movingobjects in a video stream is background subtraction , which involvescomputing a reference image for each new frame. Then, bycomparing the next input frame with the reference image, regions ofthe image which have changed are identified. Thresholding the resultproduces a binary segmentation which is used for discriminating themoving objects from the background regions.
D(x, y, t) = | I(x, y, t) B(x, y, t-1) |T(x, y, t) = Thresholded (D(x, y, t))
Due to several factors, the background image must be temporallyadaptive and it should be updated continuously in order to be kept
up to date.
7/22/2019 BS Presentation1
7/20
Background SubtractionExample
Fld249 Backgnd248
Dif_249 Threshld_249 Thresh_Sizefilt249
7/22/2019 BS Presentation1
8/20
Background Generation
Requirements Illumination changes
Gradual variations in lighting conditions.
Sudden illumination changes (such as turning alamp on or off in an indoor environment).
Motion changes
Camera oscillations.Small movements of background objects suchas trees waving in the wind or sea waves, rain,snow, etc.
7/22/2019 BS Presentation1
9/20
Background Generation
Requirements (continued) Changes in the background geometry
Introducing or removing objects in the scene
(such as a door is opened and then is leftopened or a parked car is moved on).
An initialisation process is required by a
number of algorithms. These problems and requirements are the constraintswhich should be considered as important characteristics byadaptive background removal algorithms.
7/22/2019 BS Presentation1
10/20
Background Removal
Algorithms Background removal (subtraction) methods can be classified into two broadcategories: non-recursive and recursive.
Non-recursive techniques A non-recursive technique uses a sliding-window approach which maintains a buffer tohold N previous input frames. The background image is estimated based on the temporalvariations of each pixel in the buffered frames. A significant storage requirement isconsidered as the weakness of these techniques.
Recursive techniquesInstead of using a buffer, in recursive techniques, a single background model is updatedusing each input frame recursively. Often more weights are given to most recent samplesand as a result, input frames from past distance usually have less effect on the currentbackground model. In comparison with non-recursive techniques, recursive techniquesrequire much less storage. However, if an error object suddenly appears in thebackground image, it may remain in the background for a much longer period of time.
7/22/2019 BS Presentation1
11/20
Non-recursive Techniques Frame differencingMean filterMode filterMedian filterBasic methods (i.e. mean, mode and median) withselectivity
Each pixel in every input frame is classified as either a foreground
or a background pixel.If the pixel is detected as a foreground point, it is ignored in thebackground update process.
Least Median of SquaresBt (x, y) = min b median t (I t (x, y) b) 2
7/22/2019 BS Presentation1
12/20
Non-recursive Techniques(continued)
Linear predictive filterThe background is modelled by:1. Pixel-by-pixel linear prediction ( Weiner filter) using colour information2. Region-level by region filling algorithm for dealing with background object relocation
problem3. Frame-level by model switching for detecting global illumination changes
Non-parametric background modelBackground pdf is obtained by the histogram of the N most recent pixel values asfollows:
where K (.) is a smoothing Gaussian kernel estimator. The pixel I t (x, y) is consideredas a foreground point if there is little possibility that f (I t (x, y)) has such a distribution,
i.e., f (I t (x, y)) is smaller than one global threshold value
.
7/22/2019 BS Presentation1
13/20
Non-recursive Techniques
(continued) Standard mean-shift based estimationIn this gradient-ascent method, the modes of a multimodal distribution are detectedusing their covariance matrix. The method uses an iterative approach so its step isdecreased until it is converged. For n data points x i , i = 1 . . . n in the d -dimensionalspace Rd , the multivariate mean shift vector computed with kernel g in the point x isgiven by:
where h is the kernel bandwidth.The major problems with the standard mean-shift method are that the algorithm isvery slow and requires the amount of N (i.e. buffer length) * size (frame) memory.
Eigenbackground subtraction
7/22/2019 BS Presentation1
14/20
Recursive Techniques Running averageBt+1 (x, y) = * I t (x, y) + (1 ) * Bt (x, y)where 0 < < 1. is called the learning rate and is typically 0.05.
Running average with selectivityBt+1 (x, y) = * It (x, y) + (1 ) * B t (x, y) : if I t (x, y) is a background pixelBt+1 (x, y) = B t (x, y) : if I t (x, y) is a foreground pixel
Approximated median filterThe running estimate of the median is incremented by one if the input pixel islarger than the estimate, and decreased by one if smaller. This estimate eventuallyconverges to a value for which half of the input pixels are larger than and half aresmaller than this value, that is, the median.
Kalman filter
7/22/2019 BS Presentation1
15/20
Recursive Techniques(continued)
Running Gaussian averageIt models the background as a textured surface, each point of which is associated with a
mean colour and a variance about the mean. It fits one Gaussian distribution ( , ) overthe histogram which gives the p .d .f of the background. It also applies the runningaverage to update the background p .d .f as follows:
The method uses a threshold for partitioning the background pixels into visible andoccluded points by examining the test: | I t (x, y) t (x, y) | > , where = k .
7/22/2019 BS Presentation1
16/20
Recursive Techniques(continued)
Mixture of GaussiansThe intensity of each background pixel is adaptively represented by the summation of kweighted Gaussians. The MOG model maintains a density function for each pixel and as aresult is capable of handling multimodal background distributions. The number of modes
(i.e. k) is usually predefined from 3 to 5. The pixel distribution is modelled as amixture of k Gaussians:
where is the i -th Gaussian component with intensity mean and standarddeviation . is the portion of the data accounted for by the i -th component. For eachinput pixel I t (x, y), the component whose mean is closest to I t (x, y) is declared as thematched component if .
At every new frame, the parameters of the matched component are then updated as
where is a user- defined learning rate with 0 1. is the learning rate for theparameters and can be approximated by / .
f (I t (x, y) = u )
7/22/2019 BS Presentation1
17/20
Recursive Techniques(continued)
In order to determine whether I t (x, y) is a foreground or background pixel, all components areranked by their . If i 1, i 2 , ., i k is the component order after sorting, the first M components thatsatisfy the following criterion are declared to be the background components:
The advantages of Gaussian mixture models (GMMs) are :Dealing with lighting changesslow-moving objectsintroducing or removing objects from the scene.
The drawbacks of GMMs are :The number of the mixture components is pre-set and fixed-value.GMM approach for foreground segmentation is a time-consuming process due to estimatingthe number of parameters which are mostly determined by the number of mixture components.
Also the application of GMMs for background subtraction requires an efficient method forlearning the GMM parameters which are computationally expensive.Therefore, the selection of the number of components and the initialisation process are two
important problems of the GMM algorithm for background subtraction.
where is the weight threshold.
7/22/2019 BS Presentation1
18/20
Recursive Techniques(continued)
Sequential kernel density approximationThe density is represented by a weighted sum of Gaussians, whose number, weights,means and covariances are updated at each time step to include the new data into themodel. For each mode, a Gaussian component is created whose mean is given by themode location. The covariance of the Gaussian is also derived from the Hessian matrixwhich is computed at the mode location. It relies on the modelling and density modeswhich are propagated by adapting them with the new samples as follows:
Pdf ( x ) = (new_mode ) + (1 ) (existing_modes )
7/22/2019 BS Presentation1
19/20
Comparison of Background Removal Algorithms
Method Non-Recursive(N) or Recursive(R)
Real-time RequiredMemory
Frame Differencing N Yes Low
Mean Filter N Yes High
Mode Filter N Yes High
Median Filter N Yes High
Basic methods withselectivity N Yes High
Least Median ofSquares (LMedS) N Yes High
Linear PredictiveFilter N Yes Intermediate
Non-parametricBackground Model N
No(Relatively Slow) High
Standard Mean-ShiftBased Estimation N
No(Very Slow) High
EigenbackgroundSubtraction N
No(Relatively Slow) Intermediate
7/22/2019 BS Presentation1
20/20
Comparison of Background Removal Algorithms (continued)
Method Non-Recursive(N) or Recursive(R)
Real-time RequiredMemory
Running Average R Yes Low
Running Average withSelectivity R Yes Low
Approximated MedianFilter R Yes High
Kalman Filter R No(Almost Fast) Intermediate
Running GaussianAverage R No(Almost Fast) Intermediate
Mixture of Gaussians(MOG)
R No(Relatively Slow) Intermediate
Sequential KernelDensity Approximation
R No(Relatively Slow) Intermediate