BS Presentation1

7/22/2019 BS Presentation1

1/20

In The Name Of God,The Most Beneficent, The Most Merciful

Dynamic Background Generation

Presented By: Bijan Shoushtarian

Computer Engineering DepartmentThe University of Isfahan

1392/6/23


2/20

Common Approaches ToMoving Target Detection

1. Optical Flow

2. Temporal Differencing

3. Background Subtraction


3/20

Optical FlowIf an observer (a camera or a human eye) moves in a 3D scene, the patternof motion of objects, surfaces, and edges in a visual scene caused by therelative motion between the observer and the scene is called optical flow .

(a) A Rubik's cube on a rotating turntable;(b) Flow vectors calculated from comparingthe two images of a Rubik's cube (Russelland Norvig, 1995)

(a)

(b)


4/20

Optical Flow Advantages andDisadvantages

Target detection and tracking can beperformed using optical flow even thoughthere is no prior knowledge about thebackground or when the camera is moving.

Optical flow methods need very complexcomputations which cannot be coded inreal-time algorithms without specialisedhardware.

Optical flow methods are sensitive to noise.Thus, many researchers usually do not prefer usingoptical flow methods for implementing real-timebackground generation techniques.


5/20

Temporal Differencing In temporal differencing method, the arithmetic difference ofcorresponding pixels in the same physical locations in two (consecutiveor non-consecutive) frames of an image sequence is obtained. Thedifference image contains non-zero values whenever objects have movedto another location but will be black when no moving object is detected.

The problems with consecutive frames are:

1. A pre-selected threshold is necessary to obtain the thresholdeddifference image. As a result, this method will be dependent on thevideo sequence and selected threshold.

2. If an object moves too slowly so that it is stationary in twoconsecutive frames, the object will not appear in the difference frame.

3. Extending the method to use more than two frames is difficult.


6/20

Background Subtraction

The most important approach to identify and segment movingobjects in a video stream is background subtraction , which involvescomputing a reference image for each new frame. Then, bycomparing the next input frame with the reference image, regions ofthe image which have changed are identified. Thresholding the resultproduces a binary segmentation which is used for discriminating themoving objects from the background regions.

D(x, y, t) = | I(x, y, t) B(x, y, t-1) |T(x, y, t) = Thresholded (D(x, y, t))

Due to several factors, the background image must be temporallyadaptive and it should be updated continuously in order to be kept

up to date.


7/20

Background SubtractionExample

Fld249 Backgnd248

Dif_249 Threshld_249 Thresh_Sizefilt249


8/20

Background Generation

Requirements Illumination changes

Gradual variations in lighting conditions.

Sudden illumination changes (such as turning alamp on or off in an indoor environment).

Motion changes

Camera oscillations.Small movements of background objects suchas trees waving in the wind or sea waves, rain,snow, etc.


9/20

Background Generation

Requirements (continued) Changes in the background geometry

Introducing or removing objects in the scene

(such as a door is opened and then is leftopened or a parked car is moved on).

An initialisation process is required by a

number of algorithms. These problems and requirements are the constraintswhich should be considered as important characteristics byadaptive background removal algorithms.


10/20

Background Removal

Algorithms Background removal (subtraction) methods can be classified into two broadcategories: non-recursive and recursive.

Non-recursive techniques A non-recursive technique uses a sliding-window approach which maintains a buffer tohold N previous input frames. The background image is estimated based on the temporalvariations of each pixel in the buffered frames. A significant storage requirement isconsidered as the weakness of these techniques.

Recursive techniquesInstead of using a buffer, in recursive techniques, a single background model is updatedusing each input frame recursively. Often more weights are given to most recent samplesand as a result, input frames from past distance usually have less effect on the currentbackground model. In comparison with non-recursive techniques, recursive techniquesrequire much less storage. However, if an error object suddenly appears in thebackground image, it may remain in the background for a much longer period of time.


11/20

Non-recursive Techniques Frame differencingMean filterMode filterMedian filterBasic methods (i.e. mean, mode and median) withselectivity

Each pixel in every input frame is classified as either a foreground

or a background pixel.If the pixel is detected as a foreground point, it is ignored in thebackground update process.

Least Median of SquaresBt (x, y) = min b median t (I t (x, y) b) 2


12/20

Non-recursive Techniques(continued)

Linear predictive filterThe background is modelled by:1. Pixel-by-pixel linear prediction ( Weiner filter) using colour information2. Region-level by region filling algorithm for dealing with background object relocation

problem3. Frame-level by model switching for detecting global illumination changes

Non-parametric background modelBackground pdf is obtained by the histogram of the N most recent pixel values asfollows:

where K (.) is a smoothing Gaussian kernel estimator. The pixel I t (x, y) is consideredas a foreground point if there is little possibility that f (I t (x, y)) has such a distribution,

i.e., f (I t (x, y)) is smaller than one global threshold value

.


13/20

Non-recursive Techniques

(continued) Standard mean-shift based estimationIn this gradient-ascent method, the modes of a multimodal distribution are detectedusing their covariance matrix. The method uses an iterative approach so its step isdecreased until it is converged. For n data points x i , i = 1 . . . n in the d -dimensionalspace Rd , the multivariate mean shift vector computed with kernel g in the point x isgiven by:

where h is the kernel bandwidth.The major problems with the standard mean-shift method are that the algorithm isvery slow and requires the amount of N (i.e. buffer length) * size (frame) memory.

Eigenbackground subtraction


14/20

Recursive Techniques Running averageBt+1 (x, y) = * I t (x, y) + (1 ) * Bt (x, y)where 0 < < 1. is called the learning rate and is typically 0.05.

Running average with selectivityBt+1 (x, y) = * It (x, y) + (1 ) * B t (x, y) : if I t (x, y) is a background pixelBt+1 (x, y) = B t (x, y) : if I t (x, y) is a foreground pixel

Approximated median filterThe running estimate of the median is incremented by one if the input pixel islarger than the estimate, and decreased by one if smaller. This estimate eventuallyconverges to a value for which half of the input pixels are larger than and half aresmaller than this value, that is, the median.

Kalman filter


15/20

Recursive Techniques(continued)

Running Gaussian averageIt models the background as a textured surface, each point of which is associated with a

mean colour and a variance about the mean. It fits one Gaussian distribution ( , ) overthe histogram which gives the p .d .f of the background. It also applies the runningaverage to update the background p .d .f as follows:

The method uses a threshold for partitioning the background pixels into visible andoccluded points by examining the test: | I t (x, y) t (x, y) | > , where = k .


16/20


Mixture of GaussiansThe intensity of each background pixel is adaptively represented by the summation of kweighted Gaussians. The MOG model maintains a density function for each pixel and as aresult is capable of handling multimodal background distributions. The number of modes

(i.e. k) is usually predefined from 3 to 5. The pixel distribution is modelled as amixture of k Gaussians:

where is the i -th Gaussian component with intensity mean and standarddeviation . is the portion of the data accounted for by the i -th component. For eachinput pixel I t (x, y), the component whose mean is closest to I t (x, y) is declared as thematched component if .

At every new frame, the parameters of the matched component are then updated as

where is a user- defined learning rate with 0 1. is the learning rate for theparameters and can be approximated by / .

f (I t (x, y) = u )


17/20


In order to determine whether I t (x, y) is a foreground or background pixel, all components areranked by their . If i 1, i 2 , ., i k is the component order after sorting, the first M components thatsatisfy the following criterion are declared to be the background components:

The advantages of Gaussian mixture models (GMMs) are :Dealing with lighting changesslow-moving objectsintroducing or removing objects from the scene.

The drawbacks of GMMs are :The number of the mixture components is pre-set and fixed-value.GMM approach for foreground segmentation is a time-consuming process due to estimatingthe number of parameters which are mostly determined by the number of mixture components.

Also the application of GMMs for background subtraction requires an efficient method forlearning the GMM parameters which are computationally expensive.Therefore, the selection of the number of components and the initialisation process are two

important problems of the GMM algorithm for background subtraction.

where is the weight threshold.


18/20


Sequential kernel density approximationThe density is represented by a weighted sum of Gaussians, whose number, weights,means and covariances are updated at each time step to include the new data into themodel. For each mode, a Gaussian component is created whose mean is given by themode location. The covariance of the Gaussian is also derived from the Hessian matrixwhich is computed at the mode location. It relies on the modelling and density modeswhich are propagated by adapting them with the new samples as follows:

Pdf ( x ) = (new_mode ) + (1 ) (existing_modes )


19/20

Comparison of Background Removal Algorithms

Method Non-Recursive(N) or Recursive(R)

Real-time RequiredMemory

Frame Differencing N Yes Low

Mean Filter N Yes High

Mode Filter N Yes High

Median Filter N Yes High

Basic methods withselectivity N Yes High

Least Median ofSquares (LMedS) N Yes High

Linear PredictiveFilter N Yes Intermediate

Non-parametricBackground Model N

No(Relatively Slow) High

Standard Mean-ShiftBased Estimation N

No(Very Slow) High

EigenbackgroundSubtraction N

No(Relatively Slow) Intermediate


20/20

Comparison of Background Removal Algorithms (continued)

Method Non-Recursive(N) or Recursive(R)

Real-time RequiredMemory

Running Average R Yes Low

Running Average withSelectivity R Yes Low

Approximated MedianFilter R Yes High

Kalman Filter R No(Almost Fast) Intermediate

Running GaussianAverage R No(Almost Fast) Intermediate

Mixture of Gaussians(MOG)

R No(Relatively Slow) Intermediate

Sequential KernelDensity Approximation

R No(Relatively Slow) Intermediate

BS Presentation1

Documents

Transcript of BS Presentation1