Presentasi ICIAST 2010
Transcript of Presentasi ICIAST 2010
-
8/6/2019 Presentasi ICIAST 2010
1/40
ExtractInformationofPolarizationImagingfromLocalMatchingStereo
MohammadIqbalMohammadIqbal
OlivierMorelOlivierMorel
FabriceMeriaudeauFabriceMeriaudeau
International Conference on Intelligent & Advanced Systems 2010
-
8/6/2019 Presentasi ICIAST 2010
2/40
Content
Background
Polarization+StereoImagingSystemResultandConclusionFutureWork
2
3
1
3
3
4
-
8/6/2019 Presentasi ICIAST 2010
3/40
Background:WhatisPolarization? Polarization (alsopolarisation)isa
propertyofwavesthatdescr ibes theor ien ta t ion o f the i r osc i l l a t ion
Lightcanbepolarizedbyseveraldifferentprocesses Absorption Polaroidfilter Reflection Brewstersangle Scattering Lightfromthesky
Unpolarized Light
has E field at any
instant can have E
in any direction.
Polarized Light
has E field in a
certain direction
3
1
-
8/6/2019 Presentasi ICIAST 2010
4/40
Background:WhatisPolarization? PolarizationlightInformation:
TotalIntensity AngleofPolarization:isadirectionofEfieldon
electromagneticwave/lightafterpolarized DegreeofPolarization:isaquantityusedto
describetheportionofanelectromagneticwavewhich
is
polarized.
Perfectlypolarizedwave=DOPof100%, unpolarizedwave=aDOPof0%. Partiallypolarized,=DOPsomewhereinbetween0
and100%. ApplicationsofPolarizedlight:
Polaroidsunglasses,3DMovies,Photography Polarizationmicroscopy,Liquidcrystaldisplay ComputerVision:segmentation,navigation
AoP
31
-
8/6/2019 Presentasi ICIAST 2010
5/40
Background:WhatisStereoSystem? Stereosystemwillprovide
expandedviewsofanobject. Obtaininformationnotonlyin
2D,butalsogetthedepthofanobject(3D).
Geta
distance
information
from
twoormoreimagestakenfromdifferentpointofview(viewpoint).
scene pointscene point
optical centeroptical center
imageplane
imageplane
31
-
8/6/2019 Presentasi ICIAST 2010
6/40
Polarization+StereoImagingImage
Acquisition
Calibration
Photometric
Calibration
Stereo
Geometric
Calibration
Extract
Polarization
Information
Remove
Outlier
Application
Feature Detection
Rectifying Image
Stereo
Matching
2
-
8/6/2019 Presentasi ICIAST 2010
7/40
OurSystem:ImageAcquisition2
-
8/6/2019 Presentasi ICIAST 2010
8/40
OurSystem:PhotometricCalibrationMethod of Acquisition:
1. Left
2. Right
1. LC 0
2. LC 90
2
-
8/6/2019 Presentasi ICIAST 2010
9/40
OurSystem:PhotometricCalibration2
2
1 cos 2 sin 2 0
cos 2 cos 2 sin 2 cos 2 01 M ( )
sin 2 sin 2 cos 2 sin 2 020 0 0 0
=
Based on Mueller Matrice for Linear Polarizer
s = Mpol1.s
s = Mpol2.s = Mpol2.Mpol1.s
Target Polarizer0-180
s s s
Mpol1 Mpol2
s0
s1s
s2
s3
=
Stokes Vector :
1 Ip( ) ( s0 cos2 .s1 sin2 .s2 )
2 = + +
1. Mueller Matrix to get scalar from
2. Least Square for get a s0, s1 and s2
For =0-180 Matrix My=M.x
x=(MtM)-1Mt.y
H=(MtM)-1Mt Matrix 3x3
For =0-180s0 = s0+H(1, )*Ip()s1 = s1+H(2, )*Ip()s2 = s2+H(3, )*Ip()
Ip = a s0 + b s1 + c s2
3. Get a predicted intensity Verify with the intensity of
real Image from acquisition (Image Observe)
4. For every pixel in status alpha =0-180 :
Ipred= a s0 + b s1 + c s2
Verify = Ipred-Iobs
result 0 is ideal
result 1-2 is toleranceresult >2 is to much noise
2
-
8/6/2019 Presentasi ICIAST 2010
10/40
OurSystem:VisualResultofPhotometricCalibration
Left Polarizer
2 pixel : A(270,497)
B(198,501)
Right Polarizer
2 pixel : A(270,497)
B(198,501)
A BLC 0 for 2 pixel : A(270,497)
B(198,501)
LC 90 for 2 pixel :
A(270,497) B(198,501)
A B
2
-
8/6/2019 Presentasi ICIAST 2010
11/40
OurSystem:GeometricCalibrationandImageRectifying
Left Right
Calibration use aBouguet Toolbox, toget a camera :- Internal parameter- External parameter
Images rectifying use all parameter to transform geometrically
2
-
8/6/2019 Presentasi ICIAST 2010
12/40
OurSystem:FeatureDetector,StereoMatchingandRemoveOutliers
Sparse Feature DetectorWe use Harris corner detector
method, detector based-on the imagegradient.
Harris and Stephens Method :
1. R = det(M)- ktr(M)2
2. E = min (Eigenvalue of M)
1 1 2 2
2 2u ,v
1 1 2 2
u ,v u ,v
I ( u,v ) I I ( u d ,v ) I NSSD
I ( u,v ) I I ( u d ,v ) I
+ = +
Dense Stereo Matching AlgorithmWe use Normalized SSD (Sum of Squared
Differences) because this algorithm gave bettermatching results when applied to our polarized
images, compared to other local matching
algorithms (NCC, SSD, SAD, census and rank).
((I0+I90)/2) I45
2
DetailDetail
-
8/6/2019 Presentasi ICIAST 2010
13/40
OurSystem:DenseStereoMatchingandRemoveOutliers
Remove Outliers
We use RANSAC. it is an iterativemethod to estimate parameters ofa mathematical model from a setof observed data which containsoutliers. This algorithm was firstpublished by Fischler and Bolles in1981.
Putative Match Inlying Matches
2
DetailDetail
-
8/6/2019 Presentasi ICIAST 2010
14/40
OurSystem:ExtractPolarizationInfoBasic technique to obtain the state of polarization of incident light is to capturethree different intensity images through a set of polarization filters. If we have I0,I45
and I90
(representation of the image intensity measurements taken at an
angle of polarizer of 0, 45and 90), we can compute : angle of polarization,degree of polarization and total intensity.
totI
I ( 1 cos( 2 2 ))2
= +
tot0
tot45
tot90
I I ( 1 cos 2 )
2
I I ( 1 sin 2 )
2
I
I ( 1 cos 2 )2
= +
=
=
45
tot
90
tot
2Isin 2 1
I
2Icos 2 1
I
=
=
45
tot
90
tot
2I1I
a tan / 22I
1I
=
45
tot
45
tot
90
tot
2I1I
2I1
Isin( a tan )
2I
1 I
=
I = Imin+ Imax
Angle ofpolarization
Degree ofpolarization
Total
Intensity
= Angle of polarizer rotation
2
-
8/6/2019 Presentasi ICIAST 2010
15/40
OurSystem:ImageAcquisitionforVerifyPolarizationInfo
55mm
400mm
Pol1Pol2
LC
2 State Liquid CrystalPolarizer : 0, 90
5 State Incident light :0, 10, 20, 30, 45
Object
right : 45
2
-
8/6/2019 Presentasi ICIAST 2010
16/40
OurSystem:ExtractPolarizationInfo
IncidentIncident
LightLight
angleangle
IntensityIntensity
OfPolOfPol
MeanofMeanof
AOPAOP
MeanofMeanof
DOPDOP
%ofAverage%ofAverage
ErrorofErrorof
AOPAOP
0 0.3863 2.1586 0.7470 0.021610 0.3674 8.7851 0.6139 0.187920 0.3409 16.0545 0.5019 0.360530 0.4062 26.6184 0.4561 0.566245 0.4042 39.7027 0.4437 0.8470
0and 90
45
Open Image left 0 and 90
and right 45
Get a graylevel value on
every pixel founded by
Local Matching Algorithm
Compute Angle of
Polarization (AoP)
Compute Degree ofPolarization (AoP)
Compute Intensity ofPolarization
Visualization Polarization
Based Stereo Matching
AOPAOP = Angle of PolAngle of Pol
DOPDOP = Degree of PolDegree of Pol
33
-
8/6/2019 Presentasi ICIAST 2010
17/40
OurSystem:VizualizePolarizationInfo(x,y)
AOP /AOP /
Angle of
Polarization
DOP /DOP /
Degree of
Polarization
(x2,y2)
(x1,y1)
33
10
20
30
-
8/6/2019 Presentasi ICIAST 2010
18/40
ResumeResult11 2233
44
55
the polarization information is
extracted from the inlier pixelsand visualized.
Each source image :
Left :(i(0)+i(90))/2 Right : i(45)
Extract Harris features
Stereo matching by NSSD
Each pixel match found in thewrong place is rejected byRANSAC method
11
22
33
44
55
33
DetailDetail
-
8/6/2019 Presentasi ICIAST 2010
19/40
Conclusion Inthiswork,wehavedone:
Implemented
of
polarization
visionImplemented
of
polarization
vision systemin
stereo
imaging.
SetupdesignofImagingsystemSetupdesignofImagingsystem withliquidcrystalpolarizerinonesideandafixedpolarizerintheothersidethatcansensepartiallinearlypolarizedlightandcomputationallyprocesspolarizationcomponent.
VisualizeapolarizationVisualizeapolarization componentintheoutput. Experimentsshowthatourmethodexhibitsgood
performanceincomplexbackground,especiallywhenthereissomelightreflectedfromspecularparts.
33
-
8/6/2019 Presentasi ICIAST 2010
20/40
FutureWorkImage
Acquisition
Calibration
Photometric
Calibration
Stereo
Geometric
Calibration
Extract
Polarization
Information
Remove
Outlier
Feature Detection
Rectifying Image
Stereo
Matching
Water region
Segmentation
3D
Recontruction
SelfCalibration
Improvement
Photometric Invariant
Feature Detector
4
-
8/6/2019 Presentasi ICIAST 2010
21/40
Remerciement Thisworkwassupportedby:
BureauofPlanningandForeignCooperation,MinistryofNationalEducation,RepublicofIndonesia.
GunadarmaUniversity,IndonesiaOpticsandallresearchequipmentsarecourtesy
ofLE2IIUTLeCreusot,France.
-
8/6/2019 Presentasi ICIAST 2010
22/40
Terima Kasih
Thank You
Question?Question?
-
8/6/2019 Presentasi ICIAST 2010
23/40
DetailVisualResultDetailVisualResult
-
8/6/2019 Presentasi ICIAST 2010
24/40
Scene 10 :SourceImages(Rectified)Epipolar
line
33
-
8/6/2019 Presentasi ICIAST 2010
25/40
Scene 10 :HarrisFeature33
-
8/6/2019 Presentasi ICIAST 2010
26/40
Scene 10 :NSSDMatching33
-
8/6/2019 Presentasi ICIAST 2010
27/40
Scene 10 :NSSDMatchingPixelPixel CorrespondenceCorrespondence
0 100 200 300 400 500 600 7000
0.5
1
1.5
2
2.5
3
3.5
1st Pixel1st Pixel 2nd Pixel2nd Pixel LastLast PixelPixel
0 200 400 600 8000
0.5
1
1.5
2
2.5
3
0 200 400 600 8000
0.5
1
1.5
2
2.5
3
3.5
Compare to
Find the lowest(NSSD)
12..
.
n
1 2 . . . n12..
.
n
1 2 . . . n
Windows11x11
Left Right
33
-
8/6/2019 Presentasi ICIAST 2010
28/40
Scene 10 :Remove OutliersPutative Match
Number of inliers was 110 (61%)Number of putative matches was 180
Inlying Matches
33
-
8/6/2019 Presentasi ICIAST 2010
29/40
Scene 10 :Visualisasi Polarization33
-
8/6/2019 Presentasi ICIAST 2010
30/40
Visualization forall Scene33
-
8/6/2019 Presentasi ICIAST 2010
31/40
Visualization forall SceneScene 0
(Yellow Box)
Scene 10
(Green Box)
33
-
8/6/2019 Presentasi ICIAST 2010
32/40
Visualization forall SceneScene 20(Blue Box)
Scene 30
(Red Box)
33
-
8/6/2019 Presentasi ICIAST 2010
33/40
Visualization forall SceneScene 45(Cyan Box)
33BackBack
-
8/6/2019 Presentasi ICIAST 2010
34/40
AppendixAppendix
-
8/6/2019 Presentasi ICIAST 2010
35/40
HarriscornerdetectorThere is many possibility detector to get a feature point fromimages. One of well-known method is Harris corner detector.
This method based on the use of the image gradient.
The squared image derivatives
should be smoothed by convolving
them with a Gaussian filter g.2 2
x x I g I =
Defined autocorrelation matrix M
for every pixel:2
x x y
w w
2
x y y
w w
I I I
M I I I
=
where Wis a 33 neighborhood
around the point.
Compute the derivatives of the
intensity image in the x and y
directions for every pixel.
x
II
x
=
y
II
y
=
2 2
x y
I ,I
M is for derive a
measure of
cornerness
rank M = 0 : the
pixel belongs to an
homogeneous regionrank M = 1 : an edge
(significant gradients
in one directions),
rank M = 2 : a
corner (significant
gradients in both
directions).
Harris and Stephens Method :1. R = det(M)- k tr(M)2
2. E = min (Eigenvalue of M)
use of the determinant
and the trace (sum of
the diagonalelements ) of M to
detect corners
Use the smaller
eigenvalue of M for
detect a corner.
2BackBack
-
8/6/2019 Presentasi ICIAST 2010
36/40
RANSAC(RAndom
SAmple
Consensus)
InlyingMatches
2BackBack
-
8/6/2019 Presentasi ICIAST 2010
37/40
ComputePolarizationInfoPartial linear polarization can be measured at a pixel level
by the transmitted radiance through a polarization filter.The radiance varies sinusoidally with filter orientation. This
is based on the work of Wolff et al. :
Total Intensity
I = Imin
+ Imax
= I0
+ I90
Angle of Polarization :
= 0.5 * arctan ( ( I0 + I90 - 2I45 ) / I90 - I0If I90 < I0 [ if (I45 < I0 ) = +90
else = +90
Partial Polarization or degree of polarization :
= ( I90 - I0 ) / ( I90 - I0 ) cos 2
StepStep :: Open Image left 0 and 90and right 45
Get a graylevel value onevery pixel founded by
Local Matching Algorithm
Compute Angle of
Polarization (AoP)
Compute Degree of
Polarization (AoP)
Compute Intensity of
Polarization
Visualization Polarization
Based Stereo Matching
2
-
8/6/2019 Presentasi ICIAST 2010
38/40
ComputePolarizationInfo1
2 2 2
1 2 3
s1arccos
2 s s s
=
+ +s0 =Itots1 =Ipol cos 2
s2 =Ipol sin 2cos
s3 =Ipol sin 2sin
3
2
sarctan
s
=
Angle of Polarization Phase of Polarization
We use a linear polarizer, so we can simplified polarization imaging without use
S3 and phase of polarization. So, Stokes vector of partially linearly polarized wave
can be written :
s0 =Itots1 = Ipol cos 2s2 = Ipol sin 2
s3 = 0
0 1 2
1 I( ) ( S S cos 2 S sin 2 )
2
= + +
totI I( ) (1 cos( 2 2 ))2
= +
= Angle of polarizer rotation
Degree of Polarization2 2 2
pol 1 2 3
tol 0
I s s s
I s
+ += =
2
-
8/6/2019 Presentasi ICIAST 2010
39/40
ComputePolarization
Info
totI I (1 cos( 2 2 ))
2 = +
tot0
tot45
tot90
I I (1 cos 2 )
2
I I (1 sin 2 )
2I
I (1 cos 2 )2
= +
=
=
45
tot
90
tot
2Isin 2 1 ( a )I
2Icos 2 1 ( b )
I
=
=
45
tot
90
tot
2I1
Ia tan / 22I
1I
=
45
tot
45
tot
90
tot
2I1
I
2I1I
sin( a tan )2I
1I
=
2BackBack
-
8/6/2019 Presentasi ICIAST 2010
40/40
VizualizePolarization
Info
Cosinus Law
(x,y)
x1=(x+b)
y1 = (y-c)
y2=(y+c)
x2=(x-b)
(x,y)AOP /AOP /
Angle of
Polarization
DOP /DOP /
Degree of
Polarization
(x2,y2)
(x1,y1)Length = DOP/2
b=length*cos(AOP)c=length*sin(AOP)
x1=x+b;y1=y-c
x2=x-b;y2=y+c(x,y) ((x+b),y)
((x+b),(y-c))
x(col)
y(row)
a
b = a cos
= a sin c
2BackBack