Introduction to R: Part VIIb2slab.upc.edu/wp-content/uploads/2014/02/CursR_VII.pdf · Introduction...
Transcript of Introduction to R: Part VIIb2slab.upc.edu/wp-content/uploads/2014/02/CursR_VII.pdf · Introduction...
Introduction to R: Part VIIA Time Series Data Session
Alexandre Perera i Lluna 1,2
1Centre de Recerca en Enginyeria Biomèdica (CREB)Departament d’Enginyeria de Sistemes, Automàtica i Informàtica Industrial (ESAII)
Universitat Politècnica de Catalunyamailto:[email protected]
2Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina(CIBER-BBN)
Jan 2011 / Introduction to RUniversitat Rovira i Virgili
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Contents I1 Introduction
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
2 Main Signal Processing FunctionsSliding WindowsAutocorrelationSpectral Analysis
3 Autoregresive ModelsARMA simulationFitting ARMA(p,q) models
4 FilteringFiltersFilters Characterization
5 IOI
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Contents IIO
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Obtaining some time Series Data
Yahoo! Finance provides with a historic finance dataAllows free access (not real-time, 15 minutes delay)
Yahoo! Finance download> library(tseries)> telefonica <- get.hist.quote("TEF.MC", start = "2006-01-01",+ end = "2008-05-07", quote = c("Open"))
time series starts 2006-01-02
> class(telefonica)
[1] "zoo"
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Index visualization
code> plot(telefonica)
1214
1618
2022
Index
tele
foni
ca
2006 2007 2008
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Getting raw data from time series (zoo)
code> teldata <- coredata(telefonica)> hist(teldata, 200)
Histogram of teldata
teldata
Fre
quen
cy
12 14 16 18 20 22
05
10
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Time Series
The zoo package allows iregular sampling times and missing data> telefonica[1:10]
2006-01-02 2006-01-03 2006-01-04 2006-01-05 2006-01-0612.75 12.76 12.90 12.99 13.09
2006-01-09 2006-01-10 2006-01-11 2006-01-12 2006-01-1313.17 13.19 13.14 13.10 13.01
The ts class from stats does not:> telts <- as.ts(telefonica)> telts[1:10]
[1] 12.75 12.76 12.90 12.99 13.09 NA NA 13.17 13.19[10] 13.14
however we can deal with NA data in several ways.
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Missing Values in Time Series
> plot(telts[1:25], type = "b")
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
5 10 15 20 25
12.4
12.6
12.8
13.0
13.2
Index
telts
[1:2
5]
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Missing Values in Time Series
> plot(telts[1:25], type = "b")> lines(na.locf(telts[1:25]), type = "b", col = "red")
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
5 10 15 20 25
12.4
12.6
12.8
13.0
13.2
Index
telts
[1:2
5]
● ●
●
●
● ● ●
●●
●
●
● ● ●
●
●
●
●
● ● ●
●
●
●
●
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Missing Values in Time Series
> plot(telts[1:25], type = "b")> lines(na.locf(telts[1:25]), type = "b", col = "red")> lines(na.approx(telts[1:25]), type = "b", col = "blue")
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
5 10 15 20 25
12.4
12.6
12.8
13.0
13.2
Index
telts
[1:2
5]
● ●
●
●
● ● ●
●●
●
●
● ● ●
●
●
●
●
● ● ●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Example DataPlotting time series objectsMissing Values and (i)regular Sampling Times
Missing Values in Time Series
> plot(telts[1:25], type = "b")> lines(na.locf(telts[1:25]), type = "b", col = "red")> lines(na.approx(telts[1:25]), type = "b", col = "blue")> lines(na.spline(telts[1:25]), type = "b", col = "green")
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
5 10 15 20 25
12.4
12.6
12.8
13.0
13.2
Index
telts
[1:2
5]
● ●
●
●
● ● ●
●●
●
●
● ● ●
●
●
●
●
● ● ●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Sliding Windows (rollers)
rolling functions (applies to both ts and zoo obects)rollapply(data, width, FUN)
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Sliding Windows
> mtelefonica <- rollapply(telefonica, 30, mean)> plot(mtelefonica, type = "l")
1416
1820
22
Index
mte
lefo
nica
2006 2007 2008
Also available as rollmean()
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Computing Autocorrelations
> coredata(telts) <- na.approx(telts)> acf(telts)
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Open
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Computing Autocorrelations
Let’s try the differenciated signal, just for fun:> coredata(telts) <- na.approx(telts)> dtel <- diff(telts)> acf(dtel)
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Open
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Periodogram
> require(signals)> plot(sunspots)
Time
suns
pots
1750 1800 1850 1900 1950
050
100
150
200
250
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Periodogram
> spectrum(sunspots)
0 1 2 3 4 5 6
1e−
021e
+00
1e+
021e
+04
frequency
spec
trum
Series: xRaw Periodogram
bandwidth = 0.00120
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
Periodogram
> spectrum(sunspots, spans = 10, xlim = c(0, 1))> abline(v = 1:3/12, lty = 3)
0.0 0.2 0.4 0.6 0.8 1.0
510
5050
050
00
frequency
spec
trum
Series: xSmoothed Periodogram
bandwidth = 0.0122 Extracted from
http://zoonek2.free.fr/
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
FFT
Let’s define a two component signal:> x <- 1:256/256> x <- sin(16 * pi * x) + 0.3 * cos(56 * pi * x)
> plot(x, type='l', lwd=3,> main="Signal", xlab="", ylab="")> plot(Mod(fft(x)[1: ceiling((length(x)+1)/2) ]),> type='l', col="blue", lwd=3,> ylab="Mof(fft(x))", xlab="",> main="DTF (Discrete Fourrier Transform)")> par(op)
Extracted from http://zoonek2.free.fr/
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
FFT
0 50 100 150 200 250
−1.
00.
01.
0
Signal
0 20 40 60 80 100 120
040
8012
0
DTF (Discrete Fourrier Transform)
Mof
(fft(
x))
Extracted from
http://zoonek2.free.fr/
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
Sliding WindowsAutocorrelationSpectral Analysis
spectrogram
Through the seewave package, we can plot spectrograms in dynamic 3Dplots:
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
AR(1)
In AR1, we can estimate thecoefficients by performing aregression of the signal againstlag(x,1)
> y <- telts[-1]> x <- telts[-length(telts)]> r <- lm(y ~ x - 1)> coefficients(r)
x1.000345
> plot(y ~ x)> abline(r, col = "red")
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
● ●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●
●
●●●●●●
● ●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●
● ●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●
●●●●●●
●●●●
● ●●●●●●●●●●●●●
●●●●●●●
●●●
● ●●●●●● ●●●●
●●●●●●●●●●●●●●●● ●
●●●●●
●●●●●●●●●●●●●●●●●●●●
●●● ●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●
●●
●●
● ●●● ●●●●
●●●●●
●●●
● ●●●●●●●●●●●● ●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●
●●●●●●●●●●●●●
● ●●●●●●●●●●
●●
●●
● ●●●●●
●●●●●●
●●
●●●●●●
●●
●●
●●
●●
●●●●●●●
●●●●●●●●●●●
●
●● ●●●
●●●●●●●●
●●
●
● ●●●●
●●●●●●●●
●●●●●●
●●●●
●●●●
●
●●
●● ●
●
● ●●
●●
●●
●●
●●
●●●●●●●●● ●
●●
●●
●●
●●●●
●●●●●
●●●●●●
● ●●●●●
● ●
●●●●
●●●
●●●●●
●
●● ●●●●●
●●●●●●●
●
●●●●
● ●
● ●●
●●●
●
●
●●●●
●
●
●
●●
●●
● ●●
● ●●●●
●
● ●●●●●● ●
●●●●●●●●
●●●●
● ●
●●●●●●●
●●●●●
●
●●●●●●●
●●●●●●●●● ●●●
●
● ●●●●●
●
●●●●
●●●
● ●●●●●
●●●●●●●●●●
●●●●●●
12 14 16 18 20 22
1214
1618
2022
x
y
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
AR(2) process
arima.sim() from stats package
Simulates an Autoregressive integrated moving average general model. AnARMA(q,p) is given by:(
1 −p∑
i−1
αiLi
)Xt =
(1 +
q∑i−1
θiLi
)εt (1)
To simulate an AR(2) process:> n <- 300> r <- arima.sim(list(ar = c(0.9, -0.2)), n)> plot(r)
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
AR(2) process
Time
r
0 50 100 150 200 250 300
−4
−2
02
4
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
MA(2) process
arima.sim() from stats package
Simulates an Autoregressive integrated moving average general model. AnARMA(q,p) is given by:(
1 −p∑
i−1
αiLi
)Xt =
(1 +
q∑i−1
θiLi
)εt (2)
To simulate an MA(2) process:> n <- 300> r <- arima.sim(list(ma = c(-0.7, -0.1)), n)> plot(r)
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
MA(2) process
Time
r
0 50 100 150 200 250 300
−4
−2
02
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
ARMA(p,q) process
arima.sim() from stats package
Simulates an Autoregressive integrated moving average general model. AnARMA(q,p) is given by:(
1 −p∑
i−1
αiLi
)Xt =
(1 +
q∑i−1
θiLi
)εt (3)
To simulate an ARMA(2,2) process:> n <- 300> r <- arima.sim(list(ma = c(-0.7, 1), ar = c(0.9,+ -0.2)), n)> plot(r)
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
ARMA(q,p) process
Time
r
0 50 100 150 200 250 300
−6
−4
−2
02
4
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
ARMA simulationFitting ARMA(p,q) models
Using arima() to fit ARMA models
> mod <- arima(r, order = c(2, 0, 2))> coef(mod)
ar1 ar2 ma1 ma2 intercept0.9274427 -0.2271395 -0.6929875 0.9999988 0.4574083
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
FiltersFilters Characterization
Defining filters
Filter objects (signal package)
Filters are created and stored as objects. The filter function applies anydefined filter to any signal.
Example (3 order, 10Hz low-pass Butterworth filter> library(signal)> bf = butter(3, 0.1)> t = seq(0, 1, len = 100)> x = sin(2 * pi * t * 2.3) + 0.25 * rnorm(length(t))> z = filter(bf, x)> class(z)
[1] "ts"
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
FiltersFilters Characterization
Filters
0.0 0.2 0.4 0.6 0.8 1.0
−1.
5−
1.0
−0.
50.
00.
51.
0
t
x
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
FiltersFilters Characterization
Filter Functions in signal package
fltfilt()
fir1(),fir2()
butter()
hamming(),hanning(),flattopwin(),gausswin(),kaiser(),blackman(),boxcar()
Savitzsky Golay filtersParks McClellan optimal FIR filter design (remez)... and more
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
FiltersFilters Characterization
Frequency response
freqz(fir1(40,0.3))
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
FiltersFilters Characterization
Filter characterization
b = c(1, 2)a = c(1, 1)w = linspace(0, 4, 128)freqs(b, a, w)
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
IO
The tuneR package
> library(tuneR)> s7 <- readWave("mysong.wav", from = 1, to = 5, units = "seconds")> s7
Wave ObjectNumber of Samples: 32000Duration (seconds): 4Samplingrate (Hertz): 8000Channels (Mono/Stereo): MonoBit (8/16/24/32): 16
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
IO
The sound package
> library(sound)> s <- loadSample("mysong.wav", from = 1, to = 5, units = "seconds")> s
type : monorate : 8000 samples / secondquality : 16 bits / samplelength : 480000 samplesR memory : 1920000 bytesHD memory : 960044 bytesduration : 60 seconds
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
IO
The audio package
> library(audio)> s <- load.wave("mysong.wav")> head(s)
sample rate: 8000Hz, mono, 16-bits[1] 0.0000000 0.7070923 0.9999695 0.7070923 0.0000000 -0.7071139
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
IO
To a file
To a .txt file:> data(tico)> export(tico, f = 22050, header = FALSE, filename = "tico_Gold.txt"))
To a .wav file (e.g.through seewave):> savewav(ticofirst, filename = "tico_firstnote.wav")
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
IO
To an audio device:
Using play from the tuneR package:> play(s)
Using listen from the seewave package:> listen(s1, f=8000, from=0.3, to=5)
Alexandre Perera i Lluna , Introduction to R: Part VII
IntroductionMain Signal Processing Functions
Autoregresive ModelsFiltering
IO
IO
End Part VII
Alexandre Perera i Lluna , Introduction to R: Part VII