Bayesian Model Selection in Non-Linear Time Series
Analysis
Non-Linear time series provides an alternative to
classical methods that make strong assumptions about the mechanism
generating a time series. Typically it is assumed that the current value
of a time series is a linear combination of past values:
which is called an autoregressive model of order p.
The term e t
represents a random shock to the system, which is assumed to have mean
0 and variance s 2 and is an
uncorrelated series.
It is common in physical systems for there to be
switching behaviour around particular threshold values. An illustration of
this behaviour is shown in Figure 1; Graham and Barnett (1987) found in
the tropics that below 29ºC there is little relationship between sea
surface temperature (SST) and rainfall. Above this threshold temperature
however there is a strong relationship between rainfall and SST flowing
from enhanced convection of moisture into the atmosphere, so SST operates
in effect as a rainfall switch. This motivates the idea of threshold
models (Tong, 1983).
Figure 1 Non-linear
relationship between rainfall and sea surface temperature (SST)
A threshold model allows for different models depending
on the value of the switching, or thresholding, variable. In the context
of Figure 1 a possible threshold model for rainfall {Yt}
might be:
where d is a delay parameter. This model then
defines two rainfall regimes, which we can think of as low and high
rainfall respectively. It would also be possible to incorporate the SST
time series in the model as an exogenous predictor.
The CMIS Environmetrics Group has been developing
methodology to select good predictors for non-linear time series models,
including the important lags for these predictors. Our approach is
described in detail in Campbell (2000), and is based on the reversible
jump Markov chain Monte Carlo (RJMCMC) methodology for Bayesian model
selection developed by Green (1995).
In conventional Bayesian inference we update prior
uncertainty about model parameters Q ,
expressed as p(Q ), given data Y
using Bayes’ theorem:
In model selection problems we assume that there are
many possible models, each having a set of parameters defined by the set {Q
k: k = 1, …, K} so that the full set of
unknown parameters may be written as
and our objective is to explore this augmented
parameter space. The key difference with conventional MCMC is that moves
between model subspaces can lead to changes in dimension of the parameter
vector. It is necessary to construct these moves to ensure reversibility,
which is the key condition ensuring convergence of MCMC samples in
distribution to the posterior distribution. Full theoretical details are
provided in Green (1995). We illustrate here our use of these ideas.
For a linear autoregressive model one possible set of
move types is shown in Figure 2, which assumes that the current model is
of order k. We could choose a ‘birth’ move with probability bk
that increases the model order to k + 1. The opposite move is a ‘death’
move, having probability dk, which reduces the model
order to k – 1. We could instead choose to explore the current
model, with probability 1 – bk – dk.
The results reported in Campbell (2000) suggest that a Poisson prior
distribution for the model order works well. If poor mixing is observed
then the Poisson mean can be given a prior gamma distribution for example.
This has been noted previously in related problems by Denison et al.
(1998).
An algorithm for selecting threshold models follows
immediately from the approach above by first selecting the regime to
update and then applying the above approach to the autoregressive model to
be updated. In our work we have found that this approach works well in the
sense that the optimal model is located efficiently and a full statement
of model uncertainty is available. One deficiency is that there is no
complete theory of convergence as yet, although some work is beginning to
appear (e.g., Brooks and Giudici, 1998).
Figure 2 A possible set
of move types for selecting an autoregressive model.
There are some practical difficulties in the use of
threshold models, principally in choosing the number of thresholds and the
delay parameter. We might envisage a range of RJMCMC approaches to these
problems, but they are inelegant, to say the least. We have instead come
to focus on a nonparametric approach to non-linear time series analysis,
adopting the general model
where the notation ≤t indicates history to
time t. The model predictors include the series itself, {Yt}
and a range of exogenous predictors {Uti: i
= 1, …, nu}. We are using nonparametric methods to
reconstruct ft :
We are at present researching the use of product basis
splines of the form
implemented in ways that aid interpretation of the
model parameters.
References
Brooks, S. P. and Giudici, P. (1998) In Bayesian Statistics 6.
Eds, Bernardo, J. M., Berger, J. O., Dawid, A. P. and Smith, A. F. M.
Campbell, E. P. (2000). Bayesian selection of threshold autoregressive
models. Submitted to J. Time. Ser. Anal.
Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998). Automatic
Bayesian curve fitting. J. R. Statist. Soc. B, 60, 333-350.
Graham, N. E. and Barnett, T. P. (1987). Sea surface temperature,
surface wind divergence, and convection over tropical oceans. Science,
238, 657-659.
Green, P. J. (1995). Reversible jump Markov chain Monte Carlo
computation and Bayesian model determination. Biometrika, 82,
711-732.
Tong, H. (1983) Threshold Models in Non-linear Time Series Analysis,
Springer-Verlag, New York.
Useful Links
Dr Dave Denison’s home page: http://stats.ma.ic.ac.uk/dgtd.html
Prof. Peter Green’s home page: http://www.stats.bris.ac.uk/~peter/Research.html
Contacts:
Eddy Campbell (eddy.campbell@cmis.csiro.au)
Yun Li (yun.li@cmis.csiro.au)
|
To
top