Abstract: The chapter gives a review of the literature on bootstrap methods for time series data. It describes various possibilities on how the bootstrap method, initially introduced for independent random variables, can be extended to a wide range of dependent variables in discrete time, including parametric or nonparametric time series models, autoregressive and Markov processes, long range dependent time series and nonlinear time series, among others. Relevant bootstrap approaches, namely the intuitive residual bootstrap and Markovian bootstrap methods, the prominent block bootstrap methods as well as frequency domain resampling procedures, are described. Further, conditions for consistent approximations of distributions of parameters of interest by these methods are presented. The presentation is deliberately kept non-technical in order to allow for an easy understanding of the topic, indicating which bootstrap scheme is advantageous under a specific dependence situation and for a given class of parameters of interest. Moreover, the chapter contains an extensive list of relevant references for bootstrap methods for time series.

Abstract: Preface. 1. Introduction. 1.1 Signals. 1.2 Systems and Models. 1.3 System Modeling. 1.4 System Identification. 1.5 How Common are Nonlinear Systems? 2. Background. 2.1 Vectors and Matrices. 2.2 Gaussian Random Variables. 2.3 Correlation Functions. 2.4 Mean-Square Parameter Estimation. 2.5 Polynomials. 2.6 Notes and References. 2.7 Problems. 2.8 Computer Exercises. 3. Models of Linear Systems. 3.1 Linear Systems. 3.2 Nonparametric Models. 3.3 Parametric Models. 3.4 State-Space Models. 3.5 Notes and References. 3.6 Theoretical Problems. 3.7 Computer Exercises. 4. Models of Nonlinear Systems. 4.1 The Volterra Series. 4.2 The Wiener Series. 4.3 Simple Block Structures. 4.4 Parallel Cascades. 4.5 The Wiener-Bose Model. 4.6 Notes and References. 4.7 Theoretical Problems. 4.8 Computer Exercises. 5. Identification of Linear Systems. 5.1 Introduction. 5.2 Nonparametric Time-Domain Models. 5.3 Frequency Response Estimation. 5.4 Parametric Methods. 5.5 Notes and References. 5.6 Computer Exercises. 6. Correlation-Based Methods. 6.1 Methods for Functional Expansions. 6.2 Block Structured Models. 6.3 Problems. 6.4 Computer Exercises. 7. Explicit Least-Squares Methods. 7.1 Introduction. 7.2 The Orthogonal Algorithms. 7.3 Expansion Bases. 7.4 Principal Dynamic Modes. 7.5 Problems. 7.6 Computer Exercises. 8. Iterative Least-Squares Methods. 8.1 Optimization Methods. 8.2 Parallel Cascade Methods. 8.3 Application: Visual Processing in the Light Adapted Fly Retina. 8.4 Problems 8.5 Computer Exercises. References. Index. IEEE Press Series in Biomedical Engineering.

Abstract: Parametric models are only occasionally used in the analysis of clinical studies of survival although they may offer advantages over Cox's model. In this paper, we report experiences that we have made fitting parametric models to data sets from different clinical trials mainly performed at the Vienna University Medical School. We emphasize the role of residuals for discriminating among candidate models and judging their goodness of fit. The effect of misspecification of the baseline distribution on parameter estimates and testing has been explored. The results from parametric analyses have always been contrasted with those from Cox's model.

Abstract: We propose a fully parametric model for the analysis of competing risks data where the types of failure may not be independent. We show how the dependence between the cause-specific survival times can be modelled with a copula function. Features include: identifiability of the problem; accessible understanding of the dependence structures; and flexibility in choosing marginal survival functions. The model is constructed in such a way that it allows us to adjust for concomitant variables and for a dependence parameter to assess the effects of these on each marginal survival model and on the relationship between the causes of death. The methods are applied to a prostate cancer data set. We find that, with the copula model, more accurate inferences are obtained than with the use of a simpler model such as the independent competing risks approach.

Abstract: This paper discusses a statistical modeling strategy based on extreme value theory to describe the behavior of an insurance portfolio, with particular emphasis on large claims. The strategy is illustrated using the 1991–92 group medical claims database maintained by the Society of Actuaries. Using extreme value theory, the modeling strategy focuses on the “excesses over threshold” approach to fit generalized Pareto distributions. The proposed strategy is compared to standard parametric modeling based on gamma, lognormal, and log-gamma distributions. Extreme value theory outperforms classical parametric fits and allows the actuary to easily estimate high quantiles and the probable maximum loss from the data.

Abstract: In certain auction, search, and related models, the boundary of the support of the observed data depends on some of the parameters of interest. For such nonregular models, standard asymptotic distribution theory does not apply. Previous work has focused on characterizing the nonstandard limiting distributions of particular estimators in these models. In contrast, we study the problem of constructing efficient point estimators. We show that the maximum likelihood estimator is generally inefficient, but that the Bayes estimator is efficient according to the local asymptotic minmax criterion for conventional loss functions. We provide intuition for this result using Le Cam's limits of experiments framework.

Abstract: The objective of this paper is to show how experimentally based modelling can be used for designing Linear Parametrically Varying (LPV) controllers. As a test system we use an industrial pick and place unit with one linear X-drive and two independent linear Ydrives. The dynamics of the Y-axes depend on the Xposition. An LPV model is derived by using measured Frequency Response Functions at different positions, fitting a parametric model on each measurement and combining these models by linking parameters via a fit as a function of operating point. Rewriting the LPV model into a LFT structure and applying model reduction in the space of the scheduling variable finalizes the modelling phase. With this model an LPV controller is calculated and shows robust performance for the whole operating range, in contrast to local H∞ controllers.

Abstract: Previous work has shown that it is often essential to account for the variation in rates at different sites in phylogenetic models in order to avoid phylogenetic artifacts such as long branch attraction. In most current models, the gamma distribution is used for the rates-across-sites distributions and is implemented as an equal-probability discrete gamma. In this article, we introduce discrete distribution estimates with large numbers of equally spaced rate categories allowing us to investigate the appropriateness of the gamma model. With large numbers of rate categories, these discrete estimates are flexible enough to approximate the shape of almost any distribution. Likelihood ratio statistical tests and a nonparametric bootstrap confidence-bound estimation procedure based on the discrete estimates are presented that can be used to test the fit of a parametric family. We applied the methodology to several different protein data sets, and found that although the gamma model often provides a good parametric model for this type of data, rate estimates from an equal-probability discrete gamma model with a small number of categories will tend to underestimate the largest rates. In cases when the gamma model assumption is in doubt, rate estimates coming from the discrete rate distribution estimate with a large number of rate categories provide a robust alternative to gamma estimates. An alternative implementation of the gamma distribution is proposed that, for equal numbers of rate categories, is computationally more efficient during optimization than the standard gamma implementation and can provide more accurate estimates of site rates.

Abstract: An accurate optical flow estimation algorithm is proposed in this paper. By combining the three-dimensional (3D) structure tensor with a parametric flow model, the optical flow estimation problem is converted to a generalized eigenvalue problem. The optical flow can be accurately estimated from the generalized eigenvectors. The confidence measure derived from the generalized eigenvalues is used to adaptively adjust the coherent motion region to further improve the accuracy. Experiments using both synthetic sequences with ground truth and real sequences illustrate our method. Comparisons with classical and recently published methods are also given to demonstrate the accuracy of our algorithm.

Abstract: To be written. I. THE STATE OF CENTRAL BANK POLICY MODELING I.1. Not probability models. I.2. Ad hoc “econometrics” layered on a “long-run” core. I.3. Lack of a language to discuss uncertainty about models and parameters. I.4. Why models are nonetheless used, and are useful. The need to bring data to bear on policy discussions. There’s a lot of data, and it’s hard to maintain accounting and statistical consistency without a model. I.4.1. Decentralization, model size. I.5. The impact of flexible inflation targeting. II. DIRECTIONS FOR PROGRESS There are promising new directions, but as we get closer to actually using these methods in real-time decision-making we will need to solve some implementation problems and broaden understanding of the contributions and limitations of these methods. II.1. Existing work. II.1.1. Using bad models. Schorfheide, Geweke, Brock-Durlauf-West? II.1.2. MCMC methods for parameter uncertainty. Smets and Wouters Date: July 9, 2003. c ©2003 by Christopher A. Sims. This material may be reproduced for educational and research purposes so long as the copies are not sold, even to recover costs, the document is not altered, and this copyright notice is included in the copies. 1 PROBABILITY MODELS 2 II.1.3. Bayes factors, odds ratios, model averaging. Brock-Durlauf-West, Smets and Wouters II.1.4. Perturbation Methods. II.2. Problems and prospects. II.2.1. Using bad models: wasted energy? II.2.2. MCMC methods. II.2.3. Nonlinearities. Stickiness, welfare evaluation, the zero bound. III. ODDS RATIOS AND MODEL AVERAGING Though most central banks focus much more attention on one primary model than on others, the banks’ economists and policy makers are well aware that other models exist, in their own institutions as well as outside them, and that the other models might have different implications and might in some sense fit as well or better than their primary model. So one aspect of the Bayesian DSGE modeling approach that central bank economists find most appealing is its ability to deal with multiple models, characterizing uncertainty about which models fit best and prescribing how to use results from multiple models in policy analysis and forecasting. But practical experience with Bayesian approaches to handling multiple models has frequently turned out to be disappointing or bizarre. One standard applied Bayesian textbook (Gelman, Carlin, Stern, and Rubin, 1995) has no entry for “model selection” in its index, only an entry for “model selection, why we do not do it”. 1Some discussions of Bayesian methods by econometricians have asserted that Bayesian model selection methods can be useful in choosing among “false models”. This is basically not true. It is true that non-Bayesian approaches cannot put probability distributions across models, conditional on the data, any more than they can put probability distributions on continuously distributed parameters. But, as Schorfheide (2000) explains, choosing among, or putting weight on, false models by fit criteria or probability calculations that assume one of the models is true can lead to large errors. Bayesian methods are likelihood-based and therefore as subject to this kind of error as any other approach to inference. Schorfheide proposes ways to mitigate this problem. PROBABILITY MODELS 3 There are several related ways the Bayesian model comparison methods tend to misbehave. • Results are sensitive to prior distributions on parameters within each model’s parameter space, even when the priors attempt to be “uninformative”. • Results can be sensitive to seemingly minor aspects of model specification. • Results tend to be implausibly sharp, with posterior probabilities of models mostly very near zero or one. When we work with a single model, with a prior distribution specified by a continuous density function over the parameter space, under mild regularity conditions in large samples posterior distributions over parameters cease to depend on the prior. There are no such results available for model comparison, when we hold the set of models fixed as sample size increases. I will argue that these pathologies of Bayesian model comparison are not inherent in the methodology, but instead arise from the ways we generate and interpret collections of parametric models. Once this is understood, the model comparison methodology can be useful, but as much for guiding the process of generating and modifying our collections of models as for choosing among or weighting a given collection of models. III.1. When the Set of Models is Too Sparse. Even in simple situations with a small number of parameters, model comparison methods will misbehave when the discrete collection of models is serving as a proxy for a more realistic continuous parameter space. For example, suppose we observe a random variable Xt distributed as N(μ, .01). One theory, model 0, asserts that μ = 0, while another, model 1, asserts that μ = 1. With equal prior probabilities, if the observed X is bigger than .55 or smaller than .45, the posterior odds ratio on the two models is greater than 100 to 1. This is a correct conclusion if we in fact know that one of the two models must be true. The practical problem is that often we will have proposed these two models as representatives of “μ is small” and “μ is what is predicted by a simple theory” classes of models. It then is worrisome when an observation such as X = .6, which might seem to slightly favor the “μ is big” class, but actually is highly unlikely under either model 0 or model 1, implies a result that suggests near-certainty, rather than skepticism. PROBABILITY MODELS 4 Here it is quite clear what the problem is and how to fix it: treat μ as a continuous parameter and report the entire likelihood. Or, having noticed that likelihood concentrates almost entirely in a region far from either of our representative models, generate new representative models that are more realistic. If we had a collection of models with μ = 0, .1, .2, . . . , .9, 1, odds ratios across models would produce roughly the same sensible implications as use of a continuous parameter space. This is the kind of example that Gelman, Carlin, Stern, and Rubin (1995) have in mind when they cite, as a condition for posterior probabilities on models being “useful”, the requirement that “each of the discrete models makes scientific sense, and there are no obvious scientific models in between”. III.2. Changing Posteriors by Changing Priors. Suppose we have a set of models i = 1, . . . , q for a vector of observations X, with model i specifying the pdf of X as pi(X | θi), and the prior pdf on θi as πi(θi), over parameter space Θi. We will assume for simplicity that priors on parameter spaces are independent across models. Take the prior probabilities on models to be equal. The posterior weight on model i is then the marginal data density

Abstract: Logistic regression is widely used to estimate relative risks (odds ratios) from case-control studies, but when the study exposure is continuous, standard parametric models may not accurately characterize the exposure-response curve. Semi-parametric generalized linear models provide a useful extension. In these models, the exposure of interest is modelled flexibly using a regression spline or a smoothing spline, while other variables are modelled using conventional methods. When coupled with a model-selection procedure based on minimizing a cross-validation score, this approach provides a non-parametric, objective, and reproducible method to characterize the exposure-response curve by one or several models with a favourable bias-variance trade-off. We applied this approach to case-control data to estimate the dose-response relationship between alcohol consumption and risk of oral cancer among African Americans. We did not find a uniquely 'best' model, but results using linear, cubic, and smoothing splines were consistent: there does not appear to be a risk-free threshold for alcohol consumption vis-a-vis the development of oral cancer. This finding was not apparent using a standard step-function model. In our analysis, the cross-validation curve had a global minimum and also a local minimum. In general, the phenomenon of multiple local minima makes it more difficult to interpret the results, and may present a computational roadblock to non-parametric generalized additive models of multiple continuous exposures. Nonetheless, the semi-parametric approach appears to be a practical advance.

Abstract: We present a thorough identifiability analysis of the soil hydraulic parameters in the parametric models of Brooks and Corey (BC; Brooks and Corey, 1964), Mualem-van Genuchten (VG; van Genuchten, 1980), and Kosugi (KC; Kosugi, 1996, 1999) using the recently developed Shuffled Complex Evolution Metropolis (SCEM-UA) algorithm (Vrugt et al., 2002b, and unpublished data). Because the SCEM-UA algorithm globally thoroughly exploits the parameter space and therefore explicitly accounts for parameter interdependence and nonlinearity of the employed parametric models, the algorithm is suited to generate a useful description of parameter uncertainty and its antithesis, parameter identifiability. A set of measured water retention characteristics of the UNSODA database (Leij et al., 1996) spanning a wide range of soil textures and three transient laboratory outflow experiments with decreasing flow rates were used to illustrate that a parameter identifiability analysis facilitates the selection of an adequate parametric model structure and provides useful information about the limitations of a model. Moreover, results suggest that one should be especially careful in establishing pedotransfer functions without knowledge of the underlying posterior uncertainty associated with the soil hydraulic parameters using direct estimation methods.

Abstract: We wish to determine the epipolar geometry of a stereo camera pair from image measurements alone. This paper describes a solution to this problem, which does not require a parametric model of the camera system, and consequently applies equally well to a wide class of stereo configurations. Examples in the paper range from a standard pinhole stereo configuration to more exotic systems combining curved mirrors and wide-angle lenses. The method described here allows epipolar curves to be learnt from multiple image pairs acquired by stereo cameras with fixed configuration. By aggregating information over the multiple image pairs, a dense map of the epipolar curves can be determined on the images. The algorithm requires a large number of images, but has the distinct benefit that the correspondence problem does not have to be explicitly solved. We show that for standard stereo configurations the results are comparable to those obtained from a state of the art parametric model method, despite the significantly weaker constraints on the non-parametric model. The new algorithm is simple to implement, so it may easily be employed on a new and possibly complex camera system.

Abstract: We consider the problem of model-checking a parametric extension of the logic TCTL over timed automata and establish its decidability. Given a timed automaton, we show that the set of durations of runs starting from a region and ending in another region is definable in the arithmetic of Presburger (when the time domain is discrete) or in the theory of the reals (when the time domain is dense). With this logical definition, we show that the parametric model-checking problem for the logic TCTL can easily be solved. More generally, we are able to effectively characterize the values of the parameters that satisfy the parametric TCTL formula.

Abstract: We consider the problem of model-checking a parametric extension of the logic TCTL over timed automata and establish its decidability. Given a timed automaton, we show that the set of durations of runs starting from a region and ending in another region is definable in the arithmetic of Presburger (when the time domain is discrete) or in the theory of the reals (when the time domain is dense). With this logical definition, we show that the parametric model-checking problem for the logic TCTL can easily be solved. More generally, we are able to effectively characterize the values of the parameters that satisfy the parametric TCTL formula.

Abstract: This paper introduces the modelling of the radiation of a sphere, part of which, S0, is pulsating with an uniform velocity while the other remains motionless. Velocity and pressure can be expressed analytically in the space outside the sphere using spherical harmonic decomposition. The radiation impedance can then be deduced, providing a model approximating the radiation of horns, accounting for the curvature of the wavefront. The angular dependence of the radiation impedance is eliminated by averaging on S0 to remain compatible with most of simplified models of horns whose equations depend on a unique space variable. This averaging provides an analytical expression representing the optimal approximation, minimizing the mean square error. Three simple parametric models, which are inexpensive to simulate and approximate this model of impedance with various precisions, are proposed. They are well adapted to real-time applications, such as the simulation of wind instruments. Their parameters are given according to the geometrical characteristics of S0 and the stability is checked. The error introduced by these models is negligible compared to the original due to averaging on S0.

Abstract: Lip segmentation is an essential stage in many multimedia systems such as videoconferencing, lip reading, or low bit rate coding communication systems. In this paper, we propose an accurate and robust lip segmentation algorithm. First, the upper mouth boundary and several characteristic points are detected by using a new kind of active contour : the "jumping snake". Unlike classic snakes, it can be initialized far from the final edge and the adjustment of its parameters is easy and intuitive. In a second step, a parametric model composed of several cubic curves is fitted on the lips. This model is flexible enough to reproduce the specificities of very different lip shapes. Compared to existing models, it brings a significant accuracy and realism improvement.

Abstract: Background: Using suitable error models for gene expression measurements is essential in the statistical analysis of microarray data. However, the true probabilistic model underlying gene expression intensity readings is generally not known. Instead, in currently used approaches some simple parametric model is assumed (usually a transformed normal distribution) or the empirical distribution is estimated. However, both these strategies may not be optimal for gene expression data, as the non-parametric approach ignores known structural information whereas the fully parametric models run the risk of misspecification. A further related problem is the choice of a suitable scale for the model (e.g. observed vs. log-scale). Results: Here a simple semi-parametric model for gene expression measurement error is presented. In this approach inference is based an approximate likelihood function (the extended quasi-likelihood). Only partial knowledge about the unknown true distribution is required to construct this function. In case of gene expression this information is available in the form of the postulated (e.g. quadratic) variance structure of the data. As the quasi-likelihood behaves (almost) like a proper likelihood, it allows for the estimation of calibration and variance parameters, and it is also straightforward to obtain corresponding approximate confidence intervals. Unlike most other frameworks, it also allows analysis on any preferred scale, i.e. both on the original linear scale as well as on a transformed scale. It can also be employed in regression approaches to model systematic (e.g. array or dye) effects. Conclusions: The quasi-likelihood framework provides a simple and versatile approach to analyze gene expression data that does not make any strong distributional assumptions about the underlying error model. For several simulated as well as real data sets it provides a better fit to the data than competing models. In an example it also improved the power of tests to identify differential expression.

Abstract: Many unequal error protection algorithms used in image communication systems need the operational distortion-rate (D/R) curve of the source coder whose computation is time-consuming. We study the use of parametric models instead of the true D/R curves for wavelet-based embedded image and video coders. We propose a Weibull model and show its superiority to the previous models for real-time applications. For unequal error protection over binary symmetric and packet erasure channels, the Weibull model yielded performance similar to the one obtained with the true D/R curve while satisfying the real-time constraint.

Abstract: The purpose of this work is to present some recent results in an ongoing research project between Ghent University and Chess Medical Technology Company Belgium. The overall aim of the project is to provide a fast method for identification of the human respiratory system in order to allow for an instantaneously diagnosis of the patient by the medical staff. A novel parametric model of the human respiratory system as well as the obtained experimental results are presented in this paper. A prototype apparatus developed by the company, based on the forced oscillation technique is used to record experimental data from 4 patients in this paper. Signal processing is based on spectral analysis and is followed by the parametric identification of a non-linear mechanistic model. The parametric model is equivalent to the structure of a simple electrical RLC-circuit, containing a non-linear capacitor. These parameters have a useful and easy-to-interprete physical meaning for the medical staff members.

Abstract: SUMMARY A hybrid method that combines Laplace's approximation and Monte Carlo simulations to evaluate integrals in the likelihood function is proposed for estimation of the parameters in nonlinear mixed effects models that assume a normal parametric family for the random effects. Simulations show that these parametric estimates of fixed effects are close to the nonparametric estimates even though the mixing distribution is far from the assumed normal parametric family. An asymptotic theory of this hybrid method for parametric estimation without requiring the true mixing distribution to belong to the assumed parametric family is developed to explain these results. This hybrid method and its asymptotic theory are also extended to generalised linear mixed effects models.

Abstract: In this paper, we present a novel two-stage framework for designing a noise-robust front-end for automatic speech recognition. In the first stage, a parametric model of acoustic distortion is used to estimate the clean speech and noise spectra in a principled way so that no heuristic parameters need to be set manually. To reduce possible flaws caused by the simplifying assumptions in the parametric model, a second-stage Wiener filtering is applied to further reduce the noise while preserving speech spectra unharmed. This front-end is evaluated on the Aurora2 task. For the multi-condition training scenario, a relative error reduction of 28.4% is achieved.

Abstract: This paper describes the development of an approach to handling qualities investigation that can be applied at an early stage in the design of the vehicle. It makes use of inverse simulation techniques, together with a pilot model to provide an integrated description of the man-machine control system. In order to incorporate pilot effects into data generated by inverse simulation, the output from an inverse simulation run is applied as input to a closed-loop system model that includes the vehicle dynamics and a simple parametric model of the pilot. Parameters of the pilot model are determined by optimisation and the pilot effect is added to the system output. Validation of the approach is achieved through a case study involving a predefined mission task involving a lateral manoeuvre. Equalisation characteristics estimated for each pilot are compared with those found by inverse simulation for the same manoeuvre. This approach may be applied using a simple real-time simulation on a desk-top computer and could be of value in identifying any potential deficiencies in a helicopter flight control system at an early stage in its development.

Abstract: Image segmentations based on maximum likelihood (ML) or maximum a posteriori (MAP) analyses of object textures usually assume parametric models (e.g., Gaussian) for distributions of these features. For real images, parameter accuracy and model stationarity may be elusive, so that model-free inference methods ought to have an advantage over those that are model-dependent. Functions of the relative entropy (RE) from information theory can produce minimum error, model-free inferences, and can detect the boundary of an image object by maximizing the RE between the pixel distributions inside and outside a flexible curve contour. A generalization of the RE–the Jensen-Renyi divergence (JRD)–computes optimal n-way decisions and can contour multiple objects in an image simultaneously. Seed regions expand naturally and multiple contours tend not to overlap. We apply these functions to contour patient anatomy in X-ray computed tomography (CT) for radiotherapy treatment planning.