Abstract: In a recent paper, Bai and Perron (1998) considered theoretical issues related to the limiting distribution of estimators and test statistics in the linear model with multiple structural changes. In this companion paper, we consider practical issues for the empirical applications of the procedures. We first address the problem of estimation of the break dates and present an efficient algorithm to obtain global minimizers of the sum of squared residuals. This algorithm is based on the principle of dynamic programming and requires at most least-squares operations of order O(T 2) for any number of breaks. Our method can be applied to both pure and partial structural-change models. Secondly, we consider the problem of forming confidence intervals for the break dates under various hypotheses about the structure of the data and the errors across segments. Third, we address the issue of testing for structural changes under very general conditions on the data and the errors. Fourth, we address the issue of estimating the number of breaks. We present simulation results pertaining to the behavior of the estimators and tests in finite samples. Finally, a few empirical applications are presented to illustrate the usefulness of the procedures. All methods discussed are implemented in a GAUSS program available upon request for non-profit academic use.

Abstract: Mixed linear models were developed by animal breeders to evaluate genetic potential of bulls. Application of mixed models has recently spread to all areas of research, spurred by availability of advanced computer software. Previously, mixed model analyses were implemented by adapting fixed-effect methods to models with random effects. This imposed limitations on applicability because the covariance structure was not modeled. This is the case with PROC GLM in the SAS® System. Recent versions of the SAS System include PROC MIXED. This procedure implements random effects in the statistical model and permits modeling the covariance structure of the data. Thereby, PROC MIXED can compute efficient estimates of fixed effects and valid standard errors of the estimates. Modeling the covariance structure is especially important for analysis of repeated measures data because measurements taken close in time are potentially more highly correlated than those taken far apart in time.

Abstract: Douglas M. Bates Department of Statistics University of Wisconsin Madison Jose C. Pinheiro Bell Laboratories Lucent Technologies 1 Recent developments in computational methods for maximum likelihood (ML) or restricted maximum likelihood (REML) estimation of parameters in general linear mixed-effects models have made the analysis of data in typical agricultural settings much easier. With software such as SAS PROC MIXED we are able to handle da~ from random-effects one-way classifications, from blocked designs including incomplete blocked designs, from hierarchical designs such as splitplot designs, and other types of data that may be described as repeated measures or longitudinal data or growth-curve data. It is especially helpful that the new computational methods do not depend on balance in the data so we are able to deal more easily with observational studies or with randomly missing data in a designed experiment. We describe some of the new computational approaches and how they are implemented in the nlme3.0 library for the S-PLUS language. One of the most powerful features of this language is the graphics capabilities, especially the trellis graphics facilities developed by Bill Cleveland and his coworkers at Bell Labs. Although most participants in this conference may be more familiar with SAS, and most of the models described here can be fit with PROC MIXED or the NLiNMIX macro or new P ROC N LM IXED in SAS version 7, some exposure to the combination of graphical display and model-fitting approaches from S-PLUS may be informative. 1 Annual Conference on Applied Statistics in Agriculture Kansas State University New Prairie Press http://newprairiepress.org/agstatconference/1998/proceedings/2 2 Kansas State University We show how data exploration with trellis graphics, followed by fitting and comparing mixedeffects models, followed by graphical assessment of the fitted model can be used in a variety of situations. On some occasions, such as modeling growth curves, a linear trend or polynomial trend or other types of linear statistical models for the within-subject time dependence are just not going to do an adequate job of representing the data. In those cases, a nonlinear model is more appropriate. We show how the concept of a random coefficient model can be extended to nonlinear models so as to fit nonlinear mixed-effects models.

Abstract: Much of the research in epidemiology and clinical science is based upon longitudinal designs which involve repeated measurements of a variable of interest in each of a series of individuals. Such designs can be very powerful, both statistically and scientifically, because they enable one to study changes within individual subjects over time or under varied conditions. However, this power arises because the repeated measurements tend to be correlated with one another, and this must be taken into proper account at the time of analysis or misleading conclusions may result. Recent advances in statistical theory and in software development mean that studies based upon such designs can now be analysed more easily, in a valid yet flexible manner, using a variety of approaches which include the use of generalized estimating equations, and mixed models which incorporate random effects. This paper provides a particularly simple illustration of the use of these two approaches, taking as a practical example the analysis of a study which examined the response of portable peak expiratory flow meters to changes in true peak expiratory flow in 12 children with asthma. The paper takes the reader through the relevant practicalities of model fitting, interpretation and criticism and demonstrates that, in a simple case such as this, analyses based upon these model-based approaches produce reassuringly similar inferences to standard analyses based upon more conventional methods.

Abstract: One-Sample Problems Introduction Location Model Geometry and Inference in the Location Model Examples Properties of Norm-Based Inference Robustness Properties of Norm-Based Inference Inference and the Wilcoxon Signed-Rank Norm Inference Based on General Signed-Rank Norms Ranked Set Sampling L1 Interpolated Confidence Intervals Two-Sample Analysis Two-Sample Problems Introduction Geometric Motivation Examples Inference Based on the Mann-Whitney-Wilcoxon General Rank Scores L1 Analyses Robustness Properties Proportional Hazards Two-Sample Rank Set Sampling (RSS) Two-Sample Scale Problem Behrens-Fisher Problem Paired Designs Linear Models Introduction Geometry of Estimation and Tests Examples Assumptions for Asymptotic Theory Theory of Rank-Based Estimates Theory of Rank-Based Tests Implementation of the R Analysis L1 Analysis Diagnostics Survival Analysis Correlation Model High Breakdown (HBR) Estimates Diagnostics for Differentiating between Fits Rank-Based Procedures for Nonlinear Models Experimental Designs: Fixed Effects Introduction One-Way Design Multiple Comparison Procedures Two-Way Crossed Factorial Analysis of Covariance Further Examples Rank Transform Models with Dependent Error Structure Introduction General Mixed Models Simple Mixed Models Arnold Transformations General Estimating Equations (GEE) Time Series Multivariate Multivariate Location Model Componentwise Spatial Methods Affine Equivariant and Invariant Methods Robustness of Estimates of Location Linear Model Experimental Designs Appendix: Asymptotic Results References Index

Abstract: In the theory of linear models, the concept of degrees of freedom plays an important role. This concept is often used for measurement of model complexity, for obtaining an unbiased estimate of the error variance, and for comparison of different models. I have developed a concept of generalized degrees of freedom (GDF) that is applicable to complex modeling procedures. The definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value. The concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures. The concept of GDF offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models. By using this framework, many difficult problems can be solved easily. For example, one can now measure the number of observations used in a variable selection process. Different modeling procedures, such as a tree-based regression and a ...

Abstract: Background Recent communications have argued that often it may not be appropriate to ana-lyse cross-sectional studies of prevalent outcomes with logistic regression models.The purpose of this communication is to compare three methods that have beenproposed for application to cross sectional studies: (1) a multiplicative generalizedlinear model, which we will call the log-binomial model, (2) a method based onlogistic regression and robust estimation of standard errors, which we will call theGEE-logistic model, and (3) a Cox regression model.Methods Five sets of simulations representing fourteen separate simulation conditions wereused to test the performance of the methods.Results All three models produced point estimates close to the true parameter, i.e. the es-timators of the parameter associated with exposure had negligible bias. The Coxregression produced standard errors that were too large, especially when the pre-valence of the disease was high, whereas the log-binomial model and the GEE-logistic model had the correct type I error probabilities. It was shown by examplethat the GEE-logistic model could produce prevalences greater than one, whereasit was proven that this could not happen with the log-binomial model. The log-binomial model should be preferred.Keywords Generalized linear model, Cox regression, cross sectional study, log-binomial model,GEE-logistic modelAccepted 28 May 1997A lively discussion about the appropriateness of estimating pre-valence proportion ratios versus prevalence odds ratios in cross-sectionai studies started when Lee,

Abstract: This paper surveys the theoretical and computational development of the restricted maximum likelihood (REML) approach for the estimation of covariance matrices in linear stochastic models. A new derivation of this approach is given, valid under very weak conditions on the noise. Then the calculation of the gradient of restricted loglikelihood functions is dis- cussed, with special emphasis on the case of large and sparse model equations with a large number of unknown covariance components and possibly incomplete data. It turns out that the gradient calculations require hardly any extra storage, and only a small multiple of the number of operations needed to calculate the function values alone. The analytic gradient procedure was integrated into the VCE package for co- variance component estimation in large animal breeding models. It resulted in dramatic improvements of performance over the previous implementation with finite difference gradients. An example with more than 250 000 normal equations and 55 covariance components took hours instead of days of CPU time, and this was not an untypical case.

Abstract: The fuzzy inference system proposed by Takagi, Sugeno, and Kang, known as the TSK model in fuzzy system literature, provides a powerful tool for modeling complex nonlinear systems. Unlike conventional modeling where a single model is used to describe the global behavior of a system, TSK modeling is essentially a multimodel approach in which simple submodels (typically linear models) are combined to describe the global behavior of the system. Most existing learning algorithms for identifying the TSK model are based on minimizing the square of the residual between the overall outputs of the real system and the identified model. Although these algorithms can generate a TSK model with good global performance (i.e., the model is capable of approximating the given system with arbitrary accuracy, provided that sufficient rules are used and sufficient training data are available), they cannot guarantee the resulting model to have a good local performance. Often, the submodels in the TSK model may exhibit an erratic local behavior, which is difficult to interpret. Since one of the important motivations of using the TSK model (also other fuzzy models) is to gain insights into the model, it is important to investigate the interpretability issue of the TSK model. We propose a new learning algorithm that integrates global learning and local learning in a single algorithmic framework. This algorithm uses the idea of local weighed regression and local approximation in nonparametric statistics, but remains the component of global fitting in the existing learning algorithms. The algorithm is capable of adjusting its parameters based on the user's preference, generating models with good tradeoff in terms of global fitting and local interpretation. We illustrate the performance of the proposed algorithm using a motorcycle crash modeling example.

Abstract: Foreword.- A Conversation with Hirotugu Akaike.- List of Publications of Hirotugu Akaike.- Papers.- 1. Precursors.- 1. On a zero-one process and some of its applications.- 2. On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method.- 2. Frequency Domain Time Series Analysis.- 1. Effect of timing-error on the power spectrum of sampled-data.- 2. On a limiting process which asymptotically produces f-2 spectral density.- 3. On the statistical estimation of frequency response function.- 3. Time Domain Time Series Analysis.- 1. On the use of a linear model for the identification of feedback systems.- 2. Fitting autoregressive models for prediction.- 3. Statistical predictor identification.- 4. Autoregressive model fitting for control.- 5. Statistical approach to computer control of cement rotary kilns.- 6. Statistical identification for optimal control of supercritical thermal power plants.- 4. AIC and Parametrization.- 1. Information theory and an extension of the maximum likelihood princilple.- 2. A new look at the statistical model identification.- 3. Markovian representation of stochastic processes and its application to the analysis of autoregressive moving average processes.- 4. Covariance matrix computation of the state variable of a stationary Gaussian process.- 5. Analysis of cross classified data by AIC.- 6. On linear intensity models for mixed doubly stochastic Poisson and self-exciting point processes.- 5. Bayesian Approach.- 1. A Baysian analysis of the minimum AIC procedure.- 2. A new look at the Bayes procedure.- 3. On the likelihood of a time series model.- 4. Likelihood and the Bayes procedure.- 5. Seasonal adjustment by a Bayesian modeling.- 6. A quasi Bayesian approach to outlier detection.- 7. On the fallacy of the likelihood principle.- 8. A Bayesian apporach to the analysis of earth tides.- 9. Factor analysis and AIC.- 6. General Views on Statistics.- 1. Prediction and entropy.- 2. Experiences on the development of time series models.- 3. Implications of informational point of view on the development of statistical science.

Abstract: This paper presents a statistical model underlying the chain-ladder technique. This is related to other statistical approaches to the chain-ladder technique which have been presented previously. The statistical model is cast in the form of a generalised linear model, and a quasi-likelihood approach is used. It is shown that this enables the method to process negative incremental claims. It is suggested that the chain-ladder technique represents a very narrow view of the possible range of models.

Abstract: Nature of Exact and Optimum Tests in Mixed Linear Models. Balanced Random and Mixed Models. Measures of Data Imbalance. Unbalanced One-Way and Two-Way Random Models. Random Models with Unequal Cell Frequencies in the Last Stage. Tests in Unbalanced Mixed Models. Recovery of Inter-Block Information. Split-Plot Designs Under Mixed and Random Models. Tests Using Generalized P-Values. Multivariate Mixed and Random Models. Appendix. General Bibliography. Indexes.

Abstract: Analysis of covariance is an effective method for addressing two considerations for randomized clinical trials. One is reduction of variance for estimates of treatment effects and thereby the production of narrower confidence intervals and more powerful statistical tests. The other is the clarification of the magnitude of treatment effects through adjustment of corresponding estimates for any random imbalances between the treatment groups with respect to the covariables. The statistical basis of covariance analysis can be either non-parametric, with reliance only on the randomization in the study design, or parametric through a statistical model for a postulated sampling process. For non-parametric methods, there are no formal assumptions for how a response variable is related to the covariables, but strong correlation between response and covariables is necessary for variance reduction. Computations for these methods are straightforward through the application of weighted least squares to fit linear models to the differences between treatment groups for the means of the response variable and the covariables jointly with a specification that has null values for the differences that correspond to the covariables. Moreover, such analysis is similarly applicable to dichotomous indicators, ranks or integers for ordered categories, and continuous measurements. Since non-parametric covariance analysis can have many forms, the ones which are planned for a clinical trial need careful specification in its protocol. A limitation of non-parametric analysis is that it does not directly address the magnitude of treatment effects within subgroups based on the covariables or the homogeneity of such effects. For this purpose, a statistical model is needed. When the response criterion is dichotomous or has ordered categories, such a model may have a non-linear nature which determines how covariance adjustment modifies results for treatment effects. Insight concerning such modifications can be gained through their evaluation relative to non-parametric counterparts. Such evaluation usually indicates that alternative ways to compare treatments for a response criterion with adjustment for a set of covariables mutually support the same conclusion about the strength of treatment effects. This robustness is noteworthy since the alternative methods for covariance analysis have substantially different rationales and assumptions. Since findings can differ in important ways across alternative choices for covariables (as opposed to methods for covariance adjustment), the critical consideration for studies with covariance analyses planned as the primary method for comparing treatments is the specification of the covariables in the protocol (or in an amendment or formal plan prior to any unmasking of the study.

Abstract: Model predictive control (MPC), also referred to as moving horizon control or receding horizon control, has become an attractive feedback strategy, especially for linear or nonlinear systems subject to input and state constraints In general, linear and nonlinear MPC are distinguished Linear MPC refers to a family of MPC schemes in which linear models are used to predict the system dynamics, even though the dynamics of the closed-loop system is nonlinear due to the presence of constraints Linear MPC approaches have found successful applications, especially in the process industries (Richalet, 1993) A complete overview on industrial MPC techniques with details and comparisons is given by Qin and Badgwell(1996), where more than 2200 applications in a very wide range from chemicals to aerospace industries are also summarized By now, linear MPC theory is quite mature Important issues such as stability are well addressed (see for example (Lee, 1996) for an overview)

Abstract: Empirical time series often contain observational noise. We investigate the effect of this noise on the estimated parameters of models fitted to the data. For data of physiological tremor, i.e. a small amplitude oscillation of the outstretched hand of healthy subjects, we compare the results for a linear model that explicitly includes additional observational noise to one that ignores this noise. We discuss problems and possible solutions for nonlinear deterministic as well as nonlinear stochastic processes. Especially we discuss the state space model applicable for modeling noisy stochastic systems and Bock's algorithm capable for modeling noisy deterministic systems.

Abstract: We compared the application of ordinary linear regression, Deming regression, standardized principal component analysis, and Passing-Bablok regression to real-life method comparison studies to investigate whether the statistical model of regression or the analytical input data have more influence on the validity of the regression estimates. We took measurements of serum potassium as an example for comparisons that cover a narrow data range and measurements of serum estradiol-17beta as an example for comparisons that cover a wide data range. We demonstrate that, in practice, it is not the statistical model but the quality of the analytical input data that is crucial for interpretation of method comparison studies. We show the usefulness of ordinary linear regression, in particular, because it gives a better estimate of the standard deviation of the residuals than the other procedures. The latter is important for distinguishing whether the observed spread across the regression line is caused by the analytical imprecision alone or whether sample-related effects also contribute. We further demonstrate the usefulness of linear correlation analysis as a first screening test for the validity of linear regression data. When ordinary linear regression (in combination with correlation analysis) gives poor estimates, we recommend investigating the analytical reason for the poor performance instead of assuming that other linear regression procedures add substantial value to the interpretation of the study. This investigation should address whether (a) the x and y data are linearly related; (b) the total analytical imprecision (s(a,tot)) is responsible for the poor correlation; (c) sample-related effects are present (standard deviation of the residuals >> s(a,tot)); (d) the samples are adequately distributed over the investigated range; and (e) the number of samples used for the comparison is adequate.

Abstract: Linear (proportional) functions are undoubtedly one of the most common models for representing and solving both pure and applied problems in elementary mathematics education. But according to several authors, different aspects of the current culture and practice of school mathematics develop in students a tendency to use these linear models also in situations in which they are not applicable. This article reports two closely related studies about this phenomenon in 12–13- and 15–16-year old students working on word problems involving lengths and areas of similar plane figures of different kinds of shapes, as well as about the influence of drawings in breaking this improper use of linearity. Generally speaking, the results provide a convincing demonstration of the predominance of the linear model in secondary students' solutions of this kind of mensurational problem.

Abstract: We introduce the spherically projected multivariate linear model for directional data. This model treats directional observations as projections onto the unit sphere of unobserved responses from a multivariate linear model. Focusing on the important case of circular data, we show that maximum likelihood estimates for the model are readily computed using iterative methods, in sharp contrast with competing approaches. Examples are given to demonstrate the resulting methodology in realistic applications.

Abstract: The linear, steady-state, baroclinic model response to a tropical heating superimposed on a three-dimensional basic state is examined in this study. The emphasis is on the relevance of the linear model solution as compared to a fully nonlinear baroclinic model. The direct response to heating in the fully nonlinear, time-dependent model is obtained as the day-30 model response, following the Jin and Hoskins approach. When a 15-day linear damping is included in addition to Rayleigh friction, Newtonian cooling, and a scale-selective biharmonic diffusion, the comparison of the linear and the nonlinear model responses to a 2°C/day tropical heating reveals a striking similarity in both the spatial distribution and amplitude. Thus nonlinearity appears to be a secondary effect and may be crudely represented by the 15-day linear damping, and the linear steady-state model can be a useful tool in diagnostic studies. Both the linear and the nonlinear model responses show an insensitivity to heating longitude...

Abstract: First, we formulate some questions posed by the procedure recently proposed by Borcard et al. (1992) and Borcard and Legendre (1994) to partition the ecological variation of a community into different portions related to spatial and environmental descriptors. Working separately on the two steps of this procedure - linear modelling and ordinations on modelled tables - allows us to propose different solutions to these questions. These solutions, which use little-known proper- ties of a linear regression model with two additive factors and no interaction, are also adapted to the case of mixed factors (qualitative and quantitative). These properties are presented in the framework of canonical correlation analysis. In particular, they allow us to propose an alternative to partial regression, which avoids confounding. A detailed illustration is presented. © Rapid Science 1998

Abstract: Mutual information (MI) analysis represents a general method to detect linear and nonlinear statistical dependencies between time series, and it can be considered as an alternative to the well-known correlation analysis. This article shows how the concept of MI can be used to quantify the coupling between two systems, X and Y. We consider systems as coupled if there are two signals, x(t) and y(t), representing successive measurements of the systems, X and Y, respectively, such that x(t) and y(t) are statistically dependent. Roughly speaking, this means that we can learn anything on x from observations of y, and vice versa. MI represents a measure for the strength of statistical dependencies, hence it could also be used as a measure of coupling. We apply our method to the cardiorespiratory system of a newborn. Here, we find significant changes in the strength of coupling with some characteristic time scales. Typical linear and nonlinear dependencies were found to undergo changes with the sleep states of human newborns. Those changes and scales are also reflected by a correlation analysis. However, we argue that there might be simultaneously rather large correlations, and weak dependencies, quantified by the MI. This can occur because correlation is rather different from M1; correlation describes only linear dependencies, where MI takes into account both linear and nonlinear dependencies.

Abstract: We present a systematic approach for detecting nonlinear components in heart rate variability (HRV). The analysis is based on twenty-three 48-h Holter recordings in healthy persons during sinus rhythm. Although many segments of 1,024 R-R intervals are stationary, only few stationary segments of 8,192-32,768 R-R intervals can be found using a test of Isliker and Kurths (Int. J. Bifurcation Chaos 3:1573-1579, 1993.). By comparing the correlation integrals from these segments and corresponding surrogate data sets, we reject the null hypothesis that these time series are realization of linear processes. On the basis of a test statistic exploring the differences of consecutive R-R intervals, we reject the hypothesis that the R-R intervals represent a static transformation of a linear process using optimized surrogate data. Furthermore, time irreversibility of the heartbeat data is demonstrated. We interpret these results as a strong evidence for nonlinear components in HRV. Thus R-R intervals from healthy persons contain more information than can be extracted by linear analysis in the time and frequency domain.

Abstract: Linear quadratic optimal control theory is applied to the automatic control of two different eight-pool irrigation canals. The model used to design the controller is derived from the Saint-Venant equations discretized through the Preissmann implicit scheme. The linear quadratic closed-loop optimal controller is obtained from steady-state solution of the matrix Riccati equation. A Kalman filter reconstructs the state variables and the unknown perturbations from a reduced number of measured variables. Both perturbation rejection and tracking aspects are incorporated in the controller. Known offtake withdrawals and future targets are anticipated through an open-loop scheme utilizing time varying solutions of the linear quadratic optimization problem. The controller and Kalman filter are tested on a full nonlinear model and prove to be stable, robust, and precise.

Abstract: An approximation to the modified profile likelihood function is proposed. This approximation is invariant under interest-respecting reparameterisations, it agrees with the modified profile likelihood function to order O(n -1 ) in the moderate-deviation sense and to order O(n -1 /2) in the large-deviation sense, and it is stable in the sense of conditional inference. For the case of a curved exponential family model this approximation agrees with the one proposed by Barndorff-Nielsen (1995). A general expression for the approximate modified profile likelihood function is given for nonlinear regression models and for generalised linear models with known dispersion parameter.