Forecasting with Econometric Methods: A Comment

TL;DR: This paper pointed out that forecasting from econometric models is not more accurate than forecasting from simple mechanical schemes or autoregressions, contrary to the evidence, and pointed out the failure of econometrics to remain stable over time.

Abstract: I have long smelled a rat with respect to econometric forecasting, and I am glad Professor Armstrong finally flushed one out. In addition to synthesizing the sciences of economics and animal psychology, Armstrong raises these two major points in his paper: (1) Forecasts from econometric models are not more accurate than forecasts from simple mechanical schemes or autoregressions; (2) econometricians still believe they are, contrary to the evidence. Armstrong's focus on forecasting accuracy is a bit misdirected, however, because it lets escape the biggest rat of all: the failure of econometric models to remain stable over time. Stability of econometric models is required both to measure forecasting accuracy and to perform policy simulations-the task for which econometric models were primarily intended. I will comment first on Armstrong's two major points concerning forecasting accuracy of econometric models and then offer an explanation of their failure to remain invariant over time. The implications I draw for the direction of future research differ considerably from Armstrong's. Most, if not all, of the studies Armstrong cites in support of the first point have two defects: They utilize questionable measures of accuracy, and the forecasts from econometric models which they analyze are really hybrid mixtures generated from judgmental and econometric techniques. In order to clarify the meaning of forecasting accuracy, it is useful to formally specify the underlying statistical problem. To begin, suppose we are interested in obtaining a forecast of the value of a particular