. Both the root mean square error (RMSE) and the mean absolute error (MAE) are regularly employed in model evaluation studies. Willmott and Matsuura (2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and thus the MAE would be a better metric for that purpose. While some concerns over using RMSE raised by Willmott and Matsuura (2005) and Willmott et al. (2009) are valid, the proposed avoidance of RMSE in favor of MAE is not the solution. Citing the aforementioned papers, many researchers chose MAE over RMSE to present their model evaluation statistics when presenting or adding the RMSE measures could be more beneficial. In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009). The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian. In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric, whereas Willmott et al. (2009) indicated that the sums-of-squares-based statistics do not satisfy this rule. In the end, we discussed some circumstances where using the RMSE will be more beneficial. However, we do not contend that the RMSE is superior over the MAE. Instead, a combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.

/pdf/root-mean-square-error-rmse-or-mean-absolute-error-mae-iv2853zp7s.pdf

Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature

Both the root mean square error (RMSE) and the mean absolute error (MAE) are regularly employed in model evaluation studies. Willmott and Matsuura(2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and thus the MAE would be a better metric for that purpose. While some concerns over using RMSE raised by Willmott and Matsuura(2005) andWillmott et al. (2009) are valid, the proposed avoidance of RMSE in favor of MAE is not the solution. Citing the aforementioned papers, many researchers chose MAE over RMSE to present their model evaluation statistics when presenting or adding the RMSE measures could be more beneficial. In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009). The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian. In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric, whereasWillmott et al. (2009) indicated that the sums-ofsquares-based statistics do not satisfy this rule. In the end, we discussed some circumstances where using the RMSE will be more beneficial. However, we do not contend that the RMSE is superior over the MAE. Instead, a combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.

/pdf/root-mean-square-error-rmse-or-mean-absolute-error-mae-4c3klwj6gx.pdf

Root mean square error (RMSE) or mean absolute error (MAE)

http://www.meas.ncsu.edu/aqforecasting/publications/Zhang_AQF_partI.proof.pdf

Real-time air quality forecasting, part I: History, techniques, and current status

. The local and regional influence of elevated point sources on summertime aerosol forcing and cloud-aerosol interactions in northeastern North America was investigated using the WRF-Chem community model. The direct effects of aerosols on incoming solar radiation were simulated using existing modules to relate aerosol sizes and chemical composition to aerosol optical properties. Indirect effects were simulated by adding a prognostic treatment of cloud droplet number and adding modules that activate aerosol particles to form cloud droplets, simulate aqueous-phase chemistry, and tie a two-moment treatment of cloud water (cloud water mass and cloud droplet number) to precipitation and an existing radiation scheme. Fully interactive feedbacks thus were created within the modified model, with aerosols affecting cloud droplet number and cloud radiative properties, and clouds altering aerosol size and composition via aqueous processes, wet scavenging, and gas-phase-related photolytic processes. Comparisons of a baseline simulation with observations show that the model captured the general temporal cycle of aerosol optical depths (AODs) and produced clouds of comparable thickness to observations at approximately the proper times and places. The model overpredicted SO2 mixing ratios and PM2.5 mass, but reproduced the range of observed SO2 to sulfate aerosol ratios, suggesting that atmospheric oxidation processes leading to aerosol sulfate formation are captured in the model. The baseline simulation was compared to a sensitivity simulation in which all emissions at model levels above the surface layer were set to zero, thus removing stack emissions. Instantaneous, site-specific differences for aerosol and cloud related properties between the two simulations could be quite large, as removing above-surface emission sources influenced when and where clouds formed within the modeling domain. When summed spatially over the finest resolution model domain (the extent of which corresponds to the typical size of a single global climate model grid cell) and temporally over a three day analysis period, total rainfall in the sensitivity simulation increased by 31% over that in the baseline simulation. Fewer optically thin clouds, arbitrarily defined as a cloud exhibiting an optical depth less than 1, formed in the sensitivity simulation. Domain-averaged AODs dropped from 0.46 in the baseline simulation to 0.38 in the sensitivity simulation. The overall net effect of additional aerosols attributable to primary particulates and aerosol precursors from point source emissions above the surface was a domain-averaged reduction of 5 W m−2 in mean daytime downwelling shortwave radiation.

/pdf/coupling-aerosol-cloud-radiative-processes-in-the-wrf-chem-18adspnwho.pdf

Coupling aerosol-cloud-radiative processes in the WRF-Chem model: Investigating the radiative impact of elevated point sources

[1] During the NEAQS-ITCT2k4 campaign in New England, anthropogenic VOCs and CO were measured downwind from New York City and Boston. The emission ratios of VOCs relative to CO and acetylene were calculated using a method in which the ratio of a VOC with acetylene is plotted versus the photochemical age. The intercept at the photochemical age of zero gives the emission ratio. The so determined emission ratios were compared to other measurement sets, including data from the same location in 2002, canister samples collected inside New York City and Boston, aircraft measurements from Los Angeles in 2002, and the average urban composition of 39 U.S. cities. All the measurements generally agree within a factor of two. The measured emission ratios also agree for most compounds within a factor of two with vehicle exhaust data indicating that a major source of VOCs in urban areas is automobiles. A comparison with an anthropogenic emission database shows less agreement. Especially large discrepancies were found for the C2-C4 alkanes and most oxygenated species. As an example, the database overestimated toluene by almost a factor of three, which caused an air quality forecast model (WRF-CHEM) using this database to overpredict the toluene mixing ratio by a factor of 2.5 as well. On the other hand, the overall reactivity of the measured species and the reactivity of the same compounds in the emission database were found to agree within 30%.

/pdf/determination-of-urban-volatile-organic-compound-emission-1evpkfbui8.pdf

Determination of urban volatile organic compound emission ratios and comparison with an emissions database

The real-time forecasts of ozone (O 3 ) from seven air quality forecast models (AQFMs) are statistically evaluated against observations collected during July and August of 2004 (53 days) through the Aerometric Information Retrieval Now (AIRNow) network at roughly 340 monitoring stations throughout the eastern United States and southern Canada. One of the first ever real-time ensemble O 3 forecasts, created by combining the seven separate forecasts with equal weighting, is also evaluated in terms of standard statistical measures, threshold statistics, and variance analysis. The ensemble based on the mean of the seven models and the ensemble based on the median are found to have significantly more temporal correlation to the observed daily maximum 1-hour average and maximum 8-hour average O 3 concentrations than any individual model. However, root-mean-square errors (RMSE) and skill scores show that the usefulness of the uncorrected ensembles is limited by positive O 3 biases in all of the AQFMs. The ensembles and AQFM statistical measures are reevaluated using two simple bias correction algorithms for forecasts at each monitor location: subtraction of the mean bias and a multiplicative ratio adjustment, where corrections are based on the full 53 days of available comparisons. The impact the two bias correction techniques have on RMSE, threshold statistics, and temporal variance is presented. For the threshold statistics a preferred bias correction technique is found to be model dependent and related to whether the model overpredicts or underpredicts observed temporal O 3 variance. All statistical measures of the ensemble mean forecast, and particularly the bias-corrected ensemble forecast, are found to be insensitive to the results of any particular model. The higher correlation coefficients, low RMSE, and better threshold statistics for the ensembles compared to any individual model point to their preference as a real-time O 3 forecast.

https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2005JD005858

Assessment of an ensemble of seven real-time ozone forecasts over eastern North America during the summer of 2004

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

A. Chan

Author Tools

Chat about Author

Papers

Assessment of an ensemble of seven real-time ozone forecasts over eastern North America during the summer of 2004