Journal Article10.1007/S10586-017-1477-0
Performance optimization and evaluation for parallel processing of big data in earth system models
15
TL;DR: An optimization algorithm for the massive data communication between the Weather Research and Forecasting model and Coupler version 7 in the Chinese Academy of Sciences-Earth System Model (CAS-ESM) is proposed and a generic performance evaluation method for ESMs from perspectives of optimal load balance and efficiency is proposed.
read more
Abstract: Big data and high performance computing in Earth System Models (ESMs) are receiving increased attention in earth science research. When scaling to large-scale multi-core computing, efficient parallelization of an ESM, which demands fast parallel computing for long-term integration or climate simulation, becomes extremely challenging because of time-consuming internal big data communication. In this paper, an optimization algorithm for the massive data communication between the Weather Research and Forecasting model and Coupler version 7 in the Chinese Academy of Sciences-Earth System Model (CAS-ESM) is proposed. The optimization strategy is to transmit data from a small packet into a larger packet. Through experiments on a multi-core cluster, the efficiency of the algorithm is confirmed. Then, the parallel performance of the CAS-ESM is evaluated fully. Results show that the parallel efficiency of the CAS-ESM on 1024 CPU cores reaches nearly 70%, indicating that the CAS-ESM has desirable parallel performance and strong scalability. In addition, a generic performance evaluation method for ESMs from perspectives of optimal load balance and efficiency is proposed. Results show that the computing speed is the fastest and computational efficiency is the highest when the CAS-ESM runs on a certain number of cores.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
GPUs-RRTMG_LW: high-efficient and scalable computing for a longwave radiative transfer model on multiple GPUs
TL;DR: This paper demonstrates an approach that runs a large-scale, computationally intensive, longwave radiative transfer model on a GPU cluster and demonstrates that the multi-GPU acceleration algorithm is valid, scalable, and highly efficient when compared to a single GPU or CPU.
17
Cloud computing model for big data processing and performance optimization of multimedia communication
Zhicheng Zhou,Liang Zhao +1 more
TL;DR: The experimental results show that compared with the existing algorithms, the cloud algorithm proposed in this paper has better acceleration speed, faster convergence speed and fewer iterations.
15
GPU-RRTMG_SW: Accelerating a Shortwave Radiative Transfer Scheme on GPU
TL;DR: In this article, two kinds of algorithms using GPU technology to accelerate the RRTMG shortwave radiation scheme (RRTMG_SW) are proposed, and an optimization method for data transmission between host and device is proposed.
Research on an advanced intelligence implementation system for engineering process in industrial field under big data
TL;DR: This paper proposes In-External (IE) algorithm to perform the assignment of big data distribution for parallel data processing, which can fully utilize the capacity of Hadoop system and attain the best efficiency of knowledge reasoning.
10
A Novel GPU-Based Acceleration Algorithm for a Longwave Radiative Transfer Model
TL;DR: This paper proposes a GPU-based acceleration algorithm for the RRTMG longwave radiation scheme (RRTMG_LW) and implements the algorithm in CUDA Fortran, and experimental results indicated that the algorithm was effective.
10
References
An Overview of CMIP5 and the Experiment Design
TL;DR: The fifth phase of the Coupled Model Intercomparison Project (CMIP5) will produce a state-of-the- art multimodel dataset designed to advance the authors' knowledge of climate variability and climate change.
A Description of the Advanced Research WRF Version 3
C. Skamarock,B. Klemp,Jimy Dudhia,O. Gill,Dale Barker,G. Duda,Xiang-Yu Huang,Wei Wang,G. Powers +8 more
- 01 Jan 2008
TL;DR: The Technical Note series provides an outlet for a variety of NCAR manuscripts that contribute in specialized ways to the body of scientific knowledge but which are not suitable for journal, monograph, or book publication.
The Community Earth System Model: A Framework for Collaborative Research
James W. Hurrell,Marika M. Holland,Peter R. Gent,Steven J. Ghan,Jennifer E. Kay,Paul J. Kushner,Jean-Francois Lamarque,William G. Large,David M. Lawrence,Keith Lindsay,William H. Lipscomb,Matthew C. Long,Natalie M. Mahowald,Daniel R. Marsh,Richard Neale,Philip J. Rasch,S. J. Vavrus,M. Vertenstein,David C. Bader,William D. Collins,James J. Hack,Jeffrey T. Kiehl,Shawn J. Marshall +22 more
TL;DR: The Community Earth System Model (CESM) as discussed by the authors is a community tool used to investigate a diverse set of Earth system interactions across multiple time and space scales, including biogeochemical cycles, a variety of atmospheric chemistry options, the Greenland Ice Sheet, and an atmosphere that extends to the lower thermosphere.
CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model
John M. Dennis,Jim Edwards,Katherine J. Evans,Oksana Guba,Peter H. Lauritzen,Arthur A. Mirin,Amik St-Cyr,Mark A. Taylor,Patrick H. Worley +8 more
- 01 Feb 2012
TL;DR: This work gives scalability and performance results from CAM running with three different dynamical core options within the Community Earth System Model, using a pre-industrial time-slice configuration, and focuses on high-resolution simulations.
A new flexible coupler for earth system modeling developed for CCSM4 and CESM1
Anthony Craig,Mariana Vertenstein,Robert Jacob +2 more
- 01 Feb 2012
TL;DR: The latest versions of the model, CCSM4 and CESM1, contain totally new coupling capabilities in the CPL7 coupler that permit additional flexibility and extensibility to address the challenges involved in earth system modeling.
308