March 17, 2016
A research group led by Dr. Seiji Tsuboi at Center for Earth Information Science and Technology (CEIST), the Japan Agency for Marine-Earth Science and Technology (JAMSTEC: Asahiko Taira, President) presented high-performance simulations of global seismic wave propagation by using the Spectral-Element Method(SEM)*1 on the K computer*2. It successfully realized an unprecedented accuracy of 1.2 seconds seismic period for a realistic three-dimensional Earth model. The result demonstrated that global seismic wave propagation with a precision of 1 second is possible for realistic three-dimensional (3-D) Earth models. It should help understand earthquake occurrence mechanisms and the Earth's internal structures more precisely, expecting to make significant contributions to disaster prevention and mitigation.
JAMSTEC has been aiming to obtain the most accurate structure of the Earth's interior for improvement of earthquake simulations. It is part of the HPIC Strategic Programs for Innovative Research Field 3*3 as JAMSTEC is acting as a strategic organization. To achieve this goal, performing theoretical simulations of global seismic wave propagation excited by earthquakes is essential. Since the Earth is not a perfect sphere, however, analytical method can't be applied. Instead, it is more common to use numerical methods for simulations of theoretical seismograms. In 2003, the study group achieved the computed theoretical seismograms with record-breaking 5 second frequency for realistic three -dimensional Earth with the Earth Simulator. To simulate seismic wave propagation for a three-dimensional Earth model, P and S waves known as “body waves” still need to run faster at a period of 1 second. By obtaining seismograms with an accuracy of 1.2 seconds for the entire group, it marked a significant step for better understanding of mechanisms of devastating earthquakes.
This project was carried out as part of Project 1, “Improvement of earthquake prediction accuracy” (Managing Director: Prof. Takashi Furumura, The University of Tokyo) in HPIC Strategic Programs for Innovative Research Field 3, “Advanced Prediction Researches for Natural Disaster Prevention and Reduction” (hp140221) by the Ministry of Education, Culture, Sports, Science and Technology. The above results were published on the International Journal of High Performance Computing Applications on February 28, 2016 (JST).
Title: A 1.8 trillion degrees-of-freedom, 1.24 petaflops global seismic wave simulation on the K computer
Authors: Seiji Tsuboi1, Kazuto Ando1, Takayuki Miyoshi1, Daniel Peter2, Dimitri Komatitsch3 and Jeroen Tromp4
1. JAMSTEC 2. King Abdullah University of Science & Technology (KAUST), Extreme Computing Research Center, Kingdom of Saudi Arabia 3. Aix-Marseille University, France 4. Princeton University, USA
*1 Spectral-Element Method (SEM)
The SEM is an optimized high-order version of the finite-element method that is highly accurate for linear hyperbolic problems such as wave propagation. In addition, its mass matrix is perfectly diagonal by construction, which makes it favorable to implement on parallel systems because no linear system needs to be inverted.
*2 K computer
Named for the Japanese word "kei," it is the first machine to achieve 10 quadrillion operations a second. It was jointly developed by RIKEN and Fujitsu as a core system for the “construction of innovative high-performance computing infrastructure (HPCI)” program by the Ministry of Education, Culture, Sports, Science and Technology. In September 2012, the joint operation was started.
*3 HPCI Strategic Programs for Innovative Research: Advanced Prediction Researches for Natural Disaster Prevention and Reduction (Field 3)
The project's strategic goals are "to globally project tropical cyclone trends in simulated global warming, demonstrate prediction of local heavy rain falls, establish the foundation for next-generation earthquake hazard maps and improve tsunami warning accuracy."
Figure 1. A global view of the mesh used at the surface in the SEM. In the SEM, the Earth is divided into six quadrangular pyramids, each of which is then divided into smaller quadrangular ones. They are allocated to individual CPU of the supercomputer for calculation. In this example of simulation performed on the Earth simulator in 2003, six quadrangular pyramids were even divided into 26 x 26, 676 slices in total. On the whole, CPU with a total of 4,053 slices (676 x 6) was used to perform the large scale numerical calculation.
Figure 2. Results of the optimized SEM. The blue line shows a result when only an automatic parallelization was applied for the original codes by the supercomputer's compiler, resulting in failure in parallelization of the processing data. On the other hand, as shown in the red, an optimized programming for effective parallelization enabled parallelization of the target program.
Figure 3. Comparison of numerical seismograms (red curves) with real recordings from the field (black curves) for a seismographic observation station at Matsushiro, Japan during the 2011 off the Pacific coast of Tohoku Earthquake. The horizontal axis is time in seconds for three minutes after the occrenece of the earthquake, and the vertical axis is ground velocity in m/s. The origin of the horizontal axis is the earthquake origin time.