^{1}

^{2}

^{1}

^{1}

^{2}

^{2}

^{2}

^{2}

^{1}

^{2}

Edited by: Katsuichiro Goda, University of Bristol, United Kingdom

Reviewed by: Tatsuya Itoi, The University of Tokyo, Japan; Manolis S. Georgioudakis, National Technical University of Athens, Greece

Specialty section: This article was submitted to Earthquake Engineering, a section of the journal Frontiers in Built Environment

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Integrated earthquake simulation (IES) is a seamless simulation of analyzing all processes of earthquake hazard and disaster. There are two difficulties in carrying out IES, namely, the requirement of large-scale computation and the requirement of numerous analysis models for structures in an urban area, and they are solved by taking advantage of high performance computing (HPC) and by developing a system of automated model construction. HPC is a key element in developing IES, as it needs to analyze wave propagation and amplification processes in an underground structure; a model of high fidelity for the underground structure exceeds a degree-of-freedom larger than 100 billion. Examples of IES for Tokyo Metropolis are presented; the numerical computation is made by using K computer, the supercomputer of Japan. The estimation of earthquake hazard and disaster for a given earthquake scenario is made by the ground motion simulation and the urban area seismic response simulation, respectively, for the target area of 10,000 m × 10,000 m.

Estimation of earthquake hazard and disaster has been a core theme of earthquake engineering, and, recently, some systems have been developed for this purpose; see HAZUS (

The two core elements of the system, namely, the attenuation relation and the fragility curves, are not often used for other purposes except for the assessment of earthquake hazard and disaster for an urban area. For the first element, numerical analysis of earthquake wave propagation is used; the ground motion distribution is obtained for a given earthquake scenario. For the second element, there are many numerical methods for structural seismic responses analysis which are used for the seismic design. Thus, arises a natural question, “why such numerical analysis methods are not used as alternative of the two core elements of the system?” Around the world, some research projects (Si and Midorikawa,

Methodology of estimating earthquake hazard and disaster.

While the question made above is natural, it is not easy to replace the empirical equations with the numerical simulation for the estimation of earthquake hazard and disaster. This is because there are two major difficulties; see Figure

Two requirements in applying numerical simulation to estimation of earthquake hazard and disaster.

The authors have been developing a system for the estimation of earthquake hazard and disaster that uses a set of numerical analysis methods. Developing such a system is a challenging problem even for modern computational science since the target is an urban area. The system is called

This paper is aimed at summarizing recent achievements of developing IES, which are made by applying HPC to IES and using a large-scale parallel computer such as K computer in Japan (Miyamura et al.,

In closing this section, we have to explain the quality of IES as numerical simulation. All the numerical methods that are implemented in IES are verified, but automatically constructed analysis models are not validated; literally no observed data are available for the purpose of validation. Highest quality is thus not expected for IES. The reliability of IES could be evaluated beside for the quality of the numerical simulation; IES employs the rational methodology of simulating the physical processes of earthquake hazard and disaster. No reduced models are used for the earthquake hazard estimation, and reduced but consistent models are sued for the earthquake disaster estimation. The resulting estimation of earthquake hazard and disaster made by IES is being compared with that made by the conventional method together with the observed data of 2011 Tohoku Earthquake.

The progress of computers, both hardware and software, enables us to utilize advanced numerical analysis methods in various fields of science and engineering. For instance, in the field of seismology, available are advanced numerical analysis methods which are capable to compute the seismic wave propagation processes in a large domain the dimension of which is in the crustal length scale (Bao et al.,

The numerical analysis methods mentioned above are not capable to be used in the numerical simulation for the estimation of earthquake hazard and disaster in an urban area. As for the ground motion simulation, there is a limitation in the temporal resolution; the temporal resolution of currently available methods do not reach 10 Hz, which is needed for the accurate computation of structural responses since its major frequency components lie in the range of 1–10 Hz. Near the surface ground, the wave velocity is of the order of 100 m/s, and hence the required spatial resolution is 1 m in order to accurately compute frequency components of 10 Hz which has the wave length of 10 m; this fine resolution is in contrast of the spatial resolution of 100 m that is required for bedrock whose wave velocity is of the order of 1,000 m/s. Accurate computation is essential to estimate the topographical effects of irregular underground structures.

As for the urban area seismic response simulation, we have to construct an analysis model for all structures which are located in a target area. The quality of the constructed analysis model ought to be assured so that the results of the numerical analysis are reliable. Manual construction is not feasible for structures the number of which exceeds 100,000. Moreover, we have to be aware of the fact that perfect digital data about material and structural properties are not available for all the structures. For instance, high-rise buildings have a complete data set for the material and structure components for the construction, but the data set are not open to the public because the buildings are private asset.

We have to mention that the difficulty of constructing an analysis model is shared by the ground motion simulation. This is because the simulating needs a three-dimensional underground structure model which consists of a few soil layers of distinct configuration and material properties. The model must have high fidelity for the configuration of the soil layers, so that the topographical effects are evaluated accurately. However, the data of the soil layers are limited in the quality and quantity. We have to guess as well for the analysis model of the underground structures.

The first difficulty, the requirement of large-scale numerical computation, is solved by making use of HPC. A model of more than 10,000,000 DOF can be analyzed by using a parallel computer of moderate class, and we need to develop a numerical analysis method which possesses sufficient performance or fast analysis of a model of such large DOF. In IES, we have developed an FEM that is capable to solve a model of 1,000,000,000,000 DOF. Numerically solving a mode of this scale is a challenge in the field of HPC; this is regarded as a challenge of capability computing that solves a problem of largest scale. The number of time steps that are needed for the ground motion simulation is of the order of 10,000, since the time increment and the time duration are 0.01 and 100 s, respectively. FEM of IES is fast in analyzing a model of large DOF in repeated times.

The second difficulty, the need of analysis model construction for a large number of structures located in a target urban area, is solved by developing a program of the automated model construction. Automated model construction is regarded as data conversion, in the sense that digital data stored in several data resources are processed to form a set of digital data which correspond to an analysis model. Data resources available to the automated morel construction are of the form of Geographical Information System (GIS), and hence the data conversion is principally possible. As explained in the preceding section, however, there are no GIS’s which have data of the material and structure properties for all structures. We have to guess these properties by interpreting data which are stored in several data resources of GIS.

In the following two subsections, we briefly explain FEM developed for the ground motion simulation and the automated model construction for the urban area response simulation. The points of the explanation are the key feature of FEM and the automated model construction, in order to solve the two difficulties.

We first mention that FEM, rather than finite difference method, is suitable to solve numerical problems of the ground motion simulation, since a major concern of the simulation for the estimation of earthquake hazard is the identification of sites at which larger ground motion is concentrated due to the topographical effects induced by the underground structures. An analysis model of high fidelity is thus needed to model complicated configuration of soil layers, and FEM is the unique solution to analyze such a model.

The major portion of the numerical computation of FEM is used in solving a matrix equation for unknown displacement. That is,

where [^{n}^{n}^{n−1}^{n}^{n}^{n}^{n}^{n}

FEM of IES has developed a fast ^{n}^{n}^{2}) for ordinary algorithms, but

The speed of solving equation (

The scalability of the solver that is implemented in FEM of IES is presented in Figure

Scalability of developed FEM: GAMERA and GAMERA^{EBE} designate the usage of the element-by-element method (GAMERA does not use it); Model 4-A, 4-B, and 4-C have different DOF; outer, inner fine, and inner coarse provide the breakdown of the developed multi-grid solver processes, with outer for the finest grid with double precision arithmetic’s, inner fine and coarse for the coarse grid with single and double precision arithmetic’s, respectively.

The automated model construction has two steps, namely, interpreting data stored in data resources, and converting data of the data resources to an analysis model (Architectural Institute of Japan,

Data resources which are currently available are commercial GIS or 3D maps, or a set of inventories operated by local government. The commercial GIS has configuration data for structures including residential buildings and road networks. The configuration data are the height and floor shape of the structure, together with the location information of a target of the data, which is given as a pair of latitude and altitude. There are some structures whose configuration data include minor errors, such as negative height. The inventories are made for specific purposes such as the registration of real estate. There are the inventories for the structure type and construction year. The location information of a target structure is given as a certain address; mailing address or lot number is mainly used, but some inventories are made as a map and the location information is specified as coordinates of the map.

Interpreting data stored in a data resource is made by understanding the data structure of the data resource. In general, the data resource has several attributes (or data) to each target item, and the data structure means the number of attributes and the property of each attribute; there are cases where an attribute consists of a few attributes. Location information is an attribute. If the data structure is understood, it is possible to make a program for reading a file of the data resource (which is often of binary format) and interpreting data. Since data resources share a similar data structure, aspect-oriented programming makes efficient and robust programming for the program for reading and interpreting, when not a small number of data resources are used.

The difficulty of converting interpreted data to an analysis model depends on the complexity of the model. That is, a fewer model parameters are converted from the interpreted data, as a simpler model is constructed. The simplest model for the structural seismic response analysis is a linear one-degree-of-freedom system, which has two model parameters, a mass and a stiffness. The quality of the model depends on the accuracy of the model parameters, and we have to make rational conversion from the interpreted data to the model parameters. A natural frequency is a key characteristic of a structure, and an empirical relation between the natural frequency and the structure height is available (Architectural Institute of Japan,

Between the step of interpreting data stored in data resources and converting data to an analysis method, we have to combine data for a target structure which are stored in different data resources. If the data include the location information of a target structure in it, combining the data is principally straightforward. However, as explained above, we have to interpret the location information in order to accurately specify the location of a target; this could be understood as conversion of the local coordinate (that is relevant to each data resource) to the global coordinate. There are data resources which have errors about location information or cases where contracting location information is found in different data resources. Combining data of different data resources for one structure is thus difficult, and manual works are needed if data resources which do not have accurate location information in them are used. A flow of the automated model construction is presented in Figure

Flow of automated model construction.

We point out that the automated model construction system is designed for easy operation; the system is coded to take advantage of object-oriented programming together with the aspect-oriented programming. As shown in Figure

Two built-in functions of automated model construction system of IES.

It is not expected that complete information that is needed to construct an analysis model is included in available data resources. To account for the limitation of the available data, IES is able to construct 10,000 or more analysis models for one structure, which are generated by the automated model construction system, suitably varying model parameters. It is another challenge of HPC in terms of capacity computing to construct and analyze numerous models for one target considering the uncertainty of the model parameters; note that the number of analysis models reaches 10,000,000,0000 if IES analyzed 1,000,000 structures located in a target area and constructs 10,000 analysis models for each structure.

As mentioned, all the numerical analysis methods implemented in IES are verified by comparing the numerical solution with analytical solution, but automatically constructed analysis models are not validated. This is because no data are available to fully validate high fidelity model for the underground structure or numerous analysis models of buildings. Thus, IES cannot have highest quality as numerical analysis. Uncertainty quantification is needed for IES.

The greatest uncertainty is an earthquake scenario. Since predicting fault mechanism (or rupture processes on a fault plane) is impossible at this moment, an alternative is to simulate earthquake hazard and disaster for numerous earthquake scenarios. Indeed, capacity computing of HPS is often used for this purpose. Strong ground motion and structural seismic responses change depending on the given scenario, but we can quantitatively estimate a range of possible ground motion and seismic responses which are obtained by capacity computing.

As for man-made structures, we can use capacity computing in which numerous analysis models are used for one structure by changing model parameters. We might use 10,000 models for one structure. It should be noted that even when design data are available, actual structural properties are better than the design values since safety factors are included in the design. Monitoring or sensing is needed for the estimation of the actual structural properties; it is extremely difficult to estimate strength of a structure, compared with its stiffness, since strength is not identified until certain failure takes place in the structure.

In this section, we present examples of IES using capability computing and capacity computing. The target is Tokyo Metropolis, and commercial GIS’s are used as data resources. The ground motion simulation is made for an underground structure consisting of three ground layers, and the urban area seismic response simulation is made by using a non-linear multi-degree-of-freedom system as an analysis model of a residential building. The last example presents the combination of the ground motion simulation and the urban area seismic response simulation.

An analysis model of surface layers is presented in Figure

A model for ground motion simulation.

Figure _{ν} being the velocity response spectra of the ground motion measured or synthesized at the site; as is seen, SI is the average of the velocity response taken over 0.1 and 2.4 s. Kobe Earthquake (JR Takatori) (Japan Meteorological Agency,

Distribution of SI.

It is of interest to compare the results of the above capability computing with the conventional analysis that uses a one-dimensional (1D) stratified model at a target site. In Figure

Comparison of ground motion simulation and conventional 1D analysis.

There are 4,066 residential buildings in the area presented in Figure

The distribution of the maximum story drift angle (MSDA) is presented in Figure

Distribution of MSDA.

Like the preceding subsection, we examine the necessity of making the 3D ground motion simulation, which provide ground motion that is amplified in ground layers and input to a structure on it. The identical analysis models are used for the residential buildings, but input ground motion is either the one computed by using the 3D ground motion simulation or the conventional 1D analysis. The results are presented in Figure

Comparison of ground motion simulation and conventional 1D analysis for estimation of earthquake hazard and disaster.

Due to the lack in relevant data resources, there is larger uncertainty in determining the strength of an analysis model for the residential buildings. While the stiffness can be determined by using empirical relations, it is not easy to determine the maximum force of the springs; the maximum force corresponds to the sum of the strength of walls and columns located on the floor which the spring represents. We apply capacity computing of generating 10,000 models for each residential building, assuming a normal distribution of the strength and assigning a randomly generated value to each spring; the mean of the maximum force is determined by using an empirical relation between the stiffness and the strength, and the SD is assumed to be 10% of the mean. Since the number of the buildings is 4,066, the total number of non-linear analysis models is 40,066,000.

A typical distribution of MSDA for 10,000 analysis models is shown in Figure

Distribution of MSDA for 10,000 analysis models for one residential building.

Distribution of mean, maximum and SD of MSDA.

Using K computer, IES is able to mate the ground motion simulation and the urban area seismic response simulation for a domain of 10,250 m × 9,250 m (Ichimura et al.,

An example of the combined simulation is presented in Figure

Distribution of SI and MSDA computed by making combined simulation of ground motion and seismic response.

It should be emphasized that new findings are never made in the combined simulation of IES. It simply combines ground motion simulation to well-established structural seismic response analysis. However, applying the combined simulation to a large area, we can surely identify spots at which SI takes on a larger value and other spots at which buildings of large MSDA’s are more densely located. The results of the combined simulation are worth being examined as it produces more rational assessment of earthquake hazard and disaster in highest resolution. Such combined simulation made by IES is applicable to any other cities in the world if suitable data resources are available and the automated model construction system generates a suitable model for the city using the data resources.

This paper presents recent achievement of developing Integrated Earthquake Simulation (IES), by taking advantage of High performance computing (HPC). Indeed, IES enhanced with HPC enables us to develop a method of making a rational estimation of earthquake hazard and disaster for Tokyo Metropolis when an earthquake scenario is given. Provided that suitable computational environment and data resources are available, IES is applicable to any urban area. The two difficulties of numerically simulating earthquake hazard and disaster processes are being solved by developing a finite element method (FEM) with a fast solver and by developing a system of automated model construction.

We are planning to extend IES to social science simulations, such as mass evacuation from tsunami, traffic simulation in damaged areas, or recovery of economic activities. This social science simulation needs numerous scenarios of earthquake disasters which are made by applying IES to a target area for various earthquake scenarios. Further spatial resolution will be needed to consider more details of earthquake disasters, and we have to improve FEM of IES. It is another challenge to apply HPC to realize the social science simulation that is needed to increase the resilience of a target area, as it helps us to consider a better recovery plan. Part of the results was obtained by using the K computer at the RIKEN Advanced Institute for Computational Science. We used KiK-net and Japan Seismic Hazard Information Station of National Research Institute for Earth Science and Disaster Prevention (NIED), and National Digital Soil Map provided by Japanese Geotechnical Society. This work was supported by JSPS KAKENHI Grant Number 25220908, MEXT’s program of Post-K project.

The first author is a primary investigator of this research. The second and third authors made numerical analysis programs for earthquake hazard and disaster assessment. The fourth, fifth, and sixth authors carried out numerical computations, constructing urban area models.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer, TI, declared a shared affiliation, though no other collaboration, with several of the authors, MH, TI, and LW, to the handling editor.