This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Web of Science (2) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Nakano, A. Articles by Yang, L. H. Search for Related Content Social Bookmarking                         What's this?

De Novo Ultrascale Atomistic Simulations On High-End Parallel Supercomputers

COLLABORATORY FOR ADVANCED COMPUTING AND SIMULATIONS, UNIVERSITY OF SOUTHERN CALIFORNIA, LOS ANGELES, CA 90089-0242, USA, ANAKANO{at}USC.EDU

Rajiv K. Kalia

COLLABORATORY FOR ADVANCED COMPUTING AND SIMULATIONS, UNIVERSITY OF SOUTHERN CALIFORNIA, LOS ANGELES, CA 90089-0242, USA

Ken-ichi Nomura

COLLABORATORY FOR ADVANCED COMPUTING AND SIMULATIONS, UNIVERSITY OF SOUTHERN CALIFORNIA, LOS ANGELES, CA 90089-0242, USA

Ashish Sharma

COLLABORATORY FOR ADVANCED COMPUTING AND SIMULATIONS, UNIVERSITY OF SOUTHERN CALIFORNIA, LOS ANGELES, CA 90089-0242, USA, DEPARTMENT OF BIOMEDICAL INFORMATICS, OHIO STATE UNIVERSITY, COLUMBUS, OH 43210, USA

Priya Vashishta

COLLABORATORY FOR ADVANCED COMPUTING AND SIMULATIONS, UNIVERSITY OF SOUTHERN CALIFORNIA, LOS ANGELES, CA 90089-0242, USA

Fuyuki Shimojo

COLLABORATORY FOR ADVANCED COMPUTING AND SIMULATIONS, UNIVERSITY OF SOUTHERN CALIFORNIA, LOS ANGELES, CA 90089-0242, USA, DEPARTMENT OF PHYSICS, KUMAMOTO UNIVERSITY, KUMAMOTO 860-8555, JAPAN

Adri C. T. van Duin

MATERIALS AND PROCESS SIMULATION CENTER, CALIFORNIA INSTITUTE OF TECHNOLOGY, PASADENA, CA 91125, USA

William A. Goddard

MATERIALS AND PROCESS SIMULATION CENTER, CALIFORNIA INSTITUTE OF TECHNOLOGY, PASADENA, CA 91125, USA

Rupak Biswas

NASA ADVANCED SUPERCOMPUTING (NAS) DIVISION, NASA AMES RESEARCH CENTER, MOFFETT FIELD, CA 94035, USA

Deepak Srivastava

NASA ADVANCED SUPERCOMPUTING (NAS) DIVISION, NASA AMES RESEARCH CENTER, MOFFETT FIELD, CA 94035, USA

Lin H. Yang

PHYSICS/H DIVISION, LAWRENCE LIVERMORE NATIONAL LABORATORY, LIVERMORE, CA 94551, USA

We present a de novo hierarchical simulation framework for first-principles based predictive simulations of materials and their validation on high-end parallel supercomputers and geographically distributed clusters. In this framework, high-end chemically reactive and non-reactive molecular dynamics (MD) simulations explore a wide solution space to discover microscopic mechanisms that govern macroscopic material properties, into which highly accurate quantum mechanical (QM) simulations are embedded to validate the discovered mechanisms and quantify the uncertainty of the solution. The framework includes an embedded divide-and-conquer (EDC) algorithmic framework for the design of linear-scaling simulation algorithms with minimal bandwidth complexity and tight error control. The EDC framework also enables adaptive hierarchical simulation with automated model transitioning assisted by graph-based event tracking. A tunable hierarchical cellular decomposition parallelization framework then maps the O(N) EDC algorithms onto petaflops computers, while achieving performance tunability through a hierarchy of parameterized cell data/ computation structures, as well as its implementation using hybrid grid remote procedure call + message passing + threads programming. High-end computing platforms such as IBM BlueGene/L, SGI Altix 3000 and the NSF TeraGrid provide an excellent test grounds for the framework. On these platforms, we have achieved unprecedented scales of quantum-mechanically accurate and well validated, chemically reactive atomistic simulations—1.06 billion-atom fast reactive force-field MD and 11.8 million-atom (1.04 trillion grid points) quantum-mechanical MD in the framework of the EDC density functional theory on adaptive multigrids— in addition to 134 billion-atom non-reactive space—time multiresolution MD, with the parallel efficiency as high as 0.998 on 65,536 dual-processor BlueGene/L nodes. We have also achieved an automated execution of hierarchical QM/MD simulation on a grid consisting of 6 supercomputer centers in the US and Japan (in total of 150,000 processor hours), in which the number of processors change dynamically on demand and resources are allocated and migrated dynamically in response to faults. Furthermore, performance portability has been demonstrated on a wide range of platforms such as BlueGene/L, Altix 3000, and AMD Opteron-based Linux clusters.

Key Words: hierarchical simulation • molecular dynamics • reactive force field • quantum mechanics • density functional theory • parallel computing • grid computing

International Journal of High Performance Computing Applications, Vol. 22, No. 1, 113-128 (2008)
DOI: 10.1177/1094342007085015


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?