Computer Science Department

Computer Science Department

Simulating Subsurface Flow and Reactive Transport using Ultrascale Computers

To provide true predictive utility, subsurface simulations often must accurately resolve--in three dimensions--complicated, multi-phase flow fields in highly heterogeneous geology with numerous chemical species and complex chemistry. This task is especially daunting because of the wide range of spatial scales involved--from the pore scale to the field scale--ranging over six orders of magnitude, and the wide range of time scales ranging from seconds or less to millions of years. This represents a true "Grand Challenge" computational problem, requiring not only the largest-scale ("ultrascale") supercomputers, but accompanying advances in algorithms for the efficient numerical solution of systems of PDEs using these machines, and in mathematical modeling techniques that can adequately capture the truly multi-scale nature of these problems.

In this talk, I will describe some of the specific challenges involved in simulating subsurface reactive flows using ultrascale supercomputers and discuss how they are being addressed in the development of the computer code PFLOTRAN. I will begin with some brief background material on the governing equations of subsurface reactive flows and on modern parallel computer architectures and programming models. I will then discuss the software and algorithmic approaches that have enabled PFLOTRAN to provide scalable parallel performance on tens of thousands of processors. I will describe ongoing work to address disparate time and spatial scales by both the development of adaptive mesh refinement methods and the use of multiple continuum formulations. Finally, I will present some examples from recent simulations conducted on Jaguar, the 23412 processor core Cray XT3/4 system at Oak Ridge National Laboratory that is currently one of the top ten fastest supercomputers in the world.