This is a continuing profile series on the directors of the Department of Energy (DOE) Office of Science User Facilities. These scientists lead a variety of research institutions that provide researchers with the most advanced tools of modern science including accelerators, colliders, supercomputers, light sources and neutron sources, as well as facilities for studying the nano world, the environment, and the atmosphere.
Contributing Author Credit: Laura Wolf is a writer and editor at Argonne National Laboratory, [email protected]
Meet the director:
Michael E. Papka likes to get his hands dirty.
That’s one way he describes his work at the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science user facility located outside Chicago. Papka, a senior scientist and deputy associate laboratory director at DOE’s Argonne National Laboratory and the ALCF’s director since 2010, joined the laboratory more than two decades ago as a young computer scientist with expertise in visualization. Today, his responsibilities include overseeing a research program of some of the biggest and most ambitious computational science campaigns in the world.
Papka’s enthusiasm for turning computational ones and zeros into useful insights is perhaps equal only to the supercomputing resources themselves. The major systems that the ALCF operates are among the fastest in the world and all have been designed in partnership with industry to support specific types of computational workloads. As director, Papka must understand what the research community wants and needs from each new generation of machine and also what the industry partners can practically deliver in terms of speed, capability, and power requirements every few years.
The director’s background:
As a physics undergrad at Northern Illinois University (NIU), Papka began a part-time job analyzing data for a professor doing research at DOE’s Fermi National Accelerator Laboratory. He enjoyed the work and the lab staff’s esprit de corps so much that he began to rethink his graduate school plans—eventually changing schools and his major to computer science.
A chance meeting between his master’s thesis advisor and Rick Stevens, the then-director of Argonne’s Mathematics and Computer Science division, resulted in a job offer and Papka began working with scientists to help translate their simulation data into images. “The opportunity to collaborate on a range of science problems, early in my career, probably prepared me the most for the job I do now,” says Papka.
Papka then earned a second master’s and a Ph.D. in computer science from the University of Chicago, supervised by Stevens, now Argonne’s associate laboratory director for Computing, Environment and Life Sciences.
Papka continues to be passionate about education. In addition to his duties at Argonne, he is a professor of computer science at NIU, where he teaches courses in visualization and data analysis and supervises graduate and undergraduate students.
The Facility:
“You must be this tall to ride,” jokes Papka about the computational demands of the projects that run on the ALCF machines. Just as amusement parks insist on minimum height standards before allowing riders in, users seeking access to the ALCF’s resources must meet certain standards—first and foremost, their problem must be large enough and capable of using petascale, soon-to-be exascale, resources. Typically, researchers have worked on their own institutions’ sizeable computer systems, then moved onto a larger machine funded by a national agency. And then, for some researchers, “You get to the point where you need the biggest machines in the world to do your science,” says Papka.
Time on the ALCF machines is awarded through two major DOE Office of Science programs, the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) and the Advanced Leadership Computing Challenge (ALCC), both part of the Advanced Scientific Computing Research program. In addition, the ALCF awards discretionary projects, primarily aimed at preparing smaller projects for the scale of the ALCF machines, and future-looking programs, like the ALCF Data Science Program (ADSP).
The ALCF was formed in 2006 and started with a five teraflop (five trillion calculations per second) machine known as the Blue Gene/L. One of the ALCF’s two current petaflop supercomputers, Mira, recently retired at the end of 2019. The other, a Cray system called Theta, will continue to support a growing number of projects through 2021 and beyond. Aurora, the ALCF’s next-generation exascale system, is expected to go online in 2021 and will be capable of calculating one quintillion calculations per second, or a billion billion calculations per second.
It can take up to a year to install, test, and debug a new supercomputer. “If your home computer breaks, you can go to Best Buy or the Apple Store and they repair it. When something goes wrong with our machine,” says Papka, “there are thousands of parts to debug.”
Papka shared an example of how the synergy between these large machines and other DOE research facilities have changed the way basic science research is done at the national labs. There was an experiment taking place on the Argonne campus at another DOE Office of Science user facility, the Advanced Photon Source (APS). Scientists collected data at the APS and then analyzed them in real time on the ALCF supercomputer, which signaled a problem. This catch saved the experiment. The researchers were able to quickly figure out what was wrong, make corrections, and continue the experiment at the APS to get good data. “That,” explains Papka, “is where a supercomputer has enough power to give researchers answers in real time.”
Typical day:
Papka’s workday typically starts at 5 A.M. at home and ends at the lab, usually long after 6 P.M., and includes a large number of meetings a day with various lab personnel, research staff, and students. He considers one of his main responsibilities as a facility director to be predicting how current and future users’ needs will continue to evolve. “I rely on an incredibly talented group of people to help me do that.”
The ALCF users have widely varying research goals and what they require to reach those goals. “As a young researcher, my job was to listen and then answer needs with deliverables,” says Papka. “Now I’m more of a facilitator and make sure others have what they need to do their work.”
Typical experiment:
At the ALCF, users’ projects cover the full spectrum from the research of very basic science (evolution of the universe) to engineering applications (new materials, engine design, and energy efficiency).
One of the many research projects employs machine learning as part of the data analysis. University of Cambridge materials scientist Jacqueline Cole, who also holds a joint appointment at Argonne, aims to discover a next-generation photovoltaic technology that mimics photosynthesis. This technology, the dye-sensitized solar cell, is a leading contender for future ‘smart windows’ that would generate electricity from sunlight. Cole and her team use the ALCF resources to identify candidates for new dye materials that would be suitable for such cells. The overarching concept of this work is to search through a representative set of all possible chemical molecules and use artificial intelligence to target molecules with optical properties that would yield optimum device function in these solar cells.
Another project combines observation with next-generation data analysis and simulation. The Computing the Sky at Extreme Scales or “ExaSky” project, led by Argonne physicists and computational scientists Salman Habib and Katrin Heitmann, will reconstruct all the processes in the formation of the universe. Habib’s and Heitmann’s team has spent time rewriting their code from scratch, making it among the fastest science production codes in use today. This rewriting is also allowing the work to move very efficiently from one computer architecture to another and from one machine generation to the next.
Best advice for a future director at the ALCF:
Papka is actively evolving the capabilities and overall utility of Leadership Computing Facilities. “Figuratively speaking, we want to blow the walls off these facilities,” says Papka. “For decades the procedure has been to write code, compile it, submit it to the facility queue, run the job, collect the data, and then take it back to your home institution for analysis,” says Papka. “I think that day is done.”
He expects real-time computing, with analysis happening while the simulations are running, to become the standard.
“The rest of the world has seen the power of supercomputing and what it can bring to their innovation. We are seeing a more diversified deployment of supercomputers worldwide—which is a good thing.”
As to Papka’s advice for his successor someday: “DOE Leadership Computing Facilities exist to serve the research community. As stewards of these resources, we must provide an environment that not only meets the needs of the user but the future they have not yet imagined.”
In Fiscal Year 2019, the ALCF and the other user facilities welcomed more than 35,000 researchers – from academic, industry, and government laboratories in all 50 states and the District of Columbia – to perform new scientific research. For details on the DOE Office of Science User Facilities, go to https://science.energy.gov/user-facilities/.
Please go to Profiles of User Facilities Directors to read more articles on the directors for the Office of Science user facilities.
The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, please visit www.energy.gov/science.
RSS