Doe Science news source
The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
  • 2017-08-24 13:05:11
  • Article ID: 680037

Big Bang - The Movie

  • Credit: Lindsey Bleem, Nan Li, and the HACC team/Argonne National Laboratory; Mike Gladders/University of Chicago

    A simulated sky image of galaxies produced by running Argonne-developed high-performance computing codes and then running a galaxy formation model. Argonne has collaborated with the University of Illinois, teaming up two supercomputers to perform simulation and data analysis of extremely large-scale, computationally intensive models of the universe.

  • Credit: Joe Insley and Silvio Rizzi/Argonne National Laboratory.

    Dark matter halo distribution from the Outer Rim simulation carried out on 32 racks of Mira with the Hardware/Hybrid Accelerated Cosmology Code (HACC). Shown is the full simulation volume with halos above a certain mass cut. The halos are colored by mass, the heavier halos are shown in red. These halos host the galaxies that we observe with large telescopes.

If you have ever had to wait those agonizing minutes in front of a computer for a movie or large file to load, you’ll likely sympathize with the plight of cosmologists at the U.S. Department of Energy’s (DOE) Argonne National Laboratory. But instead of watching TV dramas, they are trying to transfer, as fast and as accurately as possible, the huge amounts of data that make up movies of the universe – computationally demanding and highly intricate simulations of how our cosmos evolved after the Big Bang.

In a new approach to enable scientific breakthroughs, researchers linked together supercomputers at the Argonne Leadership Computing Facility (ALCF) and at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UI). This link enabled scientists to transfer massive amounts of data and to run two different types of demanding computations in a coordinated fashion – referred to technically as a workflow.

“We talk about building the ‘universe in the lab,’ and simulations are a huge component of that.” - Katrin Heitmann, Argonne cosmologist

What distinguishes the new work from typical workflows is the scale of the computation, the associated data generation and transfer and the scale and complexity of the final analysis. Researchers also tapped the unique capabilities of each supercomputer: They performed cosmological simulations on the ALCF’s Mira supercomputer, and then sent huge quantities of data to UI’s Blue Waters, which is better suited to perform the required data analysis tasks because of its processing power and memory balance.

For cosmology, observations of the sky and computational simulations go hand in hand, as each informs the other. Cosmological surveys are becoming ever more complex as telescopes reach deeper into space and time, mapping out the distributions of galaxies at farther and farther distances, at earlier epochs of the evolution of the universe.

The very nature of cosmology precludes carrying out controlled lab experiments, so scientists rely instead on simulations to provide a unique way to create a virtual cosmological laboratory. “The simulations that we run are a backbone for the different kinds of science that can be done experimentally, such as the large-scale experiments at different telescope facilities around the world,” said Argonne cosmologist Katrin Heitmann. “We talk about building the ‘universe in the lab,’ and simulations are a huge component of that.”

Not just any computer is up to the immense challenge of generating and dealing with datasets that can exceed many petabytes a day, according to Heitmann. “You really need high-performance supercomputers that are capable of not only capturing the dynamics of trillions of different particles, but also doing exhaustive analysis on the simulated data,” she said. “And sometimes, it’s advantageous to run the simulation and do the analysis on different machines.”

Typically, cosmological simulations can only output a fraction of the frames of the computational movie as it is running because of data storage restrictions. In this case, Argonne sent every data frame to NCSA as soon it was generated, allowing Heitmann and her team to greatly reduce the storage demands on the ALCF file system. “You want to keep as much data around as possible,” Heitmann said. “In order to do that, you need a whole computational ecosystem to come together: the fast data transfer, having a good place to ultimately store that data and being able to automate the whole process.”

In particular, Argonne transferred the data produced immediately to Blue Waters for analysis. The first challenge was to set up the transfer to sustain the bandwidth of one petabyte per day.

Once Blue Waters performed the first pass of data analysis, it reduced the raw data – with high fidelity – into a manageable size. At that point, researchers sent the data to a distributed repository at Argonne, the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory and the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. Cosmologists can access and further analyze the data through a system built by researchers in Argonne’s Mathematics and Computer Science Division in collaboration with Argonne’s High Energy Physics Division.

Argonne and University of Illinois built one such central repository on the Supercomputing ’16 conference exhibition floor in November 2016, with memory units supplied by DDN Storage. The data moved over 1,400 miles to the conference’s SciNet network. The link between the computers used high-speed networking through the Department of Energy’s Energy Science Network (ESnet). Researchers sought, in part, to take full advantage of the fast SciNET infrastructure to do real science; typically it is used for demonstrations of technology rather than solving real scientific problems.

“External data movement at high speeds significantly impacts a supercomputer’s performance,” said Brandon George, systems engineer at DDN Storage. “Our solution addresses that issue by building a self-contained data transfer node with its own high-performance storage that takes in a supercomputer’s results and the responsibility for subsequent data transfers of said results, leaving supercomputer resources free to do their work more efficiently.”

The full experiment ran successfully for 24 hours without interruption and led to a valuable new cosmological data set that Heitmann and other researchers started to analyze on the SC16 show floor.

Argonne senior computer scientist Franck Cappello, who led the effort, likened the software workflow that the team developed to accomplish these goals to an orchestra. In this “orchestra,” Cappello said, the software connects individual sections, or computational resources, to make a richer, more complex sound.

He added that his collaborators hope to improve the performance of the software to make the production and analysis of extreme-scale scientific data more accessible. “The SWIFT workflow environment and the Globus file transfer service were critical technologies to provide the effective and reliable orchestration and the communication performance that were required by the experiment,” Cappello said.

“The idea is to have data centers like we have for the commercial cloud. They will hold scientific data and will allow many more people to access and analyze this data, and develop a better understanding of what they’re investigating,” said Cappello, who also holds an affiliate position at NCSA and serves as director of the international Joint Laboratory on Extreme Scale Computing, based in Illinois. “In this case, the focus was cosmology and the universe. But this approach can aid scientists in other fields in reaching their data just as well.”

Argonne computer scientist Rajkumar Kettimuthu and David Wheeler, lead network engineer at NCSA, were instrumental in establishing the configuration that actually reached this performance. Maxine Brown from University of Illinois provided the Sage environment to display the analysis result at extreme resolution. Justin Wozniak from Argonne developed the whole workflow environment using SWIFT to orchestrate and perform all operations.

The Argonne Leadership Computing Facility, the Oak Ridge Leadership Computing Facility, the Energy Science Network and the National Energy Research Scientific Computing Center are DOE Office of Science User Facilities. Blue Waters is the largest leadership-class supercomputer funded by the National Science Foundation. Part of this work was funded by DOE’s Office of Science.

The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign provides supercomputing and advanced digital resources for the nation’s science enterprise. At NCSA, University of Illinois faculty, staff, students, and collaborators from around the globe use advanced digital resources to address research grand challenges for the benefit of science and society. NCSA has been advancing one third of the Fortune 50 for more than 30 years by bringing industry, researchers, and students together to solve grand challenges at rapid speed and scale.

The National Science Foundation (NSF) is an independent federal agency that supports fundamental research and education across all fields of science and engineering. In fiscal year (FY) 2016, its budget is $7.5 billion. NSF funds reach all 50 states through grants to nearly 2,000 colleges, universities and other institutions. Each year, NSF receives more than 48,000 competitive proposals for funding and makes about 12,000 new funding awards. NSF also awards about $626 million in professional and service contracts yearly.

Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy's Office of Science.

The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.

X
X
X
  • Filters

  • × Clear Filters

Relax, Just Break It

Argonne scientists and their collaborators are helping to answer long-held questions about a technologically important class of materials called relaxor ferroelectrics.

Putting Bacteria to Work

Bacteria are diverse and complex creatures that are demonstrating the ability to communicate organism-to-organism and even interact with the moods and perceptions of their hosts (human or otherwise). Scientists call this behavior "bacterial cognition," a systems biology concept that treats these microscopic creatures as beings that can behave like information processing systems.

New Computer Model Predicts How Fracturing Metallic Glass Releases Energy at the Atomic Level

Metallic glasses are an exciting research target for tantalizing applications; however, the difficulties associated with predicting how much energy these materials release when they fracture is slowing down development of metallic glass-based products. Recently, researchers developed a way of simulating to the atomic level how metallic glasses behave as they fracture. This modeling technique could improve computer-aided materials design and help researchers determine the properties of metallic glasses. The duo reports their findings in the Journal of Applied Physics.

The Relationship Between Charge Density Waves and Superconductivity? It's Complicated.

For a long time, physicists have tried to understand the relationship between a periodic pattern of conduction electrons called a charge density wave (CDW), and another quantum order, superconductivity, or zero electrical resistance, in the same material. Do they compete? Co-exist? Co-operate? Do they go their separate ways?

Splitting Water: Nanoscale Imaging Yields Key Insights

In the quest to realize artificial photosynthesis to convert sunlight, water, and carbon dioxide into fuel - just as plants do - researchers need to not only identify materials to efficiently perform photoelectrochemical water splitting, but also to understand why a certain material may or may not work. Now scientists at Lawrence Berkeley National Laboratory have pioneered a technique that uses nanoscale imaging to understand how local, nanoscale properties can affect a material's macroscopic performance.

Feeding Plants to This Algae Could Fuel Your Car

The research shows that a freshwater production strain of microalgae, Auxenochlorella protothecoides, is capable of directly degrading and utilizing non-food plant substrates, such as switchgrass, for improved cell growth and lipid productivity, useful for boosting the algae's potential value as a biofuel.

No More Zigzags: Scientists Uncover Mechanism That Stabilizes Fusion Plasmas

Article describes simulation of physics behind elimination of sawtooth instabilities.

Solutions to Water Challenges Reside at the Interface

Leading Argonne National Laboratory researcher Seth Darling describes the most advanced research innovations that could address global clean water accessibility.

New Cost-Effective Instrument Measures Molecular Dynamics on a Picosecond Timescale

Studying the photochemistry has shown that ultraviolet radiation can set off harmful chemical reactions in the human body and, alternatively, can provide "photo-protection" by dispersing extra energy. To better understand the dynamics of these photochemical processes, a group of scientists irradiated the RNA base uracil with ultraviolet light and documented its behavior on a picosecond timescale. They discuss their work this week in The Journal of Chemical Physics.

Exploding Waves from Colliding Dissipative Pulses

The interaction of traveling waves in dissipative systems, physical systems driven by energy dissipation, can yield unexpected and sometimes chaotic results. These waves, known as dissipative pulses are driving experimental studies in a variety of areas that involve matter and energy flows. In the journal Chaos, researchers discuss their work studying collisions between three types of DSs to determine what happens when these traveling waves interact.


  • Filters

  • × Clear Filters

Professor Miao Yu Named the Priti and Mukesh Chatter '82 Career Development Professor

Miao Yu, associate professor in the Howard P. Isermann Department of Chemical and Biological Engineering at Rensselaer Polytechnic Institute, has been named the Priti and Mukesh Chatter Career Development Professor. His research focuses on developing advanced nanomaterials for energy and environmental applications.

Funding for New DOE Energy Frontier Research Center at Brookhaven Lab

UPTON, NY--The U.S. Department of Energy (DOE) has announced funding for a new Energy Frontier Research Center (EFRC) to be led by DOE's Brookhaven National Laboratory. The Brookhaven EFRC, named "Molten Salts in Extreme Environments," will focus on understanding the properties of a class of materials with potential applications in energy technologies--particularly in nuclear power.

Two Stony Brook Researchers Receive Energy Frontier Research Center Awards Totaling $21.75M

Stony Brook University received notification from the U.S. Department of Energy (DOE) that two proposals directed by SBU faculty to expand or develop Energy Frontier Research Centers (EFRCs) designed to accelerate scientific breakthroughs needed to strengthen U.S. economic leadership and energy security will receive funding totaling $21.75 million. The two Stony Brook EFRCs are the Center for Mesoscale Transport Properties (m2M), led by renowned energy storage researcher, Esther Takeuchi, PhD, which will receive a four-year $12 million grant for the existing center; and the creation of a new EFRC, A Next Generation Synthesis Center (GENESIS) led by John Parise, PhD, which will receive a four-year $9.75 million grant.

Seth Davidovits Wins 2018 Marshall N. Rosenbluth Dissertation Award

Article describes dissertation award won by Seth Davidovits.

DOE Launches New Lab Partnering Service

The U.S. Department of Energy officially launched the Lab Partnering Service (LPS), an on-line, single access point platform for investors, innovators, and institutions to identify, locate, and obtain information from DOE's 17 national laboratories.

Department of Energy Announces $75 Million for High Energy Physics Research

The U.S. Department of Energy (DOE) announced $75 million in funding for 77 university research awards on a range of topics in high energy physics to advance knowledge of how the universe works at its most fundamental level.

Thesis Prize Winner's Calculations Characterize Neutrino Interactions

Alessandro Baroni is helping demystify one of the most mysterious particles. His work is contributing to our understanding of neutrinos, and it has earned him the 2017 Jefferson Science Associates Thesis Prize for work performed on a thesis related to research at the Department of Energy's Thomas Jefferson National Accelerator Facility

10 Questions for Steven Cowley, New Director of the Princeton Plasma Physics Laboratory

Steven Cowley, a theoretical physicist and international authority on fusion energy, became the seventh Director of the Princeton Plasma Physics Laboratory (PPon July 1 and will be Princeton professor of astrophysical sciences on September 1.

Ames Laboratory to lead new Center for Advancement of Topological Semimetals

Ames Laboratory will receive $10.75 million over four yearrs for a new Center for Advancement of Topological Semimetals as one of the Department of Energy's Energy Frontier Research Centers.

DOE Awards $100 Million for Energy Frontier Research Centers

U.S. Secretary of Energy Rick Perry announced $100 million in funding for 42 Energy Frontier Research Centers (EFRCs) to accelerate the scientific breakthroughs needed to strengthen U.S. economic leadership and energy security.


  • Filters

  • × Clear Filters

Steering Light with Dynamic Lens-on-MEMS

Scientists add active control to design capabilities for new lightweight flat optical devices.

Sugar-Coated Sheets Selectively Target Pathogens

Researchers design self-assembling nanosheets that mimic the surface of cells.

Tracking Down Helium-4's Quarks and Gluons

Scientists obtain the first exclusive measurement of deeply virtual Compton scattering of electrons off helium-4, vital to obtaining an unambiguous 3-D view of quarks and gluons within nuclei.

Predicting Magnetic Explosions: From Plasma Current Sheet Disruption to Fast Magnetic Reconnection

Supercomputer simulations and theoretical analysis shed new light on when and how fast reconnection occurs.

Is Nature Exclusively Left Handed? Using Chilled Atoms to Find Out

Elegant techniques of trapping and polarizing atoms open vistas for beta-decay tests of fundamental symmetries, key to understanding the most basic forces and particles constituting our universe.

As Future Batteries, Hybrid Supercapacitors Are Super-Charged

A new supercapacitor could be a competitive alternative to lithium-ion batteries.

Forever Young Catalyst Reduces Diesel Emissions

Atom probe tomography reveals key explanations for stable performance over a cutting-edge diesel-exhaust catalyst's lifetime.

Sense Like a Shark: Saltwater-Submersible Films

A nickelate thin film senses electric field changes analogous to the electroreception sensing organ in sharks, which detects the bioelectric fields of prey.

A Bit of Quantum Logic--What Did the Atom Say to the Quantum Dot?

Let's talk! Scientists demonstrate coherent coupling between a quantum dot and a donor atom in silicon, vital for moving information inside quantum computers.

New Tech Uses Isomeric Beams to Study How and Where the Galaxy Makes One of Its Most Common Elements

A new measurement using a beam of aluminum-26 prepared in a metastable state allows researchers to better understand the creation of the elements in our galaxy.


Spotlight

Tuesday July 03, 2018, 11:05 AM

2018 RHIC & AGS Annual Users' Meeting: 'Illuminating the QCD Landscape'

Brookhaven National Laboratory

Friday June 29, 2018, 06:05 PM

Argonne welcomes The Martian author Andy Weir

Argonne National Laboratory

Monday June 18, 2018, 09:55 AM

Creating STEM Knowledge and Innovations to Solve Global Issues Like Water, Food, and Energy

Illinois Mathematics and Science Academy (IMSA)

Friday June 15, 2018, 10:00 AM

Professor Emily Liu Receives $1.8 Million DoE Award for Solar Power Systems Research

Rensselaer Polytechnic Institute (RPI)

Thursday June 07, 2018, 03:05 PM

Celebrating 40 years of empowerment in science

Argonne National Laboratory

Monday May 07, 2018, 10:30 AM

Introducing Graduate Students Across the Globe to Photon Science

Brookhaven National Laboratory

Wednesday May 02, 2018, 04:05 PM

Students from Massachusetts and Washington Win DOE's 28th National Science Bowl(r)

Department of Energy, Office of Science

Thursday April 12, 2018, 07:05 PM

The Race for Young Scientific Minds

Argonne National Laboratory

Wednesday March 14, 2018, 02:05 PM

Q&A: Al Ashley Reflects on His Efforts to Diversify SLAC and Beyond

SLAC National Accelerator Laboratory

Thursday February 15, 2018, 12:05 PM

Insights on Innovation in Energy, Humanitarian Aid Highlight UVA Darden's Net Impact Week

University of Virginia Darden School of Business

Friday February 09, 2018, 11:05 AM

Ivy League Graduate, Writer and Activist with Dyslexia Visits CSUCI to Reframe the Concept of Learning Disabilities

California State University, Channel Islands

Wednesday January 17, 2018, 12:05 PM

Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

Fermi National Accelerator Laboratory (Fermilab)

Wednesday January 17, 2018, 12:05 PM

Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

Fermi National Accelerator Laboratory (Fermilab)

Wednesday December 20, 2017, 01:05 PM

Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

SLAC National Accelerator Laboratory

Monday December 18, 2017, 01:05 PM

The Future of Today's Electric Power Systems

Rensselaer Polytechnic Institute (RPI)

Monday December 18, 2017, 12:05 PM

Supporting the Development of Offshore Wind Power Plants

Rensselaer Polytechnic Institute (RPI)

Tuesday October 03, 2017, 01:05 PM

Stairway to Science

Argonne National Laboratory

Thursday September 28, 2017, 12:05 PM

After-School Energy Rush

Argonne National Laboratory

Thursday September 28, 2017, 10:05 AM

Bringing Diversity Into Computational Science Through Student Outreach

Brookhaven National Laboratory

Thursday September 21, 2017, 03:05 PM

From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

SLAC National Accelerator Laboratory

Thursday September 07, 2017, 02:05 PM

Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

SLAC National Accelerator Laboratory

Thursday August 31, 2017, 05:05 PM

Binghamton University Opens $70 Million Smart Energy Building

Binghamton University, State University of New York

Wednesday August 23, 2017, 05:05 PM

Widening Horizons for High Schoolers with Code

Argonne National Laboratory

Saturday May 20, 2017, 12:05 PM

Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

Rensselaer Polytechnic Institute (RPI)

Monday May 15, 2017, 01:05 PM

ORNL, University of Tennessee Launch New Doctoral Program in Data Science

Oak Ridge National Laboratory

Friday April 07, 2017, 11:05 AM

Champions in Science: Profile of Jonathan Kirzner

Department of Energy, Office of Science

Wednesday April 05, 2017, 12:05 PM

High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

Argonne National Laboratory

Tuesday March 28, 2017, 12:05 PM

Champions in Science: Profile of Jenica Jacobi

Department of Energy, Office of Science

Friday March 24, 2017, 10:40 AM

Great Neck South High School Wins Regional Science Bowl at Brookhaven Lab

Brookhaven National Laboratory

Wednesday February 15, 2017, 04:05 PM

Middle Schoolers Test Their Knowledge at Science Bowl Competition

Argonne National Laboratory

Friday January 27, 2017, 04:00 PM

Haslam Visits ORNL to Highlight State's Role in Discovering Tennessine

Oak Ridge National Laboratory

Tuesday November 08, 2016, 12:05 PM

Internship Program Helps Foster Development of Future Nuclear Scientists

Oak Ridge National Laboratory

Friday May 13, 2016, 04:05 PM

More Than 12,000 Explore Jefferson Lab During April 30 Open House

Thomas Jefferson National Accelerator Facility

Monday April 25, 2016, 05:05 PM

Giving Back to National Science Bowl

Ames Laboratory

Friday March 25, 2016, 12:05 PM

NMSU Undergrad Tackles 3D Particle Scattering Animations After Receiving JSA Research Assistantship

Thomas Jefferson National Accelerator Facility

Tuesday February 02, 2016, 10:05 AM

Shannon Greco: A Self-Described "STEM Education Zealot"

Princeton Plasma Physics Laboratory

Monday November 16, 2015, 04:05 PM

Rare Earths for Life: An 85th Birthday Visit with Mr. Rare Earth

Ames Laboratory

Tuesday October 20, 2015, 01:05 PM

Meet Robert Palomino: 'Give Everything a Shot!'

Brookhaven National Laboratory

Tuesday April 22, 2014, 11:30 AM

University of Utah Makes Solar Accessible

University of Utah

Wednesday March 06, 2013, 03:40 PM

Student Innovator at Rensselaer Polytechnic Institute Seeks Brighter, Smarter, and More Efficient LEDs

Rensselaer Polytechnic Institute (RPI)

Friday November 16, 2012, 10:00 AM

Texas Tech Energy Commerce Students, Community Light up Tent City

Texas Tech University

Wednesday November 23, 2011, 10:45 AM

Don't Get 'Frosted' Over Heating Your Home This Winter

Temple University

Wednesday July 06, 2011, 06:00 PM

New Research Center To Tackle Critical Challenges Related to Aircraft Design, Wind Energy, Smart Buildings

Rensselaer Polytechnic Institute (RPI)

Friday April 22, 2011, 09:00 AM

First Polymer Solar-Thermal Device Heats Home, Saves Money

Wake Forest University

Friday April 15, 2011, 12:25 PM

Like Superman, American University Will Get Its Energy from the Sun

American University

Thursday February 10, 2011, 05:00 PM

ARRA Grant to Help Fund Seminary Building Green Roof

University of Chicago





Showing results

0-4 Of 2215