DOE News
    Doe Science news source
    The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
    • 2017-08-24 13:05:11
    • Article ID: 680037

    Big Bang - The Movie

    • Credit: Lindsey Bleem, Nan Li, and the HACC team/Argonne National Laboratory; Mike Gladders/University of Chicago

      A simulated sky image of galaxies produced by running Argonne-developed high-performance computing codes and then running a galaxy formation model. Argonne has collaborated with the University of Illinois, teaming up two supercomputers to perform simulation and data analysis of extremely large-scale, computationally intensive models of the universe.

    • Credit: Joe Insley and Silvio Rizzi/Argonne National Laboratory.

      Dark matter halo distribution from the Outer Rim simulation carried out on 32 racks of Mira with the Hardware/Hybrid Accelerated Cosmology Code (HACC). Shown is the full simulation volume with halos above a certain mass cut. The halos are colored by mass, the heavier halos are shown in red. These halos host the galaxies that we observe with large telescopes.

    If you have ever had to wait those agonizing minutes in front of a computer for a movie or large file to load, you’ll likely sympathize with the plight of cosmologists at the U.S. Department of Energy’s (DOE) Argonne National Laboratory. But instead of watching TV dramas, they are trying to transfer, as fast and as accurately as possible, the huge amounts of data that make up movies of the universe – computationally demanding and highly intricate simulations of how our cosmos evolved after the Big Bang.

    In a new approach to enable scientific breakthroughs, researchers linked together supercomputers at the Argonne Leadership Computing Facility (ALCF) and at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UI). This link enabled scientists to transfer massive amounts of data and to run two different types of demanding computations in a coordinated fashion – referred to technically as a workflow.

    “We talk about building the ‘universe in the lab,’ and simulations are a huge component of that.” - Katrin Heitmann, Argonne cosmologist

    What distinguishes the new work from typical workflows is the scale of the computation, the associated data generation and transfer and the scale and complexity of the final analysis. Researchers also tapped the unique capabilities of each supercomputer: They performed cosmological simulations on the ALCF’s Mira supercomputer, and then sent huge quantities of data to UI’s Blue Waters, which is better suited to perform the required data analysis tasks because of its processing power and memory balance.

    For cosmology, observations of the sky and computational simulations go hand in hand, as each informs the other. Cosmological surveys are becoming ever more complex as telescopes reach deeper into space and time, mapping out the distributions of galaxies at farther and farther distances, at earlier epochs of the evolution of the universe.

    The very nature of cosmology precludes carrying out controlled lab experiments, so scientists rely instead on simulations to provide a unique way to create a virtual cosmological laboratory. “The simulations that we run are a backbone for the different kinds of science that can be done experimentally, such as the large-scale experiments at different telescope facilities around the world,” said Argonne cosmologist Katrin Heitmann. “We talk about building the ‘universe in the lab,’ and simulations are a huge component of that.”

    Not just any computer is up to the immense challenge of generating and dealing with datasets that can exceed many petabytes a day, according to Heitmann. “You really need high-performance supercomputers that are capable of not only capturing the dynamics of trillions of different particles, but also doing exhaustive analysis on the simulated data,” she said. “And sometimes, it’s advantageous to run the simulation and do the analysis on different machines.”

    Typically, cosmological simulations can only output a fraction of the frames of the computational movie as it is running because of data storage restrictions. In this case, Argonne sent every data frame to NCSA as soon it was generated, allowing Heitmann and her team to greatly reduce the storage demands on the ALCF file system. “You want to keep as much data around as possible,” Heitmann said. “In order to do that, you need a whole computational ecosystem to come together: the fast data transfer, having a good place to ultimately store that data and being able to automate the whole process.”

    In particular, Argonne transferred the data produced immediately to Blue Waters for analysis. The first challenge was to set up the transfer to sustain the bandwidth of one petabyte per day.

    Once Blue Waters performed the first pass of data analysis, it reduced the raw data – with high fidelity – into a manageable size. At that point, researchers sent the data to a distributed repository at Argonne, the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory and the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. Cosmologists can access and further analyze the data through a system built by researchers in Argonne’s Mathematics and Computer Science Division in collaboration with Argonne’s High Energy Physics Division.

    Argonne and University of Illinois built one such central repository on the Supercomputing ’16 conference exhibition floor in November 2016, with memory units supplied by DDN Storage. The data moved over 1,400 miles to the conference’s SciNet network. The link between the computers used high-speed networking through the Department of Energy’s Energy Science Network (ESnet). Researchers sought, in part, to take full advantage of the fast SciNET infrastructure to do real science; typically it is used for demonstrations of technology rather than solving real scientific problems.

    “External data movement at high speeds significantly impacts a supercomputer’s performance,” said Brandon George, systems engineer at DDN Storage. “Our solution addresses that issue by building a self-contained data transfer node with its own high-performance storage that takes in a supercomputer’s results and the responsibility for subsequent data transfers of said results, leaving supercomputer resources free to do their work more efficiently.”

    The full experiment ran successfully for 24 hours without interruption and led to a valuable new cosmological data set that Heitmann and other researchers started to analyze on the SC16 show floor.

    Argonne senior computer scientist Franck Cappello, who led the effort, likened the software workflow that the team developed to accomplish these goals to an orchestra. In this “orchestra,” Cappello said, the software connects individual sections, or computational resources, to make a richer, more complex sound.

    He added that his collaborators hope to improve the performance of the software to make the production and analysis of extreme-scale scientific data more accessible. “The SWIFT workflow environment and the Globus file transfer service were critical technologies to provide the effective and reliable orchestration and the communication performance that were required by the experiment,” Cappello said.

    “The idea is to have data centers like we have for the commercial cloud. They will hold scientific data and will allow many more people to access and analyze this data, and develop a better understanding of what they’re investigating,” said Cappello, who also holds an affiliate position at NCSA and serves as director of the international Joint Laboratory on Extreme Scale Computing, based in Illinois. “In this case, the focus was cosmology and the universe. But this approach can aid scientists in other fields in reaching their data just as well.”

    Argonne computer scientist Rajkumar Kettimuthu and David Wheeler, lead network engineer at NCSA, were instrumental in establishing the configuration that actually reached this performance. Maxine Brown from University of Illinois provided the Sage environment to display the analysis result at extreme resolution. Justin Wozniak from Argonne developed the whole workflow environment using SWIFT to orchestrate and perform all operations.

    The Argonne Leadership Computing Facility, the Oak Ridge Leadership Computing Facility, the Energy Science Network and the National Energy Research Scientific Computing Center are DOE Office of Science User Facilities. Blue Waters is the largest leadership-class supercomputer funded by the National Science Foundation. Part of this work was funded by DOE’s Office of Science.

    The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign provides supercomputing and advanced digital resources for the nation’s science enterprise. At NCSA, University of Illinois faculty, staff, students, and collaborators from around the globe use advanced digital resources to address research grand challenges for the benefit of science and society. NCSA has been advancing one third of the Fortune 50 for more than 30 years by bringing industry, researchers, and students together to solve grand challenges at rapid speed and scale.

    The National Science Foundation (NSF) is an independent federal agency that supports fundamental research and education across all fields of science and engineering. In fiscal year (FY) 2016, its budget is $7.5 billion. NSF funds reach all 50 states through grants to nearly 2,000 colleges, universities and other institutions. Each year, NSF receives more than 48,000 competitive proposals for funding and makes about 12,000 new funding awards. NSF also awards about $626 million in professional and service contracts yearly.

    Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy's Office of Science.

    The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.

    X
    X
    X
    • Filters

    • × Clear Filters

    Getting To Know the Microbes that Drive Climate Change

    The genetics of viruses living along a permafrost thaw gradient may help scientists better predict the pace of climate change.

    Taking magnetism for a spin:

    Scientists at the U.S. Department of Energy's Ames Laboratory have discovered the relaxation dynamics of a zero-field state in skyrmions, a spinning magnetic phenomenon that has potential applications in data storage and spintronic devices.

    A Challenging Future for Tropical Forests

    Mortality rates of moist tropical forests are on the rise due to environmental drivers and related mechanisms.

    Stronger, lighter, greener

    A new award-winning magnet technology invented at the U.S. Department of Energy's Argonne National Laboratory could help drive the nation's transition from gas-powered vehicles to electric and hybrid power more rapidly, at lower cost, and in a more environmentally friendly way.

    Science Up-Close: Developing a Cookbook for Efficient Fusion Energy

    To develop a future fusion reactor, scientists need to understand how and why plasma in fusion experiments moves into a "high-confinement mode" where particles and heat can't escape. Scientists at the Department of Energy's Princeton Plasma Physics Laboratory simulated the transition into that mode starting from the most basic physics principles.

    Peering into the Mist: How Water Vapor Changes Metal at the Atomic Level

    New insights into molecular-level processes could help prevent corrosion and improve catalytic conversion.

    Neutron science publications reach new highs at ORNL's flagship facilities

    The High Flux Isotope Reactor and the Spallation Neutron Source at the Department of Energy's Oak Ridge National Laboratory have reached new levels of increased science productivity. In 2018, a record high of more than 500 scientific instrument publications were produced between HFIR and SNS--based on neutron beamline experiments conducted by more than 1,200 US and international researchers who used the world-leading facilities.

    Fiery sighting: A new physics of eruptions that damage fusion experiments

    Feature describes first direct sighting of a trigger for bursts of heat that can disrupt fusion reactions.

    Microbial Types May Prove Key to Gas Releases from Thawing Permafrost

    Scientists discover key types of microbes that degrade organic matter and release carbon dioxide and methane into the atmosphere.

    An effect that Einstein helped discover 100 years ago offers new insight into a puzzling magnetic phenomenon

    Experiments at the Department of Energy's SLAC National Accelerator Laboratory have seen for the first time what happens when magnetic materials are demagnetized at ultrafast speeds of millionths of a billionth of a second: The atoms on the surface of the material move, much like the iron bar did. The work, done at SLAC's Linac Coherent Light Source (LCLS) X-ray laser, was published in Nature earlier this month.


    • Filters

    • × Clear Filters

    Remote-Control Plasma Physics Experiment is Named One of Top Webcams of 2018

    EarthCam names remote-control experiment at PPPL one of 25 most interesting Webcams of 2018.

    Jefferson Lab Scientist Awarded Distinguished Lectureship

    Cynthia Keppel, leader of Jefferson Lab's Halls A&C, has been honored with the APS 2019 Distinguished Lectureship Award on the Applications of Physics.

    Journal Special Issues Honor Chemists Radoslav Adzic and Jan Hrbek

    The Journal of the Electrochemical Society and Surface Science recognized the contributions of Brookhaven Lab chemists Radoslav Adzic and Jan Hrbek to electrocatalysis and catalysis.

    Argonne scientist elected as SAE Fellow

    Scientist Michael Wang from the U.S. Department of Energy's (DOE) Argonne National Laboratory was recently inducted as a Fellow of the professional engineering organization SAE (Society of Automotive Engineers). The organization reserves this prestigious grade of membership for thosewho have made significant contributions to mobility technology and have demonstrated leadership in their field.

    Top 10 Discoveries of 2018

    Every year, the U.S. Department of Energy's (DOE) Brookhaven National Laboratory compiles a list of the biggest advances made by the Lab's staff scientists, engineers, and visiting researchers. From uncovering mysteries of the universe to building better batteries, here, in no particular order, are our picks for the top 10 discoveries of 2018.

    U.S. Department of Energy Announces $33 Million for Small Business Research and Development Grants

    The U.S. Department of Energy announced it will award 189 grants totaling $33 million to 149 small businesses in 32 states.

    DOE to Provide $16 Million for New Research into Atmospheric and Terrestrial Processes

    The U.S. Department of Energy (DOE) announced a plan to provide $16 million for new observational research aimed at improving the accuracy of today's climate and earth system models.

    Machine learning award powers Argonne leadership in engine design

    When attempting to design engines to be more fuel-efficient and emissions-free, automotive manufacturers have to take into account all the complexity inherent in the combustion process.

    ORNL partners with industry to address multiple nuclear technology challenges

    The Department of Energy's Oak Ridge National Laboratory is collaborating with industry on six new projects focused on advancing commercial nuclear energy technologies that offer potential improvements to current nuclear reactors and move new reactor designs closer to deployment.

    Lithium earns honors for three physicists working to bring the energy that powers the sun to Earth

    Feature describes research of three PPPL physicists who have won the laboratory's 2018 outstanding research awards


    • Filters

    • × Clear Filters

    Getting To Know the Microbes that Drive Climate Change

    The genetics of viruses living along a permafrost thaw gradient may help scientists better predict the pace of climate change.

    Observing Clouds in Four Dimensions

    Six cameras are revolutionizing observations of shallow cumulus clouds.

    A Challenging Future for Tropical Forests

    Mortality rates of moist tropical forests are on the rise due to environmental drivers and related mechanisms.

    Rapid Lake Draining on Ice Sheets Changes How Water Moves in Unexpected Ways

    Widespread fracturing during lake drainage triggers vertical shafts to form that affect the Greenland Ice Sheet.

    New Historical Emissions Trends Estimated with the Community Emissions Data System

    The data system will allow for more detailed, consistent, and up-to-date global emissions trends that will aid in understanding aerosol effects.

    Peering into the Mist: How Water Vapor Changes Metal at the Atomic Level

    New insights into molecular-level processes could help prevent corrosion and improve catalytic conversion.

    Microbial Types May Prove Key to Gas Releases from Thawing Permafrost

    Scientists discover key types of microbes that degrade organic matter and release carbon dioxide and methane into the atmosphere.

    New Method Knocks Out Yeast Genes with Single-Point Precision

    Researchers can precisely study how different genes affect key properties in a yeast used industrially to produce fuel and chemicals.

    How Plants Regulate Sugar Deposition in Cell Walls

    Identified genes involved in plant cell wall polysaccharide production and restructuring could aid in engineering bioenergy crops.

    Scientists Identify Gene Cluster in Budding Yeasts with Major Implications for Renewable Energy

    How yeast partition carbon into a metabolite may offer insights into boosting production for biofuels.


    Spotlight

    Friday January 18, 2019, 05:05 PM

    Chemistry intern inspired by Argonne's real-world science

    Argonne National Laboratory

    Friday January 18, 2019, 04:05 PM

    Chasing a supernova

    Argonne National Laboratory

    Tuesday January 08, 2019, 02:05 PM

    Argonne intern streamlines the beamline

    Argonne National Laboratory

    Thursday October 11, 2018, 04:00 PM

    Research on Light-Matter Interaction Could Lead to Improved Electronic and Optoelectronic Devices

    Rensselaer Polytechnic Institute (RPI)

    Wednesday October 03, 2018, 07:05 PM

    Innovating Our Energy Future

    Oregon State University, College of Engineering

    Tuesday October 02, 2018, 03:05 PM

    Physics graduate student takes her thesis research to a Department of Energy national lab

    University of Alabama at Birmingham

    Friday September 21, 2018, 01:05 PM

    "Model" students enjoy Argonne campus life

    Argonne National Laboratory

    Thursday September 06, 2018, 01:05 PM

    Writing Code for a More Skilled and Diverse STEM Workforce

    Brookhaven National Laboratory

    Tuesday September 04, 2018, 11:30 AM

    New graduate student summer school launches at Princeton Plasma Physics Laboratory

    Princeton Plasma Physics Laboratory

    Friday August 31, 2018, 06:05 PM

    The Gridlock State

    California State University (CSU) Chancellor's Office

    Friday August 31, 2018, 02:05 PM

    Meet Jasmine Hatcher and Trishelle Copeland-Johnson

    Brookhaven National Laboratory

    Friday August 24, 2018, 11:05 AM

    Argonne hosts Modeling, Experimentation and Validation Summer School

    Argonne National Laboratory

    Wednesday August 22, 2018, 01:05 PM

    Students affected by Hurricane Maria bring their research to SLAC

    SLAC National Accelerator Laboratory

    Wednesday August 22, 2018, 10:05 AM

    Brookhaven Lab Pays Tribute to 2018 Summer Interns

    Brookhaven National Laboratory

    Monday August 20, 2018, 12:05 PM

    Changing How Buildings Are Made

    Washington University in St. Louis

    Thursday August 16, 2018, 12:05 PM

    CSUMB Selected to Host Architecture at Zero Competition in 2019

    California State University, Monterey Bay

    Friday July 20, 2018, 03:00 PM

    Department of Energy Invests $64 Million in Advanced Nuclear Technology

    Rensselaer Polytechnic Institute (RPI)

    Thursday July 19, 2018, 05:00 PM

    Professor Miao Yu Named the Priti and Mukesh Chatter '82 Career Development Professor

    Rensselaer Polytechnic Institute (RPI)

    Tuesday July 03, 2018, 11:05 AM

    2018 RHIC & AGS Annual Users' Meeting: 'Illuminating the QCD Landscape'

    Brookhaven National Laboratory

    Friday June 29, 2018, 06:05 PM

    Argonne welcomes The Martian author Andy Weir

    Argonne National Laboratory

    Monday June 18, 2018, 09:55 AM

    Creating STEM Knowledge and Innovations to Solve Global Issues Like Water, Food, and Energy

    Illinois Mathematics and Science Academy (IMSA)

    Friday June 15, 2018, 10:00 AM

    Professor Emily Liu Receives $1.8 Million DoE Award for Solar Power Systems Research

    Rensselaer Polytechnic Institute (RPI)

    Thursday June 07, 2018, 03:05 PM

    Celebrating 40 years of empowerment in science

    Argonne National Laboratory

    Monday May 07, 2018, 10:30 AM

    Introducing Graduate Students Across the Globe to Photon Science

    Brookhaven National Laboratory

    Wednesday May 02, 2018, 04:05 PM

    Students from Massachusetts and Washington Win DOE's 28th National Science Bowl(r)

    Department of Energy, Office of Science

    Thursday April 12, 2018, 07:05 PM

    The Race for Young Scientific Minds

    Argonne National Laboratory

    Wednesday March 14, 2018, 02:05 PM

    Q&A: Al Ashley Reflects on His Efforts to Diversify SLAC and Beyond

    SLAC National Accelerator Laboratory

    Thursday February 15, 2018, 12:05 PM

    Insights on Innovation in Energy, Humanitarian Aid Highlight UVA Darden's Net Impact Week

    University of Virginia Darden School of Business

    Friday February 09, 2018, 11:05 AM

    Ivy League Graduate, Writer and Activist with Dyslexia Visits CSUCI to Reframe the Concept of Learning Disabilities

    California State University, Channel Islands

    Wednesday January 17, 2018, 12:05 PM

    Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday January 17, 2018, 12:05 PM

    Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday December 20, 2017, 01:05 PM

    Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

    SLAC National Accelerator Laboratory

    Monday December 18, 2017, 01:05 PM

    The Future of Today's Electric Power Systems

    Rensselaer Polytechnic Institute (RPI)

    Monday December 18, 2017, 12:05 PM

    Supporting the Development of Offshore Wind Power Plants

    Rensselaer Polytechnic Institute (RPI)

    Tuesday October 03, 2017, 01:05 PM

    Stairway to Science

    Argonne National Laboratory

    Thursday September 28, 2017, 12:05 PM

    After-School Energy Rush

    Argonne National Laboratory

    Thursday September 28, 2017, 10:05 AM

    Bringing Diversity Into Computational Science Through Student Outreach

    Brookhaven National Laboratory

    Thursday September 21, 2017, 03:05 PM

    From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

    SLAC National Accelerator Laboratory

    Thursday September 07, 2017, 02:05 PM

    Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

    SLAC National Accelerator Laboratory

    Thursday August 31, 2017, 05:05 PM

    Binghamton University Opens $70 Million Smart Energy Building

    Binghamton University, State University of New York

    Wednesday August 23, 2017, 05:05 PM

    Widening Horizons for High Schoolers with Code

    Argonne National Laboratory

    Saturday May 20, 2017, 12:05 PM

    Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

    Rensselaer Polytechnic Institute (RPI)

    Monday May 15, 2017, 01:05 PM

    ORNL, University of Tennessee Launch New Doctoral Program in Data Science

    Oak Ridge National Laboratory

    Friday April 07, 2017, 11:05 AM

    Champions in Science: Profile of Jonathan Kirzner

    Department of Energy, Office of Science

    Wednesday April 05, 2017, 12:05 PM

    High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

    Argonne National Laboratory





    Showing results

    0-4 Of 2215