DOE News
    Doe Science news source
    The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
    • 2015-07-07 07:30:00
    • Article ID: 636737

    Big PanDA and Titan Merge to Tackle Torrent of LHC's Full-Energy Collision Data

    Workload handling software has broad potential to maximize use of available supercomputing resources

    • Credit: Oak Ridge National Laboratory

      The PanDA workload management system developed at Brookhaven Lab and the University of Texas, Arlington, has been integrated on the Titan supercomputer at the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory.

    • Credit: Brookhaven National Laboratory

      Brookhaven physicists Alexei Klimentov and Torre Wenaus have helped to design computational strategies for handling a torrent of data from the ATLAS experiment at the LHC.

    Big PanDA and Titan Merge to Tackle Torrent of LHC's Full-Energy Collision Data

    Workload handling software has broad potential to maximize use of available supercomputing resources

    July 7, 2015

    With the successful restart of the Large Hadron Collider (LHC), now operating at nearly twice its former collision energy, comes an enormous increase in the volume of data physicists must sift through to search for new discoveries. Thanks to planning and a pilot project funded by the offices of Advanced Scientific Computing Research and High-Energy Physics within the Department of Energy’s Office of Science, a remarkable data-management tool developed by physicists at DOE’s Brookhaven National Laboratory and the University of Texas at Arlington is evolving to meet the big-data challenge.

    The workload management system, known as PanDA (for Production and Distributed Analysis), was designed by high-energy physicists to handle data analysis jobs for the LHC’s ATLAS collaboration. During the LHC’s first run, from 2010 to 2013, PanDA made ATLAS data available for analysis by 3000 scientists around the world using the LHC’s global grid of networked computing resources. The latest rendition, known as Big PanDA, schedules jobs opportunistically on Titan—the world’s most powerful supercomputer for open scientific research, located at the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science User Facility at Oak Ridge National Laboratory—in a manner that does not conflict with Titan’s ability to schedule its traditional, very large, leadership-class computing jobs.

    This integration of the workload management system on Titan—the first large-scale use of leadership class supercomputing facilities fully integrated with PanDA to assist in the analysis of experimental high-energy physics data—will have immediate benefits for ATLAS.

    “Titan is ready to help with new discoveries at the LHC,” said Brookhaven physicist Alexei Klimentov, a leader on the development of Big PanDA.

    The workload management system will likely also help meet big data challenges in many areas of science by maximizing the use of limited supercomputing resources.

    “As a DOE leadership computing facility, OLCF was designed to tackle large complex computing problems that cannot be readily performed using smaller facilities—things like modeling climate and nuclear fusion,” said Jack Wells, Director of Science for the National Center for Computational Science at ORNL. OLCF prioritizes the scheduling of these leadership jobs, which can take up 20, 60, or even greater than 90 percent of Titan’s computational resources. One goal is to make the most of the available running time and get as close to 100 percent utilization of the system as possible.

    “But even when Titan is fully loaded and large jobs are standing in the queue to run, we are typically using about 90 percent of the machine averaging over long periods of time,” Wells said. “That means, on average, there’s 10 percent of the machine that we are unable to use that could be made available to handle a mix of smaller jobs, essentially ‘filling in the cracks’ between the very large jobs.”

    As Klimentov explained, “Applications from high-energy physics don’t require a huge allocation of resources on a supercomputer. If you imagine a glass filled with stones to represent the supercomputing capacity and how much ‘space’ is taken up by the big computing jobs, we use the small spaces between the stones.”

    A workload-management system like PanDA could help fill those spaces with other types of jobs as well.

    New territory for experimental physicists

    While supercomputers have been absolutely essential for the complex calculations of theoretical physics, distributed grid resources have been the workhorses for analyzing experimental high-energy physics data. PanDA, as designed by Kaushik De, a professor of physics at UT, Arlington, and Torre Wenaus of Brookhaven Lab, helped to integrate these worldwide computing centers by introducing common workflow protocols and access to the entire ATLAS data set.

    But as the volume of data increases with the LHC collision energy, so does the need for running simulations that help scientists interpret their experimental results, Klimentov said. These simulations are perfectly suited for running on supercomputers, and Big PanDA makes it possible to do so without eating up valuable computing time.

    The cutting-edge prototype Big PanDA software, which has been significantly modified from its original design, “backfills” simulations of the collisions taking place at the LHC into spaces between typically large supercomputing jobs.

    “We can insert jobs at just the right time and in just the right size chunks so they can run without competing in any way with the mission leadership jobs, making use of computing power that would otherwise sit idle,” Wells said.

    In early June, as the LHC ramped up to 13 trillion electron volts of energy per proton, Titan ramped up to 10,000 core processing units (CPUs) simultaneously calculating LHC collisions, and has tested scalability successfully up to 90,000 concurrent cores.

    “These simulations provide a clear path to understanding the complex physical phenomena recorded by the ATLAS detector,” Klimentov said.

    He noted that during one 10-day period just after the LHC restart, the group ran ATLAS simulations on Titan for 60,000 Titan core-hours in backfill mode. (30 Titan cores used over a period of one hour consume 30 Titan core-hours of computing resource.)

    “This is a great achievement of the pilot program,” said De of UT Arlington, co-leader of the Big PanDA project.

    “We’ll be able to reach far greater heights when the pilot matures into daily operations at Titan in the next phase of this project,” he added.

    The Big PanDA team is now ready to bring its expertise to advancing the use of supercomputers for fields beyond high-energy physics. Already they have plans to use Big PanDA to help tackle the data challenges presented by the LHC’s nuclear physics research using the ALICE detector—a program that complements the exploration of quark-gluon plasma and the building blocks of visible matter at Brookhaven’s Relativistic Heavy Ion Collider (RHIC). But they see widespread applicability in other data-intensive fields, including molecular dynamics simulations and studies of genes and proteins in biology, the development of new energy technologies and materials design, and understanding global climate change.

    “Our goal is to work with Jack and our other colleagues at OLCF to develop Big PanDA as a general workload tool available to all users of Titan and other supercomputers to advance fundamental discovery and understanding in a broad range of scientific and engineering disciplines,” Klimentov said. Supercomputing groups in the Czech Republic, UK, and Switzerland have already been making inquiries.

    Brookhaven’s role in this work was supported by the DOE Office of Science. The Oak Ridge Leadership Computing Facility is supported by the DOE Office of Science.

    Brookhaven National Laboratory and Oak Ridge National Laboratory are supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

    One of ten national laboratories overseen and primarily funded by the Office of Science of the U.S. Department of Energy (DOE), Brookhaven National Laboratory conducts research in the physical, biomedical, and environmental sciences, as well as in energy technologies and national security. Brookhaven Lab also builds and operates major scientific facilities available to university, industry and government researchers. Brookhaven is operated and managed for DOE's Office of Science by Brookhaven Science Associates, a limited-liability company founded by the Research Foundation for the State University of New York on behalf of Stony Brook University, the largest academic user of Laboratory facilities, and Battelle, a nonprofit applied science and technology organization.

    X
    X
    X
    • Filters

    • × Clear Filters

    Stronger, lighter, greener

    A new award-winning magnet technology invented at the U.S. Department of Energy's Argonne National Laboratory could help drive the nation's transition from gas-powered vehicles to electric and hybrid power more rapidly, at lower cost, and in a more environmentally friendly way.

    Science Up-Close: Developing a Cookbook for Efficient Fusion Energy

    To develop a future fusion reactor, scientists need to understand how and why plasma in fusion experiments moves into a "high-confinement mode" where particles and heat can't escape. Scientists at the Department of Energy's Princeton Plasma Physics Laboratory simulated the transition into that mode starting from the most basic physics principles.

    Peering into the Mist: How Water Vapor Changes Metal at the Atomic Level

    New insights into molecular-level processes could help prevent corrosion and improve catalytic conversion.

    Neutron science publications reach new highs at ORNL's flagship facilities

    The High Flux Isotope Reactor and the Spallation Neutron Source at the Department of Energy's Oak Ridge National Laboratory have reached new levels of increased science productivity. In 2018, a record high of more than 500 scientific instrument publications were produced between HFIR and SNS--based on neutron beamline experiments conducted by more than 1,200 US and international researchers who used the world-leading facilities.

    Fiery sighting: A new physics of eruptions that damage fusion experiments

    Feature describes first direct sighting of a trigger for bursts of heat that can disrupt fusion reactions.

    Microbial Types May Prove Key to Gas Releases from Thawing Permafrost

    Scientists discover key types of microbes that degrade organic matter and release carbon dioxide and methane into the atmosphere.

    An effect that Einstein helped discover 100 years ago offers new insight into a puzzling magnetic phenomenon

    Experiments at the Department of Energy's SLAC National Accelerator Laboratory have seen for the first time what happens when magnetic materials are demagnetized at ultrafast speeds of millionths of a billionth of a second: The atoms on the surface of the material move, much like the iron bar did. The work, done at SLAC's Linac Coherent Light Source (LCLS) X-ray laser, was published in Nature earlier this month.

    Found: A precise method for determining how waves and particles affect fusion reactions

    Like surfers catching ocean waves, particles within plasma can ride waves oscillating through the plasma during fusion energy experiments. Now a team of physicists led by PPPL has devised a faster method to determine how much this interaction contributes to efficiency loss in tokamaks.

    Discovery adapts natural membrane to make hydrogen fuel from water

    In a recent study from the U.S. Department of Energy's (DOE) Argonne National Laboratory, scientists have combined two membrane-bound protein complexes to perform a complete conversion of water molecules to hydrogen and oxygen.

    How Plants Regulate Sugar Deposition in Cell Walls

    Identified genes involved in plant cell wall polysaccharide production and restructuring could aid in engineering bioenergy crops.


    • Filters

    • × Clear Filters

    Top 10 Discoveries of 2018

    Every year, the U.S. Department of Energy's (DOE) Brookhaven National Laboratory compiles a list of the biggest advances made by the Lab's staff scientists, engineers, and visiting researchers. From uncovering mysteries of the universe to building better batteries, here, in no particular order, are our picks for the top 10 discoveries of 2018.

    U.S. Department of Energy Announces $33 Million for Small Business Research and Development Grants

    The U.S. Department of Energy announced it will award 189 grants totaling $33 million to 149 small businesses in 32 states.

    DOE to Provide $16 Million for New Research into Atmospheric and Terrestrial Processes

    The U.S. Department of Energy (DOE) announced a plan to provide $16 million for new observational research aimed at improving the accuracy of today's climate and earth system models.

    Machine learning award powers Argonne leadership in engine design

    When attempting to design engines to be more fuel-efficient and emissions-free, automotive manufacturers have to take into account all the complexity inherent in the combustion process.

    ORNL partners with industry to address multiple nuclear technology challenges

    The Department of Energy's Oak Ridge National Laboratory is collaborating with industry on six new projects focused on advancing commercial nuclear energy technologies that offer potential improvements to current nuclear reactors and move new reactor designs closer to deployment.

    Lithium earns honors for three physicists working to bring the energy that powers the sun to Earth

    Feature describes research of three PPPL physicists who have won the laboratory's 2018 outstanding research awards

    DOE approves technical plan and cost estimate to upgrade Argonne facility; Project will create X-rays that illuminate the atomic scale, in 3D

    The U.S. Department of Energy has approved the technical scope, cost estimate and plan of work for an upgrade of the Advanced Photon Source, a major storage-ring X-ray source at Argonne.

    Costas Soukoulis elected to National Academy of Inventors

    Costas Soukoulis, Ames Laboratory senior scientist and Iowa State University Frances M. Craig Endowed Chair and Distinguished Professor, has been named as a 2018 National Academy of Inventors (NAI) Fellow.

    Biophysicist F. William Studier Elected Fellow of the National Academy of Inventors

    F. William Studier, a Senior Biophysicist Emeritus at the U.S. Department of Energy's Brookhaven National Laboratory and Adjunct Professor of Biochemistry at Stony Brook University, has been elected as a Fellow of the National Academy of Inventors (NAI). He is among 148 renowned academic inventors being recognized by NAI for 2018.

    Blast to the future

    A grant from DOE's Technology Commercialization Fund will help researchers at Argonne and industry partners seek improvements to U.S. manufacturing by making discovery and design of new materials more efficient.


    • Filters

    • × Clear Filters

    Rapid Lake Draining on Ice Sheets Changes How Water Moves in Unexpected Ways

    Widespread fracturing during lake drainage triggers vertical shafts to form that affect the Greenland Ice Sheet.

    New Historical Emissions Trends Estimated with the Community Emissions Data System

    The data system will allow for more detailed, consistent, and up-to-date global emissions trends that will aid in understanding aerosol effects.

    Peering into the Mist: How Water Vapor Changes Metal at the Atomic Level

    New insights into molecular-level processes could help prevent corrosion and improve catalytic conversion.

    Microbial Types May Prove Key to Gas Releases from Thawing Permafrost

    Scientists discover key types of microbes that degrade organic matter and release carbon dioxide and methane into the atmosphere.

    New Method Knocks Out Yeast Genes with Single-Point Precision

    Researchers can precisely study how different genes affect key properties in a yeast used industrially to produce fuel and chemicals.

    How Plants Regulate Sugar Deposition in Cell Walls

    Identified genes involved in plant cell wall polysaccharide production and restructuring could aid in engineering bioenergy crops.

    Scientists Identify Gene Cluster in Budding Yeasts with Major Implications for Renewable Energy

    How yeast partition carbon into a metabolite may offer insights into boosting production for biofuels.

    More Designer Peptides, More Possibilities

    A combined experimental and modeling approach contributes to understanding small proteins with potential use in industrial, therapeutic applications.

    Deep Learning for Electron Microscopy

    Artificial intelligence on Summit to discover atomic-scale structures.

    Clarifying Rates of Methylmercury Production

    New model provides more accurate estimates of how fast microbes produce a mercury-based neurotoxin.


    Spotlight

    Friday January 18, 2019, 05:05 PM

    Chemistry intern inspired by Argonne's real-world science

    Argonne National Laboratory

    Friday January 18, 2019, 04:05 PM

    Chasing a supernova

    Argonne National Laboratory

    Tuesday January 08, 2019, 02:05 PM

    Argonne intern streamlines the beamline

    Argonne National Laboratory

    Thursday October 11, 2018, 04:00 PM

    Research on Light-Matter Interaction Could Lead to Improved Electronic and Optoelectronic Devices

    Rensselaer Polytechnic Institute (RPI)

    Wednesday October 03, 2018, 07:05 PM

    Innovating Our Energy Future

    Oregon State University, College of Engineering

    Tuesday October 02, 2018, 03:05 PM

    Physics graduate student takes her thesis research to a Department of Energy national lab

    University of Alabama at Birmingham

    Friday September 21, 2018, 01:05 PM

    "Model" students enjoy Argonne campus life

    Argonne National Laboratory

    Thursday September 06, 2018, 01:05 PM

    Writing Code for a More Skilled and Diverse STEM Workforce

    Brookhaven National Laboratory

    Tuesday September 04, 2018, 11:30 AM

    New graduate student summer school launches at Princeton Plasma Physics Laboratory

    Princeton Plasma Physics Laboratory

    Friday August 31, 2018, 06:05 PM

    The Gridlock State

    California State University (CSU) Chancellor's Office

    Friday August 31, 2018, 02:05 PM

    Meet Jasmine Hatcher and Trishelle Copeland-Johnson

    Brookhaven National Laboratory

    Friday August 24, 2018, 11:05 AM

    Argonne hosts Modeling, Experimentation and Validation Summer School

    Argonne National Laboratory

    Wednesday August 22, 2018, 01:05 PM

    Students affected by Hurricane Maria bring their research to SLAC

    SLAC National Accelerator Laboratory

    Wednesday August 22, 2018, 10:05 AM

    Brookhaven Lab Pays Tribute to 2018 Summer Interns

    Brookhaven National Laboratory

    Monday August 20, 2018, 12:05 PM

    Changing How Buildings Are Made

    Washington University in St. Louis

    Thursday August 16, 2018, 12:05 PM

    CSUMB Selected to Host Architecture at Zero Competition in 2019

    California State University, Monterey Bay

    Friday July 20, 2018, 03:00 PM

    Department of Energy Invests $64 Million in Advanced Nuclear Technology

    Rensselaer Polytechnic Institute (RPI)

    Thursday July 19, 2018, 05:00 PM

    Professor Miao Yu Named the Priti and Mukesh Chatter '82 Career Development Professor

    Rensselaer Polytechnic Institute (RPI)

    Tuesday July 03, 2018, 11:05 AM

    2018 RHIC & AGS Annual Users' Meeting: 'Illuminating the QCD Landscape'

    Brookhaven National Laboratory

    Friday June 29, 2018, 06:05 PM

    Argonne welcomes The Martian author Andy Weir

    Argonne National Laboratory

    Monday June 18, 2018, 09:55 AM

    Creating STEM Knowledge and Innovations to Solve Global Issues Like Water, Food, and Energy

    Illinois Mathematics and Science Academy (IMSA)

    Friday June 15, 2018, 10:00 AM

    Professor Emily Liu Receives $1.8 Million DoE Award for Solar Power Systems Research

    Rensselaer Polytechnic Institute (RPI)

    Thursday June 07, 2018, 03:05 PM

    Celebrating 40 years of empowerment in science

    Argonne National Laboratory

    Monday May 07, 2018, 10:30 AM

    Introducing Graduate Students Across the Globe to Photon Science

    Brookhaven National Laboratory

    Wednesday May 02, 2018, 04:05 PM

    Students from Massachusetts and Washington Win DOE's 28th National Science Bowl(r)

    Department of Energy, Office of Science

    Thursday April 12, 2018, 07:05 PM

    The Race for Young Scientific Minds

    Argonne National Laboratory

    Wednesday March 14, 2018, 02:05 PM

    Q&A: Al Ashley Reflects on His Efforts to Diversify SLAC and Beyond

    SLAC National Accelerator Laboratory

    Thursday February 15, 2018, 12:05 PM

    Insights on Innovation in Energy, Humanitarian Aid Highlight UVA Darden's Net Impact Week

    University of Virginia Darden School of Business

    Friday February 09, 2018, 11:05 AM

    Ivy League Graduate, Writer and Activist with Dyslexia Visits CSUCI to Reframe the Concept of Learning Disabilities

    California State University, Channel Islands

    Wednesday January 17, 2018, 12:05 PM

    Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday January 17, 2018, 12:05 PM

    Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday December 20, 2017, 01:05 PM

    Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

    SLAC National Accelerator Laboratory

    Monday December 18, 2017, 01:05 PM

    The Future of Today's Electric Power Systems

    Rensselaer Polytechnic Institute (RPI)

    Monday December 18, 2017, 12:05 PM

    Supporting the Development of Offshore Wind Power Plants

    Rensselaer Polytechnic Institute (RPI)

    Tuesday October 03, 2017, 01:05 PM

    Stairway to Science

    Argonne National Laboratory

    Thursday September 28, 2017, 12:05 PM

    After-School Energy Rush

    Argonne National Laboratory

    Thursday September 28, 2017, 10:05 AM

    Bringing Diversity Into Computational Science Through Student Outreach

    Brookhaven National Laboratory

    Thursday September 21, 2017, 03:05 PM

    From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

    SLAC National Accelerator Laboratory

    Thursday September 07, 2017, 02:05 PM

    Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

    SLAC National Accelerator Laboratory

    Thursday August 31, 2017, 05:05 PM

    Binghamton University Opens $70 Million Smart Energy Building

    Binghamton University, State University of New York

    Wednesday August 23, 2017, 05:05 PM

    Widening Horizons for High Schoolers with Code

    Argonne National Laboratory

    Saturday May 20, 2017, 12:05 PM

    Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

    Rensselaer Polytechnic Institute (RPI)

    Monday May 15, 2017, 01:05 PM

    ORNL, University of Tennessee Launch New Doctoral Program in Data Science

    Oak Ridge National Laboratory

    Friday April 07, 2017, 11:05 AM

    Champions in Science: Profile of Jonathan Kirzner

    Department of Energy, Office of Science

    Wednesday April 05, 2017, 12:05 PM

    High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

    Argonne National Laboratory





    Showing results

    0-4 Of 2215