DOE News
    Doe Science news source
    The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
    • 2018-12-19 12:05:53
    • Article ID: 705697

    Lighting the Way to Centralized Computing Support for Photon Science

    Brookhaven Lab's Computational Science Initiative hosted a workshop for scientists and information technology specialists to discuss best practices for managing and processing data generated at light source facilities

    • Credit: Brookhaven National Laboratory

      On Sept. 24, scientists and information technology specialists from various labs in the United States and Europe participated in a full-day workshop—hosted by the Scientific Data and Computing Center at Brookhaven Lab—to share challenges and solutions to providing centralized computing support for photon science. From left to right, seated: Eric Lancon, Ian Collier, Kevin Casella, Jamal Irving, Tony Wong, and Abe Singer. Standing: Yee-Ting Li, Shigeki Misawa, Amedeo Perazzo, David Yu, Hironori Ito, Krishna Muriki, Alex Zaytsev, John DeStefano, Stuart Campbell, Martin Gasthuber, Andrew Richards, and Wei Yang.

    • Credit: Brookhaven National Laboratory

      An aerial photo of NSLS-II.

    • Credit: Brookhaven National Laboratory

      The amount of data that need to be archived and retrieved from tape storage has significantly increased over the past decade, as seen in the above graph. "Hot" storage refers to storing data that are frequently accessed, while "cold" storage refers to storing data that are rarely used.

    • Credit: ESNet

      ESnet provides network connections across the world to enable sharing of big data for scientific discovery.

    Large particle accelerator–based facilities known as synchrotron light sources provide intense, highly focused photon beams in the infrared, visible, ultraviolet, and x-ray regions of the electromagnetic spectrum. The photons, or tiny bundles of light energy, can be used to probe the structure, chemical composition, and properties of a wide range of materials on the atomic scale. For example, scientists direct the brilliant light at batteries to resolve charge and discharge processes, at protein-drug complexes to understand how the molecules bind, and at soil samples to identify environmental contaminants.

    As these facilities continue to become more advanced through upgrades to light sources, detectors, optics, and other technologies, they are producing data at a higher rate and with increasing complexity. These big data present a challenge to facility users, who have to be able to quickly analyze the data in real time to make sure their experiments are functioning as they should be. Once they have concluded their experiments, users also need ways to store, retrieve, and distribute the data for further analysis. High-performance computing hardware and software are critical to supporting such immediate analysis and post-acquisition requirements.

    The U.S. Department of Energy’s (DOE) Brookhaven National Laboratory hosted a one-day workshop on Sept. 24 for information technology (IT) specialists and scientists from various labs around the world to discuss best practices and share experiences in providing centralized computing support to photon science. Many institutions provide limited computing resources (e.g., servers, disk/tape storage systems) within their respective light source facilities for data acquisition and a quick check and feedback on the quality of the collected data. Though these facilities have computing infrastructure (e.g., login access, network connectivity, data management software) to support usage, access to computing resources is often time-constrained because of the high number and frequency of experiments being conducted at any given time. For example, the Diamond Light Source in the United Kingdom hosts about 9,000 experiments in a single year. Because of the limited computing resources, extensive (or multiple attempts at) data reconstruction and analysis must typically be performed outside of the facilities. But centralized computing centers can provide the resources needed to manage and process data being generated by such experiments.   

    Continuing a legacy of computing support

    Brookhaven Lab is home to the National Synchrotron Light Source II (NSLS-II), a DOE Office of Science User Facility, that began operating in 2014 and is 10,000 times brighter than the original NSLS. Currently, 28 beamlines are in operation or commissioning and one beamline is under construction, and there is space to accommodate an additional 30 beamlines. NSLS-II is expected to generate tens of petabytes of data (one petabyte is equivalent to a stack of CDs standing nearly 10,000 feet tall) per year in the next decade.

    Brookhaven is also home to the Scientific Data and Computing Center (SDCC), part of the Computational Science Initiative (CSI). The centralized data storage, computing, and networking infrastructure that SDCC provides has historically supported the RHIC and ATLAS Computing Facility (RACF). This facility provides the necessary resources to store, process, analyze, and distribute experimental data from the Relativistic Heavy Ion Collider (RHIC)—another DOE Office of Science User Facility at Brookhaven—and the ATLAS detector at CERN’s Large Hadron Collider in Europe.

    “Brookhaven has a long tradition of providing centralized computing support to the nuclear and high-energy physics communities,” said workshop organizer Tony Wong, deputy director of SDCC. “A standard approach for dealing with their computing requirements has been developed for more than 50 years. New and advanced photon science facilities such as NSLS-II have very different requirements, and therefore we need to reconsider our approach. The purpose of the workshop was to gain insights from labs with a proven track record of providing centralized computing support for photon science, and to apply those insights at SDCC and other centralized computing centers. There are a lot of research organizations around the world who are similar to Brookhaven in the sense that they have a long history in data-intensive nuclear and high-energy physics experiments and are now branching out to newer data-intensive areas, such as photon science.”

    Nearly 30 scientists and IT specialists from several DOE national laboratories—Brookhaven, Argonne, Lawrence Berkeley, and SLAC—and research institutions in Europe, including the Diamond Light Source and Science and Technology Facilities Council in the United Kingdom and the PETRA III x-ray light source at the Deutsches Elektronen-Synchrotron (DESY) in Germany, participated in this first-of-its-kind workshop. They discussed common challenges in storing, archiving, retrieving, sharing, and analyzing photon science data, and techniques to overcome these challenges.

    Meeting different computing requirements 

    One of the biggest differences in computing requirements between nuclear and high-energy physics and photon science is the speed with which the data must be analyzed upon collection.

    “In nuclear and high-energy physics, the data-taking period spans weeks, months, or even years, and the data are analyzed at a later date,” said Wong. “But in photon science, experiments sometimes only last a few hours to a couple of days. When your time at a beamline is this limited, every second counts. Therefore, it is vitally important for the users to be able to immediately check their data as it is collected to ensure it is of value. It is through these data checks that scientists can confirm whether the detectors and instruments are working properly.”

    Photon science also has unique networking requirements, both internally within the light sources and central computing centers, and externally across the internet and remote facilities. For example, in the past, scientists could load their experimental results onto portable storage devices such as removable drives. However, because of the proliferation of big data, this take-it-home approach is often not feasible. Instead, scientists are investigating cloud-based data storage and distribution technology. While the DOE-supported Energy Sciences Network (ESnet)—a DOE Office of Science User Facility stewarded by Lawrence Berkeley National Laboratory—provides high-bandwidth connections for national labs, universities, and research institutions to share their data, no such vehicle exists for private companies. Additionally, sending, storing, and accessing data over the internet can pose security concerns in cases where the data are proprietary or involve confidential information, such as corporate entities.

    Even nonproprietary academic research requires that some security measures are in place to ensure that the appropriate personnel are accessing the computing resources and data. The workshop participants discussed authentication and authorization infrastructure and mechanisms to address these concerns.

    Identifying opportunities and challenges

    According to Wong, the workshop raised both concern and optimism. Many of the world’s light sources will be undergoing upgrades between 2020 and 2025 that will increase today’s data collection rates by three to 10 times.

    “If we are having trouble coping with data challenges today, even taking into account advancements in technology, we will continue to have problems in the future with respect to moving data from detectors to storage and performing real-time analysis on the data,” said Wong. “On the other hand, SDCC has extensive experience in providing software visualization, cloud computing, authentication and authorization, scalable disk storage, and other infrastructure for nuclear and high-energy physics research. This experience can be leveraged to tackle the unique challenges of managing and processing data for photon science.”

    Going forward, SDCC will continue to engage with the larger community of IT experts in scientific computing through existing information-exchange forums, such as HEPiX. Established in 1991, HEPiX comprises more than 500 scientists and IT system administrators, engineers, and managers who meet twice a year to discuss scientific computing and data challenges in nuclear and high-energy physics. Recently, HEPiX has been extending these discussions to other scientific areas, with scientists and IT professionals from various light sources in attendance. Several of the Brookhaven workshop participants attended the recent HEPiX Autumn/Fall 2018 Workshop in Barcelona, Spain.

    “The seeds have already been planted for interactions between the two communities,” said Wong. “It is our hope that the exchange of information will be mutually beneficial.”

    With this knowledge sharing, SDCC hopes to expand the amount of support provided to NSLS-II, as well as the Center for Functional Nanomaterials (CFN)—another DOE Office of Science User Facility at Brookhaven. In fact, several scientists from NSLS-II and CFN attended the workshop, providing a comprehensive view of their computing needs.

    “SDCC already supports these user facilities but we would like to make this support more encompassing,” said Wong. “For instance, we provide offline computing resources for post-data acquisition analysis but we are not yet providing a real-time data quality IT infrastructure. Events like this workshop are part of SDCC’s larger ongoing effort to provide adequate computing support to scientists, enabling them to carry out the world-class research that leads to scientific discoveries.” 

    Brookhaven National Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit

    Follow @BrookhavenLab on Twitter or find us on Facebook.

    • Filters

    • × Clear Filters

    A Challenging Future for Tropical Forests

    Mortality rates of moist tropical forests are on the rise due to environmental drivers and related mechanisms.

    Stronger, lighter, greener

    A new award-winning magnet technology invented at the U.S. Department of Energy's Argonne National Laboratory could help drive the nation's transition from gas-powered vehicles to electric and hybrid power more rapidly, at lower cost, and in a more environmentally friendly way.

    Science Up-Close: Developing a Cookbook for Efficient Fusion Energy

    To develop a future fusion reactor, scientists need to understand how and why plasma in fusion experiments moves into a "high-confinement mode" where particles and heat can't escape. Scientists at the Department of Energy's Princeton Plasma Physics Laboratory simulated the transition into that mode starting from the most basic physics principles.

    Peering into the Mist: How Water Vapor Changes Metal at the Atomic Level

    New insights into molecular-level processes could help prevent corrosion and improve catalytic conversion.

    Neutron science publications reach new highs at ORNL's flagship facilities

    The High Flux Isotope Reactor and the Spallation Neutron Source at the Department of Energy's Oak Ridge National Laboratory have reached new levels of increased science productivity. In 2018, a record high of more than 500 scientific instrument publications were produced between HFIR and SNS--based on neutron beamline experiments conducted by more than 1,200 US and international researchers who used the world-leading facilities.

    Fiery sighting: A new physics of eruptions that damage fusion experiments

    Feature describes first direct sighting of a trigger for bursts of heat that can disrupt fusion reactions.

    Microbial Types May Prove Key to Gas Releases from Thawing Permafrost

    Scientists discover key types of microbes that degrade organic matter and release carbon dioxide and methane into the atmosphere.

    An effect that Einstein helped discover 100 years ago offers new insight into a puzzling magnetic phenomenon

    Experiments at the Department of Energy's SLAC National Accelerator Laboratory have seen for the first time what happens when magnetic materials are demagnetized at ultrafast speeds of millionths of a billionth of a second: The atoms on the surface of the material move, much like the iron bar did. The work, done at SLAC's Linac Coherent Light Source (LCLS) X-ray laser, was published in Nature earlier this month.

    Found: A precise method for determining how waves and particles affect fusion reactions

    Like surfers catching ocean waves, particles within plasma can ride waves oscillating through the plasma during fusion energy experiments. Now a team of physicists led by PPPL has devised a faster method to determine how much this interaction contributes to efficiency loss in tokamaks.

    Discovery adapts natural membrane to make hydrogen fuel from water

    In a recent study from the U.S. Department of Energy's (DOE) Argonne National Laboratory, scientists have combined two membrane-bound protein complexes to perform a complete conversion of water molecules to hydrogen and oxygen.

    • Filters

    • × Clear Filters

    Argonne scientist elected as SAE Fellow

    Scientist Michael Wang from the U.S. Department of Energy's (DOE) Argonne National Laboratory was recently inducted as a Fellow of the professional engineering organization SAE (Society of Automotive Engineers). The organization reserves this prestigious grade of membership for thosewho have made significant contributions to mobility technology and have demonstrated leadership in their field.

    Top 10 Discoveries of 2018

    Every year, the U.S. Department of Energy's (DOE) Brookhaven National Laboratory compiles a list of the biggest advances made by the Lab's staff scientists, engineers, and visiting researchers. From uncovering mysteries of the universe to building better batteries, here, in no particular order, are our picks for the top 10 discoveries of 2018.

    U.S. Department of Energy Announces $33 Million for Small Business Research and Development Grants

    The U.S. Department of Energy announced it will award 189 grants totaling $33 million to 149 small businesses in 32 states.

    DOE to Provide $16 Million for New Research into Atmospheric and Terrestrial Processes

    The U.S. Department of Energy (DOE) announced a plan to provide $16 million for new observational research aimed at improving the accuracy of today's climate and earth system models.

    Machine learning award powers Argonne leadership in engine design

    When attempting to design engines to be more fuel-efficient and emissions-free, automotive manufacturers have to take into account all the complexity inherent in the combustion process.

    ORNL partners with industry to address multiple nuclear technology challenges

    The Department of Energy's Oak Ridge National Laboratory is collaborating with industry on six new projects focused on advancing commercial nuclear energy technologies that offer potential improvements to current nuclear reactors and move new reactor designs closer to deployment.

    Lithium earns honors for three physicists working to bring the energy that powers the sun to Earth

    Feature describes research of three PPPL physicists who have won the laboratory's 2018 outstanding research awards

    DOE approves technical plan and cost estimate to upgrade Argonne facility; Project will create X-rays that illuminate the atomic scale, in 3D

    The U.S. Department of Energy has approved the technical scope, cost estimate and plan of work for an upgrade of the Advanced Photon Source, a major storage-ring X-ray source at Argonne.

    Costas Soukoulis elected to National Academy of Inventors

    Costas Soukoulis, Ames Laboratory senior scientist and Iowa State University Frances M. Craig Endowed Chair and Distinguished Professor, has been named as a 2018 National Academy of Inventors (NAI) Fellow.

    Biophysicist F. William Studier Elected Fellow of the National Academy of Inventors

    F. William Studier, a Senior Biophysicist Emeritus at the U.S. Department of Energy's Brookhaven National Laboratory and Adjunct Professor of Biochemistry at Stony Brook University, has been elected as a Fellow of the National Academy of Inventors (NAI). He is among 148 renowned academic inventors being recognized by NAI for 2018.

    • Filters

    • × Clear Filters

    Observing Clouds in Four Dimensions

    Six cameras are revolutionizing observations of shallow cumulus clouds.

    A Challenging Future for Tropical Forests

    Mortality rates of moist tropical forests are on the rise due to environmental drivers and related mechanisms.

    Rapid Lake Draining on Ice Sheets Changes How Water Moves in Unexpected Ways

    Widespread fracturing during lake drainage triggers vertical shafts to form that affect the Greenland Ice Sheet.

    New Historical Emissions Trends Estimated with the Community Emissions Data System

    The data system will allow for more detailed, consistent, and up-to-date global emissions trends that will aid in understanding aerosol effects.

    Peering into the Mist: How Water Vapor Changes Metal at the Atomic Level

    New insights into molecular-level processes could help prevent corrosion and improve catalytic conversion.

    Microbial Types May Prove Key to Gas Releases from Thawing Permafrost

    Scientists discover key types of microbes that degrade organic matter and release carbon dioxide and methane into the atmosphere.

    New Method Knocks Out Yeast Genes with Single-Point Precision

    Researchers can precisely study how different genes affect key properties in a yeast used industrially to produce fuel and chemicals.

    How Plants Regulate Sugar Deposition in Cell Walls

    Identified genes involved in plant cell wall polysaccharide production and restructuring could aid in engineering bioenergy crops.

    Scientists Identify Gene Cluster in Budding Yeasts with Major Implications for Renewable Energy

    How yeast partition carbon into a metabolite may offer insights into boosting production for biofuels.

    More Designer Peptides, More Possibilities

    A combined experimental and modeling approach contributes to understanding small proteins with potential use in industrial, therapeutic applications.


    Friday January 18, 2019, 05:05 PM

    Chemistry intern inspired by Argonne's real-world science

    Argonne National Laboratory

    Friday January 18, 2019, 04:05 PM

    Chasing a supernova

    Argonne National Laboratory

    Tuesday January 08, 2019, 02:05 PM

    Argonne intern streamlines the beamline

    Argonne National Laboratory

    Thursday October 11, 2018, 04:00 PM

    Research on Light-Matter Interaction Could Lead to Improved Electronic and Optoelectronic Devices

    Rensselaer Polytechnic Institute (RPI)

    Wednesday October 03, 2018, 07:05 PM

    Innovating Our Energy Future

    Oregon State University, College of Engineering

    Tuesday October 02, 2018, 03:05 PM

    Physics graduate student takes her thesis research to a Department of Energy national lab

    University of Alabama at Birmingham

    Friday September 21, 2018, 01:05 PM

    "Model" students enjoy Argonne campus life

    Argonne National Laboratory

    Thursday September 06, 2018, 01:05 PM

    Writing Code for a More Skilled and Diverse STEM Workforce

    Brookhaven National Laboratory

    Tuesday September 04, 2018, 11:30 AM

    New graduate student summer school launches at Princeton Plasma Physics Laboratory

    Princeton Plasma Physics Laboratory

    Friday August 31, 2018, 06:05 PM

    The Gridlock State

    California State University (CSU) Chancellor's Office

    Friday August 31, 2018, 02:05 PM

    Meet Jasmine Hatcher and Trishelle Copeland-Johnson

    Brookhaven National Laboratory

    Friday August 24, 2018, 11:05 AM

    Argonne hosts Modeling, Experimentation and Validation Summer School

    Argonne National Laboratory

    Wednesday August 22, 2018, 01:05 PM

    Students affected by Hurricane Maria bring their research to SLAC

    SLAC National Accelerator Laboratory

    Wednesday August 22, 2018, 10:05 AM

    Brookhaven Lab Pays Tribute to 2018 Summer Interns

    Brookhaven National Laboratory

    Monday August 20, 2018, 12:05 PM

    Changing How Buildings Are Made

    Washington University in St. Louis

    Thursday August 16, 2018, 12:05 PM

    CSUMB Selected to Host Architecture at Zero Competition in 2019

    California State University, Monterey Bay

    Friday July 20, 2018, 03:00 PM

    Department of Energy Invests $64 Million in Advanced Nuclear Technology

    Rensselaer Polytechnic Institute (RPI)

    Thursday July 19, 2018, 05:00 PM

    Professor Miao Yu Named the Priti and Mukesh Chatter '82 Career Development Professor

    Rensselaer Polytechnic Institute (RPI)

    Tuesday July 03, 2018, 11:05 AM

    2018 RHIC & AGS Annual Users' Meeting: 'Illuminating the QCD Landscape'

    Brookhaven National Laboratory

    Friday June 29, 2018, 06:05 PM

    Argonne welcomes The Martian author Andy Weir

    Argonne National Laboratory

    Monday June 18, 2018, 09:55 AM

    Creating STEM Knowledge and Innovations to Solve Global Issues Like Water, Food, and Energy

    Illinois Mathematics and Science Academy (IMSA)

    Friday June 15, 2018, 10:00 AM

    Professor Emily Liu Receives $1.8 Million DoE Award for Solar Power Systems Research

    Rensselaer Polytechnic Institute (RPI)

    Thursday June 07, 2018, 03:05 PM

    Celebrating 40 years of empowerment in science

    Argonne National Laboratory

    Monday May 07, 2018, 10:30 AM

    Introducing Graduate Students Across the Globe to Photon Science

    Brookhaven National Laboratory

    Wednesday May 02, 2018, 04:05 PM

    Students from Massachusetts and Washington Win DOE's 28th National Science Bowl(r)

    Department of Energy, Office of Science

    Thursday April 12, 2018, 07:05 PM

    The Race for Young Scientific Minds

    Argonne National Laboratory

    Wednesday March 14, 2018, 02:05 PM

    Q&A: Al Ashley Reflects on His Efforts to Diversify SLAC and Beyond

    SLAC National Accelerator Laboratory

    Thursday February 15, 2018, 12:05 PM

    Insights on Innovation in Energy, Humanitarian Aid Highlight UVA Darden's Net Impact Week

    University of Virginia Darden School of Business

    Friday February 09, 2018, 11:05 AM

    Ivy League Graduate, Writer and Activist with Dyslexia Visits CSUCI to Reframe the Concept of Learning Disabilities

    California State University, Channel Islands

    Wednesday January 17, 2018, 12:05 PM

    Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday January 17, 2018, 12:05 PM

    Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday December 20, 2017, 01:05 PM

    Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

    SLAC National Accelerator Laboratory

    Monday December 18, 2017, 01:05 PM

    The Future of Today's Electric Power Systems

    Rensselaer Polytechnic Institute (RPI)

    Monday December 18, 2017, 12:05 PM

    Supporting the Development of Offshore Wind Power Plants

    Rensselaer Polytechnic Institute (RPI)

    Tuesday October 03, 2017, 01:05 PM

    Stairway to Science

    Argonne National Laboratory

    Thursday September 28, 2017, 12:05 PM

    After-School Energy Rush

    Argonne National Laboratory

    Thursday September 28, 2017, 10:05 AM

    Bringing Diversity Into Computational Science Through Student Outreach

    Brookhaven National Laboratory

    Thursday September 21, 2017, 03:05 PM

    From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

    SLAC National Accelerator Laboratory

    Thursday September 07, 2017, 02:05 PM

    Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

    SLAC National Accelerator Laboratory

    Thursday August 31, 2017, 05:05 PM

    Binghamton University Opens $70 Million Smart Energy Building

    Binghamton University, State University of New York

    Wednesday August 23, 2017, 05:05 PM

    Widening Horizons for High Schoolers with Code

    Argonne National Laboratory

    Saturday May 20, 2017, 12:05 PM

    Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

    Rensselaer Polytechnic Institute (RPI)

    Monday May 15, 2017, 01:05 PM

    ORNL, University of Tennessee Launch New Doctoral Program in Data Science

    Oak Ridge National Laboratory

    Friday April 07, 2017, 11:05 AM

    Champions in Science: Profile of Jonathan Kirzner

    Department of Energy, Office of Science

    Wednesday April 05, 2017, 12:05 PM

    High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

    Argonne National Laboratory

    Showing results

    0-4 Of 2215