Doe Science news source
The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
  • 2017-06-29 08:05:06
  • Article ID: 677253

Brookhaven Lab's Scientific Data and Computing Center Reaches 100 Petabytes of Recorded Data

Total reflects 17 years of experimental physics data collected by scientists to understand the fundamental nature of matter and the basic forces that shape our universe

  • Credit: Brookhaven National Laboratory

    (Back row) Ognian Novakov, Christopher Pinkenburg, Jérôme Lauret, Eric Lançon, (front row) Tim Chou, David Yu, Guangwei Che, and Shigeki Misawa at Brookhaven Lab's Scientific Data and Computing Center, which houses the Oracle StorageTek tape storage system where experimental data are recorded.

  • Credit: Brookhaven National Laboratory

    Inside one of the automated tape libraries at the Scientific Data and Computing Center (SDCC), Eric Lançon, director of SDCC, holds a magnetic tape cartridge. When scientists need data, a robotic arm (the piece of equipment in front of Lançon) retrieves the relevant cartridges from their slots and loads them into drives in the back of the library.

Imagine storing approximately 1300 years’ worth of HDTV video, nearly six million movies, or the entire written works of humankind in all languages since the start of recorded history—twice over. Each of these quantities is equivalent to 100 petabytes of data: the amount of data now recorded by the Relativistic Heavy Ion Collider (RHIC) and ATLAS Computing Facility (RACF) Mass Storage Service, part of the Scientific Data and Computing Center (SDCC) at the U.S. Department of Energy’s (DOE) Brookhaven National Laboratory. One petabyte is defined as 10245 bytes, or 1,125,899,906,842,624 bytes, of data. 

“This is a major milestone for SDCC, as it reflects nearly two decades of scientific research for the RHIC nuclear physics and ATLAS particle physics experiments, including the contributions of thousands of scientists and engineers,” said Brookhaven Lab technology architect David Yu, who leads the SDCC’s Mass Storage Group.

SDCC is at the core of a global computing network connecting more than 2,500 researchers around the world with data from the STAR and PHENIX experiments at RHIC—a DOE Office of Science User Facility at Brookhaven—and the ATLAS experiment at the Large Hadron Collider (LHC) in Europe. In these particle collision experiments, scientists recreate conditions that existed just after the Big Bang, with the goal of understanding the fundamental forces of nature—gravitational, electromagnetic, strong nuclear, and weak nuclear—and the basic structure of matter, energy, space, and time. 

Big Data Revolution

The RHIC and ATLAS experiments are part of the big data revolution. These experiments involve collecting extremely large datasets that reduce statistical uncertainty to make high-precision measurements and search for extremely rare processes and particles.

For example, only one Higgs boson—an elementary particle whose energy field is thought to give mass to all the other elementary particles—is produced for every billion proton-proton collisions at the LHC. More, once produced, the Higgs boson almost immediately decays into other particles. So detecting the particle is a rare event, with around one trillion collisions required to detect a single instance. When scientists first discovered the Higgs boson at the LHC in 2012, they observed about 20 instances, recording and analyzing more than 300 trillion collisions to confirm the particle’s discovery.

At the end of 2016, the ATLAS collaboration released its first measurement of the mass of the W boson particle (another elementary particle that, together with the Z boson, is responsible for the weak nuclear force). This measurement, which is based on a sample of 15 million W boson candidates collected at LHC in 2011, has a relative precision of 240 parts per million (ppm)—a result that matches the best single-experiment measurement announced in 2007 by the Collider Detector at Fermilab collaboration, whose measurement is based on several years’ worth of collected data. A highly precise measurement is important because a deviation from the mass predicted by the Standard Model could point to new physics. More data samples are required to achieve the level of accuracy (80 ppm) that scientists need to significantly test this model.

The volume of data collected by these experiments will grow significantly in the near future as new accelerator programs deliver higher-intensity beams. The LHC will be upgraded to increase its luminosity (rate of collisions) by a factor of 10. This High-Luminosity LHC, which should be operational by 2025, will provide a unique opportunity for particle physicists to look for new and unexpected phenomena within the exabytes (one exabyte equals 1000 petabytes) of data that will be collected.  

Data archiving is the first step in making available the results from such experiments. Thousands of physicists then need to calibrate and analyze the archived data and compare the data to simulations. To this end, computational scientists, computer scientists, and mathematicians in Brookhaven Lab’s Computational Science Initiative, which encompasses SDCC, are developing programming tools, numerical models, and data-mining algorithms. Part of SDCC’s mission is to provide computing and networking resources in support of these activities.

A Data Storage, Computing, and Networking Infrastructure

Housed inside SDCC are more than 60,000 computing cores, 250 computer racks, and tape libraries capable of holding up to 90,000 magnetic storage tape cartridges that are used to store, process, analyze, and distribute the experimental data. The facility provides approximately 90 percent of the computing capacity for analyzing data from the STAR and PHENIX experiments, and serves as the largest of the 12 Tier 1 computing centers worldwide that support the ATLAS experiment. As a Tier 1 center, SDCC contributes nearly 23 percent of the total computing and storage capacity for the ATLAS experiment and delivers approximately 200 terabytes of data (picture 62 million photos) per day to more than 100 data centers globally.

At SDCC, the High Performance Storage System (HPSS) has been providing mass storage services to the RHIC and LHC experiments since 1997 and 2006, respectively. This data archiving and retrieval software, developed by IBM and several DOE national laboratories, manages petabytes of data on disk and in robot-controlled tape libraries. Contained within the libraries are magnetic tape cartridges that encode the data and tape drives that read and write the data. Robotic arms load the cartridges into the drives and unload them upon request.

When ranked by the volume of data stored in a single HPSS, Brookhaven’s system is the second largest in the nation and the fourth largest in the world. Currently, the RACF operates nine Oracle robotic tape libraries that constitute the largest Oracle tape storage system in the New York tri-state area. Contained within this system are nearly 70,000 active cartridges with capacities ranging from 800 gigabytes to 8.5 terabytes, and more than 100 tape drives. As the volume of scientific data to be stored increases, more libraries, tapes, and drives can be added accordingly. In 2006, this scalability was exercised when HPSS was expanded to accommodate data from the ATLAS experiment at LHC.

“The HPSS system was deployed in the late 1990s, when the RHIC accelerator was coming on line. It allowed data from RHIC experiments to be transmitted via network to the data center for storage—a relatively new idea at the time,” said Shigeki Misawa, manager of Mass Storage and General Services at Brookhaven Lab. Misawa played a key role in the initial evaluation and configuration of HPSS, and has guided the system through significant changes in hardware (network equipment, storage systems, and servers) and operational requirements (tape drive read/write rate, magnetic tape cartridge capacity, and data transfer speed). “Prior to this system, data was recorded on magnetic tape at the experiment and physically moved to the data center,” he continued.

Over the years, SDCC’s HPSS has been augmented with a suite of optimization and monitoring tools developed at Brookhaven Lab. One of these tools is David Yu’s scheduling software that optimizes the retrieval of massive amounts of data from tape storage. Another, developed by Jérôme Lauret, software and computing project leader for the STAR experiment, is software for organizing multiple user requests to retrieve data more efficiently.

Engineers in the Mass Storage Group—including Tim Chou, Guangwei Che, and Ognian Novakov—have created other software tools customized for Brookhaven Lab’s computing environment to enhance data management and operation abilities and to improve the effectiveness of equipment usage.

STAR experiment scientists have demonstrated the capabilities of SDCC’s enhanced HPSS, retrieving more than 4,000 files per hour (a rate of 6,000 gigabytes per hour) while using a third of HPSS resources. On the data archiving side, HPSS can store data in excess of five gigabytes per second.

As demand for mass data storage spreads across Brookhaven, access to HPSS is being extended to other research groups. In the future, SDCC is expected to provide centralized mass storage services to multi-experiment facilities, such as the Center for Functional Nanomaterials and the National Synchrotron Light Source II—two more DOE Office of Science User Facilities at Brookhaven.

“The tape library system of SDCC is a clear asset for Brookhaven’s current and upcoming big data science programs,” said SDCC Director Eric Lançon. “Our expertise in the field of data archiving is acknowledged worldwide.”

Brookhaven National Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit

  • Filters

  • × Clear Filters

Theorists Publish Highest-Precision Prediction of Muon Magnetic Anomaly

UPTON, NY--Theoretical physicists at the U.S. Department of Energy's (DOE's) Brookhaven National Laboratory and their collaborators have just released the most precise prediction of how subatomic particles called muons--heavy cousins of electrons--"wobble" off their path in a powerful magnetic field.

How Gold Nanoparticles Could Improve Solar Energy Storage

Star-shaped gold nanoparticles, coated with a semiconductor, can produce hydrogen from water over four times more efficiently than other methods - opening the door to improved storage of solar energy and other advances that could boost renewable energy use and combat climate change, according to Rutgers University-New Brunswick researchers.

National Ignition Facility Sets New Energy Record

Lawrence Livermore National Laboratory's National Ignition Facility (NIF) laser system has set a new record, firing 2.15 megajoules (MJ) of energy to its target chamber - a 15 percent improvement over NIF's design specification of 1.8 MJ, and more than 10 percent higher than the previous 1.9 MJ energy record set in March 2012. Increasing NIF's energy limit will expand the parameter space for stockpile stewardship experiments and provide a significant boost to the pursuit of ignition.

Generating Electrical Power From Waste Heat

Researchers from Sandia National Laboratories have developed a tiny silicon-based device that can harness what was previously called waste heat and turn it into DC power.

Extracting Signals of Elusive Particles from Giant Chambers Filled with Liquefied Argon

In two new papers, the MicroBooNE collaboration describes how they use this detector to pick up the telltale signs of neutrinos. The papers include details of the signal processing algorithms that are critical to accurately reconstruct neutrinos' subtle interactions with atoms in the detector.

SLAC's Ultra-High-Speed 'Electron Camera' Catches Molecules at a Crossroads

An extremely fast "electron camera" at the Department of Energy's SLAC National Accelerator Laboratory has produced the most detailed atomic movie of the decisive point where molecules hit by light can either stay intact or break apart. The results could lead to a better understanding of how molecules respond to light in processes that are crucial for life, like photosynthesis and vision, or that are potentially harmful, such as DNA damage from ultraviolet light.

Merging Antenna and Electronics Boosts Energy and Spectrum Efficiency

By integrating the design of antenna and electronics, researchers have boosted the energy and spectrum efficiency for a new class of millimeter wave transmitters, allowing improved modulation and reduced generation of waste heat. The result could be longer talk time and higher data rates in millimeter wave wireless communication devices for future 5G applications.

New Experimental Results from the Largest and Most Sophisticated Stellerator

An international team is running tests on the largest and most sophisticated stellerator, the Wendelstein 7-X fusion experiment. This complex machine is housed at the Max-Planck-Institute of Plasma Physics, and researchers are analyzing data from the first experiment campaign that took place in 2016, hoping to understand the science of fusion reactors. In a new report in Physics of Plasma, the scientists recount the first detailed characterization of plasma turbulence at the outer edge of the stellerator.

X-Ray Experiment Confirms Theoretical Model for Making New Materials

Experiments at the Department of Energy's SLAC National Accelerator Laboratory have confirmed the predictive power of a new computational approach to materials synthesis. Researchers say that this approach, developed at the DOE's Lawrence Berkeley National Laboratory, could streamline the creation of novel materials for solar cells, batteries and other sustainable technologies.

Diesel Doesn't Float This Boat

Marine research could soon be possible without the risk of polluting either the air or the ocean. It's thanks to a new ship design and feasibility study led by Sandia National Laboratories. Despite many advantages, the feasibility of a hydrogen-powered research vessel has never been studied or proven. Until now.

  • Filters

  • × Clear Filters

Seth Davidovits Wins 2018 Marshall N. Rosenbluth Dissertation Award

Article describes dissertation award won by Seth Davidovits.

DOE Launches New Lab Partnering Service

The U.S. Department of Energy officially launched the Lab Partnering Service (LPS), an on-line, single access point platform for investors, innovators, and institutions to identify, locate, and obtain information from DOE's 17 national laboratories.

Department of Energy Announces $75 Million for High Energy Physics Research

The U.S. Department of Energy (DOE) announced $75 million in funding for 77 university research awards on a range of topics in high energy physics to advance knowledge of how the universe works at its most fundamental level.

Thesis Prize Winner's Calculations Characterize Neutrino Interactions

Alessandro Baroni is helping demystify one of the most mysterious particles. His work is contributing to our understanding of neutrinos, and it has earned him the 2017 Jefferson Science Associates Thesis Prize for work performed on a thesis related to research at the Department of Energy's Thomas Jefferson National Accelerator Facility

10 Questions for Steven Cowley, New Director of the Princeton Plasma Physics Laboratory

Steven Cowley, a theoretical physicist and international authority on fusion energy, became the seventh Director of the Princeton Plasma Physics Laboratory (PPon July 1 and will be Princeton professor of astrophysical sciences on September 1.

Ames Laboratory to lead new Center for Advancement of Topological Semimetals

Ames Laboratory will receive $10.75 million over four yearrs for a new Center for Advancement of Topological Semimetals as one of the Department of Energy's Energy Frontier Research Centers.

DOE Awards $100 Million for Energy Frontier Research Centers

U.S. Secretary of Energy Rick Perry announced $100 million in funding for 42 Energy Frontier Research Centers (EFRCs) to accelerate the scientific breakthroughs needed to strengthen U.S. economic leadership and energy security.

Argonne welcomes <em>The Martian</em> author Andy Weir

Best-selling science fiction author Andy Weir visited Argonne to give a series of standing-room-only talks, inspiring students and scientists alike.

United States and Italy Sign Agreement to Collaborate on Sterile Neutrino Research

Yesterday, the U.S. Department of Energy and the Italian Embassy, on behalf of the Italian Ministry of Education, Universities and Research, signed an agreement for collaboration on research with the international Short-Baseline Neutrino (SBN) program hosted at DOE's Fermi National Accelerator Laboratory.

UW Professor and Clean Energy Institute Director Daniel Schwartz Wins Highest U.S. Award for STEM Mentors

Daniel Schwartz, University of Washington Professor and Clean Energy Institute Director, received the Presidential Award for Excellence in Science, Mathematics and Engineering Mentoring (PAESMEM) from the White House Office of Science and Technology Policy (OSTP) and the National Science Foundation (NSF) this week.

  • Filters

  • × Clear Filters

Tracking Down Helium-4's Quarks and Gluons

Scientists obtain the first exclusive measurement of deeply virtual Compton scattering of electrons off helium-4, vital to obtaining an unambiguous 3-D view of quarks and gluons within nuclei.

Predicting Magnetic Explosions: From Plasma Current Sheet Disruption to Fast Magnetic Reconnection

Supercomputer simulations and theoretical analysis shed new light on when and how fast reconnection occurs.

Is Nature Exclusively Left Handed? Using Chilled Atoms to Find Out

Elegant techniques of trapping and polarizing atoms open vistas for beta-decay tests of fundamental symmetries, key to understanding the most basic forces and particles constituting our universe.

As Future Batteries, Hybrid Supercapacitors Are Super-Charged

A new supercapacitor could be a competitive alternative to lithium-ion batteries.

Forever Young Catalyst Reduces Diesel Emissions

Atom probe tomography reveals key explanations for stable performance over a cutting-edge diesel-exhaust catalyst's lifetime.

Sense Like a Shark: Saltwater-Submersible Films

A nickelate thin film senses electric field changes analogous to the electroreception sensing organ in sharks, which detects the bioelectric fields of prey.

A Bit of Quantum Logic--What Did the Atom Say to the Quantum Dot?

Let's talk! Scientists demonstrate coherent coupling between a quantum dot and a donor atom in silicon, vital for moving information inside quantum computers.

New Tech Uses Isomeric Beams to Study How and Where the Galaxy Makes One of Its Most Common Elements

A new measurement using a beam of aluminum-26 prepared in a metastable state allows researchers to better understand the creation of the elements in our galaxy.

Simulations of Magnetically Confined Plasmas Reveal a Self-Regulating Stabilizing Mechanism

A mysterious mechanism that prevents instabilities may be similar to the process that maintains the Earth's magnetic field.

Seeing All the Colors of the Plasma Wind

2-D velocity imaging helps fusion researchers understand the role of ion winds (aka flows) in the boundary of tokamak plasmas.


Tuesday July 03, 2018, 11:05 AM

2018 RHIC & AGS Annual Users' Meeting: 'Illuminating the QCD Landscape'

Brookhaven National Laboratory

Friday June 29, 2018, 06:05 PM

Argonne welcomes The Martian author Andy Weir

Argonne National Laboratory

Monday June 18, 2018, 09:55 AM

Creating STEM Knowledge and Innovations to Solve Global Issues Like Water, Food, and Energy

Illinois Mathematics and Science Academy (IMSA)

Friday June 15, 2018, 10:00 AM

Professor Emily Liu Receives $1.8 Million DoE Award for Solar Power Systems Research

Rensselaer Polytechnic Institute (RPI)

Thursday June 07, 2018, 03:05 PM

Celebrating 40 years of empowerment in science

Argonne National Laboratory

Monday May 07, 2018, 10:30 AM

Introducing Graduate Students Across the Globe to Photon Science

Brookhaven National Laboratory

Wednesday May 02, 2018, 04:05 PM

Students from Massachusetts and Washington Win DOE's 28th National Science Bowl(r)

Department of Energy, Office of Science

Thursday April 12, 2018, 07:05 PM

The Race for Young Scientific Minds

Argonne National Laboratory

Wednesday March 14, 2018, 02:05 PM

Q&A: Al Ashley Reflects on His Efforts to Diversify SLAC and Beyond

SLAC National Accelerator Laboratory

Thursday February 15, 2018, 12:05 PM

Insights on Innovation in Energy, Humanitarian Aid Highlight UVA Darden's Net Impact Week

University of Virginia Darden School of Business

Friday February 09, 2018, 11:05 AM

Ivy League Graduate, Writer and Activist with Dyslexia Visits CSUCI to Reframe the Concept of Learning Disabilities

California State University, Channel Islands

Wednesday January 17, 2018, 12:05 PM

Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

Fermi National Accelerator Laboratory (Fermilab)

Wednesday January 17, 2018, 12:05 PM

Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

Fermi National Accelerator Laboratory (Fermilab)

Wednesday December 20, 2017, 01:05 PM

Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

SLAC National Accelerator Laboratory

Monday December 18, 2017, 01:05 PM

The Future of Today's Electric Power Systems

Rensselaer Polytechnic Institute (RPI)

Monday December 18, 2017, 12:05 PM

Supporting the Development of Offshore Wind Power Plants

Rensselaer Polytechnic Institute (RPI)

Tuesday October 03, 2017, 01:05 PM

Stairway to Science

Argonne National Laboratory

Thursday September 28, 2017, 12:05 PM

After-School Energy Rush

Argonne National Laboratory

Thursday September 28, 2017, 10:05 AM

Bringing Diversity Into Computational Science Through Student Outreach

Brookhaven National Laboratory

Thursday September 21, 2017, 03:05 PM

From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

SLAC National Accelerator Laboratory

Thursday September 07, 2017, 02:05 PM

Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

SLAC National Accelerator Laboratory

Thursday August 31, 2017, 05:05 PM

Binghamton University Opens $70 Million Smart Energy Building

Binghamton University, State University of New York

Wednesday August 23, 2017, 05:05 PM

Widening Horizons for High Schoolers with Code

Argonne National Laboratory

Saturday May 20, 2017, 12:05 PM

Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

Rensselaer Polytechnic Institute (RPI)

Monday May 15, 2017, 01:05 PM

ORNL, University of Tennessee Launch New Doctoral Program in Data Science

Oak Ridge National Laboratory

Friday April 07, 2017, 11:05 AM

Champions in Science: Profile of Jonathan Kirzner

Department of Energy, Office of Science

Wednesday April 05, 2017, 12:05 PM

High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

Argonne National Laboratory

Tuesday March 28, 2017, 12:05 PM

Champions in Science: Profile of Jenica Jacobi

Department of Energy, Office of Science

Friday March 24, 2017, 10:40 AM

Great Neck South High School Wins Regional Science Bowl at Brookhaven Lab

Brookhaven National Laboratory

Wednesday February 15, 2017, 04:05 PM

Middle Schoolers Test Their Knowledge at Science Bowl Competition

Argonne National Laboratory

Friday January 27, 2017, 04:00 PM

Haslam Visits ORNL to Highlight State's Role in Discovering Tennessine

Oak Ridge National Laboratory

Tuesday November 08, 2016, 12:05 PM

Internship Program Helps Foster Development of Future Nuclear Scientists

Oak Ridge National Laboratory

Friday May 13, 2016, 04:05 PM

More Than 12,000 Explore Jefferson Lab During April 30 Open House

Thomas Jefferson National Accelerator Facility

Monday April 25, 2016, 05:05 PM

Giving Back to National Science Bowl

Ames Laboratory

Friday March 25, 2016, 12:05 PM

NMSU Undergrad Tackles 3D Particle Scattering Animations After Receiving JSA Research Assistantship

Thomas Jefferson National Accelerator Facility

Tuesday February 02, 2016, 10:05 AM

Shannon Greco: A Self-Described "STEM Education Zealot"

Princeton Plasma Physics Laboratory

Monday November 16, 2015, 04:05 PM

Rare Earths for Life: An 85th Birthday Visit with Mr. Rare Earth

Ames Laboratory

Tuesday October 20, 2015, 01:05 PM

Meet Robert Palomino: 'Give Everything a Shot!'

Brookhaven National Laboratory

Tuesday April 22, 2014, 11:30 AM

University of Utah Makes Solar Accessible

University of Utah

Wednesday March 06, 2013, 03:40 PM

Student Innovator at Rensselaer Polytechnic Institute Seeks Brighter, Smarter, and More Efficient LEDs

Rensselaer Polytechnic Institute (RPI)

Friday November 16, 2012, 10:00 AM

Texas Tech Energy Commerce Students, Community Light up Tent City

Texas Tech University

Wednesday November 23, 2011, 10:45 AM

Don't Get 'Frosted' Over Heating Your Home This Winter

Temple University

Wednesday July 06, 2011, 06:00 PM

New Research Center To Tackle Critical Challenges Related to Aircraft Design, Wind Energy, Smart Buildings

Rensselaer Polytechnic Institute (RPI)

Friday April 22, 2011, 09:00 AM

First Polymer Solar-Thermal Device Heats Home, Saves Money

Wake Forest University

Friday April 15, 2011, 12:25 PM

Like Superman, American University Will Get Its Energy from the Sun

American University

Thursday February 10, 2011, 05:00 PM

ARRA Grant to Help Fund Seminary Building Green Roof

University of Chicago

Showing results

0-4 Of 2215