Doe Science news source
The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
  • 2017-06-29 08:05:06
  • Article ID: 677253

Brookhaven Lab's Scientific Data and Computing Center Reaches 100 Petabytes of Recorded Data

Total reflects 17 years of experimental physics data collected by scientists to understand the fundamental nature of matter and the basic forces that shape our universe

  • Credit: Brookhaven National Laboratory

    (Back row) Ognian Novakov, Christopher Pinkenburg, Jérôme Lauret, Eric Lançon, (front row) Tim Chou, David Yu, Guangwei Che, and Shigeki Misawa at Brookhaven Lab's Scientific Data and Computing Center, which houses the Oracle StorageTek tape storage system where experimental data are recorded.

  • Credit: Brookhaven National Laboratory

    Inside one of the automated tape libraries at the Scientific Data and Computing Center (SDCC), Eric Lançon, director of SDCC, holds a magnetic tape cartridge. When scientists need data, a robotic arm (the piece of equipment in front of Lançon) retrieves the relevant cartridges from their slots and loads them into drives in the back of the library.

Imagine storing approximately 1300 years’ worth of HDTV video, nearly six million movies, or the entire written works of humankind in all languages since the start of recorded history—twice over. Each of these quantities is equivalent to 100 petabytes of data: the amount of data now recorded by the Relativistic Heavy Ion Collider (RHIC) and ATLAS Computing Facility (RACF) Mass Storage Service, part of the Scientific Data and Computing Center (SDCC) at the U.S. Department of Energy’s (DOE) Brookhaven National Laboratory. One petabyte is defined as 10245 bytes, or 1,125,899,906,842,624 bytes, of data. 

“This is a major milestone for SDCC, as it reflects nearly two decades of scientific research for the RHIC nuclear physics and ATLAS particle physics experiments, including the contributions of thousands of scientists and engineers,” said Brookhaven Lab technology architect David Yu, who leads the SDCC’s Mass Storage Group.

SDCC is at the core of a global computing network connecting more than 2,500 researchers around the world with data from the STAR and PHENIX experiments at RHIC—a DOE Office of Science User Facility at Brookhaven—and the ATLAS experiment at the Large Hadron Collider (LHC) in Europe. In these particle collision experiments, scientists recreate conditions that existed just after the Big Bang, with the goal of understanding the fundamental forces of nature—gravitational, electromagnetic, strong nuclear, and weak nuclear—and the basic structure of matter, energy, space, and time. 

Big Data Revolution

The RHIC and ATLAS experiments are part of the big data revolution. These experiments involve collecting extremely large datasets that reduce statistical uncertainty to make high-precision measurements and search for extremely rare processes and particles.

For example, only one Higgs boson—an elementary particle whose energy field is thought to give mass to all the other elementary particles—is produced for every billion proton-proton collisions at the LHC. More, once produced, the Higgs boson almost immediately decays into other particles. So detecting the particle is a rare event, with around one trillion collisions required to detect a single instance. When scientists first discovered the Higgs boson at the LHC in 2012, they observed about 20 instances, recording and analyzing more than 300 trillion collisions to confirm the particle’s discovery.

At the end of 2016, the ATLAS collaboration released its first measurement of the mass of the W boson particle (another elementary particle that, together with the Z boson, is responsible for the weak nuclear force). This measurement, which is based on a sample of 15 million W boson candidates collected at LHC in 2011, has a relative precision of 240 parts per million (ppm)—a result that matches the best single-experiment measurement announced in 2007 by the Collider Detector at Fermilab collaboration, whose measurement is based on several years’ worth of collected data. A highly precise measurement is important because a deviation from the mass predicted by the Standard Model could point to new physics. More data samples are required to achieve the level of accuracy (80 ppm) that scientists need to significantly test this model.

The volume of data collected by these experiments will grow significantly in the near future as new accelerator programs deliver higher-intensity beams. The LHC will be upgraded to increase its luminosity (rate of collisions) by a factor of 10. This High-Luminosity LHC, which should be operational by 2025, will provide a unique opportunity for particle physicists to look for new and unexpected phenomena within the exabytes (one exabyte equals 1000 petabytes) of data that will be collected.  

Data archiving is the first step in making available the results from such experiments. Thousands of physicists then need to calibrate and analyze the archived data and compare the data to simulations. To this end, computational scientists, computer scientists, and mathematicians in Brookhaven Lab’s Computational Science Initiative, which encompasses SDCC, are developing programming tools, numerical models, and data-mining algorithms. Part of SDCC’s mission is to provide computing and networking resources in support of these activities.

A Data Storage, Computing, and Networking Infrastructure

Housed inside SDCC are more than 60,000 computing cores, 250 computer racks, and tape libraries capable of holding up to 90,000 magnetic storage tape cartridges that are used to store, process, analyze, and distribute the experimental data. The facility provides approximately 90 percent of the computing capacity for analyzing data from the STAR and PHENIX experiments, and serves as the largest of the 12 Tier 1 computing centers worldwide that support the ATLAS experiment. As a Tier 1 center, SDCC contributes nearly 23 percent of the total computing and storage capacity for the ATLAS experiment and delivers approximately 200 terabytes of data (picture 62 million photos) per day to more than 100 data centers globally.

At SDCC, the High Performance Storage System (HPSS) has been providing mass storage services to the RHIC and LHC experiments since 1997 and 2006, respectively. This data archiving and retrieval software, developed by IBM and several DOE national laboratories, manages petabytes of data on disk and in robot-controlled tape libraries. Contained within the libraries are magnetic tape cartridges that encode the data and tape drives that read and write the data. Robotic arms load the cartridges into the drives and unload them upon request.

When ranked by the volume of data stored in a single HPSS, Brookhaven’s system is the second largest in the nation and the fourth largest in the world. Currently, the RACF operates nine Oracle robotic tape libraries that constitute the largest Oracle tape storage system in the New York tri-state area. Contained within this system are nearly 70,000 active cartridges with capacities ranging from 800 gigabytes to 8.5 terabytes, and more than 100 tape drives. As the volume of scientific data to be stored increases, more libraries, tapes, and drives can be added accordingly. In 2006, this scalability was exercised when HPSS was expanded to accommodate data from the ATLAS experiment at LHC.

“The HPSS system was deployed in the late 1990s, when the RHIC accelerator was coming on line. It allowed data from RHIC experiments to be transmitted via network to the data center for storage—a relatively new idea at the time,” said Shigeki Misawa, manager of Mass Storage and General Services at Brookhaven Lab. Misawa played a key role in the initial evaluation and configuration of HPSS, and has guided the system through significant changes in hardware (network equipment, storage systems, and servers) and operational requirements (tape drive read/write rate, magnetic tape cartridge capacity, and data transfer speed). “Prior to this system, data was recorded on magnetic tape at the experiment and physically moved to the data center,” he continued.

Over the years, SDCC’s HPSS has been augmented with a suite of optimization and monitoring tools developed at Brookhaven Lab. One of these tools is David Yu’s scheduling software that optimizes the retrieval of massive amounts of data from tape storage. Another, developed by Jérôme Lauret, software and computing project leader for the STAR experiment, is software for organizing multiple user requests to retrieve data more efficiently.

Engineers in the Mass Storage Group—including Tim Chou, Guangwei Che, and Ognian Novakov—have created other software tools customized for Brookhaven Lab’s computing environment to enhance data management and operation abilities and to improve the effectiveness of equipment usage.

STAR experiment scientists have demonstrated the capabilities of SDCC’s enhanced HPSS, retrieving more than 4,000 files per hour (a rate of 6,000 gigabytes per hour) while using a third of HPSS resources. On the data archiving side, HPSS can store data in excess of five gigabytes per second.

As demand for mass data storage spreads across Brookhaven, access to HPSS is being extended to other research groups. In the future, SDCC is expected to provide centralized mass storage services to multi-experiment facilities, such as the Center for Functional Nanomaterials and the National Synchrotron Light Source II—two more DOE Office of Science User Facilities at Brookhaven.

“The tape library system of SDCC is a clear asset for Brookhaven’s current and upcoming big data science programs,” said SDCC Director Eric Lançon. “Our expertise in the field of data archiving is acknowledged worldwide.”

Brookhaven National Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

X
X
X
  • Filters

  • × Clear Filters

Ames Laboratory, UConn Discover Superconductor with Bounce

The U.S. Department of Energy's Ames Laboratory has discovered extreme "bounce," or super-elastic shape-memory properties in a material that could be applied for use as an actuator in the harshest of conditions, such as outer space, and might be the first in a whole new class of shape memory materials.

Experiment Provides Deeper Look into the Nature of Neutrinos

The first glimpse of data from the full array of a deeply chilled particle detector operating beneath a mountain in Italy sets the most precise limits yet on where scientists might find a theorized process to help explain why there is more matter than antimatter in the universe.

Transparent Solar Technology Represents 'Wave of the Future'

See-through solar materials that can be applied to windows represent a massive source of untapped energy and could harvest as much power as bigger, bulkier rooftop solar units, scientists report today in Nature Energy.

Electricity From Shale Gas vs. Coal: Lifetime Toxic Releases From Coal Much Higher

Despite widespread concern about potential human health impacts from hydraulic fracturing, the lifetime toxic chemical releases associated with coal-generated electricity are 10 to 100 times greater than those from electricity generated with natural gas obtained via fracking, according to a new University of Michigan study.

Tiny Tornados at the Dawn of the Universe

Swirling soup of matter's fundamental building blocks spins ten billion trillion times faster than the most powerful tornado, setting new record for "vorticity."

Taming 'Wild' Electrons in Graphene

Graphene - a one-atom-thick layer of the stuff in pencils - is a better conductor than copper and is very promising for electronic devices, but with one catch: Electrons that move through it can't be stopped. Until now, that is. Scientists at Rutgers University-New Brunswick have learned how to tame the unruly electrons in graphene, paving the way for the ultra-fast transport of electrons with low loss of energy in novel systems. Their study was published online in Nature Nanotechnology.

On-Demand 3-D Printing of Tiny Magic Wands

Direct writing of pure-metal structures may advance novel light sources, sensors and information storage technologies.

Heavy Quarks Probe the Early Universe

New studies of behaviors of particles containing heavy quarks shed light into what the early universe looked like in its first microseconds.

Cool Roofs Have Water Saving Benefits Too

The energy and climate benefits of cool roofs have been well established: By reflecting rather than absorbing the sun's energy, light-colored roofs keep buildings, cities, and even the entire planet cooler. Now a new study by the Department of Energy's Lawrence Berkeley National Laboratory has found that cool roofs can also save water by reducing how much is needed for urban irrigation.

The Blob That Ate the Tokamak: Physicists Gain Understanding of How Bubbles at the Edge of Plasmas Can Drain Heat and Reduce Fusion Reaction Efficiency

Scientists at PPPL have completed new simulations that could provide insight into how blobs at the plasma edge behave. The simulations, produced by a code called XGC1 developed by a national team based at PPPL, performed kinetic simulations of two different regions of the plasma edge simultaneously.


  • Filters

  • × Clear Filters

Four Argonne Researchers Appointed Fellows of Scientific Societies

A select group of scientists at the U.S. Department of Energy's (DOE) Argonne National Laboratory has been honored as fellows of the American Physical Society and the Electrochemical Society. Physicists Kawtar Hafidi and Michael Carpenter have been appointed as American Physical Society fellows and Materials Scientist Khalil Amine and Chemist Chris Johnson have been elected as Electrochemical Society fellows.

Berkeley Lab and Hydro-Quebec Announce Partnership for Transportation Electrification and Energy Storage

Hydro-Quebec and the U.S. Department of Energy's Lawrence Berkeley National Laboratory (Berkeley Lab) have agreed to explore collaborations toward the research and development of manufacturing and scale-up technology to advance transportation electrification and energy storage.

Two ORNL-Led Research Teams Receive $10.5 Million to Advance Quantum Computing for Scientific Applications

DOE's Office of Science has awarded two research teams, each headed by a member of ORNL's Quantum Information Science Group, more than $10 million over 5 years to both assess the feasibility of quantum architectures in addressing big science problems and to develop algorithms capable of harnessing the massive power predicted of quantum computing systems. The two projects are intended to work in concert to ensure synergy across DOE's quantum computing research spectrum and maximize mutual benefits.

Department of Energy Awards Flow Into Argonne

DOE Secretary Rick Perry awarded Argonne with nearly $4.7 million in projects as part of the DOE's Office of Technology Transition's Technology Commercialization Fund (TCF) in September.

NIH Awards $6.5 Million to Berkeley Lab for Augmenting Structural Biology Research Experience

The NIH has awarded $6.5 million to Berkeley Lab to integrate existing synchrotron structural biology resources to better serve researchers. The grant will establish a center based at the Lab's Advanced Light Source (ALS) called ALS-ENABLE that will guide users through the most appropriate routes for answering their specific biological questions.

LIGO Announces Detection of Gravitational Waves From Colliding Neutron Stars

The U.S.-based Laser Interferometer Gravitational-Wave Observatory and the Virgo detector in Italy announced on Oct. 16 that all three of their detectors had picked up the ripples, or gravitational waves, from two neutron stars that collided 130 million years ago. Among other discoveries, the detection allowed scientists to use gravitational waves to directly calculate the rate at which the universe is expanding.

WVU Energy Conference to Address State's Economic Opportunities

West Virginia University will look at the state's emerging energy economy through industry experts, public policy organizations, environmental groups and academic institutions at the sixth annual National Energy Conference Oct. 20.

Exploring the Exotic World of Quarks and Gluons at the Dawn of the Exascale

As nuclear physicists delve ever deeper into the heart of matter, they require the tools to reveal the next layer of nature's secrets. Nowhere is that more true than in computational nuclear physics. A new research effort led by theorists at DOE's Thomas Jefferson National Accelerator Facility (Jefferson Lab) is now preparing for the next big leap forward in their studies thanks to funding under the 2017 SciDAC Awards for Computational Nuclear Physics.

Matthew Latimer Receives 2017 Lytle Award

A staff member at the Department of Energy's SLAC National Acceleratory Laboratory, Matthew Latimer is in charge of seven spectroscopy beamlines at SSRL. He was recently selected for the 2017 Farrel W. Lytle Award, established by the SSRL Users' Organization Executive Committee. The award promotes accomplishments in synchrotron science and supports collaboration among visiting scientists and staff who conduct research at SSRL.

Jefferson Lab Completes 12 GeV Upgrade

Nuclear physicists are now poised to embark on a new journey of discovery into the fundamental building blocks of the nucleus of the atom. The completion of the 12 GeV Upgrade Project of the Continuous Electron Beam Accelerator Facility (CEBAF) at the Department of Energy's Thomas Jefferson National Accelerator Facility (Jefferson Lab) heralds this new era to image nuclei at their deepest level.


  • Filters

  • × Clear Filters

Hybrid Material Glows Like Jellyfish

Scientists combine biology, nanotechnology into composites that light up upon chemical stimulation.

Tiny Tornados at the Dawn of the Universe

Swirling soup of matter's fundamental building blocks spins ten billion trillion times faster than the most powerful tornado, setting new record for "vorticity."

On-Demand 3-D Printing of Tiny Magic Wands

Direct writing of pure-metal structures may advance novel light sources, sensors and information storage technologies.

Heavy Quarks Probe the Early Universe

New studies of behaviors of particles containing heavy quarks shed light into what the early universe looked like in its first microseconds.

Discovering the Genetic Timekeepers in Bioenergy Crops

A new class of plant-specific genes required for flowering control in temperate grasses is found.

New Technology Illuminates Microbial Dark Matter

Demonstrating the microfluidic-based, mini-metagenomics approach on samples from hot springs shows how scientists can delve into microbes that can't be cultivated in a laboratory.

Tiny Green Algae Reveal Large Genomic Variation

First complete picture of genetic variations in a natural algal population could help explain how environmental changes affect global carbon cycles.

A Complex Little Alga that Lives by the Sea

The genetic material of Porphyra umbilicalis reveals the mechanisms by which it thrives in the stressful intertidal zone at the edge of the ocean.

Precise Radioactivity Measurements: A Controversy Settled

Simultaneous measurements of x-rays and gamma rays emitted in radioactive nuclear decays show that the vacancy left by an electron's departure, not the atomic structure, influences whether gamma rays are released.

OLYMPUS Experiment Sheds Light on Inner Workings of Protons

Seven-year study explains how packets of light are exchanged when protons meet electrons.


Spotlight

Tuesday October 03, 2017, 01:05 PM

Stairway to Science

Argonne National Laboratory

Thursday September 28, 2017, 12:05 PM

After-School Energy Rush

Argonne National Laboratory

Thursday September 28, 2017, 10:05 AM

Bringing Diversity Into Computational Science Through Student Outreach

Brookhaven National Laboratory

Thursday September 21, 2017, 03:05 PM

From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

SLAC National Accelerator Laboratory

Thursday September 07, 2017, 02:05 PM

Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

SLAC National Accelerator Laboratory

Thursday August 31, 2017, 05:05 PM

Binghamton University Opens $70 Million Smart Energy Building

Binghamton University, State University of New York

Wednesday August 23, 2017, 05:05 PM

Widening Horizons for High Schoolers with Code

Argonne National Laboratory

Saturday May 20, 2017, 12:05 PM

Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

Rensselaer Polytechnic Institute (RPI)

Monday May 15, 2017, 01:05 PM

ORNL, University of Tennessee Launch New Doctoral Program in Data Science

Oak Ridge National Laboratory

Friday April 07, 2017, 11:05 AM

Champions in Science: Profile of Jonathan Kirzner

Department of Energy, Office of Science

Wednesday April 05, 2017, 12:05 PM

High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

Argonne National Laboratory

Tuesday March 28, 2017, 12:05 PM

Champions in Science: Profile of Jenica Jacobi

Department of Energy, Office of Science

Friday March 24, 2017, 10:40 AM

Great Neck South High School Wins Regional Science Bowl at Brookhaven Lab

Brookhaven National Laboratory

Wednesday February 15, 2017, 04:05 PM

Middle Schoolers Test Their Knowledge at Science Bowl Competition

Argonne National Laboratory

Friday January 27, 2017, 04:00 PM

Haslam Visits ORNL to Highlight State's Role in Discovering Tennessine

Oak Ridge National Laboratory

Tuesday November 08, 2016, 12:05 PM

Internship Program Helps Foster Development of Future Nuclear Scientists

Oak Ridge National Laboratory

Friday May 13, 2016, 04:05 PM

More Than 12,000 Explore Jefferson Lab During April 30 Open House

Thomas Jefferson National Accelerator Facility

Monday April 25, 2016, 05:05 PM

Giving Back to National Science Bowl

Ames Laboratory

Friday March 25, 2016, 12:05 PM

NMSU Undergrad Tackles 3D Particle Scattering Animations After Receiving JSA Research Assistantship

Thomas Jefferson National Accelerator Facility

Tuesday February 02, 2016, 10:05 AM

Shannon Greco: A Self-Described "STEM Education Zealot"

Princeton Plasma Physics Laboratory

Monday November 16, 2015, 04:05 PM

Rare Earths for Life: An 85th Birthday Visit with Mr. Rare Earth

Ames Laboratory

Tuesday October 20, 2015, 01:05 PM

Meet Robert Palomino: 'Give Everything a Shot!'

Brookhaven National Laboratory

Tuesday April 22, 2014, 11:30 AM

University of Utah Makes Solar Accessible

University of Utah

Wednesday March 06, 2013, 03:40 PM

Student Innovator at Rensselaer Polytechnic Institute Seeks Brighter, Smarter, and More Efficient LEDs

Rensselaer Polytechnic Institute (RPI)

Friday November 16, 2012, 10:00 AM

Texas Tech Energy Commerce Students, Community Light up Tent City

Texas Tech University

Wednesday November 23, 2011, 10:45 AM

Don't Get 'Frosted' Over Heating Your Home This Winter

Temple University

Wednesday July 06, 2011, 06:00 PM

New Research Center To Tackle Critical Challenges Related to Aircraft Design, Wind Energy, Smart Buildings

Rensselaer Polytechnic Institute (RPI)

Friday April 22, 2011, 09:00 AM

First Polymer Solar-Thermal Device Heats Home, Saves Money

Wake Forest University

Friday April 15, 2011, 12:25 PM

Like Superman, American University Will Get Its Energy from the Sun

American University

Thursday February 10, 2011, 05:00 PM

ARRA Grant to Help Fund Seminary Building Green Roof

University of Chicago

Tuesday December 07, 2010, 05:00 PM

UC San Diego Installing 2.8 Megawatt Fuel Cell to Anchor Energy Innovation Park

University of California San Diego

Monday November 01, 2010, 12:50 PM

Rensselaer Smart Lighting Engineering Research Center Announces First Deployment of New Technology on Campus

Rensselaer Polytechnic Institute (RPI)

Friday September 10, 2010, 12:40 PM

Ithaca College Will Host Regional Clean Energy Summit

Ithaca College

Tuesday July 27, 2010, 10:30 AM

Texas Governor Announces $8.4 Million Award to Create Renewable Energy Institute

Texas Tech University

Friday May 07, 2010, 04:20 PM

Creighton University to Offer New Alternative Energy Program

Creighton University

Wednesday May 05, 2010, 09:30 AM

National Engineering Program Seeks Subject Matter Experts in Energy

JETS Junior Engineering Technical Society

Wednesday April 21, 2010, 12:30 PM

Students Using Solar Power To Create Sustainable Solutions for Haiti, Peru

Rensselaer Polytechnic Institute (RPI)

Wednesday March 03, 2010, 07:00 PM

Helping Hydrogen: Student Inventor Tackles Challenge of Hydrogen Storage

Rensselaer Polytechnic Institute (RPI)

Thursday February 04, 2010, 02:00 PM

Turning Exercise into Electricity

Furman University

Thursday November 12, 2009, 12:45 PM

Campus Leaders Showing the Way to a Sustainable, Clean Energy Future

National Wildlife Federation (NWF)

Tuesday November 03, 2009, 04:20 PM

Furman University Receives $2.5 Million DOE Grant for Geothermal Project

Furman University

Thursday September 17, 2009, 02:45 PM

Could Sorghum Become a Significant Alternative Fuel Source?

Salisbury University

Wednesday September 16, 2009, 11:15 AM

Students Navigating the Hudson River With Hydrogen Fuel Cells

Rensselaer Polytechnic Institute (RPI)

Wednesday September 16, 2009, 10:00 AM

College Presidents Flock to D.C., Urge Senate to Pass Clean Energy Bill

National Wildlife Federation (NWF)

Wednesday July 01, 2009, 04:15 PM

Northeastern Announces New Professional Master's in Energy Systems

Northeastern University

Friday October 12, 2007, 09:35 AM

Kansas Rural Schools To Receive Wind Turbines

Kansas State University

Thursday August 17, 2006, 05:30 PM

High Gas Prices Here to Stay, Says Engineering Professor

Rowan University





Showing results

0-4 Of 2215