DOE News
    Doe Science news source
    The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
    • 2018-07-24 12:05:20
    • Article ID: 697831

    Berkeley Lab-Developed Digital Library is a Game Changer for Environmental Research

    ESS-DIVE is a new digital library for the Department of Energy's environmental science data

    • Credit: (Photo credit: Berkeley Lab)

      Data collection in Rifle, Colorado, for EESA’s Sustainable Systems Scientific Focus Area 2.0 research project.

    • Credit: (Credit: Berkeley Lab)

      This is a screenshot of the ESS-DIVE Data Access Portal, which enables search of the entire text within the metadata submitted by contributors, and allows filtering by geographic location through a map, and other criteria such as DOI, authors, keywords and publication year.

    Environmental data are crucial for planning our water and energy future, safeguarding against environmental threats and building resilient infrastructure. By using high-quality observations collected over years to power computer models, researchers can examine and predict ecosystem and watershed behaviors over the course of seasons to decades to centuries.  However, storing, accessing and incorporating environmental data into models is challenging due to the diversity of the datasets, which include measurement of properties associated with bedrock, groundwater, soils, vegetation and atmospheric compartments of environmental systems.

    Now accessing archival data generated by environmental field, experimental and modeling activities has gotten much easier with the April 1 launch of ESS-DIVE (Environmental System Science – Data Infrastructure for a Virtual Ecosystem)—a digital archive that serves as a repository for hundreds of U.S. Department of Energy (DOE)-funded research projects under the agency’s Environmental System Science umbrella, which includes the Subsurface Biogeochemical Research and Terrestrial Ecosystem Sciences programs. The digital library also serves datasets that were previously stored in DOE’s Carbon Dioxide Information Analysis Center archive.

    The digital library was built by a collaboration of scientists in Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) Computational Research Division (CRD) and Earth & Environmental Sciences Area (EESA), the National Energy Research Scientific Computing (NERSC) and digital librarians at the National Center for Ecological Analysis and Synthesis (NCEAS)—a research center based at UC Santa Barbara.  

    “By moving to a digital library, we are going from a one-time use to a reusable data paradigm. Because ESS-DIVE stores ‘data packages,’ which are essentially collections of related data files and metadata, we no longer just have a research paper to reference but also the underlying data that led to those results. This means that other scientists can review the analyses, validate results, study different scenarios and reuse the data for other purposes,” says Deb Agarwal, a scientist in Berkeley Lab’s CRD and lead of the ESS-DIVE project.  

    And when ESS-DIVE becomes a DataONE member node in the near future, she notes that it will become an even more powerful tool, as the library’s DOE-funded data contents will be discoverable in cross-catalogue searches. DataONE supports enhanced search and discovery of earth and environmental data across a number of repositories managed by its members.  

    “This digital library is a total game-changer for scientists like me. ESS-DIVE will make it much easier for scientists to archive our data using standardized formats, to make data publicly available and to find and integrate data from various research studies. Although some projects had independent systems to manage their data, those efforts were not coordinated, and there was no central location where data across the DOE’s environmental research program were available,” says Charuleka Varadharajan, a research scientist in Berkeley Lab’s EESA and deputy lead for the ESS-DIVE project.

    “A lot of our datasets require really intensive fieldwork to collect. No field season is ever going to be the same because the environmental conditions are always changing. And to accurately predict long-term ecosystem behavior, we need to feed and validate our models with years to decades of diverse observations. This means that every single data point is incredibly valuable and has the potential to be reused for purposes that go way beyond the intent for which it was originally collected,” Varadharajan adds. “That’s why this work is so important.”

    It Takes a Community to Build an Effective Archive

    Interdisciplinary research is a hallmark of Berkeley Lab, and over the years Lab computer and environmental scientists have collaborated to help a number of large ESS projects—like the Berkeley Water Center, AmeriFlux, Watershed Function SFA, and Next Generation Ecosystem Experiment (NGEE)-Tropics—curate, manage, store and publicly share their data. From these collaborations, Agarwal learned early on that the most effective way to build useful scientific tools and accelerate adoption of any kind of data standard or technology is to work in partnership with the scientific community.  

    “We have to embed ourselves with the project’s scientists. We have to sit down and work with them to understand their challenges and define an infrastructure that fits with their process of curating and publishing data, such that it is a natural part of their workflow,” says Agarwal. “Our end goal is to make these data publicly available and easily accessible. If researchers only upload their data once their paper is published and the research is complete, then our archive becomes an afterthought and a burden.”

    In the case of ESS-DIVE, Agarwal and her team included functionalities that allow researchers to upload their data and keep it proprietary as they work on the paper. Once the paper is published, researchers can go into the library and easily make their data public.

    ESS-DIVE also includes tools that allow users to access data, contribute and publish data and track data downloads. And because each data package is assigned a unique Digital Object Identifier (DOI) when it’s uploaded, researchers can now cite the dataset that they used.  The DOIs are assigned with the help of the Office of Scientific and Technical Information, which also helped with the transition of pre-existing data into ESS-DIVE. 

    “The DOI is extremely useful for data generators, who typically work hard to collect their data and clean and process it. With this identifier, the data set itself becomes a product that can be cited,” says Varadharajan. “That way, if the data were used in a publication that the data generators are not co-authors on, they would still be able to get credit for producing the data.”

    The current implementation of ESS-DIVE’s web portal is built on top of the Metacat repository software developed by NCEAS. Metacat stores metadata using the Ecological Metadata Standard, provides an application programming interface to facilitate the input and retrieval of data packages and includes an engine to query across a number of metadata attributes.

    All basic components of the architecture run inside Docker containers, which allow multiple redundant instances of ESS-DIVE to be available in case of outages at a site. The primary ESS-DIVE instance runs on NERSC’s Spin, a Docker-based edge-services technology that can access the facility’s supercomputers and storage systems on the back end. In addition, copies of the library’s data and metadata are stored on a server in Berkeley Lab’s Information Technology Division and will soon be replicated via Metacat to NCEAS and the DataONE federation, which guarantees broad redundant availability.    

    “Spin is great because it effectively allows us to run a next-generation science gateway at NERSC,” says Agarwal. “We can install all of the different nuanced interaction capabilities in an easily deployable space, which is then held in a Docker container. This means that users can upload and interact with data in a controlled environment, but the infrastructure can access all of the local libraries that we’ve stored out in NERSC’s project space. And because everything is contained within a Docker container, we can also create backup instances relatively easily.”

    In addition to Agarwal and Varadharajan, other key Berkeley Lab contributors to ESS-DIVE include Shreyas Cholia (CRD), Cory Snavely (NERSC), Valerie Hendrix (CRD), Fianna O’Brien (CRD), Abdelrahman Elbashandy (CRD), Karen Whitenack (CRD), Yeongshnn Ong (CRD) and William Riley (EESA). NERSC is a DOE Office of Science user facility.


    More information about ESS-DIVE:

    • Filters

    • × Clear Filters

    Microbes Eat the Same in Labs and the Desert

    Analyses of natural communities forming soil crusts agree with laboratory studies of isolated microbe-metabolite relationships.

    Scientists Produce 3-D Chemical Maps of Single Bacteria

    Scientists at the National Synchrotron Light Source II (NSLS-II)--a U.S. Department of Energy (DOE) Office of Science User Facility at DOE's Brookhaven National Laboratory--have used ultrabright x-rays to image single bacteria with higher spatial resolution than ever before. Their work, published in Scientific Reports, demonstrates an x-ray imaging technique, called x-ray fluorescence microscopy (XRF), as an effective approach to produce 3-D images of small biological samples.

    Self-Sensing Materials Are Here

    Oak Ridge National Laboratory researchers invented a way to make a nanomaterial-embedded composite that is stronger than other fiber-reinforced composites and imbued with a new capability--the ability to monitor its own structural health.

    Diverse Biofeedstocks Have High Ethanol Yields and Offer Biorefineries Flexibility

    Evidence suggests that biorefineries can accept various feedstocks without negatively impacting the amount of ethanol produced per acre.

    Opening Access to Explore the Synthetic Chemistry of Neptunium

    New, easily prepared starting material opens access to learning more about a difficult-to-control element in nuclear waste.

    Symbiosis a Driver of Truffle Diversity

    Truffles are thought of as dining delicacies but they play an important role in soil ecosystem services as the fruiting bodies of the ectomycorrhizal (ECM) fungal symbionts residing on host plant roots. An international team sought insights into the ECM lifestyle of truffle-forming species through a comparative analysis of eight fungal genomes.

    Climate Simulations Project Wetter, Windier Hurricanes

    New supercomputer simulations by climate scientists at the Department of Energy's Lawrence Berkeley National Laboratory (Berkeley Lab) have shown that climate change intensified the amount of rainfall in recent hurricanes such as Katrina, Irma, and Maria by 5 to 10 percent. They further found that if those hurricanes were to occur in a future world that is warmer than present, those storms would have even more rainfall and stronger winds.

    Tiny Titanium Barrier Halts Big Problem in Fuel-Producing Solar Cells

    New design coats molecular components and dramatically improves stability under tough, oxidizing conditions.

    X-Rays Show How Periods of Stress Changed an Ice Age Hyena to the Bone

    An international team has unearthed what life might have been like for a now-extinct subspecies of spotted hyena. They found that despite their massive size, some cave hyenas experienced times of hardship that affected them to the bone, causing areas of arrested growth that appear as dark lines, like rings on a tree trunk.

    Turning Wood Scraps into Tape

    A new chemical process converts a component of wasted wood pulp and other biomass into high-value pressure-sensitive adhesives.

    • Filters

    • × Clear Filters

    Argonne's Min Si receives early career award from IEEE Computer Society

    Argonne's Min Si wins Award for Excellence for Early Career Researchers in High Performance Computing through the Institute of Electrical and Electronics Engineers.

    Jefferson Lab Director Appointed to Distinguished Professorship

    Stuart Henderson, director of the Department of Energy's Thomas Jefferson National Accelerator Facility, has been appointed the Governor's Distinguished CEBAF professor at Old Dominion University. The position is supported by the Commonwealth of Virginia and is named for the Continuous Electron Beam Accelerator Facility, which is the main research facility located at Jefferson Lab.

    DOE issues call for HPC for Energy Innovation proposals

    The U.S. Department of Energy's (DOE) High Performance Computing for Energy Innovation (HPC4EI) Initiative today issued its first joint solicitation for the High Performance Computing for Manufacturing Program (HPC4Mfg) and the High Performance Computing for Materials Program (HPC4Mtls).

    DOE funding advances project to turn captured CO2 into key chemicals

    The U.S. Department of Energy has selected Southern Research for an award of up to $1.5 million to advance technology for carbon dioxide utilization.

    Sierra Reaches Higher Altitudes, Takes Number Two Spot on List of Fastest Supercomputers

    Sierra, Lawrence Livermore National Laboratory's newest supercomputer, rose to second place on the list of the world's fastest computing systems, TOP500 List representatives announced Monday at the International Conference for High Performance Computing, Networking, Storage and Analysis conference (SC18) in Dallas.

    Green energy: Wind energy agreement will provide savings, 50 percent of electricity needs for Kansas State University Manhattan campus

    Kansas State University has signed an agreement with Westar Energy to provide approximately 50 percent of the energy needs for the university's main Manhattan campus from a wind farm in Nemaha County and save the university nearly $200,000 annually.

    INCITE grants awarded to 62 computational research projects

    The U.S. Department of Energy announced new projects for 2019 through its Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program.

    Argonne's Raj Kettimuthu Named ACM Distinguished Member

    Argonne computer scientist Raj Kettimuthu recently was named a Distinguished Member of the Association for Computing Machinery for his development of tools to analyze and enhance end-to-end data transfer performance.

    Jefferson Lab-Affiliated Researchers Honored as APS Fellows

    The Department of Energy's Thomas Jefferson National Accelerator Facility now has a few more fellows on campus. The American Physical Society, a professional membership society that works on behalf of the physics community, recently announced its list of 2018 fellowships.

    Jefferson Lab Receives DOE Award for Energy Efficient Upgrade

    On Oct. 23, a team from the Department of Energy's Thomas Jefferson National Accelerator Facility was honored at the 2018 Federal Energy and Water Management Award Ceremony for upgrades made to the lab's data center, ultimately improving its energy efficiency.

    • Filters

    • × Clear Filters

    Microbes Eat the Same in Labs and the Desert

    Analyses of natural communities forming soil crusts agree with laboratory studies of isolated microbe-metabolite relationships.

    Diverse Biofeedstocks Have High Ethanol Yields and Offer Biorefineries Flexibility

    Evidence suggests that biorefineries can accept various feedstocks without negatively impacting the amount of ethanol produced per acre.

    Opening Access to Explore the Synthetic Chemistry of Neptunium

    New, easily prepared starting material opens access to learning more about a difficult-to-control element in nuclear waste.

    Tiny Titanium Barrier Halts Big Problem in Fuel-Producing Solar Cells

    New design coats molecular components and dramatically improves stability under tough, oxidizing conditions.

    Turning Wood Scraps into Tape

    A new chemical process converts a component of wasted wood pulp and other biomass into high-value pressure-sensitive adhesives.

    Very Heavy Elements Deliver More Electrons

    Scientists revise understanding of the limits of bonding for very electron-rich heavy elements.

    Probing Water's "No-Man's Land" Temperature Region

    Measuring the physical properties of water at previously unexplored temperatures offers insights into one of the world's essential liquids.

    Novel Soil Bacteria with Unusual Genes Synthesize Unique Antibiotic Precursors

    A large-scale soil project uncovered genetic information from bacteria with the capacity to make specialized molecules that could lead to new pharmaceuticals.

    Warmer Temperatures Lengthen Growing Season, Increase Plants' Vulnerability to Frost

    Experimental warming treatments show how peatland forests may respond to future environmental change.

    Rising Stars Seek to Learn from the Master: Mother Nature

    A trio of scientists was recognized for their early career successes in uncovering how microbes produce fuel, insights that could change our energy portfolio


    Thursday October 11, 2018, 04:00 PM

    Research on Light-Matter Interaction Could Lead to Improved Electronic and Optoelectronic Devices

    Rensselaer Polytechnic Institute (RPI)

    Wednesday October 03, 2018, 07:05 PM

    Innovating Our Energy Future

    Oregon State University, College of Engineering

    Tuesday October 02, 2018, 03:05 PM

    Physics graduate student takes her thesis research to a Department of Energy national lab

    University of Alabama at Birmingham

    Friday September 21, 2018, 01:05 PM

    "Model" students enjoy Argonne campus life

    Argonne National Laboratory

    Thursday September 06, 2018, 01:05 PM

    Writing Code for a More Skilled and Diverse STEM Workforce

    Brookhaven National Laboratory

    Tuesday September 04, 2018, 11:30 AM

    New graduate student summer school launches at Princeton Plasma Physics Laboratory

    Princeton Plasma Physics Laboratory

    Friday August 31, 2018, 06:05 PM

    The Gridlock State

    California State University (CSU) Chancellor's Office

    Friday August 31, 2018, 02:05 PM

    Meet Jasmine Hatcher and Trishelle Copeland-Johnson

    Brookhaven National Laboratory

    Friday August 24, 2018, 11:05 AM

    Argonne hosts Modeling, Experimentation and Validation Summer School

    Argonne National Laboratory

    Wednesday August 22, 2018, 01:05 PM

    Students affected by Hurricane Maria bring their research to SLAC

    SLAC National Accelerator Laboratory

    Wednesday August 22, 2018, 10:05 AM

    Brookhaven Lab Pays Tribute to 2018 Summer Interns

    Brookhaven National Laboratory

    Monday August 20, 2018, 12:05 PM

    Changing How Buildings Are Made

    Washington University in St. Louis

    Thursday August 16, 2018, 12:05 PM

    CSUMB Selected to Host Architecture at Zero Competition in 2019

    California State University, Monterey Bay

    Friday July 20, 2018, 03:00 PM

    Department of Energy Invests $64 Million in Advanced Nuclear Technology

    Rensselaer Polytechnic Institute (RPI)

    Thursday July 19, 2018, 05:00 PM

    Professor Miao Yu Named the Priti and Mukesh Chatter '82 Career Development Professor

    Rensselaer Polytechnic Institute (RPI)

    Tuesday July 03, 2018, 11:05 AM

    2018 RHIC & AGS Annual Users' Meeting: 'Illuminating the QCD Landscape'

    Brookhaven National Laboratory

    Friday June 29, 2018, 06:05 PM

    Argonne welcomes The Martian author Andy Weir

    Argonne National Laboratory

    Monday June 18, 2018, 09:55 AM

    Creating STEM Knowledge and Innovations to Solve Global Issues Like Water, Food, and Energy

    Illinois Mathematics and Science Academy (IMSA)

    Friday June 15, 2018, 10:00 AM

    Professor Emily Liu Receives $1.8 Million DoE Award for Solar Power Systems Research

    Rensselaer Polytechnic Institute (RPI)

    Thursday June 07, 2018, 03:05 PM

    Celebrating 40 years of empowerment in science

    Argonne National Laboratory

    Monday May 07, 2018, 10:30 AM

    Introducing Graduate Students Across the Globe to Photon Science

    Brookhaven National Laboratory

    Wednesday May 02, 2018, 04:05 PM

    Students from Massachusetts and Washington Win DOE's 28th National Science Bowl(r)

    Department of Energy, Office of Science

    Thursday April 12, 2018, 07:05 PM

    The Race for Young Scientific Minds

    Argonne National Laboratory

    Wednesday March 14, 2018, 02:05 PM

    Q&A: Al Ashley Reflects on His Efforts to Diversify SLAC and Beyond

    SLAC National Accelerator Laboratory

    Thursday February 15, 2018, 12:05 PM

    Insights on Innovation in Energy, Humanitarian Aid Highlight UVA Darden's Net Impact Week

    University of Virginia Darden School of Business

    Friday February 09, 2018, 11:05 AM

    Ivy League Graduate, Writer and Activist with Dyslexia Visits CSUCI to Reframe the Concept of Learning Disabilities

    California State University, Channel Islands

    Wednesday January 17, 2018, 12:05 PM

    Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday January 17, 2018, 12:05 PM

    Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

    Fermi National Accelerator Laboratory (Fermilab)

    Wednesday December 20, 2017, 01:05 PM

    Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

    SLAC National Accelerator Laboratory

    Monday December 18, 2017, 01:05 PM

    The Future of Today's Electric Power Systems

    Rensselaer Polytechnic Institute (RPI)

    Monday December 18, 2017, 12:05 PM

    Supporting the Development of Offshore Wind Power Plants

    Rensselaer Polytechnic Institute (RPI)

    Tuesday October 03, 2017, 01:05 PM

    Stairway to Science

    Argonne National Laboratory

    Thursday September 28, 2017, 12:05 PM

    After-School Energy Rush

    Argonne National Laboratory

    Thursday September 28, 2017, 10:05 AM

    Bringing Diversity Into Computational Science Through Student Outreach

    Brookhaven National Laboratory

    Thursday September 21, 2017, 03:05 PM

    From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

    SLAC National Accelerator Laboratory

    Thursday September 07, 2017, 02:05 PM

    Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

    SLAC National Accelerator Laboratory

    Thursday August 31, 2017, 05:05 PM

    Binghamton University Opens $70 Million Smart Energy Building

    Binghamton University, State University of New York

    Wednesday August 23, 2017, 05:05 PM

    Widening Horizons for High Schoolers with Code

    Argonne National Laboratory

    Saturday May 20, 2017, 12:05 PM

    Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

    Rensselaer Polytechnic Institute (RPI)

    Monday May 15, 2017, 01:05 PM

    ORNL, University of Tennessee Launch New Doctoral Program in Data Science

    Oak Ridge National Laboratory

    Friday April 07, 2017, 11:05 AM

    Champions in Science: Profile of Jonathan Kirzner

    Department of Energy, Office of Science

    Wednesday April 05, 2017, 12:05 PM

    High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

    Argonne National Laboratory

    Tuesday March 28, 2017, 12:05 PM

    Champions in Science: Profile of Jenica Jacobi

    Department of Energy, Office of Science

    Friday March 24, 2017, 10:40 AM

    Great Neck South High School Wins Regional Science Bowl at Brookhaven Lab

    Brookhaven National Laboratory

    Wednesday February 15, 2017, 04:05 PM

    Middle Schoolers Test Their Knowledge at Science Bowl Competition

    Argonne National Laboratory

    Showing results

    0-4 Of 2215