Doe Science news source
The DOE Science News Source is a Newswise initiative to promote research news from the Office of Science of the DOE to the public and news media.
  • 2017-06-07 12:05:38
  • Article ID: 676049

Berkeley Lab's Open-Source Spinoff Serves Science

Singularity allows scientists to use "containers" in high-performance computing

  • Credit: Marillyn Chung/Berkeley Lab

    Greg Kurtzer invented software called Singularity to enable the use of containers in high-performance computing

Scientists used to come to Gregory Kurtzer of Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) IT department a lot, asking for a better way to use software containers in a high-performance computing (HPC) environment. After a while he got tired of saying, “Sorry, not possible.” So he invented a solution and named it Singularity.

Within a few months of its release last year, Singularity took off. Computing-heavy scientific institutions worldwide—from Stanford University to the Massachusetts Institute of Technology to various sites on the European Grid e-Infrastructure—flocked to the software. Singularity was also recently recognized by HPCwire editors as one of five new technologies to watch.

“Singularity has been making huge strides in the computing community,” Kurtzer said, with some surprise, adding that Open Science Grid, a consortium that provides distributed computing resources for scientific research, has served over 20 million containers with Singularity.

It’s now on its seventh release (version 2.2.1) and has caught on so quickly that Kurtzer has launched SingularityWare LLC to further develop and support the open-source software. The company is being funded by RStor Inc., a startup based in Saratoga, California. Kurtzer, the long-time technical lead and architect for the HPC Services group at Berkeley Lab with a joint appointment at UC Berkeley, will shift to an advisory role at the Lab in order to focus on Singularity.

“Berkeley Lab makes some Lab-developed software available at no cost to maximize its impact and to participate in the open-source software community,” said Elsie Quaite-Randall, Berkeley Lab’s Chief Technology Transfer Officer. “Singularity fosters innovation as open-source software, and now SingularityWare LLC—like other Berkeley Lab startups—will set out to expand the reach and adoption of an important technology.”

Who needs containers, anyway?

A typical case where users might need Singularity is if they want to run an application such as Google’s TensorFlow. “They may need a very specific version of Tensor Flow installed,” Kurtzer said. “They can create a container to do that in about five minutes. Then they can take that container, bring it to our environment and run it, even if we don’t have that version of Tensor Flow installed.”

Software containers make it possible to take your entire computing environment, including your files and all the applications you want to run, and encapsulate it so it can be easily replicated on another machine without worrying whether the new machine has a compatible operating system, libraries, applications, and so forth.

“Containers share some of the use cases of virtual machines but without the code redundancy and performance hit associated with virtualization,” Kurtzer said. “Singularity containers allow a user to encapsulate an entire OS (operating system) environment and use it on a shared HPC system like any other program, without an admin doing anything.”

Another example where Singularity would be useful would be allowing other scientists to reproduce experiments. “Say you just published an article. Wouldn’t it be nice to have a location you can cite where someone can download the Singularity container and replicate all the experiments?” Kurtzer asked. “Someone can enter the container, and now they’re sitting in the exact same environment as you were.”

Containerization was developed for enterprise environments, where it has become very popular, especially with the rise of Docker’s container technology. “Docker’s container solution is for the enterprise. But the scientific use case is quite different,” Kurtzer said. “Our goal isn’t to run as many containers as we can on a single host, with each having the illusion of sole occupancy and isolation, but to run maybe one, and enable it to utilize all the resources on that host. It’s kind of the opposite of isolation!”

So Kurtzer started working on his own solution, and four months later, the first version was released last spring. “When I started working on it, I asked, what do scientists really need from containers? They need reproducibility, mobility, and also freedom¾the ability to install their own applications and run in their own environment, and store it just like any other data file,” Kurtzer said. “That’s what Singularity solves for scientific computing.”

Long tail of science

Kurtzer chose the name Singularity for its meaning in astronomy. “As I understand it, it’s the culmination of a whole bunch of matter in the universe forming a single infinitely dense point,” he said. “That’s what I was thinking when I was creating Singularity—taking everything necessary to create a reproducible scientific environment and putting it in one file.”

Singularity also enables users to run legacy workflows easily. Kurtzer cites one example of how his group saved an 18-year-old workflow from failing hardware and was able to convert it to a Singularity container that is still being used in production today.

Kurtzer believes Singularity will benefit scientists who may not even know they need it. “We’re trying to reach out to more scientists and engage with additional groups, especially those who are not traditional HPC users, also known as the computational ‘long tail of science,’” Kurtzer said. “We have a lot of users that are running computationally intensive jobs on their laptops and workstations and not making use of the dedicated computational cycles that are designed specifically for computing and available to them. With Singularity we can easily make these large computing resources tangible.”

# # #

Lawrence Berkeley National Laboratory addresses the world’s most urgent scientific challenges by advancing sustainable energy, protecting human health, creating new materials, and revealing the origin and fate of the universe. Founded in 1931, Berkeley Lab’s scientific expertise has been recognized with 13 Nobel Prizes. The University of California manages Berkeley Lab for the U.S. Department of Energy’s Office of Science. For more, visit www.lbl.gov.

DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

X
X
X
  • Filters

  • × Clear Filters

Rutgers Scientists Discover 'Legos of Life'

Rutgers scientists have found the "Legos of life" - four core chemical structures that can be stacked together to build the myriad proteins inside every organism - after smashing and dissecting nearly 10,000 proteins to understand their component parts. The four building blocks make energy available for humans and all other living organisms, according to a study published online today in the Proceedings of the National Academy of Sciences.

Small Hydroelectric Dams Increase Globally with Little Research, Regulations

University of Washington researchers have published the first major assessment of small hydropower dams around the world -- including their potential for growth -- and highlight the incredibly variability in how dams of varying sizes are categorized, regulated and studied.

Researchers Reveal How Microbes Cope in Phosphorus-Deficient Tropical Soil

A team led by the Department of Energy's Oak Ridge National Laboratory has uncovered how certain soil microbes cope in a phosphorus-poor environment to survive in a tropical ecosystem. Their novel approach could be applied in other ecosystems to study various nutrient limitations and inform agriculture and terrestrial biosphere modeling.

Scientists Discover Material Ideal for Smart Photovoltaic Windows

Researchers at Berkeley Lab discovered that a form of perovskite, one of the hottest materials in solar research due to its high conversion efficiency, works surprisingly well as a stable and photoactive semiconductor material that can be reversibly switched between a transparent state and a non-transparent state, without degrading its electronic properties.

Biofuels Feedstock Study Supports Billion-Ton Estimate

Can farmers produce at least 1 billion tons of biomass per year that can be used as biofuels feedstock? The answer is yes.

On the Rebound

New research from the U.S. Department of Energy's Argonne National Laboratory and Stanford University has found that palladium nanoparticles can repair atomic dislocations in their crystal structure, potentially leading to other advances in material science.

Coupling Experiments to Theory to Build a Better Battery

A Berkeley Lab-led team of researchers has reported that a new lithium-sulfur battery component allows a doubling in capacity compared to a conventional lithium-sulfur battery, even after more than 100 charge cycles.

DRIFTing to Fast, Precise Data

Non-destructive technique identifies key variations in Alaskan soils, quickly providing insights into carbon levels.

A Shortcut to Modeling Sickle Cell Disease

Using Oak Ridge National Laboratory's Titan supercomputer, a team led by Brown University's George Karniadakis devised a multiscale model of sickle cell disease that captures what happens inside a red blood cell affected by the disease.

Remotely Predicting Leaf Age in Tropical Forests

New approach offers data across species, sites, and canopies, providing insights into carbon uptake by forests.


  • Filters

  • × Clear Filters

Theoretical Physicist Elena Belova Named to Editorial Board of Physics of Plasmas

Theoretical physicist Elena Belova named to editorial board of Physics of Plasmas

Superconducting X-Ray Laser Takes Shape in Silicon Valley

An area known for high-tech gadgets and innovation will soon be home to an advanced superconducting X-ray laser that stretches 3 miles in length, built by a collaboration of national laboratories. On January 19, the first section of the machine's new accelerator arrived by truck at SLAC National Accelerator Laboratory in Menlo Park after a cross-country journey that began in Batavia, Illinois, at Fermi National Accelerator Laboratory.

Kelsey Stoerzinger Earns Young Investigator Lectureship

Kelsey Stoerzinger, Pauling Fellow at Pacific Northwest National Laboratory, is one of the 2018 Caltech Young Investigator Lecturers in Engineering and Applied Physics.

North Dakota State University Joins Two National Distributed Computing Groups

The NDSU Center for Computationally Assisted Science and Technology (CCAST) joins OSG (Open Science Grid) and XSEDE (Extreme Science and Engineering Discovery Environment).

DOE Announces Funding for New HPC4Manufacturing Industry Projects

The Department of Energy's Advanced Manufacturing Office (AMO) today announced the funding of $1.87 million for seven new industry projects under an ongoing initiative designed to utilize DOE's high-performance computing (HPC) resources and expertise to advance U.S. manufacturing and clean energy technologies.

DOE Announces First Awardees for New HPC4Materials for Severe Environments Program

The Department of Energy's Office of Fossil Energy (FE) today announced the funding of $450,000 for the first two private-public partnerships under a brand-new initiative aimed at discovering, designing and scaling up production of novel materials for severe environments.

Two Argonne Scientists Recognized for a Decade of Breakthroughs

Two scientists with the U.S. Department of Energy's (DOE) Argonne National Laboratory have been named to the Web of Science's Highly Cited List of 2017, ranking in the top 1 percent of their peers by citations and subject area. Materials Scientist Khalil Amine and Energy and Environmental Policy Scientist David Streets say they are thrilled to see their work -- and the laboratory -- recognized in such a way.

Argonne Welcomes Department of Energy Secretary Perry

U.S. Department of Energy Secretary Rick Perry visited Argonne National Laboratory yesterday, getting a first-hand view of the multifaceted and interdisciplinary research program laboratory of the Department.

Argonne names John Quintana Deputy Laboratory Director for Operations and COO

John Quintana has been named Deputy Laboratory Director for Operations and Chief Operations Officer (COO) of the U.S. Department of Energy's (DOE) Argonne National Laboratory.

Developing Next-Generation Sensing Technologies

Recently, the Advanced Research Projects Agency-Energy (ARPA-E) announced $20 million in funding for 15 projects that will develop a new class of sensor systems to enable significant energy savings via reduced demand for heating and cooling in residential and commercial buildings.


  • Filters

  • × Clear Filters

Exploring Past, Present, and Future Water Availability Regionally, Globally

New open-source software simulates river and runoff resources.

Arctic Photosynthetic Capacity and Carbon Dioxide Assimilation Underestimated by Terrestrial Biosphere Models

New measurements offer data vital to projecting plant response to environmental changes.

DRIFTing to Fast, Precise Data

Non-destructive technique identifies key variations in Alaskan soils, quickly providing insights into carbon levels.

Superconducting Tokamaks Are Standing Tall

Plasma physicists significantly improve the vertical stability of a Korean fusion device.

Graphene Flexes Its Muscle

Crumpling reduces rigidity in an otherwise stiff material, making it less prone to catastrophic failure.

Remotely Predicting Leaf Age in Tropical Forests

New approach offers data across species, sites, and canopies, providing insights into carbon uptake by forests.

What's the Noise Eating Quantum Bits?

The magnetic noise caused by adsorbed oxygen molecules is "eating at" the phase stability of quantum bits, mitigating the noise is vital for future quantum computers.

Rewritable Wires Could Mean No More Obsolete Circuitry

An electric field switches the conductivity on and off in atomic-scale channels, which could allow for upgrades at will.

Filtering Water Better than Nature

Water passes through human-made straws faster than the "gold standard" protein, allowing us to filter seawater.

Machine Learning Provides a Bridge to the Texture of the Quantum World

Machine learning and neural networks are the foundation of artificial intelligence and image recognition, but now they offer a bridge to see and recognize exotic insulating phases in quantum materials.


Spotlight

Wednesday January 17, 2018, 12:05 PM

Photographer Adam Nadel Selected as Fermilab's New Artist-in-Residence for 2018

Fermi National Accelerator Laboratory (Fermilab)

Wednesday January 17, 2018, 12:05 PM

Fermilab Computing Partners with Argonne, Local Schools for Hour of Code

Fermi National Accelerator Laboratory (Fermilab)

Wednesday December 20, 2017, 01:05 PM

Q&A: Sam Webb Teaches X-Ray Science from a Remote Classroom

SLAC National Accelerator Laboratory

Monday December 18, 2017, 01:05 PM

The Future of Today's Electric Power Systems

Rensselaer Polytechnic Institute (RPI)

Monday December 18, 2017, 12:05 PM

Supporting the Development of Offshore Wind Power Plants

Rensselaer Polytechnic Institute (RPI)

Tuesday October 03, 2017, 01:05 PM

Stairway to Science

Argonne National Laboratory

Thursday September 28, 2017, 12:05 PM

After-School Energy Rush

Argonne National Laboratory

Thursday September 28, 2017, 10:05 AM

Bringing Diversity Into Computational Science Through Student Outreach

Brookhaven National Laboratory

Thursday September 21, 2017, 03:05 PM

From Science to Finance: SLAC Summer Interns Forge New Paths in STEM

SLAC National Accelerator Laboratory

Thursday September 07, 2017, 02:05 PM

Students Discuss 'Cosmic Opportunities' at 45th Annual SLAC Summer Institute

SLAC National Accelerator Laboratory

Thursday August 31, 2017, 05:05 PM

Binghamton University Opens $70 Million Smart Energy Building

Binghamton University, State University of New York

Wednesday August 23, 2017, 05:05 PM

Widening Horizons for High Schoolers with Code

Argonne National Laboratory

Saturday May 20, 2017, 12:05 PM

Rensselaer Polytechnic Institute Graduates Urged to Embrace Change at 211th Commencement

Rensselaer Polytechnic Institute (RPI)

Monday May 15, 2017, 01:05 PM

ORNL, University of Tennessee Launch New Doctoral Program in Data Science

Oak Ridge National Laboratory

Friday April 07, 2017, 11:05 AM

Champions in Science: Profile of Jonathan Kirzner

Department of Energy, Office of Science

Wednesday April 05, 2017, 12:05 PM

High-Schooler Solves College-Level Security Puzzle From Argonne, Sparks Interest in Career

Argonne National Laboratory

Tuesday March 28, 2017, 12:05 PM

Champions in Science: Profile of Jenica Jacobi

Department of Energy, Office of Science

Friday March 24, 2017, 10:40 AM

Great Neck South High School Wins Regional Science Bowl at Brookhaven Lab

Brookhaven National Laboratory

Wednesday February 15, 2017, 04:05 PM

Middle Schoolers Test Their Knowledge at Science Bowl Competition

Argonne National Laboratory

Friday January 27, 2017, 04:00 PM

Haslam Visits ORNL to Highlight State's Role in Discovering Tennessine

Oak Ridge National Laboratory

Tuesday November 08, 2016, 12:05 PM

Internship Program Helps Foster Development of Future Nuclear Scientists

Oak Ridge National Laboratory

Friday May 13, 2016, 04:05 PM

More Than 12,000 Explore Jefferson Lab During April 30 Open House

Thomas Jefferson National Accelerator Facility

Monday April 25, 2016, 05:05 PM

Giving Back to National Science Bowl

Ames Laboratory

Friday March 25, 2016, 12:05 PM

NMSU Undergrad Tackles 3D Particle Scattering Animations After Receiving JSA Research Assistantship

Thomas Jefferson National Accelerator Facility

Tuesday February 02, 2016, 10:05 AM

Shannon Greco: A Self-Described "STEM Education Zealot"

Princeton Plasma Physics Laboratory

Monday November 16, 2015, 04:05 PM

Rare Earths for Life: An 85th Birthday Visit with Mr. Rare Earth

Ames Laboratory

Tuesday October 20, 2015, 01:05 PM

Meet Robert Palomino: 'Give Everything a Shot!'

Brookhaven National Laboratory

Tuesday April 22, 2014, 11:30 AM

University of Utah Makes Solar Accessible

University of Utah

Wednesday March 06, 2013, 03:40 PM

Student Innovator at Rensselaer Polytechnic Institute Seeks Brighter, Smarter, and More Efficient LEDs

Rensselaer Polytechnic Institute (RPI)

Friday November 16, 2012, 10:00 AM

Texas Tech Energy Commerce Students, Community Light up Tent City

Texas Tech University

Wednesday November 23, 2011, 10:45 AM

Don't Get 'Frosted' Over Heating Your Home This Winter

Temple University

Wednesday July 06, 2011, 06:00 PM

New Research Center To Tackle Critical Challenges Related to Aircraft Design, Wind Energy, Smart Buildings

Rensselaer Polytechnic Institute (RPI)

Friday April 22, 2011, 09:00 AM

First Polymer Solar-Thermal Device Heats Home, Saves Money

Wake Forest University

Friday April 15, 2011, 12:25 PM

Like Superman, American University Will Get Its Energy from the Sun

American University

Thursday February 10, 2011, 05:00 PM

ARRA Grant to Help Fund Seminary Building Green Roof

University of Chicago

Tuesday December 07, 2010, 05:00 PM

UC San Diego Installing 2.8 Megawatt Fuel Cell to Anchor Energy Innovation Park

University of California San Diego

Monday November 01, 2010, 12:50 PM

Rensselaer Smart Lighting Engineering Research Center Announces First Deployment of New Technology on Campus

Rensselaer Polytechnic Institute (RPI)

Friday September 10, 2010, 12:40 PM

Ithaca College Will Host Regional Clean Energy Summit

Ithaca College

Tuesday July 27, 2010, 10:30 AM

Texas Governor Announces $8.4 Million Award to Create Renewable Energy Institute

Texas Tech University

Friday May 07, 2010, 04:20 PM

Creighton University to Offer New Alternative Energy Program

Creighton University

Wednesday May 05, 2010, 09:30 AM

National Engineering Program Seeks Subject Matter Experts in Energy

JETS Junior Engineering Technical Society

Wednesday April 21, 2010, 12:30 PM

Students Using Solar Power To Create Sustainable Solutions for Haiti, Peru

Rensselaer Polytechnic Institute (RPI)

Wednesday March 03, 2010, 07:00 PM

Helping Hydrogen: Student Inventor Tackles Challenge of Hydrogen Storage

Rensselaer Polytechnic Institute (RPI)

Thursday February 04, 2010, 02:00 PM

Turning Exercise into Electricity

Furman University

Thursday November 12, 2009, 12:45 PM

Campus Leaders Showing the Way to a Sustainable, Clean Energy Future

National Wildlife Federation (NWF)

Tuesday November 03, 2009, 04:20 PM

Furman University Receives $2.5 Million DOE Grant for Geothermal Project

Furman University

Thursday September 17, 2009, 02:45 PM

Could Sorghum Become a Significant Alternative Fuel Source?

Salisbury University

Wednesday September 16, 2009, 11:15 AM

Students Navigating the Hudson River With Hydrogen Fuel Cells

Rensselaer Polytechnic Institute (RPI)





Showing results

0-4 Of 2215