Newswise — ESnet, the Energy Sciences Network, is the Internet connection you wish you had – but it is so much more. It is a high performance network created for the U.S. Department of Energy’s Office of Science (SC), to speed discoveries and innovations by connecting scientists at national labs, universities, and other research institutions. Funded primarily by the SC’s office of Advanced Scientific Computing Research (ASCR) and managed by Lawrence Berkeley National Laboratory, ESnet has grown to a global network capable of delivering data at speeds up to 100 gigabits per second through the network’s direct connections to more than 40 DOE sites. ESnet also connects to 140 research and commercial networks, creating a fast and reliable infrastructure. Its high speed and reliability make ESnet itself an instrument for collaboration and discovery; it empowers researchers around the globe to send, share, and collectively analyze vast data sets generated by sophisticated scientific instruments, such as the Large Hadron Collider, supercomputers, and digital sky surveys. Currently ESnet carries around 20 petabytes of data each month – an amount that would fill approximately 400 million 4-drawer filing cabinets full of text – and expects to scale up to 100 petabytes of data by 2016.
Weathering the Flood of Big Data in Climate ResearchAn extreme weather researcher recently used the speed and collaborative power of ESnet to gather massive climate datasets from 21 different sites around the world to look at how extreme weather is likely to change in the future, in frequency and intensity. He estimated it would take more than two and a half years, using a desktop computer and commercial network speed to download the datasets. He turned to ESnet for help. It took three months of work by the ESnet staff to transfer the data – accessing each site, solving problems, overseeing the process one file at a time. Next, ESnet staff sent the data to Berkeley Lab’s National Energy Research Scientific Computing Center (NERSC) where data were standardized and cleaned in approximately 15 days. The data were transferred to the supercomputer at Argonne Leadership Computing Facility using the 100 Gbps speed of the ESnet network – in only two days. The ESnet facilities and staff had drawn data from far-flung locations, amassing data so that the weather researcher obtained a valuable dataset, organized and ready for analysis in a fraction of the time it would have otherwise taken. http://www.es.net/news-and-publications/esnet-news/2014/weathering-the-flood-of-big-data-in-climate-research/
RSS