Newswise — For utilities to efficiently integrate solar energy into the electric power grid, they need to be able to anticipate the amount of energy that will be generated on any given day in a particular area. This quantity depends on uncontrollable and highly variable environmental factors, such as clouds and atmospheric particles, which can reduce the amount of sunlight that reaches solar panels.

Capabilities for accurate solar energy forecasting are needed to reliably balance electricity supply and demand. Because of the current lack of cost-effective energy storage methods, electricity generated by solar and other renewable resources such as wind must be consumed as it is generated. By knowing when to expect drops and surges in these renewable resources, utilities can efficiently allocate them.

For example, if given sufficient notice (e.g., 24–48 hours) that the amount of available solar irradiance will be reduced due to overcast skies, utilities can negotiate the purchase of cost-effective alternative sources of electricity (hydro, coal, oil, and nuclear power) to keep up with demand. For short-term shortages caused by variability in cloud cover, they could power up natural gas peaking power plants (i.e., plants that only operate when high, or “peak,” demand for electricity exists) to maintain equilibrium power on the grid. Conversely, they could power down these facilities when clear, cloud-free conditions are present and solar production is optimal.

Without accurate forecasting capabilities, utilities cannot maximize the purchase of economic alternative power, maintain a stable grid, or prevent excess generated power from going to waste. 

“Predicting when and where clouds will appear, how long they will persist, and what impact they may have on solar energy production is no easy task and is the subject of several solar forecasting development efforts at Brookhaven,” said Allison McComiskey, chair of the Environmental and Climate Sciences Department (ECSD) at the U.S. Department of Energy’s (DOE) Brookhaven National Laboratory.

Clouds are impacted by several environmental factors, including winds, humidity, air pressure, and temperature. Cloud cover predictions require a combination of meteorological measurements from satellites and ground-based field sensors, and calculations by numerical weather prediction models.

A site for solar research

In 2011, the Long Island Solar Farm (LISF)—a 32-megawatt solar photovoltaic power plant, the largest such plant in the Eastern United States at that time—was installed at Brookhaven Lab through a collaboration between DOE, BP Solar, and the Long Island Power Authority (LIPA). In addition to providing power for about 4,500 homes on Long Island, New York, LISF has been helping scientists study how weather impacts the amount of power being generated by the solar array.

“Access to the facility for research purposes was part of the agreement,” said Paul Kalb, deputy chair of ECSD and principal investigator for nowcasting projects. “Staff in our department have a lot of experience with meteorological instrumentation in support of climate research. Working closely with colleagues in the Sustainable Energy Technologies Department, we have installed a broad array of instruments for solar research and development.”

To support this research and development effort, Brookhaven Lab established a solar base station with instruments for precisely measuring solar irradiance, or the amount of energy incoming from the sun onto the earth’s surface at a particular location and time.

“We placed an array of solar irradiance sensors throughout the nearly 200-acre LISF, but these field-based instruments—called pyranometers—do not provide the most accurate data for research purposes,” explained John Heiser, group leader of Brookhaven Lab’s Meteorological (Met) Services, which is responsible for maintaining, calibrating, and collecting and archiving data (e.g., rainfall, temperature, wind speed) from a network of meteorological sensors for environmental protection and solar energy research. “Another more expensive grade of instruments is available for research, and we purchased a set of those for the base station in 2012.”

The solar base station includes a solar tracker, which follows the sun as it moves across the sky throughout the day. A Global Positioning System (GPS) automatically configures the location and time. Mounted on top of the tracker are four research-grade radiometers, each of which measures the amount of radiation reaching the earth’s surface in a different way. The measurements are continuously recorded every second, transmitted to a computer database, and displayed on the Met Services website.   

“These are the standard instruments and measurements expected by the solar research community,” said Heiser.

A new and improved home base

Though the solar base station has been providing high-quality data for the past six years, its infrastructure—a wooden platform on the roof of one of Brookhaven’s buildings—was outdated and not designed to support multiple research projects at once. This past summer, Brookhaven Lab relocated the station to the roof of another building on site. The relocation was accompanied by several improvements.

“The original platform was about 25 years old,” said Heiser, who oversaw the new platform’s design and construction and is its primary user. “It had very rudimentary shelving that was built out over time, and shelf space to mount the sensors was limited. Network cables to transmit the recorded sensor data had to be run outside the building, and we only had one electric circuit to power everything. In addition, the location itself was problematic. A smoke stack for an on-site medical facility shadowed the pyranometers in the spring and fall, resulting in missing data for a 15-minute window each day in those seasons.”

The new platform—three times the size of the previous one—is made out of aluminum and has weatherproof data logger boxes and internet connections and multiple electric outlets. It was designed and built at a height off the building’s roof that is above the tree line to provide an unobstructed 360-degree view of the sky for solar research and emergency protection.

In addition to these improvements, several new features were added, including a calibration plate that can calibrate up to 25 pyranometers at a time. In the past, Met Services had to annually send the pyranometers to DOE’s National Renewable Energy Laboratory for calibration.

Another new feature is a pad with the appropriate electrical requirements for lidars, which are devices that measure the distance to objects by sensing reflected light pulses. Among other applications, lidar measurements are used to study wind flow in onshore and offshore environments. 

Scientists in ECSD who are conducting research through DOE’s Atmospheric Radiation Measurement (ARM) User Facility have already installed instruments on the base station, and there is plenty of space to accommodate additional researchers. 

“Through these improvements, the platform can now accommodate multiple research projects simultaneously,” said Heiser. “Our hope is that researchers from other organizations will make use of the solar base station, which, as far as I am aware, is the only complete station of its kind within our region.”

Ongoing solar energy research

The solar base station is part of a larger ongoing solar power forecasting effort at Brookhaven Lab.

“Brookhaven Lab has a significant commitment to solar energy research,” said Kalb. “Early on, it provided program development support to invest in the sensors for LISF and the solar base station and built the Northeast Solar Energy Research Center (NSERC) with contributions from BP Solar, the developer of LISF.”

Equipped with solar panels, pyranometers, temperature sensors, and other instruments, NSERC supplies electricity to the Brookhaven campus and serves as a DOE research and testing facility for the solar community to explore different technologies.

“The solar base station goes hand in hand with NSERC—anyone doing solar research needs to know precisely what the solar resources were at that exact time,” said Kalb.

At Brookhaven, data from NSERC and the solar base station are being used to support grid modernization research being carried out by the Sustainable Energy Technologies Department.

“There is an ongoing national effort to modernize the electric grid, and the increased use of renewable energy will be one of the key elements for the modern grid,” explained Robert Lofaro, leader of the department’s Renewable Energy Group. “Brookhaven is performing research on how to mitigate the impacts of the variability of renewable resources using advanced sensors. Data from NSERC and the solar base station can be used to characterize solar energy variability, which will be an important input for this research.” 

Outside Brookhaven, the data are relevant to industry, academia, and government.

“I receive quite a few requests for data from the solar base station,” said Heiser. “Companies that are building solar farms for utilities need to properly size and angle solar arrays, and scientists around the country want high-quality solar energy data for their research.”

According to Heiser, the station generates about a terabyte (1000 gigabytes) of data per year, at 30-second resolution. The collected data must be analyzed to identify any clouds present in the sky, track their movement over time, and assess their impact on the solar power output. This data can be used as one of the inputs to train computational models via machine learning.

Shortly after Brookhaven invested in the solar research equipment, ECSD and Brookhaven’s Computational Science Initiative (CSI)—in collaboration with the National Center for Atmospheric Research (NCAR)—began a three-year project funded by the U.S. Department of Energy Solar Energy Technologies Office under the “Improving the Accuracy of Solar Forecasting” funding program.

“The question is, how do you accurately predict the variability in solar energy generation over minutes and days?” said Kalb. “Our project addressed this issue, with Brookhaven developing nowcasting algorithms for near-term forecasts (within 30 minutes) and NCAR developing and running numerical weather models to forecast up to 48 hours.”

Brookhaven’s approach is to capture images of the sky using ground-based imagers, and then apply software to identify and track the movement of clouds and estimate their impact on available solar energy. These prediction capabilities were successfully tested at LISF during various seasons. Initially, Brookhaven used Total Sky Imagers (TSIs), similar to those in use by the ARM User Facility to image the sky. These instruments have cameras with ultrawide-angle (fisheye) lenses, which look down at a hemispherical mirror that reflects a picture of the sky above. Three TSIs took 360-degree snapshots of the sky every 30 seconds, and using the geometry and distance between the imagers, scientists calculated the base height and thickness of clouds via triangulation. They also determined the direction and speed of the clouds through image processing techniques.  

However, TSIs are expensive and difficult to maintain. The mirror needs to be cleaned regularly, and the rotating parts used to block the direct image of the sun are prone to mechanical failures.

“John Heiser came up with the idea to use inexpensive, smaller high-definition (HD) digital cameras instead,” said Kalb. “These cameras are not only ten times cheaper than the TSIs but also ten times the resolution, with a field of view almost 40 degrees more. Rain is enough to keep them clean.”

Following the successful completion of this project in 2016, the team was approached by the Electrical Power Research Institute and the New York Power Authority (NYPA) to extend the previously developed solar forecasting technologies to a regional scale.

The ongoing project is currently in its second phase, which is co-funded by the DOE Solar Energy Technologies Office and NYPA. The team recently placed 12 HD sky imagers on site and off site across Long Island to expand the near-term nowcasting range from approximately one square mile to 20 square miles.

“The imager sites include a fire department, self-storage facility, retirement community, and the Tesla Science Center,” said Kalb. “There has been great interest and cooperation among outside organizations who are very willing to grant us access to their facilities for our research. It is nice to see such community involvement.”

The team also modified the nowcasting model to develop a working prototype. At any given time, wide variability exists among the available solar radiation across the forecast region, and large changes can occur within minutes.

Next, they will deploy and test the updated regional nowcasting model, making improvements as necessary. Preliminary results indicate that improvements in mean absolute error over the currently used “persistence” model of 20 to 30 percent for five-minute forecasts and more than 50 percent for 30-minute forecasts can be achieved. Eventually, the capabilities will be mature enough to be extended to additional regions in New York State, covering a larger and more diverse geographical area.

Through a separate research award from the Solar Energy Technologies Office, a team led by Yangang Liu, a senior scientist and leader of the Climate and Process Modeling Group at Brookhaven, will further improve forecasting of solar irradiance in cloudy conditions. The team will build upon the NCAR-developed Weather Research Forecasting (WRF)-Solar, one of the premier numerical weather prediction models for forecasting solar energy resources. In particular, they will improve the representation of cloud physics and the way that solar radiation interacts with clouds. Researchers will use data from measurement systems in other parts of the country that are similar to those used at the solar base station to evaluate the performance of the model forecasts.

“These research projects and the expanded measurement capabilities provided by the new base station platform will allow us to continue connecting laboratory research with industry and the public to provide the best technologies of the future to serve society and the environment,” said McComiskey.

Brookhaven National Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit

Follow @BrookhavenLab on Twitter or find us on Facebook.