Open Sustainable Technology

A curated list of open technology projects to sustain a stable climate, energy supply, biodiversity and natural resources.

Browse accepted projects | Review proposed projects | Propose new project | Open Issues

Quartz Solar Forecast

The aim of the project is to build an open source PV forecast that is free and easy to use.
https://github.com/openclimatefix/open-source-quartz-solar-forecast

Last synced: about 11 hours ago
JSON representation

Repository metadata

Open Source Solar Site Level Forecast

README

        

# Quartz Solar Forecast

[![All Contributors](https://img.shields.io/badge/all_contributors-9-orange.svg?style=flat-square)](#contributors-)

The aim of the project is to build an open source PV forecast that is free and easy to use.
The forecast provides the expected generation in `kw` for 0 to 48 hours for a single PV site.

Open Climate Fix also provides a commercial PV forecast, please get in touch at [email protected]

The current model uses GFS or ICON NWPs to predict the solar generation at a site

```python
from quartz_solar_forecast.forecast import run_forecast
from quartz_solar_forecast.pydantic_models import PVSite

# make a pv site object
site = PVSite(latitude=51.75, longitude=-1.25, capacity_kwp=1.25)

# run model, uses ICON NWP data by default
predictions_df = run_forecast(site=site, ts='2023-11-01')
```

Which gives the following prediction

![https://github.com/openclimatefix/Open-Source-Quartz-Solar-Forecast/blob/main/predictions.png?raw=true](https://github.com/openclimatefix/Open-Source-Quartz-Solar-Forecast/blob/main/predictions.png?raw=true)

## Installation

The source code is currently hosted on GitHub at: https://github.com/openclimatefix/Open-Source-Quartz-Solar-Forecast

Binary installers for the latest released version are available at the Python Package Index (PyPI)
```bash
pip install quartz-solar-forecast
```
You might need to install the following packages first
```bash
conda install -c conda-forge pyresample
````
This can solve the [bug: ___kmpc_for_static_fini](https://github.com/openclimatefix/Open-Source-Quartz-Solar-Forecast/issues/32).

## Model

The model is a gradient boosted tree model and uses 9 NWP variables.
It is trained on 25,000 PV sites with over 5 years of PV history, which is available [here](https://huggingface.co/datasets/openclimatefix/uk_pv).
The training of this model is handled in [pv-site-prediction](https://github.com/openclimatefix/pv-site-prediction)
TODO - we need to benchmark this forecast.

The 9 NWP variables, from Open-Meteo documentation, are mentioned above with their appropariate units.

1. **Visibility (km)**, or vis: Distance at which objects can be clearly seen. Can affect the amount of sunlight reaching solar panels.
2. **Wind Speed at 10 meters (km/h)**, or si10 : Wind speed measured at a height of 10 meters above ground level. Important for understanding weather conditions and potential impacts on solar panels.
3. **Temperature at 2 meters (°C)**, or t : Air temperature measure at 2 meters above the ground. Can affect the efficiency of PV systems.
4. **Precipiration (mm)**, or prate : Precipitation (rain, snow, sleet, etc.). Helps to predict cloud cover and potentiel reductions in solar irradiance.
5. **Shortwave Radiation (W/m²)**, or dswrf: Solar radiation in the shortwave spectrum reaching the Earth's surface. Measure of the potential solar energy available for PV systems.
6. **Direct Radiation (W/m²)** or dlwrf: Longwave (infrared) radiation emitted by the Earth back into the atmosphere. **confirm it is correct**
7. **Cloud Cover low (%)**, or lcc: Percentage of the sky covered by clouds at low altitudes. Impacts the amount of solar radiation reachign the ground, and similarly the PV system.
8. **Cloud Cover mid (%)**, or mcc : Percentage of the sky covered by clouds at mid altitudes.
9. **Cloud Cover high (%)**, or lcc : Percentage of the sky covered by clouds at high altitude

We also use the following features
- poa_global: The plane of array irradiance, which is the amount of solar radiation that strikes a solar panel.
- poa_global_now_is_zero: A boolean variable that is true if the poa_global is zero at the current time. This is used to help the model learn that the PV generation is zero at night.
- capacity (kw): The capacity of the PV system in kw.
- The model also has a feature to check if these variables are NaNs or not.

The model also uses the following variables, which are currently all set to nan
- recent_power: The mean power over the last 30 minutes
- h_mean: The mean of the recent pv data over the last 7 days
- h_median: The median of the recent pv data over the last 7 days
- h_max: The max of the recent pv data over the last 7 days

## Known restrictions

- The model is trained on [UK MetOffice](https://www.metoffice.gov.uk/services/data/met-office-weather-datahub) NWPs, but when running inference we use [GFS](https://www.ncei.noaa.gov/products/weather-climate-models/global-forecast) data from [Open-meteo](https://open-meteo.com/). The differences between GFS and UK MetOffice could led to some odd behaviours.
- It looks like the GFS data on Open-Meteo is only available for free for the last 3 months.

## Evaluation

To evaluate the model we use the [UK PV](https://huggingface.co/datasets/openclimatefix/uk_pv) dataset and the [ICON NWP](https://huggingface.co/datasets/openclimatefix/dwd-icon-eu) dataset.
All the data is publicly available and the evaluation script can be run with the following command

```bash
python scripts/run_evaluation.py
```

The test dataset we used is defined in `quartz_solar_forecast/dataset/testset.csv`.
This contains 50 PV sites, which 50 unique timestamps. The data is from 2021.

The results of the evaluation are as follows
The MAE is 0.1906 kw across all horizons.

| Horizons | MAE [kw] | MAE [%] |
|----------|---------------| ------- |
| 0 | 0.202 +- 0.03 | 6.2 |
| 1 | 0.211 +- 0.03 | 6.4 |
| 2 | 0.216 +- 0.03 | 6.5 |
| 3 - 4 | 0.211 +- 0.02 |6.3 |
| 5 - 8 | 0.191 +- 0.01 | 6 |
| 9 - 16 | 0.161 +- 0.01 | 5 |
| 17 - 24 | 0.173 +- 0.01 | 5.3 |
| 24 - 48 | 0.201 +- 0.01 | 6.1 |

If we exclude nighttime, then the average MAE [%] from 0 to 36 forecast hours is 13.0%.

Notes:
- The MAE in % is the MAE divided by the capacity of the PV site. We acknowledge there are a number of different ways to do this.
- It is slightly surprising that the 0-hour forecast horizon and the 24-48 hour horizon have a similar MAE.
This may be because the model is trained expecting live PV data, but currently in this project we provide no live PV data.

## Abbreviations

- NWP: Numerical Weather Predictions
- GFS: Global Forecast System
- PV: Photovoltaic
- MAE: Mean Absolute Error
- [ICON](https://www.dwd.de/EN/ourservices/nwp_forecast_data/nwp_forecast_data.html): ICOsahedral Nonhydrostatic
- KW: Kilowatt

## Contribution

We welcome other models.

## Contributors ✨

Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):



Peter Dudfield
Peter Dudfield

💻
Megawattz
Megawattz

🤔 📢
EdFage
EdFage

📖 💻
Chloe Pilon Vaillancourt
Chloe Pilon Vaillancourt

📖
rachel tipton
rachel tipton

📢
armenbod
armenbod

🖋 💻
Shreyas Udaya
Shreyas Udaya

📖


Aryan Bhosale
Aryan Bhosale

📖
Francesco
Francesco

💻

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!


Owner metadata


GitHub Events

Total
Last Year

Committers metadata

Last synced: 27 days ago

Total Commits: 125
Total Committers: 7
Avg Commits per committer: 17.857
Development Distribution Score (DDS): 0.44

Commits in past year: 125
Committers in past year: 7
Avg Commits per committer in past year: 17.857
Development Distribution Score (DDS) in past year: 0.44

Name Email Commits
peterdudfield p****d@h****m 70
EdFage 8****e 25
BumpVersion Action b****n@g****s 11
allcontributors[bot] 4****] 10
Megawattz 4****s 5
Chloe Pilon Vaillancourt 1****v 3
chloepilonv c****v@g****m 1

Committer domains:


Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 41
Total pull requests: 50
Average time to close issues: 20 days
Average time to close pull requests: about 21 hours
Total issue authors: 5
Total pull request authors: 12
Average comments per issue: 3.8
Average comments per pull request: 1.52
Merged pull request: 44
Bot issues: 0
Bot pull requests: 19

Past year issues: 41
Past year pull requests: 50
Past year average time to close issues: 20 days
Past year average time to close pull requests: about 21 hours
Past year issue authors: 5
Past year pull request authors: 12
Past year average comments per issue: 3.8
Past year average comments per pull request: 1.52
Past year merged pull request: 44
Past year bot issues: 0
Past year bot pull requests: 19

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/openclimatefix/open-source-quartz-solar-forecast

Top Issue Authors

  • peterdudfield (35)
  • EdFage (2)
  • zakwatts (2)
  • Anmirazik (1)
  • patrick-zippenfenig (1)

Top Pull Request Authors

  • allcontributors[bot] (19)
  • peterdudfield (7)
  • EdFage (7)
  • 0xFrama (3)
  • armenbod (2)
  • chloepilonv (2)
  • aryanbhosale (2)
  • shreyasudaya (2)
  • raiiasingh19 (2)
  • zakwatts (2)
  • bikramb98 (1)
  • roshnaeem (1)

Top Issue Labels

  • bug (6)
  • help wanted (6)
  • good first issue (5)
  • discussion (1)

Top Pull Request Labels


Package metadata

pypi.org: quartz-solar-forecast

Open Source Solar Forecasting for a Site

  • Homepage:
  • Documentation: https://quartz-solar-forecast.readthedocs.io/
  • Licenses: MIT
  • Latest release: 1.0.12 (published 1 day ago)
  • Last Synced: 2024-02-27T18:01:53.486Z (1 day ago)
  • Versions: 24
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 5,621 Last month
  • Rankings:
    • Dependent packages count: 10.131%
    • Average: 38.489%
    • Dependent repos count: 66.847%
  • Maintainers (1)

Dependencies

requirements.txt pypi
  • pandas *
  • psp *
  • pydantic *
  • xarray *
setup.py pypi
.github/workflows/pytest.yaml actions

Score: 14.472173198632417