brightsky
A JSON API for Germany's meteorological service as part of their Open Data program.
https://github.com/jdemaeyer/brightsky
Category: Atmosphere
Sub Category: Meteorological Observation and Forecast
Keywords
api dwd open-data weather
Keywords from Contributors
deutscherwetterdienst wetter
Last synced: about 7 hours ago
JSON representation
Repository metadata
JSON API for DWD's open weather data.
- Host: GitHub
- URL: https://github.com/jdemaeyer/brightsky
- Owner: jdemaeyer
- License: mit
- Created: 2020-03-13T08:34:19.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2025-03-25T08:43:25.000Z (about 1 month ago)
- Last Synced: 2025-04-25T13:43:24.213Z (5 days ago)
- Topics: api, dwd, open-data, weather
- Language: Python
- Homepage: https://brightsky.dev/
- Size: 2.77 MB
- Stars: 325
- Watchers: 10
- Forks: 17
- Open Issues: 23
- Releases: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
README.md
Bright Sky
JSON API for DWD's open weather data.
The DWD (Deutscher Wetterdienst), as Germany's
meteorological service, publishes a myriad of meteorological observations and
calculations as part of their Open Data
program.
Bright Sky is an open-source project aiming to
make some of the more popular data — in particular weather observations from
the DWD station network and weather forecasts from the MOSMIX model — available
in a free, simple JSON API.
Looking for something specific?
I just want to retrieve some weather data
You can use the free public Bright Sky instance!
I want to run my own instance of Bright Sky
Check out the infrastructure
repo!
I want to parse DWD weather files from the command line or in Python
The parsing core for Bright Sky is maintained in a separate package named
dwdparse
, which has no dependencies
outside the standard library. If you find that's not quite serving your needs,
check out wetterdienst
.
I want to contribute to Bright Sky's source code
Read on. :)
On Bright Sky's versioning
Starting from version 2.0, where we extracted the parsing core into a separate
package, Bright Sky is no longer
intended to be used as a Python library, but only as the service available at
brightsky.dev
.
Consequentially, we adjust our version numbers from the perspective of that
service and its users – i.e., we will increase the major version number only
when we introduce backwards-incompatible (or otherwise very major) changes to
the actual JSON API interface, e.g. by changing URLs or parameters. This means
that increases of the minor version number may introduce
backwards-incompatible changes to the internals of the brightsky
package,
including the database structure. If you use brightsky
as a Python library,
please version-pin to a minor version, e.g. by putting brightsky==2.0.*
in
your requirements.txt
.
Quickstart
Running a full-fledged API instance
Note: These instructions are aimed at running a Bright Sky instance for
development and testing. Check out our infrastructure
repository if you want
to set up a production-level API instance.
Just run docker-compose up
and you should be good to go. This will set up a
PostgreSQL database (with persistent storage in .data
), run a Redis server,
and start the Bright Sky worker and webserver. The worker periodically polls
the DWD Open Data Server for updates, parses them, and stores them in the
database. The webserver will be listening to API requests on port 5000.
Architecture
Bright Sky is a rather simple project consisting of four components:
-
The
brightsky
worker, which leverages the logic contained in the
brightsky
Python package to retrieve weather records from the DWD server,
parse them, and store them in a database. It will periodically poll the DWD
servers for new data. -
The
brightsky
webserver (API), which serves as gate to our database and
processes all queries for weather records coming from the outside world. -
A PostgreSQL database consisting of two relevant tables:
sources
contains information on the locations for which we hold weather
records, andweather
contains the history of actual meteorological measurements (or
forecasts) for these locations.
The database structure can be set up by running the
migrate
command, which
will simply apply all.sql
files found in themigrations
folder. -
A Redis server, which is used as the backend of the worker's task queue.
Most of the tasks performed by the worker and webserver can also be performed
independently. Run docker-compose run --rm brightsky
to get a list of
available commands.
Hacking
Constantly rebuilding the brightsky
container while working on the code can
become cumbersome, and the default setting of parsing records dating all the
way back to 2010 will make your development database unnecessarily large. You
can set up a more lightweight development environment as follows:
-
Create a virtual environment and install our dependencies:
python -m virtualenv .venv && source .venv/bin/activate && pip install -r requirements.txt && pip install -e .
-
Start a PostgreSQL container:
docker-compose run --rm -p 5432:5432 postgres
-
Start a Redis container:
docker-compose run --rm -p 6379:6379 redis
-
Point
brightsky
to your containers, and configure a tighter date
threshold for parsing DWD data, by adding the following.env
file:BRIGHTSKY_DATABASE_URL=postgres://postgres:pgpass@localhost BRIGHTSKY_BENCHMARK_DATABASE_URL=postgres://postgres:pgpass@localhost/benchmark BRIGHTSKY_REDIS_URL=redis://localhost BRIGHTSKY_MIN_DATE=2020-01-01
You should now be able to directly run brightsky
commands via python -m brightsky
, and changes to the source code should be effective immediately.
Tests
Large parts of our test suite run against a real Postgres database. By default,
these tests will be skipped. To enable them, make sure the
BRIGHTSKY_TEST_DATABASE_URL
environment variable is set when calling tox
,
e.g. via:
BRIGHTSKY_TEST_DATABASE_URL=postgres://postgres:pgpass@localhost/brightsky_test tox
Beware that adding this environment variable to your .env
file will not work
as that file is not read by tox
. The database will be dropped and
recreated on every test run, so don't use your normal Bright Sky database. ;)
Acknowledgements
Bright Sky's development is boosted by the priceless guidance and support of
the Open Knowledge Foundation's Prototype
Fund program, and is generously funded by Germany's
Federal Ministry of Education and Research. Obvious as
it may be, it should be mentioned that none of this would be possible without
the painstaking, never-ending effort of the Deutscher
Wetterdienst.
Owner metadata
- Name: Jakob de Maeyer
- Login: jdemaeyer
- Email:
- Kind: user
- Description: Freelance Data and Product Engineer
- Website: https://naboa.de
- Location: Münster, Germany
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/10531844?u=70329c64212c69738144799f9892b6842bb5877e&v=4
- Repositories: 18
- Last ynced at: 2025-04-20T12:48:20.401Z
- Profile URL: https://github.com/jdemaeyer
GitHub Events
Total
- Issues event: 8
- Watch event: 27
- Delete event: 1
- Issue comment event: 19
- Push event: 13
- Pull request event: 2
- Create event: 7
Last Year
- Issues event: 8
- Watch event: 27
- Delete event: 1
- Issue comment event: 19
- Push event: 13
- Pull request event: 2
- Create event: 7
Committers metadata
Last synced: 1 day ago
Total Commits: 370
Total Committers: 3
Avg Commits per committer: 123.333
Development Distribution Score (DDS): 0.008
Commits in past year: 33
Committers in past year: 1
Avg Commits per committer in past year: 33.0
Development Distribution Score (DDS) in past year: 0.0
Name | Commits | |
---|---|---|
Jakob de Maeyer | j****1@g****m | 367 |
Gerald Pape | u****t | 2 |
Tobias | t****h@p****e | 1 |
Committer domains:
- posteo.de: 1
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 114
Total pull requests: 13
Average time to close issues: 3 months
Average time to close pull requests: 10 days
Total issue authors: 40
Total pull request authors: 4
Average comments per issue: 2.74
Average comments per pull request: 0.85
Merged pull request: 5
Bot issues: 0
Bot pull requests: 7
Past year issues: 7
Past year pull requests: 1
Past year average time to close issues: about 19 hours
Past year average time to close pull requests: 34 minutes
Past year issue authors: 6
Past year pull request authors: 1
Past year average comments per issue: 2.71
Past year average comments per pull request: 0.0
Past year merged pull request: 1
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- jdemaeyer (58)
- poetaster (4)
- tawissus (3)
- smilingchemist (3)
- ubergesundheit (3)
- MHz000 (3)
- mweinelt (3)
- BlackScreen (3)
- martinfricke (2)
- ptoews (2)
- jwillmer (1)
- MatthiasDod (1)
- rfpaiva (1)
- papjul (1)
- bjoernh (1)
Top Pull Request Authors
- dependabot[bot] (7)
- jdemaeyer (4)
- tobi-laa (1)
- ubergesundheit (1)
Top Issue Labels
- documentation (4)
- experimental (4)
- enhancement (3)
- bug (2)
- data quality (2)
- housekeeping (1)
Top Pull Request Labels
- dependencies (7)
Package metadata
- Total packages: 1
-
Total downloads:
- pypi: 1,995 last-month
- Total dependent packages: 0
- Total dependent repositories: 1
- Total versions: 58
- Total maintainers: 1
pypi.org: brightsky
JSON API for DWD's open weather data.
- Homepage: https://brightsky.dev/
- Documentation: https://brightsky.dev/docs/
- Licenses: MIT
- Latest release: 2.2.6 (published about 12 hours ago)
- Last Synced: 2025-04-30T11:00:58.585Z (about 12 hours ago)
- Versions: 58
- Dependent Packages: 0
- Dependent Repositories: 1
- Downloads: 1,995 Last month
-
Rankings:
- Stargazers count: 4.211%
- Forks count: 8.893%
- Dependent packages count: 10.119%
- Average: 12.26%
- Downloads: 16.53%
- Dependent repos count: 21.545%
- Maintainers (1)
Dependencies
- astral ==2.2
- certifi ==2022.6.15
- charset-normalizer ==2.1.0
- click ==8.1.3
- coloredlogs ==15.0.1
- cssselect ==1.1.0
- falcon ==2.0.0
- falcon-cors ==1.1.7
- gunicorn ==20.1.0
- huey ==2.4.3
- humanfriendly ==10.0
- idna ==3.3
- lxml ==4.9.1
- parsel ==1.6.0
- psycopg2-binary ==2.9.3
- python-dateutil ==2.8.2
- pytz ==2022.1
- redis ==3.5.3
- requests ==2.28.1
- sentry-sdk ==1.7.2
- six ==1.16.0
- urllib3 ==1.26.10
- w3lib ==1.22.0
- astral *
- click *
- coloredlogs *
- falcon ==2.
- falcon-cors *
- gunicorn *
- huey *
- parsel *
- psycopg2-binary *
- python-dateutil *
- redis <4
- requests *
- sentry-sdk *
- actions/checkout v2 composite
- actions/setup-python v1 composite
- docker/build-push-action v2 composite
- docker/login-action v2 composite
- pypa/gh-action-pypi-publish release/v1 composite
- postgres 12 docker
- python 3.10-slim build
Score: 14.5502161018584