{"id":300281,"name":"AtmoRep","description":"A stochastic model of atmosphere dynamics using large scale representation learning.","url":"https://github.com/clessig/atmorep","last_synced_at":"2026-06-14T06:30:21.654Z","repository":{"id":199344764,"uuid":"702682964","full_name":"clessig/atmorep","owner":"clessig","description":"AtmoRep model code","archived":false,"fork":false,"pushed_at":"2025-03-10T13:43:08.000Z","size":1441,"stargazers_count":45,"open_issues_count":46,"forks_count":15,"subscribers_count":8,"default_branch":"main","last_synced_at":"2026-06-01T00:04:51.178Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clessig.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-09T19:42:46.000Z","updated_at":"2026-02-24T21:31:22.000Z","dependencies_parsed_at":"2023-12-29T09:21:58.086Z","dependency_job_id":"52c66578-1c33-4eb8-baa6-92d3571878ec","html_url":"https://github.com/clessig/atmorep","commit_stats":{"total_commits":95,"total_committers":5,"mean_commits":19.0,"dds":0.6421052631578947,"last_synced_commit":"3efcb122842b7cd285cb3074f84b98ef9665a306"},"previous_names":["clessig/atmorep"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/clessig/atmorep","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clessig","download_url":"https://codeload.github.com/clessig/atmorep/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34183109,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"owner":{"login":"clessig","name":"Christian Lessig","uuid":"29652766","kind":"user","description":"","email":"","website":"http://graphics.cs.uni-magdeburg.de","location":"Potsdam","twitter":null,"company":"European Center for Medium Range Weather Forecasts","icon_url":"https://avatars.githubusercontent.com/u/29652766?u=a9fc16f4d6c9250bdd5a303bf0eac246cc8c8072\u0026v=4","repositories_count":1,"last_synced_at":"2023-10-09T20:39:46.161Z","metadata":{"has_sponsors_listing":false},"html_url":"https://github.com/clessig","funding_links":[],"total_stars":null,"followers":null,"following":null,"created_at":"2023-10-09T20:39:46.252Z","updated_at":"2023-10-09T20:39:46.252Z","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clessig","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clessig/repositories"},"packages":[],"commits":{"id":1749190,"full_name":"clessig/atmorep","default_branch":"main","total_commits":96,"total_committers":4,"total_bot_commits":0,"total_bot_committers":0,"mean_commits":24.0,"dds":0.6458333333333333,"past_year_total_commits":0,"past_year_total_committers":0,"past_year_total_bot_commits":0,"past_year_total_bot_committers":0,"past_year_mean_commits":0.0,"past_year_dds":0.0,"last_synced_at":"2026-06-11T05:26:31.609Z","last_synced_commit":"055f858e68f5e0151eb6fece9f6b574d3da4af8d","created_at":"2024-09-17T00:12:52.810Z","updated_at":"2026-06-11T05:26:04.819Z","committers":[{"name":"Ilaria Luise","email":"ilaria.luise@cern.ch","login":null,"count":34},{"name":"Christian Lessig","email":"christian.lessig@ovgu.de","login":null,"count":23},{"name":"Christian Lessig","email":"christian.lessig@gmail.com","login":"clessig","count":20},{"name":"iluise","email":"luise.ilaria@gmail.com","login":"iluise","count":19}],"past_year_committers":[],"commits_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep/commits","host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-06-13T00:00:12.063Z","repositories_count":6258055,"commits_count":882341812,"contributors_count":35076159,"owners_count":1165875,"icon_url":"https://github.com/github.png","host_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories"}},"issues_stats":{"full_name":"clessig/atmorep","html_url":"https://github.com/clessig/atmorep","last_synced_at":"2026-05-23T20:01:23.360Z","status":"error","issues_count":60,"pull_requests_count":37,"avg_time_to_close_issue":5867941.9130434785,"avg_time_to_close_pull_request":1278897.423076923,"issues_closed_count":23,"pull_requests_closed_count":26,"pull_request_authors_count":8,"issue_authors_count":14,"avg_comments_per_issue":3.25,"avg_comments_per_pull_request":0.7567567567567568,"merged_pull_requests_count":23,"bot_issues_count":0,"bot_pull_requests_count":0,"past_year_issues_count":38,"past_year_pull_requests_count":26,"past_year_avg_time_to_close_issue":1625753.1666666667,"past_year_avg_time_to_close_pull_request":1009500.125,"past_year_issues_closed_count":12,"past_year_pull_requests_closed_count":16,"past_year_pull_request_authors_count":8,"past_year_issue_authors_count":12,"past_year_avg_comments_per_issue":3.1052631578947367,"past_year_avg_comments_per_pull_request":0.9230769230769231,"past_year_bot_issues_count":0,"past_year_bot_pull_requests_count":0,"past_year_merged_pull_requests_count":13,"created_at":"2024-09-17T00:13:21.605Z","updated_at":"2026-05-23T20:01:23.361Z","repository_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep","issues_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/clessig%2Fatmorep/issues","issue_labels_count":{"enhancement":10,"bug":7,"core model":6,"I/O":5,"good first issue":4,"scientific":2,"help wanted":2,"triaged":1,"performance":1,"question":1,"analysis":1},"pull_request_labels_count":{"core model":6,"bug":5,"enhancement":4,"I/O":2,"dependencies":1},"issue_author_associations_count":{"COLLABORATOR":48,"NONE":7,"OWNER":5},"pull_request_author_associations_count":{"COLLABORATOR":49,"OWNER":8,"NONE":2,"CONTRIBUTOR":1},"issue_authors":{"iluise":13,"sbAsma":8,"mlangguth89":7,"kacpnowak":6,"clessig":5,"grassesi":4,"nish03":4,"ankitpatnala":4,"sascholle":3,"maruf-anu":2,"dancivitarese":1,"jpolz":1,"Sindhu-Vasireddy":1,"javak87":1},"pull_request_authors":{"iluise":26,"grassesi":8,"clessig":8,"sbAsma":6,"kacpnowak":4,"mlangguth89":3,"zalbanob":2,"jpolz":2,"dependabot[bot]":1},"host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-06-11T00:00:18.937Z","repositories_count":14790658,"issues_count":33482849,"pull_requests_count":110124176,"authors_count":11299612,"icon_url":"https://github.com/github.png","host_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories","owners_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/owners","authors_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors"},"past_year_issue_labels_count":{},"past_year_pull_request_labels_count":{},"past_year_issue_author_associations_count":{},"past_year_pull_request_author_associations_count":{},"past_year_issue_authors":{},"past_year_pull_request_authors":{},"maintainers":[{"login":"iluise","count":39,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/iluise"},{"login":"sbAsma","count":14,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/sbAsma"},{"login":"clessig","count":13,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/clessig"},{"login":"grassesi","count":12,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/grassesi"},{"login":"kacpnowak","count":10,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/kacpnowak"},{"login":"mlangguth89","count":9,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/mlangguth89"},{"login":"nish03","count":4,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/nish03"},{"login":"ankitpatnala","count":4,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/ankitpatnala"},{"login":"jpolz","count":3,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/jpolz"},{"login":"Sindhu-Vasireddy","count":1,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/Sindhu-Vasireddy"},{"login":"javak87","count":1,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/javak87"}],"active_maintainers":[]},"events":{"total":{"DeleteEvent":5,"MemberEvent":2,"PullRequestEvent":21,"ForkEvent":5,"IssuesEvent":42,"WatchEvent":8,"IssueCommentEvent":87,"PushEvent":103,"GollumEvent":10,"PullRequestReviewCommentEvent":27,"PullRequestReviewEvent":15,"CreateEvent":25,"CommitCommentEvent":2},"last_year":{}},"keywords":[],"dependencies":[{"ecosystem":"pypi","filepath":"setup.py","sha":null,"kind":"manifest","created_at":"2023-11-27T22:46:43.716Z","updated_at":"2023-11-27T22:46:43.716Z","repository_link":"https://github.com/clessig/atmorep/blob/main/setup.py","dependencies":[{"id":14764745385,"package_name":"numpy","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764745386,"package_name":"matplotlib","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764745387,"package_name":"zarr","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746192,"package_name":"pandas","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746193,"package_name":"typing_extensions","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746194,"package_name":"pathlib","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746195,"package_name":"wandb","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746196,"package_name":"cloudpickle","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746197,"package_name":"ecmwflibs","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746207,"package_name":"cfgrib","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746208,"package_name":"netcdf4","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746209,"package_name":"xarray","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746210,"package_name":"pytz","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":14764746211,"package_name":"torchinfo","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false}]}],"score":5.8971538676367405,"created_at":"2024-09-17T00:12:23.906Z","updated_at":"2026-06-14T06:30:21.675Z","avatar_url":"https://github.com/clessig.png","language":"Python","category":"Atmosphere","sub_category":"Atmospheric Composition and Dynamics","monthly_downloads":0,"total_dependent_repos":0,"total_dependent_packages":0,"readme":"\n# IMPORTANT NOTE: \n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/50518092-5386-4ef5-98f3-e2abf147556f\" /\u003e\n\u003c/p\u003e\n\nPlease note that that the folder is **not maintained anymore since March 1st**. \\\nPlease use the **WeatherGenerator** code instead: https://github.com/ecmwf/WeatherGenerator \n\n\n\n# AtmoRep\n\nThis repository contains the source code for the [AtmoRep](https://www.atmorep.org) models for large scale representation learning of atmospheric dynamics as well as links to the pre-trained models and the required model input data.\n\nThe pre-print for the work is available on ArXiv: https://arxiv.org/abs/2308.13280.\n\n```\n@misc{Lessig2023atmorep,\n  title = {AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning},\n\tauthor = {Christian Lessig and Ilaria Luise and Bing Gong and Michael Langguth and Scarlet Stadler and Martin Schultz},\n  eprint = {2308.13280},\n  primaryclass = {physics.ao-ph},\n  url = {https://arxiv.org/abs/2308.13280},\n  year = {2023},\n```\n\n# Starter README\n\n## 1. Pull code\n\n`````\n%\u003e wget git@github.com:clessig/atmorep.git\n`````\nThis creates a directory ``atmorep`` with the code that contains the source code including the python scripts for model training and evaluation.\n\nAfter following the steps described below, the final directory structure will look as follows:\n````\n└── atmorep/\n    ├── atmorep/\n    │     └── ... \n    ├── data/                         \u003c- top level data directory\n    │    ├── normalisation/           \u003c- directory for data normalisations \n    │    ├── vorticity/\n    │    │       ├── ml105/           \u003c- model levels with monthly GRIB files\n    │    │       │     ├── era5_vorticity_y2021_m03_ml137.grib   \u003c- grib data file\n    │    │       │     ├── ...\n    │    │       ├── ml114/\n    │    │       ├── ml123/\n    │    │       ├── ml137/\n    │    │       ├── ml96/\n    .    .       .\n    │    ├── temperature/\n    .    .\n    ├── models\n    │    ├── id4nvwbetz   \u003c- Directory containing model weights and config\n    │    │       ├──  model_id4nvwbetz.json     \n    │    │       └──  AtmoRep_id4nvwbetz.mod\n    │    ├── id\u003cmodel_id\u003e\n    .    .\n    └── results\n         ├── id4nvwbetz\n         ...\n````\nThe directories ``data``, ``models``, and ``results`` need to be created if they do not exist. All directories might be large and should thus be on a directory with sufficient storage space; in this case they can be soft-linked to the default ones above or they can be set in ``atmorep/config/config``. \n\n## 2. Download the data \n\n### 2.1 Download pre-trained models\n\nModels can be downloaded from: https://datapub.fz-juelich.de/atmorep/trained-models.html\n\nAn example for downloading the pre-trained models is given here, in this case for the vorticity model.\n\n`````\n% atmorep/\u003e mkdir models\n% atmorep/\u003e cd models\n% atmorep/data/\u003e wget https://datapub.fz-juelich.de/atmorep/models/model_id4nvwbetz.tar.gz\n% atmorep/data/\u003e tar xvzf model_id4nvwbetz.tar.gz\n% atmorep/data/\u003e ls id4nvwbetz\nAtmoRep_id4nvwbetz.mod  model_id4nvwbetz.json\n`````\n\n\n### 2.2 Download model input data (ERA5)\n\nThe input data in the required structure can be downloaded from the [Jülich datapub](https://datapub.fz-juelich.de/atmorep/era5-data.html) server. Direct link to WebDAV [https://datapub.fz-juelich.de/atmorep/data/](https://datapub.fz-juelich.de/atmorep/data/). Alternatively, it can be directly downloaded from MARS using the following [script](https://www.atmorep.org/code/mars_era5_download.py).\n\n#### Download a subset of files\n\nAll data files (fields and normalizations) should be downloaded into the ``data`` directory. Un-taring the files will generate the correct folder structure. For example (we will use the vorticity example also below to run the first model so it is recommended to download it as a first step):\n`````\n% atmorep/\u003e mkdir data\n% atmorep/\u003e cd data\n% atmorep/data/\u003e wget https://datapub.fz-juelich.de/atmorep/data/vorticity/ml137/era5_vorticity_y2021_ml137.tar\n% atmorep/data/\u003e tar xvf era5_vorticity_y2021_ml137.tar\n% atmorep/data/\u003e ls -lah vorticity/ml137/\ntotal 18G\nera5_vorticity_y2021_m01_ml137.grib\nera5_vorticity_y2021_m02_ml137.grib\n...\nera5_vorticity_y2021_m12_ml137.grib\n`````\nFor efficiency reasons, AtmoRep takes monthly ERA5 data as input. Therefore, each tar file contains 12 GRIB files of about 1.5 GBytes each.\n\nCoefficients for data normalization per field and level can be downloaded here: https://datapub.fz-juelich.de/atmorep/data/normalization/. They should also be located in the ```data``` directory:\n`````\n% atmorep/data/\u003e wget https://datapub.fz-juelich.de/atmorep/data/normalization/normalization_vorticity_ml137.tar.gz\n% atmorep/data/\u003e tar xvzf normalization_vorticity_ml137.tar.gz\n`````\n\n## 3. Install python packages\n\nCreate a python environment, e.g.\n\n`````\n% atmorep/\u003e python3 -m venv pyenv\n`````\n\nand activate the environment:\n\n`````\n% atmorep/\u003e source pyenv/bin/activate\n`````\nconda is also possible, no environment is strictly required although we would recommend it. Please make sure to use a recent python version (we tested with python3.10).\nThen install the AtmoRep package:\n`````\n% atmorep/\u003e\n% atmorep/\u003e pip install -e .\n`````\n\ntorch is currently not included (since it is often available or has particular dependencies, e.g. a specific Cuda version). In the simplest case, it can just be installed by:\n\n`````\n% atmorep/\u003e pip install torch\n`````\nWe require torch 2.x. (A container solution allows to run even on systems where torch 2.x is not available.)\n\n## 4. Run model:\nPre-trained models can normally be run by:\n`````\n% atmorep/\u003e python atmorep/core/evaluate.py\n`````\nYou can easily adapt the configuration by selecting the corresponding _model_id_ in ``evaluate.py`` (see below). It defaults to the single-field configuration of vorticity, of which we have downloaded the data above.\n\nDepending on your compute hardware, you might also have to run the computations by submitting the job using a batch system or allocate a compute node in interactive mode (if an interactive seesion is possible, then this is recommended). If you run an interactive session you will likely need to use the following:\n`````\n%  atmorep/\u003e export CUDA_VISIBLE_DEVICES=0,1,2,3\n%  atmorep/\u003e MASTER_ADDR=\"$(scontrol show hostnames \"$SLURM_JOB_NODELIST\" | head -n 1)\"\n`````\n\nThe default evaluation mode is currently global forecast. The output will be (similar to) this:\n````\ndevices : ['cuda:0', 'cuda:1', 'cuda:2', 'cuda:3']\nWandb run: atmorep-ztvyw7k6-8932958\nRunning Evaluate.evaluate with mode = global_forecast\nLoaded AtmoRep id=4nvwbetz, ignoring/missing 2 elements.\nLoaded model id = 4nvwbetz at epoch = -2.\nNumber of batches per global forecast: 14\nINFO:: data stats vorticity : 5.374998363549821e-05 / 0.9978392720222473\nnum_accs_per_task : 1\nwith_hvd : True\nhvd_rank : 0\n\n...\n\nwandb_id : ztvyw7k6\ndates : [[2021, 2, 10, 12]]\ntoken_overlap : [0, 0]\nforecast_num_tokens : 1\nvalidation loss for strategy=forecast at epoch 0 : 0.12402566522359848\nvalidation loss for vorticity : 0.12402566522359848\nwandb: Waiting for W\u0026B process to finish... (success).\nwandb: \nwandb: Run history:\nwandb:        val. loss forecast ▁\nwandb: val., forecast, vorticity ▁\nwandb: \nwandb: Run summary:\nwandb:        val. loss forecast 0.12403\nwandb: val., forecast, vorticity 0.12403\nwandb: \nwandb: You can sync this run to the cloud by running:\nwandb: wandb sync /p/project/atmo-rep/lessig/atmorep/atmorep/lessig-cleanup/atmorep/wandb/offline-run-20231124_095428-ztvyw7k6\n````\nFor the vorticity example above, we evaluate with ``global_forecast`` for a specific date and using only a single model level:\n````\nmode, options = 'global_forecast', { 'fields[0][2]' : [137],\n                                     'dates' : [ [2021, 2, 10, 12] ],\n                                     'token_overlap' : [0, 0],\n                                     'forecast_num_tokens' : 1, \n                                     'attention' : False}\n````\nWe perform a 3 hour forecast, since 1 token is 3 hours wide. Another mode is the BERT masked token model mode used for pre-training:\n`````\nmode, options = 'BERT', {'years_test' : [2021], 'fields[0][2]' : [123, 137]}\n`````\nAgain, we chose some custom options by using two levels instead of the five ones that are default and were used during pre-training and by using 2021 as the test year (since we downloaded the data).\n\nThe generated model output (stored in ``./results/id{wandbid}``) for the ```global_forecast``` example can be post-processed into a spatial map with the [following code](https://www.atmorep.org/code/plot_forecast.py). The run_id at the top needs to be replaced by the wandb_id of your run, it can be read off from the console output. Results will be stored as ``example_0000{0,1,2}.png``. The code is also an as-simple-as-possible example with many parameters hard-coded, see our analysis code for a proper handling. \n","funding_links":[],"readme_doi_urls":[],"works":{},"citation_counts":{},"total_citations":0,"keywords_from_contributors":[],"project_url":"https://ost.ecosyste.ms/api/v1/projects/300281","html_url":"https://ost.ecosyste.ms/projects/300281"}