{"id":1122,"name":"SustainBench","description":"Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning.","url":"https://github.com/sustainlab-group/sustainbench","last_synced_at":"2026-04-23T02:30:18.704Z","repository":{"id":38839317,"uuid":"400761386","full_name":"sustainlab-group/sustainbench","owner":"sustainlab-group","description":null,"archived":false,"fork":false,"pushed_at":"2022-08-30T06:01:17.000Z","size":29002,"stargazers_count":121,"open_issues_count":10,"forks_count":28,"subscribers_count":6,"default_branch":"main","last_synced_at":"2026-04-13T01:16:50.586Z","etag":null,"topics":["deep-learning","machine-learning","sdgs","sustainability"],"latest_commit_sha":null,"homepage":"https://sustainlab-group.github.io/sustainbench","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sustainlab-group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-28T10:11:50.000Z","updated_at":"2026-03-10T11:09:33.000Z","dependencies_parsed_at":"2022-07-20T01:17:46.607Z","dependency_job_id":null,"html_url":"https://github.com/sustainlab-group/sustainbench","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sustainlab-group/sustainbench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sustainlab-group","download_url":"https://codeload.github.com/sustainlab-group/sustainbench/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32117798,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-22T00:31:26.853Z","status":"online","status_checked_at":"2026-04-22T02:00:05.693Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"owner":{"login":"sustainlab-group","name":"sustainlab-group","uuid":"43426653","kind":"organization","description":null,"email":null,"website":null,"location":null,"twitter":null,"company":null,"icon_url":"https://avatars.githubusercontent.com/u/43426653?v=4","repositories_count":7,"last_synced_at":"2023-03-05T00:31:45.439Z","metadata":{"has_sponsors_listing":false},"html_url":"https://github.com/sustainlab-group","funding_links":[],"total_stars":null,"followers":null,"following":null,"created_at":"2022-11-15T03:11:39.483Z","updated_at":"2023-03-05T00:31:45.523Z","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sustainlab-group","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sustainlab-group/repositories"},"packages":[],"commits":{"id":1254622,"full_name":"sustainlab-group/sustainbench","default_branch":"main","total_commits":7,"total_committers":2,"total_bot_commits":0,"total_bot_committers":0,"mean_commits":3.5,"dds":0.1428571428571429,"past_year_total_commits":0,"past_year_total_committers":0,"past_year_total_bot_commits":0,"past_year_total_bot_committers":0,"past_year_mean_commits":0.0,"past_year_dds":0.0,"last_synced_at":"2026-04-18T00:01:23.686Z","last_synced_commit":"4b3cdb9671d697c52520a4b8bc69a621a288eb57","created_at":"2023-03-27T11:50:14.017Z","updated_at":"2026-04-18T00:01:23.617Z","committers":[{"name":"Christopher Yeh","email":"chrisyeh96","login":"chrisyeh96","count":6},{"name":"Jihyeon Lee","email":"jihyeon@cs.stanford.edu","login":"jlee24","count":1}],"past_year_committers":[],"commits_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench/commits","host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-04-22T00:00:10.876Z","repositories_count":6215767,"commits_count":898846616,"contributors_count":34903541,"owners_count":1144007,"icon_url":"https://github.com/github.png","host_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories"}},"issues_stats":{"full_name":"sustainlab-group/sustainbench","html_url":"https://github.com/sustainlab-group/sustainbench","last_synced_at":"2026-02-15T18:01:20.212Z","status":"error","issues_count":10,"pull_requests_count":1,"avg_time_to_close_issue":null,"avg_time_to_close_pull_request":2912133.0,"issues_closed_count":0,"pull_requests_closed_count":1,"pull_request_authors_count":1,"issue_authors_count":10,"avg_comments_per_issue":0.3,"avg_comments_per_pull_request":1.0,"merged_pull_requests_count":0,"bot_issues_count":0,"bot_pull_requests_count":0,"past_year_issues_count":0,"past_year_pull_requests_count":0,"past_year_avg_time_to_close_issue":null,"past_year_avg_time_to_close_pull_request":null,"past_year_issues_closed_count":0,"past_year_pull_requests_closed_count":0,"past_year_pull_request_authors_count":0,"past_year_issue_authors_count":0,"past_year_avg_comments_per_issue":null,"past_year_avg_comments_per_pull_request":null,"past_year_bot_issues_count":0,"past_year_bot_pull_requests_count":0,"past_year_merged_pull_requests_count":0,"created_at":"2023-05-09T10:36:39.140Z","updated_at":"2026-02-15T18:01:20.213Z","repository_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench","issues_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/sustainlab-group%2Fsustainbench/issues","issue_labels_count":{},"pull_request_labels_count":{},"issue_author_associations_count":{"NONE":9},"pull_request_author_associations_count":{"NONE":1},"issue_authors":{"dpaiton":1,"gabrieltseng":1,"Spiruel":1,"MarcCoru":1,"tesfaldet":1,"robin-j412":1,"XuanfengLiuNo1":1,"yenniejun":1,"krag-harsh":1},"pull_request_authors":{"dengweihuan":1},"host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-04-22T00:00:10.827Z","repositories_count":14351443,"issues_count":34435118,"pull_requests_count":112724026,"authors_count":11242200,"icon_url":"https://github.com/github.png","host_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories","owners_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/owners","authors_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors"},"past_year_issue_labels_count":{},"past_year_pull_request_labels_count":{},"past_year_issue_author_associations_count":{},"past_year_pull_request_author_associations_count":{},"past_year_issue_authors":{},"past_year_pull_request_authors":{},"maintainers":[],"active_maintainers":[]},"events":{"total":{"ForkEvent":5,"WatchEvent":14},"last_year":{"ForkEvent":2,"WatchEvent":7}},"keywords":["deep-learning","machine-learning","sdgs","sustainability"],"dependencies":[{"ecosystem":"pypi","filepath":"setup.py","sha":null,"kind":"manifest","created_at":"2022-07-20T01:17:46.546Z","updated_at":"2022-07-20T01:17:46.546Z","repository_link":"https://github.com/sustainlab-group/sustainbench/blob/main/setup.py","dependencies":[{"id":280414131,"package_name":"numpy","ecosystem":"pypi","requirements":"\u003e=1.19.1","direct":true,"kind":"runtime","optional":false},{"id":280414132,"package_name":"pandas","ecosystem":"pypi","requirements":"\u003e=1.1.0","direct":true,"kind":"runtime","optional":false},{"id":280414133,"package_name":"scikit-learn","ecosystem":"pypi","requirements":"\u003e=0.20.0","direct":true,"kind":"runtime","optional":false},{"id":280414134,"package_name":"pillow","ecosystem":"pypi","requirements":"\u003e=7.2.0","direct":true,"kind":"runtime","optional":false},{"id":280414135,"package_name":"torch","ecosystem":"pypi","requirements":"\u003e=1.7.0","direct":true,"kind":"runtime","optional":false},{"id":280414136,"package_name":"tqdm","ecosystem":"pypi","requirements":"\u003e=4.53.0","direct":true,"kind":"runtime","optional":false}]}],"score":5.568344503761097,"created_at":"2023-09-11T11:54:36.991Z","updated_at":"2026-04-23T02:30:18.714Z","avatar_url":"https://github.com/sustainlab-group.png","language":"Python","category":"Sustainable Development","sub_category":"Sustainable Development Goals","monthly_downloads":0,"total_dependent_repos":0,"total_dependent_packages":0,"readme":"[**Datasets**](https://sustainlab-group.github.io/sustainbench/docs/datasets/) |\n[**Website**](https://sustainlab-group.github.io/sustainbench/) |\n[**Raw Data**](https://drive.google.com/drive/folders/1jyjK5sKGYegfHDjuVBSxCoj49TD830wL?usp=sharing) |\n[**OpenReview**](https://openreview.net/forum?id=5HR3vCylqD)\n\n# SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning\n\n[Christopher Yeh](https://chrisyeh96.github.io/), [Chenlin Meng](https://cs.stanford.edu/~chenlin/), [Sherrie Wang](http://stanford.edu/~sherwang/), Anne Driscoll, [Erik Rozi](https://www.linkedin.com/in/erik-rozi/), Patrick Liu, [Jihyeon Lee](https://jlee24.github.io/), [Marshall Burke](http://web.stanford.edu/~mburke/), [David B. Lobell](https://fse.fsi.stanford.edu/people/david_lobell), [Stefano Ermon](https://cs.stanford.edu/~ermon/)\n\n**California Institute of Technology, Stanford University, and UC Berkeley**\n\nSustainBench is a collection of 15 benchmark tasks across 7 SDGs, including tasks related to economic development, agriculture, health, education, water and sanitation, climate action, and life on land. **Datasets for 11 of the 15 tasks are released publicly for the first time.** Our goals for SustainBench are to\n1. lower the barriers to entry for the machine learning community to contribute to measuring and achieving the SDGs;\n2. provide standard benchmarks for evaluating machine learning models on tasks across a variety of SDGs; and\n3. encourage the development of novel machine learning methods where improved model performance facilitates progress towards the SDGs.\n\n\n## Table of Contents\n\n* [Overview](#overview)\n* [Dataloaders](#dataloaders)\n* [Running Baseline Models](#running-baseline-models)\n* [Dataset Preprocessing](#dataset-preprocessing)\n* [Computing Requirements](#computing-requirements)\n* [Code Formatting and Type Checking](#code-formatting-and-type-checking)\n* [Citation](#citation)\n\n\n## Overview\n\nSustainBench provides datasets and standardized benchmarks for 15 SDG-related tasks, listed below. Details for each dataset and task can be found in our [**paper**](https://openreview.net/forum?id=5HR3vCylqD) and on our [**website**](https://sustainlab-group.github.io/sustainbench/). The raw data can be downloaded from [**Google Drive**](https://drive.google.com/drive/folders/1jyjK5sKGYegfHDjuVBSxCoj49TD830wL?usp=sharing) and is released under a [CC-BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/).\n\n\u003cimg src=\"https://github.com/sustainlab-group/sustainbench/blob/gh-pages/assets/images/fig1.png\" width=\"600\"\u003e\n\n- **SDG 1: No Poverty**\n  - [Task 1A](https://sustainlab-group.github.io/sustainbench/docs/datasets/dhs.html): Predicting poverty over space\n  - [Task 1B](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg1/change_in_poverty.html): Predicting change in poverty over time\n- **SDG 2: Zero Hunger**\n  - [Task 2A](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg2/weakly_supervised_cropland.html): Cropland mapping\n  - [Task 2B1](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg2/crop_type_mapping_ghana-ss.html): Crop type mapping, in Ghana in South Sudan\n  - [Task 2B2](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg2/crop_type_mapping_kenya.html): Crop type mapping, in Kenya\n  - [Task 2C](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg2/crop_yield.html): Crop yield prediction\n  - [Task 2D](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg2/field_delineation.html): Field delineation\n- **SDG 3: Good Health and Well-being**\n  - [Task 3A](https://sustainlab-group.github.io/sustainbench/docs/datasets/dhs.html): Child mortality rate\n  - [Task 3B](https://sustainlab-group.github.io/sustainbench/docs/datasets/dhs.html): Women BMI\n- **SDG 4: Quality Education**\n  - [Task 4A](https://sustainlab-group.github.io/sustainbench/docs/datasets/dhs.html): Women educational attainment\n- **SDG 6: Clean Water and Sanitation**\n  - [Task 6A](https://sustainlab-group.github.io/sustainbench/docs/datasets/dhs.html): Clean water\n  - [Task 6B](https://sustainlab-group.github.io/sustainbench/docs/datasets/dhs.html): Sanitation\n- **SDG 13: Climate Action**\n  - [Task 13A](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg13/brick_kiln.html): Brick kiln classification\n- **SDG 15: Life on Land**\n  - [Task 15A](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg15/land_cover_representation.html): Feature learning for land cover classification\n  - [Task 15B](https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg15/out_of_domain_land_cover.html): Out-of-domain land cover classification\n\n\n## Dataloaders\n\nFor each dataset, we provide Python dataloaders that load the data as PyTorch tensors. Please see the `sustainbench` folder as well as our [website](https://sustainlab-group.github.io/sustainbench/) for detailed documentation.\n\n\n## Running Baseline Models\n\nWe provide baseline models for many of the benchmark tasks included in SustainBench. See the `baseline_models` folder for the code and detailed instructions to reproduce our results.\n\n\n## Dataset Preprocessing\n\n11 of the 15 SustainBench benchmark tasks involve data that is being publicly released for the first time. We release the processed versions of our datasets on [Google Drive](https://drive.google.com/drive/folders/1jyjK5sKGYegfHDjuVBSxCoj49TD830wL?usp=sharing). However, we also provide code and detailed instructions for how we preprocessed the datasets in the `dataset_preprocessing` folder. You do NOT need anything from the `dataset_preprocessing` folder for downloading the processed datasets or running our baseline models.\n\n\n## Computing Requirements\n\nThis code was tested on a system with the following specifications:\n\n- operating system: Ubuntu 16.04.7 LTS\n- CPU: Intel(R) Xeon(R) CPU E5-2620 v4\n- memory (RAM): 125 GB\n- disk storage: 5 TB\n- GPU: NVIDIA P100 GPU\n\nThe main software requirements are Python 3.7 with TensorFlow r1.15, PyTorch 1.9, and R 4.1. The complete list of required packages and library are listed in the two conda environment YAML files (`env_create.yml` and `env_bench.yml`), which are meant to be used with `conda` (version 4.10). See [here](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for instructions on installing conda via Miniconda. Once conda is installed, run one of the following commands to set up the desired conda environment:\n\n```bash\nconda env update -f env_create.yml --prune\nconda env update -f env_bench.yml --prune\n```\n\nThe conda environment files default to CPU-only packages. If you have a GPU, please comment/uncomment the appropriate lines in the environment files; you may need to also install CUDA 10 or 11 and cuDNN 7.\n\n\n## Code Formatting and Type Checking\n\nThis repo uses [flake8](https://flake8.pycqa.org/) for Python linting and [mypy](https://mypy.readthedocs.io/) for type-checking. Configuration files for each are included in this repo: `.flake8` and `mypy.ini`.\n\nTo run either code linting or type checking, set the current directory to the repo root directory. Then run any of the following commands:\n\n```bash\n# LINTING\n# =======\n\n# entire repo\nflake8\n\n# all modules within utils directory\nflake8 utils\n\n# a single module\nflake8 path/to/module.py\n\n# a jupyter notebook - ignore these error codes, in addition to the ignored codes in .flake8:\n# - E305: expected 2 blank lines after class or function definition\n# - E402: Module level import not at top of file\n# - F404: from __future__ imports must occur at the beginning of the file\n# - W391: Blank line at end of file\njupyter nbconvert path/to/notebook.ipynb --stdout --to script | flake8 - --extend-ignore=E305,E402,F404,W391\n\n\n# TYPE CHECKING\n# =============\n\n# entire repo\nmypy .\n\n# all modules within utils directory\nmypy -p utils\n\n# a single module\nmypy path/to/module.py\n\n# a jupyter notebook\nmypy -c \"$(jupyter nbconvert path/to/notebook.ipynb --stdout --to script)\"\n```\n\n\n## Citation\n\nPlease cite this article as follows, or use the BibTeX entry below.\n\n\u003e C. Yeh, C. Meng, S. Wang, A. Driscoll, E. Rozi, P. Liu, J. Lee, M. Burke, D. B. Lobell, and S. Ermon, \"SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning,\" in _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_, Dec. 2021. [Online]. Available: [https://openreview.net/forum?id=5HR3vCylqD](https://openreview.net/forum?id=5HR3vCylqD).\n\n```tex\n@inproceedings{\n    yeh2021sustainbench,\n    title = {{SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning}},\n    author = {Christopher Yeh and Chenlin Meng and Sherrie Wang and Anne Driscoll and Erik Rozi and Patrick Liu and Jihyeon Lee and Marshall Burke and David B. Lobell and Stefano Ermon},\n    booktitle = {Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)},\n    year = {2021},\n    month = {12},\n    url = {https://openreview.net/forum?id=5HR3vCylqD}\n}\n```\n","funding_links":[],"readme_doi_urls":[],"works":{},"citation_counts":{},"total_citations":0,"keywords_from_contributors":[],"project_url":"https://ost.ecosyste.ms/api/v1/projects/1122","html_url":"https://ost.ecosyste.ms/projects/1122"}