etl
A compute graph for loading and transforming Our World in Data's data.
https://github.com/owid/etl
Category: Sustainable Development
Sub Category: Data Catalogs and Interfaces
Keywords from Contributors
coronavirus covid covid-19 sars-cov-2 co2-emissions greenhouse-gas-emissions
Last synced: about 6 hours ago
JSON representation
Repository metadata
A compute graph for loading and transforming OWID's data
- Host: GitHub
- URL: https://github.com/owid/etl
- Owner: owid
- License: mit
- Created: 2021-09-17T12:41:57.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2026-08-25T19:50:48.000Z (6 days ago)
- Last Synced: 2026-08-26T00:17:46.206Z (5 days ago)
- Language: Python
- Homepage: https://docs.owid.io/projects/etl
- Size: 145 MB
- Stars: 156
- Watchers: 3
- Forks: 30
- Open Issues: 66
- Releases: 0
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.md
- License: LICENSE
- Agents: AGENTS.md
- Claude: CLAUDE.md
Owner metadata
- Name: Our World in Data
- Login: owid
- Email:
- Kind: organization
- Description: Research and data to make progress against the world’s largest problems
- Website: https://ourworldindata.org
- Location:
- Twitter: OurWorldInData
- Company:
- Icon url: https://avatars.githubusercontent.com/u/14187135?v=4
- Repositories: 45
- Last ynced at: 2024-04-20T00:04:54.367Z
- Profile URL: https://github.com/owid
GitHub Events
Total
- Issues event: 246
- Watch event: 35
- Delete event: 970
- Issue comment event: 2027
- Push event: 8906
- Pull request review event: 1408
- Pull request review comment event: 1071
- Pull request event: 2244
- Fork event: 4
- Create event: 1147
Last Year
- Issues event: 246
- Watch event: 35
- Delete event: 970
- Issue comment event: 2027
- Push event: 8906
- Pull request review event: 1408
- Pull request review comment event: 1071
- Pull request event: 2244
- Fork event: 4
- Create event: 1147
Committers metadata
Last synced: 10 days ago
Total Commits: 15,884
Total Committers: 39
Avg Commits per committer: 407.282
Development Distribution Score (DDS): 0.525
Commits in past year: 3,367
Committers in past year: 22
Avg Commits per committer in past year: 153.045
Development Distribution Score (DDS) in past year: 0.44
| Name | Commits | |
|---|---|---|
| owidbot | t****h@o****g | 7551 |
| Pablo Rosado | p****o@g****m | 2092 |
| Mojmir Vinkler | m****r@g****m | 1665 |
| Lucas Rodés-Guirao | l****s | 1154 |
| Pablo Arriagada | 6****p | 805 |
| Fiona Spooner | f****a@o****g | 663 |
| Veronika Samborska | 3****4 | 620 |
| Lars Yencken | l****s@y****g | 403 |
| Edouard Mathieu | e****t@p****e | 224 |
| analytics | a****s@o****g | 219 |
| Tuna Acisu | 5****4 | 96 |
| lucas rg | l****g@k****e | 78 |
| Daniel Bachler | d****l@d****e | 70 |
| dependabot[bot] | 4****] | 46 |
| Tuna Acisu | t****u@o****m | 40 |
| Sophia Mersmann | s****1@g****m | 34 |
| Marcel Gerber | m****9@g****m | 25 |
| Bobbie Macdonald | b****d@g****m | 16 |
| Martin Račák | r****i | 14 |
| Fiona Spooner | f****r@f****e | 12 |
| Billy Cox | w****5@g****m | 8 |
| claude[bot] | 2****] | 8 |
| Pablo Rosado | p****o@i****s | 8 |
| Bertha Rohenkohl | 9****c | 6 |
| Hannah Ritchie | 3****e | 4 |
| Claude | n****y@a****m | 4 |
| Bastian Herre | b****n@o****g | 3 |
| William Cox | w****x@c****u | 2 |
| Matthieu Bergel | m****u@o****g | 2 |
| Ike Saunders | 1****u | 2 |
| and 9 more... | ||
Committer domains:
- ourworldindata.org: 5
- ymail.com: 1
- college.harvard.edu: 1
- anthropic.com: 1
- inetum.altia.es: 1
- fionas-mbp.home: 1
- ourworldindata.com: 1
- danielbachler.de: 1
- kth.se: 1
- pm.me: 1
- yencken.org: 1
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 380
Total pull requests: 3,871
Average time to close issues: about 2 months
Average time to close pull requests: 6 days
Total issue authors: 21
Total pull request authors: 30
Average comments per issue: 2.16
Average comments per pull request: 1.19
Merged pull request: 2,652
Bot issues: 0
Bot pull requests: 151
Past year issues: 28
Past year pull requests: 402
Past year average time to close issues: 2 months
Past year average time to close pull requests: 6 days
Past year issue authors: 7
Past year pull request authors: 17
Past year average comments per issue: 1.04
Past year average comments per pull request: 2.31
Past year merged pull request: 202
Past year bot issues: 0
Past year bot pull requests: 41
Top Issue Authors
- Marigold (102)
- pabloarosado (89)
- lucasrodes (73)
- larsyencken (55)
- spoonerf (32)
- paarriagadap (10)
- edomt (3)
- ikesau (2)
- antea04 (2)
- eseidel (1)
- marcelgerber (1)
- exploreriii (1)
- jorisvandenbossche (1)
- heliodoro8243 (1)
- veronikasamborska1994 (1)
Top Pull Request Authors
- lucasrodes (917)
- Marigold (693)
- pabloarosado (614)
- spoonerf (466)
- veronikasamborska1994 (412)
- paarriagadap (346)
- dependabot[bot] (146)
- antea04 (101)
- owidbot (52)
- edomt (26)
- sophiamersmann (23)
- danyx23 (19)
- larsyencken (17)
- marcelgerber (10)
- rakyi (5)
Top Issue Labels
- needs triage (88)
- wontfix (69)
- priority 3 - nice to have (69)
- priority 2 - important (59)
- wizard (33)
- bug (27)
- metadata (23)
- priority 2 - medium (14)
- needs discussion (14)
- enhancement (11)
- priority 3 - low (9)
- pinned (9)
- data (9)
- correctness (9)
- devex (7)
- documentation (6)
- stale (5)
- walkthrough (4)
- priority 1 - essential (4)
- tracking (4)
- maintenance (4)
- ci (3)
- workflow (3)
- help wanted (2)
- dependencies (2)
- priority 1 - high (1)
- catalog (1)
- ops (1)
- needs proposal (1)
Top Pull Request Labels
- dependencies (146)
- wontfix (38)
- wizard (23)
- python:uv (22)
- data (18)
- javascript (15)
- codex (11)
- stale (11)
- merge-schedule-failed (10)
- enhancement (8)
- bug (6)
- documentation (6)
- staging (5)
- staging-bake (4)
- devex (3)
- priority 3 - nice to have (3)
- metadata (3)
- pinned (2)
- coderabbit (2)
- catalog (1)
- needs triage (1)
- priority 2 - important (1)
- api: cloudprofiler (1)
- size: xs (1)
- external (1)
- maintenance (1)
- github_actions (1)
- tagpr (1)
Package metadata
- Total packages: 2
-
Total downloads:
- pypi: 4,644 last-month
- Total dependent packages: 0 (may contain duplicates)
- Total dependent repositories: 6 (may contain duplicates)
- Total versions: 41
- Total maintainers: 1
proxy.golang.org: github.com/owid/etl
- Homepage:
- Status: removed
- Documentation: https://pkg.go.dev/github.com/owid/etl#section-documentation
- Licenses: mit
- Latest release: v0.0.0-20260811192259-4ca048c58580 (published 20 days ago)
- Last Synced: 2026-08-30T02:07:15.679Z (1 day ago)
- Versions: 1
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent packages count: 4.494%
- Average: 4.645%
- Dependent repos count: 4.796%
pypi.org: owid-catalog
Core data types used by OWID for managing data.
- Homepage: https://ourworldindata.org
- Documentation: https://docs.owid.io/projects/etl/libraries/catalog/
- Licenses: MIT
- Latest release: 1.2.4 (published about 1 month ago)
- Last Synced: 2026-08-30T02:06:03.601Z (1 day ago)
- Versions: 40
- Dependent Packages: 0
- Dependent Repositories: 6
- Downloads: 4,644 Last month
-
Rankings:
- Dependent repos count: 6.019%
- Dependent packages count: 10.052%
- Average: 11.394%
- Downloads: 12.422%
- Forks count: 13.288%
- Stargazers count: 15.188%
- Maintainers (1)
Dependencies
.github/workflows/project-automations.yml
actions
- andymckay/labeler e6c4322d0397f3240f0e7e30a33b5c5df2d39e90 composite
- owid/actions/assign-priority main composite
- owid/actions/set-project-status main composite
lib/catalog/pyproject.toml
pypi
- PyYAML >=5.4.1
- Unidecode >=1.3.4
- boto3 >=1.21.13
- dataclasses-json >=0.5.6
- dynamic-yaml ^1.3.4
- ipdb >=0.13.9
- jsonschema >=3.2.0
- owid-repack >=0.1.1
- pandas >=1.3.3,<2.0
- pyarrow >=10.0.1
- pytest-cov >=2.12.1
- python >=3.8.1
- requests >=2.26.0
- structlog >=21.5.0
lib/datautils/pyproject.toml
pypi
- boto3 >=1.21.16
- colorama >=0.4.4
- gdown >=4.5.2
- gsheets >=0.6.1
- owid-catalog *
- pandas >=1.3.3
- pydrive2 >=1.15.0
- python >=3.8.1,<4.0
- structlog >=21.5.0
- urllib3 <2
lib/repack/pyproject.toml
pypi
- numpy >=1.24.0
- pandas >=1.5.2
- python >=3.8.1
pyproject.toml
pypi
- PyPDF2 >=2.11.1
- PyYAML >=5.4.1
- SPARQLWrapper >=1.8.5
- Unidecode >=1.3.2
- bugsnag >=4.2.1
- click >=8.0.1
- dvc >=2.58.2,<3.0.0
- frictionless ^4.40.8
- fsspec 2022.11.0
- gitpython ^3.1.30
- jupyterlab >=3.1.13
- mysqlclient 2.1.1
- numpy >=1.22.1
- odfpy ^1.4.1
- openai ^0.27.7
- openpyxl >=3.0.9
- owid-catalog *
- owid-datautils *
- owid-repack *
- pandas 1.5.2
- papermill >=2.3.3
- pdfplumber ^0.9.0
- pydantic >=1.9.0
- pyhumps ^3.8.0
- python ^3.10
- python-dotenv >=0.19.0
- questionary ^2.0.0
- rapidfuzz ^2.13.7
- regex >=2022.1.18
- rich >=12.1.0
- rich-click >=1.5.1
- ruamel.yaml >=0.17.21
- sh 1.14.3
- simplejson >=3.17.6
- sqlmodel ^0.0.6
- st-pages ^0.4.4
- streamlit ^1.26.0
- streamlit-ace ^0.1.1
- streamlit-extras ^0.3.0
- structlog >=21.5.0
- tenacity >=8.0.1
- typing-extensions ^4.7.1
- walden *
- wbgapi ^1.0.12
- wikipedia >=1.4.0
- world-bank-data ^0.1.3
- xlrd >=2.0.1
dag/environment.yml
conda
lib/owl/uv.lock
pypi
- 119 dependencies
lib/repack/uv.lock
pypi
- asttokens 3.0.1
- colorama 0.4.6
- decorator 5.2.1
- executing 2.2.1
- iniconfig 2.3.0
- ipdb 0.13.13
- ipython 9.10.0
- ipython 9.11.0
- ipython-pygments-lexers 1.1.1
- jedi 0.19.2
- matplotlib-inline 0.2.1
- numpy 2.4.3
- owid-repack 0.1.11
- packaging 26.0
- pandas 2.3.3
- parso 0.8.6
- pexpect 4.9.0
- pluggy 1.6.0
- prompt-toolkit 3.0.52
- ptyprocess 0.7.0
- pure-eval 0.2.3
- pyarrow 23.0.1
- pygments 2.20.0
- pytest 9.0.3
- python-dateutil 2.9.0.post0
- pytz 2026.1.post1
- ruff 0.15.8
- six 1.17.0
- stack-data 0.6.3
- traitlets 5.14.3
- ty 0.0.55
- typing-extensions 4.15.0
- tzdata 2025.3
- wcwidth 0.6.0
uv.lock
pypi
- 422 dependencies
vscode_extensions/find-latest-etl-step/package.json
npm
- @types/lodash ^4.17.24 development
- @types/mocha ^10.0.10 development
- @types/node ^20.19.37 development
- @types/vscode ^1.110.0 development
- @typescript-eslint/eslint-plugin ^8.57.1 development
- @typescript-eslint/parser ^8.57.1 development
- @vscode/test-cli ^0.0.10 development
- @vscode/test-electron ^2.5.2 development
- esbuild ^0.28.1 development
- eslint ^9.39.4 development
- npm-run-all ^4.1.5 development
- typescript ^5.9.3 development
- ignore ^6.0.2
- lodash ^4.18.1
.github/workflows/deploy-docs-cf.yml
actions
- actions/checkout df4cb1c069e1874edd31b4311f1884172cec0e10 composite
- actions/setup-python a309ff8b426b58ec0e2a45f0f869d46889d02405 composite
- astral-sh/setup-uv fac544c07dec837d0ccb6301d7b5580bf5edae39 composite
- cloudflare/wrangler-action ebbaa1584979971c8614a24965b4405ff95890e0 composite
dag/archive/environment.yml
conda
vscode_extensions/dod-syntax/package-lock.json
npm
- 388 dependencies
.github/workflows/buildkite.yml
actions
- actions/checkout df4cb1c069e1874edd31b4311f1884172cec0e10 composite
- actions/setup-python a309ff8b426b58ec0e2a45f0f869d46889d02405 composite
.github/workflows/zizmor.yml
actions
- actions/checkout df4cb1c069e1874edd31b4311f1884172cec0e10 composite
- zizmorcore/zizmor-action 5f14fd08f7cf1cb1609c1e344975f152c7ee938d composite
vscode_extensions/dod-syntax/package.json
npm
- @types/mocha ^10.0.10 development
- @types/node ^20.19.37 development
- @types/vscode ^1.109.0 development
- @typescript-eslint/eslint-plugin ^8.57.1 development
- @typescript-eslint/parser ^8.57.1 development
- @vscode/test-cli ^0.0.10 development
- @vscode/test-electron ^2.5.2 development
- esbuild ^0.28.1 development
- eslint ^9.39.4 development
- npm-run-all ^4.1.5 development
- typescript ^5.9.3 development
vscode_extensions/run-until-cursor/package.json
npm
- @types/vscode ^1.97.0 development
- esbuild ^0.28.1 development
vscode_extensions/meta-yaml-hover/package.json
npm
- @types/js-yaml ^4.0.9 development
- @types/node ^20.19.37 development
- @types/vscode ^1.109.0 development
- @typescript-eslint/eslint-plugin ^8.57.1 development
- @typescript-eslint/parser ^8.57.1 development
- esbuild ^0.28.1 development
- eslint ^9.39.4 development
- npm-run-all ^4.1.5 development
- typescript ^5.9.3 development
- js-yaml ^4.3.0
lib/owl/pyproject.toml
pypi
- click >=8.0
- graphviz >=0.21
- owid-catalog *
- pandas >=2.2
- pyarrow >=16.0
- python-dotenv >=1.0
- pyyaml >=6.0
- rich >=12.1.0
- sqlalchemy >=2.0.30
- tomli >=2.0
- tqdm >=4.66.0
vscode_extensions/clickable-dag-steps/package-lock.json
npm
- 393 dependencies
vscode_extensions/compare-previous-version/package.json
npm
- @types/mocha ^10.0.8 development
- @types/node 20.x development
- @types/vscode ^1.93.0 development
- @typescript-eslint/eslint-plugin ^8.7.0 development
- @typescript-eslint/parser ^8.7.0 development
- @vscode/test-cli ^0.0.10 development
- @vscode/test-electron ^2.4.1 development
- esbuild ^0.28.1 development
- eslint ^9.11.1 development
- npm-run-all ^4.1.5 development
- typescript ^5.6.2 development
- ignore ^6.0.2
vscode_extensions/detect-outdated-practices/package.json
npm
- @types/node 20.x development
- @types/vscode ^1.100.0 development
- esbuild ^0.28.1 development
- typescript ^5.6.2 development
vscode_extensions/meta-yaml-hover/package-lock.json
npm
- 267 dependencies
vscode_extensions/chart-preview/package-lock.json
npm
- @esbuild/aix-ppc64 0.28.1 development
- @esbuild/android-arm 0.28.1 development
- @esbuild/android-arm64 0.28.1 development
- @esbuild/android-x64 0.28.1 development
- @esbuild/darwin-arm64 0.28.1 development
- @esbuild/darwin-x64 0.28.1 development
- @esbuild/freebsd-arm64 0.28.1 development
- @esbuild/freebsd-x64 0.28.1 development
- @esbuild/linux-arm 0.28.1 development
- @esbuild/linux-arm64 0.28.1 development
- @esbuild/linux-ia32 0.28.1 development
- @esbuild/linux-loong64 0.28.1 development
- @esbuild/linux-mips64el 0.28.1 development
- @esbuild/linux-ppc64 0.28.1 development
- @esbuild/linux-riscv64 0.28.1 development
- @esbuild/linux-s390x 0.28.1 development
- @esbuild/linux-x64 0.28.1 development
- @esbuild/netbsd-arm64 0.28.1 development
- @esbuild/netbsd-x64 0.28.1 development
- @esbuild/openbsd-arm64 0.28.1 development
- @esbuild/openbsd-x64 0.28.1 development
- @esbuild/openharmony-arm64 0.28.1 development
- @esbuild/sunos-x64 0.28.1 development
- @esbuild/win32-arm64 0.28.1 development
- @esbuild/win32-ia32 0.28.1 development
- @esbuild/win32-x64 0.28.1 development
- @types/node 20.19.33 development
- @types/vscode 1.109.0 development
- esbuild 0.28.1 development
- typescript 5.9.3 development
- undici-types 6.21.0 development
etl/steps/data/garden/demography/2022-12-08/population/meta.yml
cpan
.github/workflows/publish-owid-packages.yml
actions
- actions/checkout df4cb1c069e1874edd31b4311f1884172cec0e10 composite
- actions/setup-python a309ff8b426b58ec0e2a45f0f869d46889d02405 composite
- astral-sh/setup-uv fac544c07dec837d0ccb6301d7b5580bf5edae39 composite
- pypa/gh-action-pypi-publish cef221092ed1bacb1cc03d23a2d87d1d172e277b composite
.github/workflows/merge-schedule.yml
actions
- gr2m/merge-schedule-action a9b6ddcf1282dfd66907bda8e7941dff59f03bad composite
.github/workflows/stale.yml
actions
- actions/stale eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 composite
.github/workflows/sync-grapher-schema.yml
actions
- actions/checkout df4cb1c069e1874edd31b4311f1884172cec0e10 composite
- actions/setup-python a309ff8b426b58ec0e2a45f0f869d46889d02405 composite
- astral-sh/setup-uv fac544c07dec837d0ccb6301d7b5580bf5edae39 composite
- peter-evans/create-pull-request 5f6978faf089d4d20b00c7766989d076bb2fc7f1 composite
etl/steps/data/garden/papers/2022-11-03/zijdeman_et_al_2015/meta.yml
cpan
etl/steps/data/garden/papers/2023-02-03/riley_2005/meta.yml
cpan
vscode_extensions/clickable-dag-steps/package.json
npm
- @types/js-yaml ^4.0.9 development
- @types/mocha ^10.0.8 development
- @types/node 20.x development
- @types/vscode ^1.100.0 development
- @typescript-eslint/eslint-plugin ^8.7.0 development
- @typescript-eslint/parser ^8.7.0 development
- @vscode/test-cli ^0.0.10 development
- @vscode/test-electron ^2.4.1 development
- esbuild ^0.28.1 development
- eslint ^9.11.1 development
- npm-run-all ^4.1.5 development
- typescript ^5.6.2 development
- js-yaml ^4.3.0
vscode_extensions/compare-previous-version/package-lock.json
npm
- 393 dependencies
vscode_extensions/detect-outdated-practices/package-lock.json
npm
- @esbuild/aix-ppc64 0.28.1 development
- @esbuild/android-arm 0.28.1 development
- @esbuild/android-arm64 0.28.1 development
- @esbuild/android-x64 0.28.1 development
- @esbuild/darwin-arm64 0.28.1 development
- @esbuild/darwin-x64 0.28.1 development
- @esbuild/freebsd-arm64 0.28.1 development
- @esbuild/freebsd-x64 0.28.1 development
- @esbuild/linux-arm 0.28.1 development
- @esbuild/linux-arm64 0.28.1 development
- @esbuild/linux-ia32 0.28.1 development
- @esbuild/linux-loong64 0.28.1 development
- @esbuild/linux-mips64el 0.28.1 development
- @esbuild/linux-ppc64 0.28.1 development
- @esbuild/linux-riscv64 0.28.1 development
- @esbuild/linux-s390x 0.28.1 development
- @esbuild/linux-x64 0.28.1 development
- @esbuild/netbsd-arm64 0.28.1 development
- @esbuild/netbsd-x64 0.28.1 development
- @esbuild/openbsd-arm64 0.28.1 development
- @esbuild/openbsd-x64 0.28.1 development
- @esbuild/openharmony-arm64 0.28.1 development
- @esbuild/sunos-x64 0.28.1 development
- @esbuild/win32-arm64 0.28.1 development
- @esbuild/win32-ia32 0.28.1 development
- @esbuild/win32-x64 0.28.1 development
- @types/node 20.19.25 development
- @types/vscode 1.106.1 development
- esbuild 0.28.1 development
- typescript 5.9.3 development
- undici-types 6.21.0 development
vscode_extensions/chart-preview/package.json
npm
- @types/node 20.x development
- @types/vscode ^1.100.0 development
- esbuild ^0.28.1 development
- typescript ^5.6.2 development
vscode_extensions/find-latest-etl-step/package-lock.json
npm
- 391 dependencies
vscode_extensions/run-until-cursor/package-lock.json
npm
- @esbuild/aix-ppc64 0.28.1 development
- @esbuild/android-arm 0.28.1 development
- @esbuild/android-arm64 0.28.1 development
- @esbuild/android-x64 0.28.1 development
- @esbuild/darwin-arm64 0.28.1 development
- @esbuild/darwin-x64 0.28.1 development
- @esbuild/freebsd-arm64 0.28.1 development
- @esbuild/freebsd-x64 0.28.1 development
- @esbuild/linux-arm 0.28.1 development
- @esbuild/linux-arm64 0.28.1 development
- @esbuild/linux-ia32 0.28.1 development
- @esbuild/linux-loong64 0.28.1 development
- @esbuild/linux-mips64el 0.28.1 development
- @esbuild/linux-ppc64 0.28.1 development
- @esbuild/linux-riscv64 0.28.1 development
- @esbuild/linux-s390x 0.28.1 development
- @esbuild/linux-x64 0.28.1 development
- @esbuild/netbsd-arm64 0.28.1 development
- @esbuild/netbsd-x64 0.28.1 development
- @esbuild/openbsd-arm64 0.28.1 development
- @esbuild/openbsd-x64 0.28.1 development
- @esbuild/openharmony-arm64 0.28.1 development
- @esbuild/sunos-x64 0.28.1 development
- @esbuild/win32-arm64 0.28.1 development
- @esbuild/win32-ia32 0.28.1 development
- @esbuild/win32-x64 0.28.1 development
- @types/vscode 1.97.0 development
- esbuild 0.28.1 development
lib/datautils/uv.lock
pypi
- 122 dependencies
lib/catalog/uv.lock
pypi
- 125 dependencies
Score: 17.511076557226023