kerchunk
A library that provides a unified way to represent a variety of chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), allowing efficient access to the data from traditional file systems or cloud object storage.
https://github.com/fsspec/kerchunk
Category: Climate Change
Sub Category: Climate Data Standards
Keywords
hacktoberfest python
Keywords from Contributors
zarr pangeo compressed ndimensional-arrays data-catalog data-access pydata meteorology open-science oceanography
Last synced: about 3 hours ago
JSON representation
Repository metadata
Cloud-friendly access to archival data
- Host: GitHub
- URL: https://github.com/fsspec/kerchunk
- Owner: fsspec
- License: mit
- Created: 2020-11-10T15:26:36.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2026-01-21T17:35:23.000Z (17 days ago)
- Last Synced: 2026-02-03T07:53:20.019Z (5 days ago)
- Topics: hacktoberfest, python
- Language: Python
- Homepage: https://fsspec.github.io/kerchunk/
- Size: 116 MB
- Stars: 360
- Watchers: 18
- Forks: 94
- Open Issues: 128
- Releases: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Owner metadata
- Name: python filesystem spec
- Login: fsspec
- Email:
- Kind: organization
- Description: data storage IO layer for python
- Website:
- Location:
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/92825505?v=4
- Repositories: 21
- Last ynced at: 2025-10-30T00:34:39.970Z
- Profile URL: https://github.com/fsspec
GitHub Events
Total
- Delete event: 1
- Pull request event: 40
- Fork event: 14
- Discussion event: 2
- Issues event: 27
- Watch event: 44
- Issue comment event: 251
- Push event: 36
- Pull request review event: 14
- Pull request review comment event: 16
- Create event: 3
Last Year
- Pull request event: 20
- Fork event: 9
- Discussion event: 2
- Issues event: 13
- Watch event: 29
- Issue comment event: 84
- Push event: 20
- Pull request review event: 10
- Pull request review comment event: 10
- Create event: 1
Committers metadata
Last synced: 6 days ago
Total Commits: 749
Total Committers: 64
Avg Commits per committer: 11.703
Development Distribution Score (DDS): 0.595
Commits in past year: 19
Committers in past year: 10
Avg Commits per committer in past year: 1.9
Development Distribution Score (DDS) in past year: 0.632
| Name | Commits | |
|---|---|---|
| Martin Durant | m****t@a****a | 303 |
| Martin Durant | m****t@u****a | 65 |
| Lucas Sterzinger | l****r@u****u | 53 |
| Anu-Ra-g | n****8@g****m | 42 |
| Rich Signell | r****l@u****v | 31 |
| Kevin Paul | k****l@n****m | 20 |
| Max Jones | 1****s | 18 |
| dcherian | d****k@c****t | 18 |
| David Stuebe | d****d@c****y | 17 |
| peterm790 | p****0@g****m | 17 |
| Alex Goodman | a****m | 17 |
| Raphael Hagen | n****n@g****m | 15 |
| Lucas Sterzinger | l****s@u****u | 13 |
| Matt Iannucci | m****i@r****m | 11 |
| Lawson Woods | l****2@a****u | 7 |
| Ray Bell | r****l@d****m | 7 |
| RichardScottOZ | 7****Z | 6 |
| Ben Mares | s****1@t****m | 6 |
| Aleksandar Jelenak | a****k@g****m | 4 |
| David Stuebe | 8****d | 4 |
| Pier Lorenzo Marasco | p****o | 4 |
| Josef Kellndorfer | j****r | 4 |
| Louis Maddox | l****x@g****m | 3 |
| Ryan Grout | g****r | 3 |
| Robert Banick | r****k@g****m | 3 |
| Ray Bell | r****0@g****m | 3 |
| Pete Gadomski | p****i@g****m | 3 |
| Kelton Halbert | k****t@w****u | 3 |
| Ian Thomas | i****3@g****m | 3 |
| Aimee Barciauskas | a****e@d****g | 2 |
| and 34 more... | ||
Committer domains:
- earthmover.io: 1
- microsoft.com: 1
- spatialys.com: 1
- maths.ox.ac.uk: 1
- noaa.gov: 1
- stevenbamford.com: 1
- research.uwa.edu.au: 1
- me.com: 1
- danielrothenberg.com: 1
- ssec.wisc.edu: 1
- vt.edu: 1
- dkrz.de: 1
- developmentseed.org: 1
- wisc.edu: 1
- tensorial.com: 1
- dtn.com: 1
- asu.edu: 1
- rpsgroup.com: 1
- ucar.edu: 1
- camus.energy: 1
- cherian.net: 1
- nvidia.com: 1
- usgs.gov: 1
- ucdavis.edu: 1
- utoronto.ca: 1
- alumni.utoronto.ca: 1
Issue and Pull Request metadata
Last synced: 5 days ago
Total issues: 173
Total pull requests: 234
Average time to close issues: about 2 months
Average time to close pull requests: 21 days
Total issue authors: 95
Total pull request authors: 47
Average comments per issue: 6.86
Average comments per pull request: 3.36
Merged pull request: 174
Bot issues: 0
Bot pull requests: 0
Past year issues: 14
Past year pull requests: 27
Past year average time to close issues: 2 months
Past year average time to close pull requests: 17 days
Past year issue authors: 14
Past year pull request authors: 10
Past year average comments per issue: 1.64
Past year average comments per pull request: 2.07
Past year merged pull request: 22
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- TomNicholas (9)
- rsignell (9)
- martindurant (8)
- rsignell-usgs (7)
- raybellwaves (6)
- keltonhalbert (4)
- norlandrhagen (3)
- sreesanjeevkg (3)
- rabernat (3)
- maxrjones (3)
- pl-marasco (3)
- maresb (3)
- kthyng (3)
- John-Ragland (3)
- ashiklom (3)
Top Pull Request Authors
- martindurant (73)
- Anu-Ra-g (33)
- emfdavid (15)
- norlandrhagen (12)
- maxrjones (9)
- raybellwaves (6)
- kmpaul (6)
- maresb (6)
- jhamman (5)
- mpiannucci (5)
- groutr (4)
- ghidalgo3 (4)
- agoodm (3)
- mannreis (2)
- mullenkamp (2)
Top Issue Labels
- GSoC-2024 (1)
Top Pull Request Labels
- GSoC-2024 (15)
Package metadata
- Total packages: 3
-
Total downloads:
- pypi: 79,648 last-month
- Total docker downloads: 369
- Total dependent packages: 11 (may contain duplicates)
- Total dependent repositories: 26 (may contain duplicates)
- Total versions: 24
- Total maintainers: 1
pypi.org: kerchunk
Functions to make reference descriptions for ReferenceFileSystem
- Homepage:
- Documentation: https://fsspec.github.io/kerchunk
- Licenses: MIT
- Latest release: 0.2.9 (published 6 months ago)
- Last Synced: 2026-02-05T00:45:53.409Z (3 days ago)
- Versions: 18
- Dependent Packages: 10
- Dependent Repositories: 13
- Downloads: 79,648 Last month
- Docker Downloads: 369
-
Rankings:
- Dependent packages count: 1.395%
- Downloads: 2.126%
- Average: 2.602%
- Docker downloads count: 2.878%
- Dependent repos count: 4.009%
- Maintainers (1)
conda-forge.org: kerchunk
- Homepage: https://github.com/fsspec/kerchunk
- Licenses: MIT
- Latest release: 0.0.9 (published over 3 years ago)
- Last Synced: 2026-02-05T00:46:01.793Z (3 days ago)
- Versions: 5
- Dependent Packages: 1
- Dependent Repositories: 13
-
Rankings:
- Dependent repos count: 9.806%
- Average: 22.47%
- Forks count: 24.876%
- Stargazers count: 26.219%
- Dependent packages count: 28.978%
spack.io: py-kerchunk
Kerchunk is a library that provides a unified way to represent a variety of chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), allowing efficient access to the data from traditional file systems or cloud object storage. It also provides a flexible way to create virtual datasets from multiple files. It does this by extracting the byte ranges, compression information and other information about the data and storing this metadata in a new, separate object. This means that you can create a virtual aggregate dataset over potentially many source files, for efficient, parallel and cloud-friendly in-situ access without having to copy or translate the originals. It is a gateway to in-the-cloud massive data processing while the data providers still insist on using legacy formats for archival storage.
Dependencies
- dask
- dask-gateway
- dask-labextension
- fsspec
- intake
- intake-xarray
- ipywidgets
- pip
- s3fs
- xarray
- zarr
- autodoc *
- fsspec *
- h5py *
- numcodecs *
- numpy *
- numpydoc ==1.2.1
- sphinx-rtd-theme *
- ujson *
- xarray *
- zarr *
- cftime *
- dask *
- h5netcdf *
- h5py *
- jinja2 *
- mypy *
- pytest *
- s3fs *
- types-ujson *
- xarray *
- fsspec *
- numcodecs *
- numpy *
- ujson *
- zarr *
- actions/checkout v2 composite
- ad-m/github-push-action master composite
- ammaraskar/sphinx-action master composite
- actions/checkout v3.1.0 composite
- actions/setup-python v4 composite
- pre-commit/action v3.0.0 composite
- actions/checkout v2 composite
- ammaraskar/sphinx-action master composite
- actions/checkout v2 composite
- mamba-org/provision-with-micromamba main composite
Score: 21.63971763104935