Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Recent Releases of Herbie

Herbie - Herbie 2025.3.1

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2025.3.0...2025.3.1

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 1 month ago

Herbie - Herbie 2025.3.0

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2025.2.1...2025.3.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 1 month ago

Herbie - Herbie 2025.2.1

https://github.com/blaylockbk/Herbie/commit/077d8726819ee5b774504c86a71386464fdc9c1c is a minor tweak to the minimum pinned dependences as stated in #408. I want to keep Herbie aligned with the latest xarray version.

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2025.2.0...2025.2.1

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk 2 months ago

Herbie - Herbie 2025.2.0

I know, it's about time for another release with some small additions.

The biggest change is dropping the pygrib dependency, because I was having problems with it and Numpy v2+. Since Herbie only used pygrib for parsing the coordinate reference system, Herbie now attempts to parse those details using additional keys from cfgrib. This feature could use refactoring, but it gets the job done for now.

The second big change is Heribe can access the latest and archived Climate Forecast System (see examples in docs)

If you have issues with this release, please open an issue. Thanks!

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.8.0...2025.2.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk 3 months ago

Herbie - Herbie 2024.8.0

Thanks for all the contributions!

What's Changed

Model template changes/updates

Other enhancements

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.7.1...2024.8.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk 9 months ago

Herbie - Herbie 2024.7.1

This release allows Numpy 2.0 to be used, since pygrib recently published support in pygrib v2.6.1.

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.7.0...2024.7.1

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk 10 months ago

Herbie - Herbie 2024.7.0

I'm excited about this release because I brought in some tools for plotting data that I've developed over the years from Carpenter_Workshop into Herbie. I hope people will test it, tell me if they are helpful for you, and tell me if I've broken anything.

Optional Dependencies

The "core" function of Herbie is to download data and read it with xarray. Thus, extra features, like plotting and extracting data at a point, now require installing "extra" dependencies that are not automatically installed. These optional dependencies include cartopy, metpy, matplotlib, and scikit-learn.

  1. Installing Herbie from conda-forge will always install extra dependencies.
  2. Install from pip pip install herbie-data will only install core dependencies.
  3. Install from pip pip install herbie-data[extras] will install dependencies for extra features.

I anticipate this might cause some breaking changes for people. If this change didn't get it right, please open an issue (and offer some help).

Numpy <2.0

Related to dependencies, it seems pygib isn't working with Numpy 2.0 https://github.com/jswhit/pygrib/issues/251, so I've tagged Herbie to require numpy<2.0. Please let me know if this changes or you have an idea to fix this. Actually, pygrib is only used by Herbie to parse the coordinate reference system from the grib files. It would be nice if Herbie didn't need to depend on pygrib (PR anyone??). Maybe pygrib should be another optional dependency.

Mature functions from Carpenter_Workshop moved into Herbie

I use my Carpenter Workshop repository as a workshop for building new stuff and testing ideas. Some of the tools I made there I use nearly every day (my EasyMap class for making Cartopy more simple), so I've migrated some of the mature features into Herbie.

  1. from herbie.toolbox import EasyMap EasyMap lets you make a cartopy axes without a lot of boilerplate code. I find it useful. Read more in the docs. 
    ax = EasyMay('50m', crs=..., figsize=[10,8]).STATES().LAND().OCEAN().BORDERS().ax
  2. from herbie.toolbox import ... Other stuff in the toolbox, like unit conversions
  3. from herbie import paint Paint is a bunch of additional colormaps I've collected over the years that I think look nice. For example, here are the National Weather Service standard color curves:
    image
    I'm not completely satisfied with how these are implemented and organized, so this is considered an unstable feature. The documentation is fresh and incomplete.

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.5.0...2024.7.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk 10 months ago

Herbie - Herbie 2024.5.0

This release of Herbie has some new features and changes I've been developing and experimenting with, so I hope to hear from you if something isn't working as expected or if you have other ideas.

💣 Deprecation / Changes

I renamed the argument searchString to search. This change only affects you if you are using the keyword argument when calling the inventory or xarray methods. Herbie will now give a warning if you use searchString, so please update your scripts.

 from herbie import Herbie
 H = Herbie('2024-01-01')

- H.inventory(searchString="TMP:2 m")
- H.xarray(searchString="TMP:2 m")

+ H.inventory(r"TMP:2 m")
+ H.xarray(r"TMP:2 m")

The reason for the name change is 1) simplicity and 2) the search string isn't an exact match search but is a regex search. (I would have named it filter, but that is a python built-in and couldn't use that). I should also mention that it is recommended to use the r or raw string representation (i.e. search=r":TMP:\d+ mb") since the string is used as a regular expression.

✨ New Feature: ds.herbie.pick_points xarray accessor

I'm very excited to share this new feature. I created a new xarray accessor to get nearest neighbor point data from a model grid. I've had this on my mind for a long time and am happy with what I have made so far. This uses the Ball Tree algorithm, and thus adds scikit-learn as a new required dependency.

Here is a very simple example getting two values from the HRRR grid:

from herbie import Herbie
import pandas as pd

# Get HRRR 2-m temperature analysis
ds = Herbie('2024-1-1', model='hrrr').xarray("TMP:2 m")


# DataFrame of points to get from model grid
points = pd.DataFrame(
    {
        "longitude": [-100.25, -99.4],
        "latitude": [44.25, 45.4],
    }
)

# Pick points from HRRR grid
ds_points = ds.herbie.pick_points(points)

See more in-depth description and examples in the docs. There are options to get values from $k$ nearest neighbors from the requested point and compute the distance-weighted average of nearby points.

If you give this new feature a try, please let me know if you see any unexpected results.

📄 Documentation

I overhauled the organization of the documentation pages. Hopefully you find them organized a little better. Feel free to contribute to the docs if you see some details missing/misleading/confusing 😄

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.3.1...2024.5.0

Future Changes

I am planning to rename the argument fxx for the forecast lead time to step to match the terminology used by cfgrib.

- Herbie('2023-01-01', model='hrrr', fxx=6)
+ Herbie('2023-01-01', model='hrrr', step=6)

Again, this isn't implemented yet, but I plan to implement it eventually and wanted to let you know. Provide any comments if you have any in https://github.com/blaylockbk/Herbie/issues/160

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk 12 months ago

Herbie - Herbie 2024.3.1

Just two minor changes; wanted to get these out there for the few it may impact...

What's Changed

Note: the ds.herbie.plot() accessor is not implemented well. It could use a lot of work to make some standard, quick-look plots, which would be really cool, if anyone has some big ideas of what that should look like. I'm open to suggestions.

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.3.0...2024.3.1

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 1 year ago

Herbie - Herbie 2024.3.0

ECMWF shared this exciting news a few days ago about their open data products for the Integrated Forecast System (IFS)

ECMWF now provides a much larger open dataset to the public, representing weather forecasts at a higher resolution and a reduction in some release times.
- ECMWF: Media Centre

This Herbie release updates the ECMWF model templates to access the new IFS and AIFS 0.25 degree datasets. Historical access to the 0.4 degree datasets is still available for dates before February 1, 2024. Thanks @alcoat!

Deprecation

Now that ECMWF provides open access for two different models, the argument model='ecmwf' is deprecated. Instead, you should access the Integrated Forecast System (IFS) with model='ifs' or model='aifs for the Artificial Intelligence IFS.

- Herbie('2024-3-1', model='ecmwf')  # Deprecated

+ Herbie('2024-3-1', model='ifs')

  Herbie('2024-3-1', model='aifs')  # New Model

New Feature

Herbie has two new experimental functions that may be helpful to some people.

  • HerbieLatest Find the most recent available model run.
  • HerbieWait Wait for a particular model run to become available.

You can see examples in the docs: Herbie Latest Data.
If these are helpful to you but you want them to be improved, I'm looking for people to help make these better, (along with FastHerbie which could use some work too).

Herbie at the American Meteorological Society

I gave a talk on Herbie at the American Meteorology Society in January. You can see my slides here. Soon, you should be able to go to the conference website and watch the recorded presentation.

It was a lot of fun talking to some of you at AMS this year. Its always a surprise to me when I'm wandering around the poster hall and someone says, "Hey Brian, Herbie was used to make this poster!" Thanks everyone for your kindness.

Please feel free to open issues when you see problems, share how you use Herbie in the discussions, and feel free to contribute your ideas and PRs.

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.4...2024.3.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 1 year ago

Herbie - Herbie 2023.12.4

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.3...2023.12.4

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 1 year ago

Herbie - Herbie 2023.12.3

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.2...2023.12.3

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 1 year ago

Herbie - Herbie 2023.12.2

What's Changed

Enhancements

Documentation

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.1...2023.12.2

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 1 year ago

Herbie - Herbie 2023.12.1

It's been a while since the last release, and I wanted to get out all the changes that have built up.

Please open an issue if you see problems. I'm trying a new "auto release on new tag" workflow action, so I may have some micro releases coming.

What's Changed

New model templates

Enhancements

Bug fixes

Documentation

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.3.0...2023.12.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 1 year ago

Herbie - Herbie 2023.3.0

Thanks again for using Herbie and your kind words. More than a few of you found me at the AMS annual meeting in Denver to thank me for publishing and maintaining Herbie. It's always encouraging to hear how I've helped someone. I'm still in shock how putting some code I wrote in grad school on the internet was discovered by so many. Hebie is still a project I work on in my free time, and I don't claim it is the most efficient or cleanly coded package, but the lessons I learn with Herbie have helped me in many ways in other work I do. So, I thank you for your involvement and use of Herbie and for helping me develop my skills.

Now I want to give a big "thank you" to these two people

  1. Ryan May (@dopplershift), thanks for instructing me on how to publish Herbie on conda-forge. This really simplifies installing Herbie given its dependency on cgfrib and cartopy. Herbie can now be installed like this:
conda install herbie-data

Note: The name of this package on pypi and conda is herbie-data because herbie is taken on PyPI.

  1. Andreas Motl (@amotl), thanks for your help with migrating the documentation to readthedocs and adding GitHub Actions to run the suite of tests with each pull request. Automating the doc builds and tests really helps.

These changes and other updates help make Herbie feel a little more "grown up" in this release.

Breaking Changes

  • I renamed the file herbie/archive.py to herbie/core.py because the name "archive" was starting to lose its meaning. Please pay attention to how you import Herbie...
# DON'T do this anymore
from herbie.archive import Herbie

# DO this
from herbie import Herbie
  • I renamed the Herbie method H.read_idx() to H.inventory() because the single word is easier for me to remember. If you use read_idx() you will get a warning for now; it will be removed later. Note that the inventory method is also available for FastHerbie. In fact, I'm migrating away from the term "index" to "inventory" because it I think it's a more accurate description of what a ".idx" file is--in inventory of the file contents. And besides, "index" means other things in python if you are a heavy Pandas user.
## DON'T do this anymore
H = Herbie(date, model)
H.read_idx()

# DO this
H = Herbie(date, model)
H.inventory()

New/Updated Model Templates

Note: There will be additional changes to the GEFS and GFS model templates to make the more consistent. I also plan to add a template for the URMA model since it exists.

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2022.09.0...2022.x.x

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 2 years ago

Herbie - Herbie 2022.9.0

Huge shoutout to Ryan May for instructing me on how to publish Herbie on conda-forge. You can now install Herbie 2022.9.0.post1 with Conda

conda install -c conda-forge herbie-data

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.10...2022.09.0

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 2 years ago

Herbie - Herbie 0.0.10

⭐ Over 110 GitHub stars!

Wow, I had no idea so many people would find Herbie useful. Thanks everyone.

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.9...0.0.10

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk almost 3 years ago

Herbie - Herbie 0.0.9

Changelog

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.8...0.0.9

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 3 years ago

Herbie - Herbie 0.0.8

Add access to ECMWF open data forecast products

The main feature of this release is the ability to retrieve ECMWF open data forecast products (see tweet)

The big change was implementing a method to read the grib_ls-style index files to get the byte ranges for specific variables/parameters. This means that the searchString argument will need to be specified differently than that used for other models. Read more about the searchString argument for grib_ls-style index files: https://blaylockbk.github.io/Herbie/_build/html/user_guide/searchString.html#grib-ls-style-index-files

For example:

from herbie.archive import Herbie
# Create Herbie object to discover ECMWF operational forecast product
H = Herbie("2022-01-26", model="ecmwf", product="oper", fxx=12)

# Download the full grib2 file
H.download()

# Download just the 10-m u and v winds
H.download(searchString=":10(u|v):")

# Retrieve the 500 hPa temperature as an xarray.Dataset
ds = H.xarray(searchString=":t:500:")

🏹 More examples for retrieving ECMWF open data

Changelog

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.7...0.0.8

Pretty Pictures

image
image
image

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 3 years ago

Herbie - Herbie 0.0.7

Changelog

  • Default config value for priority is now None, which will make the download source priority to be the order of the SOURCE in the models template files.
  • isort all imports
  • Fixed Issue #24: Use hash labels to name subset files to make unique file names so filenames don't get too long (Pull Request #26).
  • When loading data into xarray, Herbie will parse CF grid_mapping from grib file with pygrib/pyproj/metpy. (see CF Appendix F: Grid Mapping for more info). 2789859120883e32592f5559a1e211696b58cf3e
  • Lay the groundwork for local GRIB2 files that are not downloaded from remote sources. Suppose you ran a model locally (like WRF) and have that data stored locally. Herbie can be configured to find and read those local files! This assumes an index file also exists in the same directory with .idx appended to the file name.
    • Documentation needed.
  • Added model template for rap_ncei. This is poorly implemented because the NCEI data doesn't match the other RAP model URLs.
  • #24, implemented hashed filename for subset by @blaylockbk in https://github.com/blaylockbk/Herbie/pull/26
  • Add a Gitter chat badge to README.md by @gitter-badger in https://github.com/blaylockbk/Herbie/pull/21
  • Remove Gitter chat badge. I'm not committed to chatting on Gitter; just focus on discussions.
  • Add support for NCEI historical RAP analyses by @djgagne in https://github.com/blaylockbk/Herbie/pull/30
  • Handle different idx styles by @blaylockbk in https://github.com/blaylockbk/Herbie/pull/35 (more changes in addition to those by djgagne)
  • Fixed #33 so that the herbie.tools.bulk_download will return a dict of Herbie objects that were successful and those that failed.

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.6...0.0.7

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 3 years ago

Herbie - Herbie 0.0.6

Changelog

  • #18 Use TOML as config file format instead of INI
  • Expanded setting that can be set in the configuration file
  • Adopt Black formatting
  • Moved PyPI project from hrrrb to herbie-data

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk over 3 years ago

Herbie - Herbie 0.0.5

New Name! HRRR-B 🠖 Herbie

I updated the GitHub repository name to Herbie and I'm slowly removing the old hrrrb API (but it's still there).

The most significant change is that the vision of Herbie has expanded. Herbie is being built do download many different model types, not just the HRRR model.

  • Rename package to herbie. "Herbie is your model output download assistant with a mind of its own." Yes, this is named after a favorite childhood movie series.
  • Implement new Herbie class
  • Drop support for hrrrx (experimental HRRR no longer archived on Pando and ESRL is now developing RRFS)
  • Added ability to download and read RAP model GRIB2 files.
  • Less reliance on Pando, more on aws and google.
  • New method for searchString index file search. Uses same regex search patterns as old API.
  • Filename for GRIB2 subset includes all GRIB message numbers.
  • Moved default download source to config file setting.
  • Check local file copy on init. (Don't need to look for file on remote if we have local copy)
  • Option to remove grib2 file when reading xarray if didn't already exist locally (don't clutter local disk).
  • Attach index file DataFrame to object if it exists.
  • If full file exists locally, use remote idx file to cURL local file instead of remote. (Can't create idx file locally because wgrib2 not available on windows)
  • Added GFS data, though it isn't implemented as cleanly as HRRR or RAP
  • Renamed 'field' argument to 'product'
  • ✨ Moved the source URL templates to their own classes in the models folder
  • Renamed GitHub repository to Herbie (changed from HRRR_archive_download)
  • Added RRFS, NBM, GFS, RAP as models Herbie can download
  • Reworked read_idx() to support index files with additional info (specifically for the NBM).

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk almost 4 years ago

Herbie - Herbie 0.0.4

New Herbie API

There are a few things about the hrrrb API that make it difficult to update, so I started changing things under the new name "herbie." But don't worry, both the hrrrb and herbie APIs are included. The setup.py file also is fixed.

To use the new Herbie API, refer to the documentation for some usage examples.

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk almost 4 years ago

Herbie - HRRR-B 0.0.3

Welcome to the world, HRRR-B 🎂

This is my first initial GitHub release ever! I have published on PyPi before, but this is my first here on GitHub. Certainly, a happy birthday for the HRRR-B package.

Be aware, this is v0.0.3, meaning it is subject to change at my leisure. The purpose of this repository is to serve as an example of how you can download HRRR data from archives, but I try to keep this package in a workable state that might be useful for you.

📔 Documentation

Atmosphere - Meteorological Observation and Forecast - Python
Published by blaylockbk about 4 years ago