What's Changed

date: 2024-03

Breaking Changes Summary

New package architecture.
New toolchain with flit and ruff.
clt standard variable uint changed.
clone method added to Indicators.

Changes Details

[maint] BREAKING CHANGE
Created a _core package which should not be used outside icclim source.
Its content may change without deprecation warnings.
[enh] Added DCSC indices under icclim.dcsc namespace.
[doc] Adapt Christian's notebooks from ISENES and add them as
tutorials in our documentation.
[doc] Add nbshpinx extension to render jupyter notebooks in
the online documentation.
[maint] Migrate from [black, blackdoc, flake8, isort, pyupgrade,
pydocstyle] to ruff
[maint] Migrate from setup.py to pyproject.toml
[maint] Make readthedocs build fail when there are warnings
[maint] Fix warnings in doc build
[maint] BREAKING CHANGE
Update architecture to have a src/ and a tests/ directory at root level
[maint] BREAKING CHANGE
Migrate build toolchain from setuptools to flit
[maint] Remove version number from constants module as it was
causing the build process to import icclim. The version number is now
statically set in src/icclim/init.py
[fix] Force xarray to read dataset sequentially to avoid a netcdf-c
threading issue causing seg faults.
[enh] Add publish-to-pypi.yml github action to automatically build
and publish icclim to pypi. This action is triggered by a github
release being published. This action requires a manual approval on
github.
[enh] Add the following ECAD indices: PP (average of pressure),
SS (sum of sunshine) and RH (average of humidity).
[fix] BREAKING CHANGE
the default unit of clt standard variable is now % as expected
(was a wind strenght unit).
[maint] BREAKING CHANGE
Rework architecture to have a _core private package
containing the core logic of icclim. Idea taken from numpy 2.0.
[doc] Add docstring to almost all functions, classes and modules.
This has been done to improve the generated documentation.
co-authored: Github's copilot.
[maint] Add missing type hints.
[doc] Make use of readthedocs' autoapi library to generate the API
documentation.
[maint] Improve maintainability of tools/extract-icclim-funs.py to
ease adding new registries (still not perfect).
[fix] Add clone method to Indicators to avoid modifying the original instance
when setting templating metadata.

Full Changelog: https://github.com/cerfacs-globc/icclim/compare/v6.5.0...v7.0.0

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah about 1 year ago

Breaking changes

Pin xclim between 0.45 and 0.47
[#281] Adapt icclim to xclim 0.45 by updating excess and deficit generic functions
Drop support for python 3.8

Fixes

[#273] Avoid resampling to compute time_bounds on SPI3 and SPI6 indices
[#256] Fix unit conversion issue for fraction_of_total generic indices (used by RxxpTOT family of indices)

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah over 1 year ago

icclim - v6.4.0

Minor release to support latest xclim version (0.43.x).

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah almost 2 years ago

BREAKING CHANGES

[fix] The indicators based on the difference between two variables (ecad: DTR, ETR, vDTR and anomaly) gave wrong values due to a bad unit conversion of the output. This was for example the case when the input variables are in Kelvin, the difference between the two variables is still in Kelvin but it cannot be converted to degree Celsius with the ususal +273.15. To workaround this issue, we first convert inputs to the expected output unit and then we compute the index.
[fix] Indices based on both a percentile threshold and a threshold_min_value (for ecad: r75p, r75pTOT, r95p, r95pTOT, r99p, r99pTOT) are now computing the exceedance rate on values above threshold_min_value as well. Previously this threshold_min_value was used to compute the percentile and the total (for rxxpTOT indices) but not the exceedance rate.
[maint] The only compatible version of icclim 6.3.0 is xclim 0.42

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah about 2 years ago

icclim - 6.2.0

Maintenance release to update icclim according to latest changes on xclim 0.40:

Moved PercentileDataArray from xclim to icclim.
Adapted the unit conversion to use the hydro context.

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah about 2 years ago

icclim - v6.1.5

icclim v6.1.5 is a small release again, to fix a problem assuming datasets having latitude and longitude not as lat and lon but as the full names, when creating the output file metadata.

Bug fix

Supports either lat or latitude, and lon or longitude, in put datasets.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 2 years ago

icclim - v6.1.3

icclim v6.1.3 is a small release to fix TnX.

Bug fix

TnX was calculating the minimum instead of the maximum.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 2 years ago

icclim - v6.1.2

icclim v6.1.2 brings new ECAD wind indices as well as new humidity-based indices.

Enhancements

ECAD wind indices were added: fxx, fg6bft, fgcalm, fg, ddnorth, ddeast, ddsouth, ddwest. Also, ddnorth and ddsouth were added but do not follow the ECAD's ATBD v11 requirements as their definition seems to be wrong in the document.[0]
Added ECAD's indices GSL, SPI3, SPI6.

Documentation

Added documentation for generic indicators stand-alone functions.
Added a recipe "how to" documentation for generic indicators.

Internal changes

Added generic indicators as stand-alone functions in icclim namespace.
Upgraded to xclim 0.39.0

[0] ATBD: https://knmi-ecad-assets-prd.s3.amazonaws.com/documents/atbd.pdf

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 2 years ago

icclim - v6.0.0

icclim v6.0.0 is a major release introducing the concept of generic indices.
Generic indices allow users to create their own indices and still benefit from:

icclim machinery to parse inputs
metadata generation
the actual numerical computation of the index

The metadata generation is based on jinja2 templates. These templates may be adjust in future releases once we get some feedback on their outputs.
Generic indices aim to be easy to use, to improve and to extend. They brings 2 concepts together:

An abstract threshold such as >= 25 degree_Celsius or >= 99 doy_per AND >= 30 degree_Celsius. These are mere examples of threshold capabilities, see Threshold documentation for more examples.
A catalog of generic operators such as count_occurrences, max_consecutive_occurrence, max_of_rolling_sum, that can be passed to index_name argument of icclim.index. A list of all possible operator will be available soon in the documentation.

Generic Index example

precip_threshold = icclim.models.threshold.build_threshold(">= 50 period_per", threshold_min_value="1 mmday")
result_dataset = icclim.index("precipitation.nc",
                             index_name="max_of_rolling_sum",
                             var_name=["precip"],
                             threshold=precip_threshold).compute()

Breaking changes

ECAD indices are no longer configurable. This choice was made to ensure that ECAD's indices truly follow the ECAD's ATBD specification[0]
To create your own index you should now use generic indices.
icclim no longer rely directly on xclim to compute indices. Instead we use xclim as a tool box for features such as units handling, calendar operations, percentiles computations etc.
The indices [CW, CD, WW, WD] were not properly computed. The computation of precipitation percentiles now use "period percentiles" instead of "doy percentiles" as the ECAD definition suggest.
clipped_season option for the slice_mode parameter has been removed.
ECAD snow indices now expect a snow (snd) variable.
Convert input data that are recognized as a precipitation amount into precipitation rate to handle e-obs dataset

Deprecation

Custom indices (a.k.a user indices) are now deprecated in favor of generic indices. For now, they are mapped to their generic indices counterpart, but they could be removed in the future.

Enhancements

Generic indices
The RR precipitation index has been added.
A dictionary can be used for in_files to describe input configuration per variable:

icclim.index(in_files={
  "tasmax": {
                "study": "tasmax-store.zarr",
                "threshold": build_threshold(operator=">", ["per-1.nc", "per-2.nc"]),
                 },
  "pr": "pr.nc"
}

Improve icclim.indices to make it possible to compute every ecad indices based on a variable name (e.g. "tasmax").

Internal changes

Upgrade to xclim 0.38 and to xarray 2022.6.
icclim no longer rely on clix-meta yaml file. We will still keep a close eye to their recommendation for the specification of climate indices.
Add BlackDoc to C.I
Add icclim logo

[0] ATBD: https://knmi-ecad-assets-prd.s3.amazonaws.com/documents/atbd.pdf

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah over 2 years ago

icclim - v5.4.0

icclim v5.4 is a small summer release with a few fixes.

What's Changed

[fix] When giving input as a list of netcdf files, the coordinate values are now merged using the override strategy.
Hence, the first file having a given dimension define this dimension for all the subsequent files.
[fix] Fixed the output unit of some indices (from "ºC" to the standard "degree_Celsius")
[fix] Fixed issue where dataset having a time bounds variable could not be processed (due to chunking). We now chunk only the studied variables thus the DataArray(s) instead of the whole Dataset.

Full Changelog: https://github.com/cerfacs-globc/icclim/compare/v5.3.0...v5.4.0

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah over 2 years ago

icclim - v5.3.0

icclim v5.3.0 is a important release with many changes.
Overall it improves icclim outputted metadata, improves slice_mode API with new modes, eases the use of dates parameter and adds the possibility to give percentiles as an input instead of a computing them.

Changelog

[enh] Add icclim version to history in outputted metadata.
[maint] breaking change Pin minimal pandas version to 1.3 to have the fix for https://github.com/pandas-dev/pandas/issues/24539
[enh] slice_mode: seasons can now be defined to be between two exact dates.
[enh] slice_mode type can now be tuple[str, list], it works similarly to the list in input of seasons but, it enforces a length of 2.
[enh] slice_mode: Added clipped_season keyword which ignores events starting before the season bounds (original behavior of season).
[maint] slice_mode: Modified season keyword to take into account events (such as in CDD) starting before the season bounds.
This should improve the scientific validity of these seasonal computations. Plus it is in accordance to xclim way of doing this.
[maint] Added dataclass ClimateIndex to ease the introduction of new indices not in the ECAD standard.
[maint] Made use the new typing syntax thanks to from __future__ import annotations.
[maint] Add docstring validation into flake8 checks.
[enh] Improve API for date related parameters {time_range, base_period_time_range, ref_time_range}
They can still be filled with a datetime object but additionally various string format are now available.
This comes with dateparser library.
[doc] Update callback doc as its outputted value is very inaccurate when dask is enable.
[enh] T(X/N/G)(10/90)p indices threshold is now configurable with threshold parameter.
Example of use: icclim.tx90p(in_files=data, threshold=[42, 99])
[enh|maint] threshold, history and source metadata have been updated to better describe what happens during icclim process.
[fix/doc] The documentation of the generated API for T(X/N/G)(10/90)p indices now properly use thier ECAD definitions instead of those from ETCCDI.
[enh/doc] Add [WSDI, CSDI, rxxp, rxxpTOT, CW, CD, WW, WD] indices in yaml definition.
Note: We no longer strictly follow the yaml given by clix-meta.
[fix] custom seasonal slice_mode was broken when it ended in december. It's now fixed and unit tested.
[enh] Make in_file accept a dictionary merging together var_name and in_file features.
[enh] in_file dictionary can now be used to pass percentiles thresholds. These thresholds will be used instead of computing them on relevant indices.
[maint/internal] Refactored IndexConfig and moved all the logic to input_parsing.
[enh] R(75/95/99)P and R(75/95/99)PTOT indices threshold is now configurable with threshold parameter.
Example of use: icclim.r75p(in_files=data, threshold=60)
[enh] Update metadata of indices when a custom threshold is used.
BREAKING CHANGE: The outputted variable name now also contains the thresholds (e.g. SU_35_42 for index_name="SU" and threshold=[35, 42])
[maint] Rename IndexConfig::threshold in IndexConfig::scalar_thresholds to avoid confusion with the thresholds given in in_files.

Diff: https://github.com/cerfacs-globc/icclim/compare/5.2.1...v5.3.0

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah almost 3 years ago

icclim - v5.2.2

Small release to update dependencies constraints:

numpy < 1.22 is no longer needed.
xclim >= 0.37 is now needed.

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah almost 3 years ago

icclim - v5.2.1

This 5.2.1 release supersedes 5.2.0 which had unpolished finishes.
As usual the package is available on pypi and conda-forge.

Breaking changes

None

Bug fixes

slice_mode seasonal samplings was giving wrong results for quite a few indices. This has been fixed and the performances should also be improved by the fix.
However, now seasonal slice_mode does not allow to use xclim missing values mechanisms.
user_index ExtremeMode config was not properly parsed when a string was used.
user_index Anomaly operator was not properly using the ref_time_range to setup a reference period as it should.
user_index Sum and Mean operators were broken due to a previous refactoring and a lack of unit tests, it is now fixed and tested.
create_optimized_zarr_store would throw an error when creating the first temp store if the chunks were not unified.

Internal changes

Made Frequency part of SliceMode union.
Changed how rechunker dependency is pinned to add flexibility. We want a version above '0.3' but not the '0.4'.
For the newly generate API, on custom_index function, the parameter user_index is now mandatory.
Update release process.

Enhancements

Improve create_optimized_zarr_store to accept a chunking schema instead of a single dim.
Make use of fsspec to generalize the storages where create_optimized_zarr_store can create its zarr stores.
Make CSDI and WSDI threshold configurable using the threshold parameter of icclim.index.
Add a function in icclim namespace for each ECA&D index for convenience.
Improved documentation about chunking.
Improved dev documentation display on readthedocs.

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah almost 3 years ago

icclim - v5.1.0

Breaking changes

Parameter out_file default value is now None. When it is None, icclim.index does not write the output on a netCDF.
Minimal required xclim version is now 0.34
Pinned dask dependency: dask<2022.01.1 for rechunker 0.3.3 to work.

API changes

Added icclim.indices to compute multiple indices at once.
Added experimental feature icclim.create_optimized_zarr_store. It is a context manager which can be used to rechunk a dataset using without any chunk on the specified dimension. It makes use of rechunker[0] library.

Bug fixes

Fixed WSDI and CSDI. They were computing percentiles on the wrong period.
Fixed WW, WD, CW, CD. Their binding with xclim was not properly made and was throwing an error.
Fixed setup.py dependency to the whole world. This make installation from sources possible in a clean environment.

Maintenance

Improved icclim CI (code coverage)
Improved code quality (unit test, refactoring)
Added contribution guide
Improve overall typing

[0] https://github.com/pangeo-data/rechunker/

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah about 3 years ago

icclim - v5.0.2

Official release of icclim 5.0.2 (a.k.a icclim v5)
For full details of the changes see https://icclim.readthedocs.io/en/latest/references/release_notes.html

In short, icclim has be rewritten to make use of xclim, xarray and dask libraries.
The scope is unchanged, icclim still computes 49 ECA&D climate indices and provides an API to create custom indices.

icclim 5.0.2 is available on both pip and conda-forge.

Climate Change - Climate Data Processing and Analysis - Python
Published by bzah about 3 years ago

icclim - 4.2.20

Removed utime since cftime to not support anymore this deprecated function.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 3 years ago

icclim - 4.2.19

Fixed bug in calc_ind for call to CWD.
Properly referencing version.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 3 years ago

icclim - 4.2.18

Fixed bugs related to user indices.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp almost 4 years ago

icclim - 4.2.17

Many fixes to setup better all import statements.
Removed support for python 2.7.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp about 4 years ago

icclim - 4.2.16

Important bug fixes related to handling of specific calendars related to leap years for percentile calculations.
Those bugs were producing an out-of-bound error for some specific calendars and start year.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp about 4 years ago

icclim - 4.2.15

Moved from netcdftime to cftime.
Fixed some python 3.8 compatibility bugs.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 4 years ago

icclim - 4.2.14

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 4 years ago

icclim - 4.2.13

New 4.2.13 release.
Many bugs fixed, large performance improvements for percentiles calculations with bootstrapping.
Full python 3 compatibility.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp about 6 years ago

icclim - 4.2.12

Critical fix related to netcdftime. Changes in netcdftime made icclim crash. Modifications in util_dt.py to take into account those changes and remove the crash.

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp over 6 years ago

icclim - 4.2.11

Print with parentheses

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp about 7 years ago

icclim - 4.2.10

Python 3 import fixes
MaskArray and Bootstrapping crash fixes

Climate Change - Climate Data Processing and Analysis - Python
Published by pagecp about 7 years ago