Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

A curated list of open technology projects to sustain a stable climate, energy supply, biodiversity and natural resources.

Recent Releases of pySTEPS

pySTEPS - pysteps v1.16.0

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.15.0...v1.16.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini 27 days ago

pySTEPS - pysteps v1.15.0

What's Changed

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.14.0...v1.15.0

Natural Resources - Water Supply and Quality - Python
Published by mats-knmi about 2 months ago

pySTEPS - pysteps v1.14.0

What's Changed

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.13.0...v1.14.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini 3 months ago

pySTEPS - pysteps v1.13.0

What's Changed

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.12.0...v1.13.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini 5 months ago

pySTEPS - pysteps v1.12.0

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.11.0...v1.12.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini 7 months ago

pySTEPS - pysteps v1.11.0

This release includes developments made during the July 2024 pySTEPS-hackathon with focus on the blending module.
Many thanks to all contributors!

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.10.0...v1.11.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini 8 months ago

pySTEPS - pysteps v1.10.0

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.9.0...v1.10.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini 10 months ago

pySTEPS - pysteps v1.9.0

What's Changed

This release mainly includes developments made during the July 2023 pySTEPS-hackathon with focus on the blending module.
Many thanks to all contributors: @mpvginde, @ladc, @RubenImhoff, Simon De Kock and Ricardo Reinoso-Rondinel!
All details can be found in PR https://github.com/pySTEPS/pysteps/pull/351 by @mpvginde.

More changes:

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.8.1+002...v1.9.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 1 year ago

pySTEPS - pysteps v1.8.1

What's Changed

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.8.0...v1.8.1+002

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 1 year ago

pySTEPS - pysteps v1.8.0

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.7.4...v1.8.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 1 year ago

pySTEPS - pysteps v1.7.4

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v.1.7.3...v1.7.4

Natural Resources - Water Supply and Quality - Python
Published by dnerini almost 2 years ago

pySTEPS - pysteps v1.7.3

What's Changed

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.7.2...v.1.7.3

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 2 years ago

pySTEPS - pysteps v1.7.2

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.7.1...v1.7.2

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 2 years ago

pySTEPS - pysteps v1.7.1

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.7.0...v1.7.1

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 2 years ago

pySTEPS - pysteps v1.7.0

What's Changed

Breaking changes

  • pysteps.downscaling.rainfarm module: the keyword argument ds_factor becomes a positional argument without default value (0af3e01e57d22c80c3c41eedea35bc2f0e0a8d75)

Future warnings

  • v1.8.0: rename input arguments in pysteps.nowcasting.linda, pysteps.nowcasting.sprog, pysteps.nowcasting.steps, pysteps.nowcasting.sseps modules (c81c848888c8eab31f1868e352377969718c9316)

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.6.3...v1.7.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 2 years ago

pySTEPS - pysteps v1.6.3

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.6.2...v1.6.3

Natural Resources - Water Supply and Quality - Python
Published by dnerini almost 3 years ago

pySTEPS - pysteps v1.6.2

What's Changed

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.6.1...v1.6.2

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 3 years ago

pySTEPS - pystep v1.6.1

  • Fix duplicate link in the README
  • Fix imports of optional dependencies in pysteps.blending.utils
  • Skip tests if missing optional dependency

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 3 years ago

pySTEPS - pysteps v1.6.0

What's Changed

New features

Docs and others

Breaking changes

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.5.1...v1.6.0

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 3 years ago

pySTEPS - pysteps v1.5.1

This release includes minor fixes and documentation updates.

What's Changed

New Contributors

Full Changelog: https://github.com/pySTEPS/pysteps/compare/v1.5.0...v1.5.1

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 3 years ago

pySTEPS - pysteps v1.5.0

Pysteps v1.5.0 is now available with some exciting new features! We would like to thank everyone who contributed to this release.

Key updates

  • Add new ensemble nowcast model: Lagrangian INtegro-Difference equation model with Autoregression (LINDA) by Pulkkinen et al. (2021). [#221, gallery]
  • Implement the local Lagrangian approach for probabilistic nowcasting by Germann and Zawadzki (2004). [#207, gallery]
  • Refactor the utils.interpolate module. The rbfinterp2d method now wraps the scipy's Rbf class, while the old version is renamed to idwinterp2d (inverse distance interpolation). [#210]
  • Implement the max_num_features keyword argument for all methods in the pysteps.feature module. With this argument, the user can specify the maximum number of detected features. [#225]

Bug fixes

  • Fix a bug in utils.spectral.rapsd function causing negative frequency for evenly-sized images. [#211]
  • Fix coordinates of the bounding box in io.importers.import_mrms_grib. [#222]

Installation

You can upgrade to the latest release using pip install pysteps --upgrade or conda update pysteps -c conda-forge. It is also possible to build from source following these instructions.

Important note

This release represents our last feature release for pysteps v1, while our efforts will now focus on the upcoming v2! See #216 to learn more about this.

Contributors

Thanks to the following developers for their contributions in this release (alphabetical order):
@aperezhortal, @dnerini, @loforest, @pulkkins, @RubenImhoff

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 3 years ago

pySTEPS - pysteps v1.4.1

Summary

  • Major refactoring of the pysteps.visualization module to improve the code quality, documentation, and coverage. As a consequence, there have been few changes in the interfaces to the plotting functionalities that will produce new deprecation warnings for v1.6 [#199]
  • Fix incompatibility that appeared with scipy>=1.6 causing pysteps.nowcasts.steps.forecast to fail with argument vel_pert_method="bps" and a zero velocity field [#203]
  • Fix bug introduced in v1.4.0 concerning the usage of the netcdf exporter in pysteps.nowcasts.steps.forecast [#200]
  • Fix bug in pysteps.utils.cleansing.decluster [#194]

For a more comprehensive list of all changes please have a look at the release v1.4.1 Milestone.

Contributors

@aperezhortal @dnerini @pulkkins

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 4 years ago

pySTEPS - pysteps v1.4

Summary

  • Introducing the thunderstorm detection and tracking (DATing) modules [example] [doc1] [doc2] [#178]
  • Introducing the RainFARM (Rebora et al. 2006) module for downscaling precipitation fields [example] [doc] [#173]
  • Introducing the pysteps importers plugins [doc] [#174]
  • Improving the semi-lagrangian extrapolation method [doc] [#176] [#191]
  • Improving pysteps netCDF exporter [doc] [#175]
  • Implementing a more general importer for ODIM-compliant data sources [doc] [#171] [#172]
  • Improving plotting of basemaps [doc] [#177] [#180] [#192]

... and plenty of smaller bug fixes and general improvements. For more info please have a look at the release v1.4 Milestone.

Breaking changes

  • pysteps.extrapolation.semilagrangian
    • D_prevargument renamed to displacement_prev [#184]
  • pysteps.utils.images
    • pysteps.utils.images.shitomasi method migrated to pysteps.feature.shitomasi [#178]
    • pysteps.utils.images.blob method migrated to pysteps.feature.blob [#178]
  • pysteps.motion.lucaskande
    • pysteps.motion.lucaskanade.track_features method migrated to pysteps.tracking.lucaskanade.track_features [#178]
  • pysteps.visualization
    • map argument renamed to plot_map [#177] [#180]

Contributors

@aperezhortal @chiara-arpae @dnerini @edigiacomo @feldmann-m @jleinonen @loforest @pulkkins @RubenImhoff

Natural Resources - Water Supply and Quality - Python
Published by aperezhortal over 4 years ago

pySTEPS - pysteps v1.3.2

Summary of main changes :

  • Improve documentation
  • Fix bug in the VET padding
  • Fix visualization routines for motion fields
  • Update default pystepsrc
  • Minor code adjustments and fixes

Pysteps v1.3.2 changelog:

  • cascade
    • Compute cascade level statistics by default in the decomposition.decomposition_fft function.
  • motion
    • Fix the issue with the padding keyword in the motion.vet function to correctly handle masked and non-masked input arrays.
  • noise
    • Add a very small offset to the tapering window to avoid zero values causing some artifacts with the ssft and nested filters.
  • nowcasts
    • Set out-of-domain values to nan in sprog.forecast and steps.forecast.
    • Set negative values in anvil.forecast to zero.
    • Add option to disable localization in anvil.forecast by passing ar_window_radius=None.
    • Change default to probmatching_method='cdf' in sprog.forecast.
  • pystepsrc
    • Convert the value of pixelsize to km for knmi data.
  • verification
    • Fix error in plot.plot_rankhist caused by a different number of ticks and tick labels.
  • visualization
    • Fix interfaces in motionfields routines with respect to basemaps.plot_geography.
    • Update docstrings in motionfields.

Future Warnings:

  • v1.4.0:
    • Rename keyword argument map to plot_map for all routines in the visualization module.

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 4 years ago

pySTEPS - pysteps v1.3.1

Summary of main changes and new features:

  • Add an interface to the feature.blob_* methods from scikit-image. A blob is defined as a local maximum of a Gaussian-filtered image. This new utils.images.blob_detection feature detection method can be used in the motion.lucaskanade module to track cells in radar data.
  • Fix installation errors in windows which caused permissions errors in datasets.download_pysteps_data and datasets.download_mrms_data functions.
  • Fix bug with the precipitation mask of nowcasts.anvil. This caused the method to produce too much rain in non-rainy areas when rain rate was used as input.
  • Run Black on the whole source code for consistent, reproducible code formatting.
  • Add pre-commit hooks as part of the development workflow.
  • Improve documentation.

Pysteps v1.3.1 changelog:

  • dataset
    • Fix permissions errors in download_pysteps_data and download_mrms_data functions
    • Update installation instructions for Windows systems
  • motion.lucaskanade.dense_lucaskanade
    • Add option to use blob_detection as feature detection method
  • nowcasts.anvil
    • Implement precipitation mask when rainrate is set to None
    • Minor changes to default values, masking and thresholding
  • utils.images
    • Add blob_detection method

Natural Resources - Water Supply and Quality - Python
Published by dnerini almost 5 years ago

pySTEPS - pysteps v1.3

Summary of main changes and new features:

  • Code formatting and bug fixes.
  • Improve documentation and test coverage.
  • New deterministic nowcasting module nowcasts.anvil implementing the ANVIL method published in Pulkkinen et al. (2020): Nowcasting of Convective Rainfall Using Volumetric Radar Observations. Learn how to use it in this example script.
  • New importer io.importers.import_mrms_grib for the for NSSL’s Multi-Radar/Multi-Sensor System (MRMS).
  • New dataset module to facilitate downloading the pysteps example data and create the configuration file.
  • New interactive notebook my_first_nowcast.ipynb available on Google Colab.

Pysteps v1.3.0 changelog:

  • dataset
    • add function download_pysteps_data to download the example data from the pysteps-data GitHub repo
    • add function create_default_pystepsrc to create a default configuration file pointing to the local pysteps data directory
    • add helper functions to easily load the example data for each data source (fmi, mch, opera, etc)
  • io.importers:
    • add function import_mrms_grib to import NSSL’s Multi-Radar/Multi-Sensor System (MRMS) grib data
    • change spatial units from meters to kilometers in import_knmi_hdf5 and fix definition of y1and y2 parameters in the metadata.
    • allow all the importers to specify the output array data type (dtype keyword) and fill invalid data with a predefined value (fillna keyword)
    • add function to safely and consistently get the rain/no rain threshold
  • motion.constant
    • constantmethod returns two-dimensional advection field instead of one motion vector
  • motion.lucaskanade
    • fix bug in dense_lucaskanade handling of MaskedArrays inputs. Previously, the method ignored the input mask, unless the masked values where set as nans
    • require ndarrays or MaskedArrays as input data to dense_lucaskanade
  • motion.vet
    • fix bug in vet keywords handling. Previously, the method argument passed to the minimizer was ignored
  • nowcasts
    • add deterministic nowcasting module nowcasts.anvil
  • utils.dimensions
    • include option trim in aggregate_fields function to allow aggregation along axes that are not perfectly divisible by a given window
  • utils.images
    • rename ShiTomasi_detection function to lowercase shitomasi_detection to comply with PEP8
    • change default value for the buffer_mask argument in the shitomasi_detection function from 0 to 5 to provide a minimum buffer to no-data in the motion.lucaskanade.dense_lucaskanade method
  • utils.interpolate
    • change default value for the epsilon argument in the rbfinterp2d function from 5 to 10 to provide a more localized interpolation of motion vectors in the motion.lucaskanade.dense_lucaskanade method

Contributors to this release:
@aperezhortal @dnerini @pulkkins @RubenImhoff

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 5 years ago

pySTEPS - pysteps v1.2

Summary of main changes and new features:

  • Code formatting and bug fixes
  • Improve documentation and test coverage
  • Implementation of advanced ARI/VARI time series models, including options for localization and differencing, as well as the estimation of the parameters by OLS
  • Implementation of S-PROG and STEPS in the spectral domain for improved performance
  • New importer for EUMETSAT SAF CRRI data
  • Fix error during gcc installation in travis-ci for osx
  • Compatibility fixes for Numpy 1.18 and pyproj 2.2.0
  • Remove Basemap as optional dependency

Pysteps v1.2.0 changelog:

  • cascade.decomposition:
    • add a method for recomposing the cascade
    • add option to normalize the cascade levels
    • make computation of the cascade level statistics (mean and std) optional
    • add options to choose the input and output domains: spatial or spectral
    • in addition, implement option to use compact output in the spectral domain
    • use lowercase names for the field and mask arguments
  • cascade.interface:
    • modify get_method so that two functions are returned: one for decomposing and one for recomposing the cascade
  • extrapolation.semilagrangian:
    • add option to specify arbitrary time steps
  • motion.lucaskanade:
    • fix error when only one sparse vector is found
  • noise.fftgenerators:
    • add option to choose the output domain for the noise: spatial or spectral
  • nowcasts.sprog and nowcasts.steps:
    • add option to choose the domain for the computations: spatial or spectral
  • nowcasts.utils:
    • remove computation of mean and std from stack_cascades
    • use utils.tapering module to compute window functions
  • io.importers
    • add import_saf_crri importer
  • timeseries.autoregression:
    • add VAR models
    • add OLS estimation of AR/VAR parameters
    • add localized estimation of AR/VAR parameters
    • add differencing option to compute integrated AR/VAR methods
    • use lowercase names for variables and revise docstrings
  • timeseries.correlation:
    • add options for localization and differencing
    • add computation of correlation coefficients in the spectral domain
    • add method for the estimation of multivariate temporal autocorrelation
  • travis:
    • run brew update-reset and brew update before installing gcc-9
  • utils.spectral:
    • add methods for computing mean, standard deviation and correlation coefficient in the spectral domain
    • add option to normalize the RAPSD
  • visualizations.basemaps
    • add future deprecation warning for Basemap dependency

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 5 years ago

pySTEPS - pysteps v1.1.1

  • Code formatting and bug fixes
  • Implement GeoTIFF exporter
  • Improve existing exporters
  • Add postprocessing method to compute the band-depth of an ensemble
  • Add method to merge verification objects
  • Modify LK default parameters to increase the number of corners to track
  • Improve documentation

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 5 years ago

pySTEPS - pysteps v1.1

  • Code formatting and bug fixes
  • Refactor build system to adopt PEP517/518 standards
  • Automate pytests with tox
  • Add Proesmans optical flow method
  • Refactor Lucas-Kanade optical flow method
  • Fix bugs in vet function
  • Optimize vet function
  • Update io interface (odim_hdf5 -> opera_hdf5)
  • Add z-r parameters to MCH, FMI and KNMI importers
  • Implement FMI GeoTIFF importer
  • Add new scores:
    • Normalized MSE
    • Matthews correlation coefficient (MCC) score
    • F1 score (harmonic mean of precision and sensitivity)
  • Implement accumulation of FSS scores from multiple forecast/observation fields
  • Distinguish between type 1 and type 2 conditional biases
  • Improve documentation:
    • Release guidelines

Natural Resources - Water Supply and Quality - Python
Published by aperezhortal over 5 years ago

pySTEPS - pysteps 1.0.1

Changelog for version 1.0.1:

  • Bug fixes, code cleanups, and syntax improvements.
  • Improve documentation.
  • Implement max correlation approach for constant motion field estimation.
  • Implement new outlier detection in LK.
  • Implement new version of SSEPS nowcasting.
  • Increase code coverage.
  • Add example scripts and tutorials.

Natural Resources - Water Supply and Quality - Python
Published by dnerini almost 6 years ago

pySTEPS - pysteps 1.0.0

Changelog for version 1.0.0:

  • Change pysteps license to the less restrictive BSD-3
  • Move documentation to Read the Docs (https://pysteps.readthedocs.io/)
  • Include a gallery of examples
  • Modify setup.py to explicitly link libraries in macOS
  • Include Travis CI and Codecov
  • Add test suites for interfaces, and the io and verification modules
  • Remove legacy config module
  • Add CONTRIBUTING.rst
  • Several docstring improvements
  • Add exporter for KINEROS2 Rainfall files (io.exporters module)
  • Add importer for MeteoSwiss AZC radar composites (io.importers module)
  • Add importer for KNMI HDF5 radar composites (io.importers module)
  • Add scripts for estimating velocity perturbator parameters
  • Add modules for deriving the auto-correlation function (ACF) and estimating precipitation lifetime
  • Implement a separate module nowcasts.sprog for S-PROG nowcasts
  • Add nowcasting module nowcasts.sseps, an experimental version of STEPS that allows for localization
  • Implement interfaces for initializing and calling different FFT methods
  • Rework of the extrapolation interface
  • Faster implementation of the rank histogram with bug fixes
  • Improve subplot handling in plot_precip_field
  • Add visualization of Fourier spectra
  • Rework interfaces for verification scores
  • Add support for MaskedArray in motion.dense_lucaskanade
  • Improve masking in nowcasts.steps module

Natural Resources - Water Supply and Quality - Python
Published by dnerini about 6 years ago

pySTEPS - pysteps 0.2

Change log since the previous release:

  • Bug fixes.
  • Performance and syntax improvements.
  • Improve documentation to several modules and methods.
  • Add the Variational Echo Tracking (VET) method from MAPLE to the motion module.
  • Replace the config module with a JSON pystepsrc configuration file.
  • More efficient computation of the FFT in the cascade decomposition and noise generation by using the real FFT instead of the complex FFT, i.e. rfft2/irfft2 instead of fft2/ifft2.
  • More sensible naming of several main modules: optflow -> motion, advection -> extrapolation and postproc -> postprocessing.
  • New utility methods to clip the domain by geographical coordinates.
  • New data transformation methods: normal quantile, square-root and logarithmic transformations.
  • New importer for MeteoSwiss' metranet data format.
  • New "shift and scale" post-processing method.
  • Implement the S-PROG method with probability matching.
  • New intensity-scale verification method.
  • New visualization methods: cartopy backend, exceedance probabilities and plotting motion fields on a map.
  • Support for Swiss projection when using cartopy.
  • New interface module for different FFT methods and allow the user to choose the method.
  • Fix incorrect separation into components when using the bps method for adding perturbations to the advection field.
  • Two different methods for adjusting the temporal autocorrelation coefficients in order to guarantee stationarity of the resulting AR(2) models.
  • Modify the STEPS method to have the same number of positional arguments as the other methods in the nowcasts module and reorganize the keyword arguments.
  • Move the deterministic S-PROG mask computation outside the stochastic part of the STEPS method to avoid repeated computations.

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 6 years ago

pySTEPS - pysteps 0.1

pysteps 0.1 is our first stable release. This beta version is being verified using radar data from the Finnish Meteorological Institute and MeteoSwiss.

Features included in pysteps 0.1:

  • Support for reading various composite radar image formats (netCDF, OPERA HDF, GIF, PGM).
  • Writing of the nowcast to netCDF following CF 1.7 conventions.
  • Optical flow methods to compute the radar echo motion fields (local and spectral approaches are available).
  • A semi-Lagrangian advection scheme to extrapolate the radar fields.
  • Spatial random field generators to create realistic correlated perturbation fields.
  • Autoregressive models to impose temporal auto-correlations.
  • Methods to generate deterministic and ensemble precipitation nowcasts.
  • Support for parallelization using Dask.
  • Scale-decomposition of precipitation fields based on the STEPS framework.
  • Verification of deterministic and spatial forecasts: RMSE, MAE, correlation, CSI, ETS, HSS, Fractions Skill Score, etc.
  • Verification of ensemble and probabilistic forecasts: reliability diagram, rank histograms, ROC curves, CRPS, ensemble spread-skill.
  • Utilities to convert, transform and aggregate precipitation fields.
  • Visualization methods to plot and animate radar rainfall fields, motion fields and ensemble nowcasts. Support for Basemap.
  • Example scripts to quickly introduce the users to the working of various modules.

Natural Resources - Water Supply and Quality - Python
Published by dnerini over 6 years ago