Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

A curated list of open technology projects to sustain a stable climate, energy supply, biodiversity and natural resources.

Recent Releases of DeepSensor

DeepSensor - v0.4.2

  • Minor fix to avoid pandas 3.0 FutureWarning

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.4.1...v0.4.2

Climate Change - Climate Downscaling - Python
Published by tom-andersson 6 months ago

DeepSensor - v0.4.1

  • Rearrange forecast dimensions from (lead_time, init_time) to (init_time, lead_time)
  • Raise error about deepsensor.plot.prediction not supporting forecasts

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.4.0...v0.4.1

Climate Change - Climate Downscaling - Python
Published by tom-andersson 6 months ago

DeepSensor - v0.4.0

What's Changed

Breaking changes

  • model.predict for models trained with target_delta_t > 0 now return Prediction objects with init_time and lead_time dimensions with a time (valid time) coordinate, rather than a single time dimension (which used to correspond to forecast initialisation time).

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.8...v0.4.0

Climate Change - Climate Downscaling - Python
Published by tom-andersson 6 months ago

DeepSensor - v0.3.8

What's changed

  • Add new DataProcessor normalisation method (`"positive_semidefinite") compatible with a Bernoulli-Gamma ConvCNP likelihood
  • Expose k and scale parameters of "bernoulli-gamma" ConvNP likelihood in low-level and high-level prediction interface

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.7...v0.3.8

Climate Change - Climate Downscaling - Python
Published by tom-andersson 9 months ago

DeepSensor - v0.3.7

What's Changed

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.6...v0.3.7

Climate Change - Climate Downscaling - Python
Published by tom-andersson 10 months ago

DeepSensor - v0.3.6

What's Changed

New Contributors

Full Changelog: https://github.com/alan-turing-institute/deepsensor/compare/v0.3.5...v0.3.6

Climate Change - Climate Downscaling - Python
Published by tom-andersson about 1 year ago

DeepSensor - v0.3.5

What's changed

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.4...v0.3.5

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.3.4

  • Fix PyPI upload by removing get-station-data dependency; instead raise error that it must be installed manually if user calls the function that uses it

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.3...v0.3.4

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.3.3

What's Changed

  • New documentation by @tom-andersson and @kallewesterling, containing steps for getting started, a user guide, learning resources, a list of research ideas, community information, and more. Uses the Jupyter Book system with fully reproducible notebooks demoing features of the package.
  • New deepsensor.data.sources module for downloading ERA5, GHCNd, topography, and land mask data directly into DeepSensor xarray/pandas format. Results can be cached to disk to avoid re-downloading. Leverages get-station-data tool by @scotthosking, @magnusross, and @tom-andersson.
  • New plotting tools: deepsensor.plot.prediction and deepsensor.plot.task.
  • Various bug fixes.

Breaking changes

  • The ConvNP hyperparameter points_per_unit has been renamed to internal_density to be more intuitive. Instances of points_per_unit in code and model JSON configuration files will need to be renamed to internal_density.

New Contributors

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.2...v0.3.3

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.3.2

  • Fix bug in ContextDist acquisition function (thanks @acocac)

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.3.1

What's Changed

  • Fix bug in ContextDist acquisition function (thanks @acocac)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.3.0...v0.3.1

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.3.0

What's Changed

Breaking changes

  • Replace any mean_ds, pred_ds, samples_ds = model.predict(...) with pred = model.predict(...). Prediction pandas/xarray objects can now be accessed with pred["<var_ID>"].
  • Replace any remove_nans_from_task_Y_t_if_present(task) with task.remove_target_nans()
  • DataProcessor now auto-normalises coordinates differently to preserve aspect ratio - see https://github.com/tom-andersson/deepsensor/issues/77. If you aren't specifying spatial coordinate normalisation mapping explicitly (through x1_map/x2_map or through config), your trained models may start receiving different data.

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.5...v0.3.0

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.2.5

What's Changed

  • Fix incompatibility with Python < 3.10 (thanks @patel-zeel)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.4...v0.2.5

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.2.4

What's Changed

  • Modularise and track Task operations (eg adding batch dim, removing nans, converting to tensor).
  • Fix bug in concat_tasks when there are NaNs in target data (thanks @nilsleh!)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.3...v0.2.4

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.2.3

What's Changed

  • Bug fixes

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.2...v0.2.3

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.2.2

What's Changed

  • Add X_t_mask kwarg to DeepSensorModel.predict when predicting on-grid (model will not predict at False/0s and will be returned as NaNs in prediction xarray objects)
  • Minor bug fixes

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.1...v0.2.2

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.2.1

What's changed

  • Saving/loading bugfixes

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.2.0...v0.2.1

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.2.0

What's Changed

  • DataProcessor, TaskLoader, and ConvNP now each have a .save(folder) method and can be re-initialised with .__init__(folder)
  • DataProcessor init kwargs x1_map and x2_map now optional (if not provided, will be auto-inferred from first dataset passed to .__call__)
  • Convert normalization parameters to a dictionary for data processor by @patel-zeel in https://github.com/tom-andersson/deepsensor/pull/48

New Contributors

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.11...v0.2.0

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.11

What's changed

  • Add Trainer class that tracks optimiser object to speed up training
  • New diff boolean kwarg for GreedyAlgorithm.__call__ to compute difference after adding query context point

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.10...v0.1.11

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.10

What's changed

  • Minor bugfix in TaskLoader aux_at_contexts functionality

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.9...v0.1.10

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.9

What's new

  • Fix bug in TaskLoader.load_dask (thanks @magnusross)
  • Handle dates with no pairs of stations in compute_pandas_data_resolution

Breaking changes

  • The TaskLoader.__init__ aux_at_contexts arg is no longer a tuple of (int, xarray.DataArray), and is now just an xarray object that will be automatically sampled at the off-grid context locations from all context sets.

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.8...v0.1.9

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.8

What's new

  • Compute analogue of resolution for scattered pandas data for inferring default ConvNP internal grid resolution
  • Allow sampling auxiliary data at locations of another context set in TaskLoader with aux_at_contexts kwarg in TaskLoader.__init__
  • deepsensor.plot.offgrid_context_observations utility to plot context observation numerical values
  • Fix bug in concatenating/batching Task objects with gridded data (thanks @patel-zeel)
  • Support active learning with aux_at_targets downscaling functionality by @RohitRathore1 in https://github.com/tom-andersson/deepsensor/pull/35
  • Unified (single) progress bar during active learning with GreedyAlgorithm

Breaking changes

  • Update DataProcessor config to store user-provided normalisation method for each variable

Contributors

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.7...v0.1.8

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.7

  • Minor bugfixes in deepsensor.active_learning.acquisition_fn

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.6...v0.1.7

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.6

  • Downscaling using auxiliary output MLP
  • Assign maximisation/minimisation attr to each AcquisitionFunction so the user doesn't need to decide (thanks @jonas-scholz123 @polpel)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.5...v0.1.6

Climate Change - Climate Downscaling - Python
Published by tom-andersson over 1 year ago

DeepSensor - v0.1.5

What's Changed

  • Fix rounding errors in DeepSensorModel.predict coordinates from normalise-unnormalise operations by @tom-andersson and @polpel in https://github.com/tom-andersson/deepsensor/pull/25
  • Support autoregressive (AR) sampling in DeepSensorModel.predict
  • Support training with multiple non-overapping targets
  • Support NaNs in context and target data for ConvNP
  • deepsensor.active_learning enhancements
  • Support numpy coordinates in TaskLoader context/target and DeepSensorModel.predict targets
  • Switch default random sampling behaviour for xarray data to use linear interpolation rather than grid-cell wise to avoid prediction artifacts away from grid cells (thanks @jonas-scholz123)

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.4...v0.1.5

Climate Change - Climate Downscaling - Python
Published by tom-andersson almost 2 years ago

DeepSensor - v0.1.4

  • Fix decoder scale not being inferred from model discretisation density (ppu). Closes https://github.com/tom-andersson/deepsensor/issues/18
  • Add deepsensor.plot.feature_maps method
  • Provisional deepsensor.active_learning functionality with acquisition functions and greedy algorithm

Breaking changes

  • ConvNP class moved from deepsensor.model.models to deepsensor.model.convnp

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.3...v0.1.4

Climate Change - Climate Downscaling - Python
Published by tom-andersson almost 2 years ago

DeepSensor - v0.1.3

Try to trigger PyPI upload with publisher set up on PyPI

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.2...v0.1.3

Climate Change - Climate Downscaling - Python
Published by tom-andersson almost 2 years ago

DeepSensor - v0.1.2

What's New

  • Breaking change: Plotting module moved and methods renamed for nicer-looking imports. E.g. now deepsensor.plot.plot.plot_context_encoding is deepsensor.plot.context_encoding.
  • Added deepsensor.plot.receptive_field method for plotting model's RF in unnormalised space with cartopy coastlines.

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.1...v0.1.2

Climate Change - Climate Downscaling - Python
Published by tom-andersson almost 2 years ago

DeepSensor - v0.1.1

What's New

  • Published on PyPI
  • ConvNP class defaults to CNP model
  • More intuitive names for ConvCNP or ConvCNP: ConvNP(..., likelihood="cnp") and ConvNP(..., likelihood="gnp")

Full Changelog: https://github.com/tom-andersson/deepsensor/compare/v0.1.0...v0.1.1

Climate Change - Climate Downscaling - Python
Published by tom-andersson almost 2 years ago

DeepSensor - v0.1.0

Pre-release with basic DeepSensor functionality for data processing, task loading, and neural process training/inference.

What's New

  • DataProcessor for normalising xarray and pandas data + standardising coordinates
  • TaskLoader for loading neural process meta-learning tasks from xarray and/or pandas data, outputting Task objects
  • TaskLoader.__call__ provides sampling schemes for generating context and target sets. Options:
    • random sampling (xarray/pandas),
    • passing all observations (xarray/pandas),
    • randomly splitting into context & target (pandas only).
  • ProbabilisticModel class providing blueprint for generic model interface
  • DeepSensorModel(ProbabilisticModel) class for outputting unnormalised model predictions in xarray (grid) or pandas (off-grid)
  • ConvNP(DeepSensorModel) model class wrapping around neuralprocesses (https://github.com/wesselb/neuralprocesses) for convolutional neural process modelling
  • train_epoch method implementing simple training scheme on a list of Tasks

Contributors

Full Changelog: https://github.com/tom-andersson/deepsensor/commits/v0.1.0

Climate Change - Climate Downscaling - Python
Published by tom-andersson almost 2 years ago