TopoPyScale - New CDS API and more

This is a lone overdue release as new features and typos/bus have been corrected. This new release now supports the new CDS API for ERA5 data download.

WARNING: There may be some compatibility issues with workflows based on previous release.

New tool to fetch Copernicus DEM [19abe84] with paralleisation [1d834bd]
improvement of path directory code [72dcd3e]
debug of masking [3713f83]
function to export to FSM2oshd [1ed7b6d]
correction of a bug when elevation was lower than the lower pressure level [e133c6e]
adding compatibility with IFS data [134f7ce] and script to feth IFS data [69c9845]
implementation of an export function to FSM2 model [749a0e3]
improvements in the routine for forecast [cd75c5a]
added rioxarray to requirements [de8ae0c]
added export function for SURFEX model [f5ad251]
added explicit get_era5() function in workflow [7b35bcd]
fixed typos and bug in topo_sim.py [327fe4f] et al.
added dask chuncking to stop memory leaks [fd2adc4]
output format of the ERA5 data is now dynamic and not hard coded to netcdf [3cb0b49]. now support for
update of new variable keywords of CDS-beta [ef13238]
replaced raise warning ... by simple prints as it led to crash [f8da401]
new feature to download ERA5 from zarr repositories [3eb160d]
added possibilities to downscale at other timestep than 1h only [d77d988]

Climate Change - Climate Downscaling - Python
Published by ArcticSnow 3 months ago

TopoPyScale - addressing multiple typos

This release includes a working version for exporting to FSM2ohd, and a number of other correction in the code. These are minor typos that led to inconsistencies.

Climate Change - Climate Downscaling - Python
Published by ArcticSnow about 1 year ago

TopoPyScale - FSM2oshd support & Arbitray domain mask

New release bringing latest development of new features into TopoPyScale:

Support to export to FSM2oshd format 3fe1e1a7bded57e282a8b35c0960881e5c6417a8
added support for masking out domain or split domain in independent regions 81298bfa8714172cf61cfbb33bdc4c456523009f
added support for custom outputs directory 951451e3e59c3cd633460ea923dd980ec803e338
rename files containign FSM simulation routines to sim_fsm.py
added support for C-implementation of solar position in pvlib library 074710a2e586d3bceadb78def108edca4e000dce
point_id can be anything from int, float to string 1a32a5d9a3003103928a243b89997613a53258b1

and a number of minor bug fixes

Climate Change - Climate Downscaling - Python
Published by ArcticSnow over 1 year ago

TopoPyScale - v0.2.3 JOSS release.

This new version of the package is justified by the release of the JOSS publication following the review process. In the meantime a number of features have been added.

support for SLURM server type
realtime processing, downloading latest ERA5 data. For operational setup c0fb49fff0f106bb39601f77b6b5a120901c18f0
git commit hash and package version added to netcdf file metadata. This ensure to know which version of TopoPyScale was used to create data.
climate input can now be located outside of the project directory. This prevent to copy multiple times the same data. 77fd990500bfe6934ad6997d70acb106fd44b6d1
Outputs directory can be named independently of default
export FSM as netcdf
streamlined the environment.yml file 5af8c8dde5c0d8ca08c9c03de7dafa85a4c543e2
fix Dask indexing: 4b00cd60bffe6fff5fa16e8f16c95b822c5238a4
option to give specifc name to cluster points and not depend on the default point_id 7695450864e3b9f978c6e22c7c5fda800941d011
function to export griddedata 420f80a55110b31abd4edede7c69fb30e608375d
update xarray functions that were deprecated 3662908f635a6c7565a6c9217934b98a09b78b1d
possibility to run FSM in parallel mode 91c8528db69499f5041f9849f739b3253b7f1d25
added snow routine for steep slope 4d31fd1a2f9c628bd2a208e3862e449ab5af847a

Climate Change - Climate Downscaling - Python
Published by ArcticSnow almost 2 years ago

TopoPyScale - Update to Python 3.9 and latest `xarray`

This release includes:

upgrade to Python 3.9 and Xarray 2.3
no more problem of memory byt using open_mfdataset
new testing script with github action
added units to default downscaled dataset
n_core option is less restrictive now.
solved a number of path issues
a number of improvements of the FSM simulation topo_sim.py routines
corrected projection issues when interpolating the climate dataset
to_cryogrid() now compatible with Cryogrid.jl
added an environment.yml file for version controlled install of VE
plotting function map_variable() has been generalized
and a number of bugs have been caught along the way.

Climate Change - Climate Downscaling - Python
Published by ArcticSnow almost 2 years ago

TopoPyScale - Another release for less bugs

New release resolving a number of bugs:

a crucial bug occurring when storing datasets during downscalng in topo_scale.py. More details in issues: https://github.com/ArcticSnow/TopoPyScale/issues/60 and commit https://github.com/ArcticSnow/TopoPyScale/commit/a33e79f77bd32fd6952c4a269024a31a9a584dae
a bug resolved in to_crocus() https://github.com/ArcticSnow/TopoPyScale/commit/4e73489e85bc567d897752f6d8cc7db0eb6c3011
a bug when computing level elevation from geopotential https://github.com/ArcticSnow/TopoPyScale/commit/6a4c68e9a0e2f9f1c66427924e75d8c126fbd963
a bug in to_musa() https://github.com/ArcticSnow/TopoPyScale/commit/81e8978cb152a7b908e53a20eea90853b63aaa18

Climate Change - Climate Downscaling - Python
Published by ArcticSnow about 2 years ago

TopoPyScale - Parallelization is up and running :)

TopoPyScale is improving and being tested by more users. New features, improvements and bug fixes justify a new release. The whole downscaling process gain speed by spreading the load on multicores. This was tested on laptops and servers.

WARNING: This new release requires Python >= 3.8. Otherwise, python runs into a memory leak and crashes.

New features:

multicore, paralleization of topo_scale
tools to explore optimal number of clusters in topo_sub
implementation of feature importance for clustering. New field in config file needed
added precipitation lapse-rate option, commit

Improvements:

simplification of path management using the library pathlib
remove all usage of open_mfdataset() or to_mfdataset that are not as efficient as expected
simplification of tmp files during downscaling. 85a4149137da173b392d5ceb336f6c18e4877ed9
added error message in case upper pressure level is not high enough

Bug fixes:

to_netcdf() default case bug fixed
plotting routines for variable mapping fixed
fixed bug in time-batch handling of files
bug in fetch_WMO() fixed

Climate Change - Climate Downscaling - Python
Published by ArcticSnow about 2 years ago

TopoPyScale - pypi install

This release includes a new options in setup.py that the package can be installed in a single line using Pypi

pip install topopyscale