The Canadian Hydrological Model - 1.5.4

• Update compatibility with Intel 2025.x
• Update docs for checkpointing and fsm
• Code clean up
• Try to fix a possible case that open ugrid files at the end of a run could stop CHM from cleanly terminating
• Remove google test submodule
• Disable vtp output of forcing points -- only geojson are written

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.3...1.5.4

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 8 months ago

The Canadian Hydrological Model - 1.5.3

• This release improves the handling of stations with multiple expansions
• Some confusing warning messages are cleaned up
• Fix the path in the PVD file creation to the vtus
• new version of bbhash

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.2...1.5.3

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 8 months ago

The Canadian Hydrological Model - 1.5.2

• Enforce that multiple netcdf input variables do not have duplicate standard_name attributes
• Ensure that string type long_names do not cause a premature exit to looking for a standard_name
• Fix a nc collective
• Chunk ugrid output per rank for performance increase
• add start_index attr

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.1...1.5.2

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 9 months ago

The Canadian Hydrological Model - 1.5.1

Try to ensure that no openmp usage is enabled if one of our dependents has openmp enabled

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 9 months ago

The Canadian Hydrological Model - 1.5.0

I think all the outstanding bugs from the major internal additions are resolved. Any further issues will be dealt with in point releases.

What's Changed

Docs

• Docs - Build instructions - fix typos by @jomey in https://github.com/Chrismarsh/CHM/pull/149
• update help documentation and runtime errors to further clarify the updated config changes

Output

• Write to ugrid netcdf files directly
• Ugrid output now defaults to shuffle+deflate level5 + lossy bitgroom (4 Sig figs precision). This results in a massive 5x-7x reduction in file size.
• std::format has been replaced with fmt::format to better support gcc@9+ as std::format was only added in gcc@13
• Fix compatibility with newer spdlog/fmt combos
• Removes output.mesh in favour of output.vtu and output.ugrid
• Defaults to the vtu/pvd and ugrid files being named the same as the output folder. This should make dealing with the outputs of multiple experiments more easily done
• Allow for checkpointing at arbitrary timesteps

Processes

• More robustly bound the Ti root in harder_precip_phase and PBSM to ensure convergence at more negative air temperatures
• Allow expansion of search for forcing input to be up to 250%
• Add scale_factor to Windninja to provide a knob to turn for highly biased input windspeeds

Misc

• enforce cmake 3.30

Met

• requires CF-compliant standard_name attributes to dynamically determine variables
• support multipart netcdf files (e.g., one nc file per day)
• the grid of each multipart does not need to remain constant, allowing for trivially changing the resolution or product used at different timesteps, for example to switch from a short term 2.5km forcing to a coarser long-term 10km forcing
• Basic unit conversion for netcdf files (needs further work and more robust error handling)
• output geojson of mesh bounding box and stations to debug any issues when using coarse met forcing
  
• Rename station_N_nearest to num_stations_to_use
• Move num_stations_to_use and interpolation to forcing section

New Contributors

• @jomey made their first contribution in https://github.com/Chrismarsh/CHM/pull/149

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.4.5...1.5.0

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 9 months ago

The Canadian Hydrological Model - 1.5.0b20

• Fix incorrect removal of ugrid x,y in lat lon

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.0b19...1.5.0b20

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 11 months ago

The Canadian Hydrological Model - 1.5.0 beta 19

• The initial ugrid code did not scale well. This release fixes that performance.
• Ugrid output now defaults to shuffle+deflate level5 + lossy bitgroom (4 Sig figs precision). This results in a massive 5x reduction in file size.
• Ugrid now only contains the timestamps written
• Ugrid output works with checkpointed and rotating files
• std::format has been replaced with fmt::format to better support gcc@9+ as std::format was only added in gcc@13

Barring any further bugs, this should hopefully be the last beta for the 1.5.0 release

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.0b18...1.5.0b19

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 11 months ago

The Canadian Hydrological Model - 1.5.0 beta 18

• Adds direct ugrid output, dramatically cutting down on memory consumption
• Removes output.mesh in favour of output.vtu and output.ugrid
• Defaults to the vtu/pvd and ugrid files being named the same as the output folder. This should make dealing with the outputs of multiple experiments more easily done

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 11 months ago

The Canadian Hydrological Model -

This reverts the ordinal mesh size changes due to some serious bugs and incompatibility with old meshes

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh 12 months ago

The Canadian Hydrological Model - 1.5.0 beta 16

• more robustly bound Ti newton iteration like in b15, but also for PBSM

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 15

More robustly bound the Ti root in harder_precip_phase to ensure convergence at more negative air temperatures

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 13

• Improve error reporting of harder precip if the newton iteration fails to converge
• Improve mesh datatype to use the global ordinal type, currently assumed to be long long
• Documentation improvement PR https://github.com/Chrismarsh/CHM/pull/149

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 12

Fix compatibility with newer spdlog/fmt combos

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.0b11...1.5.0b12

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 11

• allow expansion of search for forcing input to be up to 250%
• update help documentation and runtime errors to further clarify the updated config changes

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 10

• Rename station_N_nearest to num_stations_to_use
• Move num_stations_to_use and interpolant to forcing section
• output geojson of mesh bounding box and stations
  
• automatically expand domain to search for met points by 25% up to three times
• enforce cmake 3.30

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 9

• output the vtu and shp points for forcing into a seperate dir

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 8

• FIx incorrect handling of netcdf multipart enddate

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 7

• FIx precip unit conversion
• Output PVD to base of output folder

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 6

• Allow for checkpointing at arbitrary timesteps
• Basic unit conversion for netcdf files

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.5.0b5...1.5.0b6

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 5

• Fix a regression with custom starting points from either user of checkpoint when using multipart nc files

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 4

Fix ascii timeseries not being correctly loaded after the multipart nc feature was added

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 3

FIx delta_t consistency check. How it just checks that the offsets are spaced consistently. Should be more reliably and more performant

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 2

Fix netcdf loading bug with hourly

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 1

• fix missing file

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.5.0 beta 0

This version includes a major rewrite of the netCDF loader:

• requires CF-compliant standard_name attributes to dynamically determine variables
• support multipart netcdf files (e.g., one nc file per day)
• the grid of each multipart does not need to remain constant, allowing for trivially changing the resolution or product used at different timesteps, for example to switch from a short term 2.5km forcing to a coarser long-term 10km forcing
• output of a per-rank grid centres used for forcing as shapefiles

This also fixes a bug associated with loading too many forcing points for a mesh.

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.4.5

• Fix the LUT code path when Boost@1.8.5
• Relax the LAI condition to <1 and set the canopy height to be grass (0.25m)
  Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.4.4...1.4.5

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 1 year ago

The Canadian Hydrological Model - 1.4.3

• The LUT regression from 1.4.2 has been resolved and renabled.
• The MPI unique tag generation was improved to respect MPI TAG upper bound to ensure portability. This directly supports Intel MPI usage.
• A high-rank race condition was resolved with the auto checkpoint code to ensure that if any rank thinks it should auto-checkpoint, all the ranks auto-checkpoint
• Avalanche snowdepth is constrained to be 20m maximum
• FSM2 increased for 3 to 6 maximum layers
• Harder phase partitioning: Bound the ratio of 3%-97% as per pers comms with Harder
• Low LAI canopies with tall vegetation are bounded to be 0 m height as the parameterization produces unrealistic values otherwise
• New frequency options for output include specific date and daily time. Please see the documentation for more etails
• The default behaviour of mesh output has been changed and the new default is not to output any vtu files. The user must specify the output frequency or timing.

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh almost 2 years ago

The Canadian Hydrological Model - 1.4.2

The thin plate spline with tension interpolant uses a look up table (LUT) optimiziation. There is a regression with how windspeeds are interpolated that may, under rare circumstances, lead to very high windspeeds. The LUT usage has been disabled until this is resolved.

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh almost 2 years ago

The Canadian Hydrological Model - 1.4.1

This release adds the ability to automatically checkpoint when X minutes of wallclock time is left. In addition, CHM can now automatically resume from the most recent checkpoint file.

Combined with the the new clean_exit file that is written on final simulation end, the intent is to allow automatic suspend/resume in HPC context that have limited wall clock time.

Please see
https://chm.readthedocs.io/en/develop/configuration.html#checkpoint
for more details.

In addition, two key bug fixes:

• protect against a known compiler bug in gcc@12 when used with boost 1.85.0
• ensure variablestorage<T>::has protects against a not-found variable lookup

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh almost 2 years ago

The Canadian Hydrological Model - 1.4.0

• Increase the number of diagnostic variables output from FSM
• OpenMP is now defaulted to off due to the difficulty in propogating exceptions out of parallel regions. Suggestion is to exclusively use MPI for parallelsim.
• The spack.yaml in the root directory can be used to produce a working development environment

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.3.1...1.4.0

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 2 years ago

The Canadian Hydrological Model - 1.3.1

Fix a missing include that impacted older boost versions

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 2 years ago

The Canadian Hydrological Model - 1.3.0

• Drop conan in favour of spack for builds
• Switch to spdlog for logging
• Update to Trilinos >=14.0 which is a breaking change against previous Trilinos versions
• Update documentation and fix Sphinx RTD builds
• Ensure windmapper wind direction is on [0,360)
• Include table lookup optimization of PR #137

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 2 years ago

The Canadian Hydrological Model - v1.2.7

• Improve Pomeroy and Li 2000 upscaled probability calculation
• Bound Harder precipi phase between 3% and 97% as per pers. comms.
• Bug fixes

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 2 years ago

The Canadian Hydrological Model - v.1.2.6

• Enforce cold FSM soils at -10deg C
• Only load partial netcdf file for lower memory pressure
• MPI aware partition
• Load mesh parameters after forcing has been pruned

Full Changelog: https://github.com/Chrismarsh/CHM/compare/1.2.5...1.2.6

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 3 years ago

The Canadian Hydrological Model - v1.2.5

Allow NaN Z values with defined lat/long as can happen when regridding NWP inputs. E.g. shown here

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 3 years ago

The Canadian Hydrological Model - v1.2.4

• Improve frequency checking point on first timestep to support SnowCast update

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 3 years ago

The Canadian Hydrological Model - v1.2.3

• global_id is saved as an int in vtk files
• allow jemalloc with conan builds
• support netcdf forcing files with NaN lat/long
• checkpointing at the end of a run
• better checkpointing documentation
• FSM can init off a soil param (hacky: will be improved in the future)

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh almost 4 years ago

The Canadian Hydrological Model - v.1.2.2

• Fix deadlock when loading complex mesh topologies
• Fix incorrect face rank
• Fix FSM Nsnow incorrect type
• Fix handling v2 meshes in non partition mode
• Add missing chkp in remaining modules
• Support jemalloc 5.3+
• Add MPI testing code to functional_tests

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh almost 4 years ago

The Canadian Hydrological Model - v1.2.0

• Support compiling on arm64 (M1 Mac)
• Initial checkpointing implementation done

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh almost 4 years ago

The Canadian Hydrological Model - v1.1.0

Major improvements to CHM

• Uses Trilinos for parallel linear algebra solver in PBSM
• Metis partitioning of the mesh for parallel computation
• Pre-partitioning of the mesh via a dedicated conversion tool
• New parallel iterative SnowSlide implementation to support massively parallel runs

Bug Fixes

• Resolved issues with high rank runs
• Resolved a SnowSlide issue with isolated triangles

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh about 4 years ago

The Canadian Hydrological Model - GMD publication release

Release matching the code for the GMD publication
https://www.geosci-model-dev-discuss.net/gmd-2019-109/

Hydrosphere - Freshwater and Hydrology - C++
Published by Chrismarsh over 6 years ago