LISFLOOD - Diffusive wave river routing, updated reservoir modelling, revised initialization of soil moisture

LISFLOOD-OS v5.0.0 includes the following changes:

1. A new optional module for river routing computations implementing the Muskingum-Cunge-Todini diffusive wave routing method (Todini (2007); Reggiani et al. (2014); Reggiani et al. (2016); Reggiani and Todini (2018)).
2. A new version of the reservoir module after Hanasaki et al. (2022) and Casado-Rodríguez et al. (2026).
3. A revised approach for the initialization of soil moisture state variables.
4. A patch in the module snow.py to promote snow redistribution and avoid non-realistically high snow cover values at very high altitudes.
5. A patch in the module waterabstraction.py for consistency with groundwater domain modelling approach.
6. A comprehensive workflow for the computation of the pre-run in temporal chunks
7. Full compatibility with python 3.10-3.11-3.12, new versions of netcdf and gdal libraries.

The conceptual changes are further detailed below:

1. Diffusive wave option for river routing computation: mct.py
   LISFLOOD v4.3.1: The modelling of river routing relies on the kinematic wave solution approximation of the shallow-water equations, with the optional Split Routing approach aiming to mimic floodplain behavior and flood wave attenuation. The kinematic wave approximation is computationally efficient and well suited for medium to high slope rivers (i.e. riverbed slope > 0.001). However, it does not include diffusion effects and this can lead to inaccuracies in the modelling of flood wave peak magnitude and arrival time in mild to low slope rivers (river slope < 0.001).
   LISFLOOD v5.0.0: The Muskingum-Cunge-Todini (MCT) wave routing method accounts for both convection and diffusion of the flood wave, while maintaining numerical efficiency (Todini (2007); Reggiani et al. (2014); Reggiani et al. (2016); Reggiani and Todini (2018)). The new optional module mct.py is designed to overcome the limitations of the kinematic wave numerical solution in the modelling of mild to low slope rivers.

2. New methodology for reservoir storage and outflow computations: reservoir.py
   LISFLOOD v4.3.1: Reservoir storage and volume followed the approach of Burek et al. (2013) in which reservoir outflow is a function of reservoir level of filling.
   LISFLOOD v5.0.0: Reservoir outflow depends on both revoir level of filling and inflow discharge. The revised module is an adaptation of the methodology of Hanasaki et al. (2022) according to Casado-Rodríguez et al. (2026).

3. Revised approach for the initialization of soil moisture state variables.
   LISFLOOD v4.3.1: Soil moisture state variables at the last time step of the prerun are used as initial values of the model run. In areas with very low precipitation amounts and thick soil layers, these initial soil moisture values might not be representative of the long-term conditions and lead to erroneous decreasing trends in soil moisture values.
   LISFLOOD v5.0.0: Initial soil moisture of the third soil layer values which are consistent with the local climatological and soil properties conditions are computed via an analytical pixel-wise solution. The implemented methodology is conceptually equivalent to the well-known strategy for the initialization of the lower groundwater zone.

4. Patch in the module snow.py to avoid non-realistic results in specific conditions.
   LISFLOOD v4.3.1: Snow melting can be underestimated in pixels at very high altitudes causing not realistically high values of snow cover.
   LISFLOOD v5.0.0: When snow cover exceeds a pre-defined threshold value of snow water equivalent, a fraction of the snow cover exceeding the threshold is ablated to the lower elevation fraction (from high to medium, or from medium to low elevation fraction) or added to snow melt (lowest elevation fraction). Both the threshold value (2000 mm water equivalent) and the fraction (1/100) were based on expert definition.

5. Patch in the module waterabstraction.py
   LISFLOOD v4.3.1: Industrial, domestic, livestock water demand are abstracted (also) from groundwater resources. While the consumptive water use leaves the system, the unused water volume is returned to the channels (return flow). This solution mimics reality (and it is consistent with LISFLOOD-EPIC approach). However, limitations of the groundwater modelling approach (i.e. lack of upper boundary for groundwater abstraction and lack of modelling of horizontal groundwater fluxes) can lead to overestimation of return flow and spurious river flow values.
   LISFLOOD v5.0.0: Only consumptive use for domestic, industrial, livestock purposes is abstracted from groundwater (return flow is zero). This patch accounts for the limitations of the groundwater modelling approach, and it avoids overestimation of return flow and spurious values of river flow.

6. Comprehensive strategy for the computation of the pre-run in temporal chunks
   LISFLOOD v4.3.1: LISFLOOD prerun simulation can be accomplished in temporal chunks to accommodate limitations (e.g. time wall) of the computational infrastructure. The prerun simulation can be split in a cold start and one or more warm starts. Average variables for the initialization of the LISFLOOD run must be computed by the user based on the results of each chunk of the prerun, and prior to the set-up of the LISFLOOD run.

LISFLOOD v5.0:0: LISFLOOD prerun simulation can be accomplished in temporal chunks, using a sequence of a cold and one or more warm starts. All the variables required for the start of the LISFLOOD run are available in the output folder of the last chunk of the prerun simulation.

LISFLOOD 5.0.0 modelling approach was used for GloFASv5 and EFASv6 calibration.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 about 2 months ago

LISFLOOD - Total water storage reporting

This release adds the computation and reporting of total water storage maps.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 over 2 years ago

LISFLOOD - Replacement of cython with numba (routing)

• use of numba instead of cython for parallel computations in the routing module: numba was already used in other parallelized functions in soil modules. Now this version extends the use of numba to the kinematic wave parallel routing module. The change allows to completely exclude the use of Cython in the whole Lisflood OS software with no performance loss and skipping further compilation steps and potential run issues when porting kinematic wave compiled library to other systems/hardware. Numba use a Just in time (JIT) compilation that compiles python routines at runtime, generating optimized code and allowing parallelization by skipping the python Global Interpreter Lock (GIL) of specific parallelizable computation functions.

• xml files and documentation files are now updated removing the variables used by kinematic wave parallel routing module and skipping the steps for the compilation of Cython routing functions. Numba parameters in xml files should be used instead of the old kinematic wave ones to allow or disable parallelization on multiple CPUs hardware systems.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 over 2 years ago

LISFLOOD - Improved NetCDF4/PCRaster outputs module

• Outputs module refactored to be more efficient and modular.
• New option "OutputMapsChunks" added, allowing to write steps in chunks in the NetCDF4 file. The option "OutputMapsChunks" requires as input a positive integer number X, meaning that the code will dump outputs to disk every X steps (default value = 1).
• New option "OutputMapsDataType" added, allowing to choose the output maps data type between float64 and float32 (can be used to save space in output maps).
• Added support to flipped input static maps and meteo maps
• The Lisflood OS version number is now recorded in the metadata of the NetCDF4 files. Institution and author metadata are updated according to the version of Lisflood used to generate the output files.
• Flags to check the maps: now lisflood command can be executed to check the correctness of input maps before running the model. The option -c is available as an additional command line argument (e.g. "lisflood settings.xml -c") and generates the table of all the input maps specified in the input xml file. The table will show name of the map, the file path (or single value of the variable), the number of non missing values (nonMV) and missing values (MV) after the comparison with the mask map and LDD map, minimum, maximum and average values for each map. In case any map have missing values, a warning will be issued on screen.
• The "ReportSteps" option now support formulas. The "ReportSteps" option available in XML file can be used to customize the output frequency of the model state variables. The desired time steps can be specified by using ranges, formulas and parameters as indicated in the documentation here: https://ec-jrc.github.io/lisflood-code/3_step3_preparing-setting-file. Note that this option only impacts the output frequency of the model state variables (activated by the "repStateMaps" option), not to the auxiliary variables. The full list of the affected variables is here: https://ec-jrc.github.io/lisflood-code/4_annex_state-variables/
• Lisflood OS User Guide documentation is now included into the same repository of the Lisflood software into the "docs" folder: in this way the documentation will be up to date with the corresponding lisflood release and users can access to it offline. The documentation files are written in Markdown (MD) format and are visible using a Markdown viewer locally (available on most IDE like Visual Studio, Notepad++, etc...). The documentation can be also rendered locally as a localhost website using Jekyll and Ruby.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 almost 3 years ago

LISFLOOD - Valid_min and valid_max value outilers filtering in meteo input maps

This release includes the following fixes:

• XArray was not filtering out values outside valid_min and valid_max range in meteo maps. Now outliers values, if any, will be replaced by NaNs.
• added flag "-s" to ignore values out of valid range in meteo maps.
• fixed issue related to variables order in reading netcdf files
• fixed pandas parse_time_string renamed function issue.
• corrected units in netcdf metadata of Theta output maps

Climate Change - Natural Hazard and Storms - Python
Published by doc78 about 3 years ago

LISFLOOD - Coords from netcdf template

Small fix to use the netcdf template to set the coordinates in the netcdf outputs rather than using a pcraster clone.
This avoids inconsistencies between the netcdf inputs and outputs.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 over 3 years ago

LISFLOOD - Hot fix to increase the robustness of the code

This release features the following changes:

• a hot fix to prevent the warm start (i.e. restarting from a previous state) to occasionally fail due to small approximation errors (order of magnitude: 10exp-12) when writing and reading the state files;
• updates to the reporting options for output maps. List of repStateMaps files is now the same as repEndMaps and it only contains files used to warm start the model. Additional reporting options are available to singularly output maps of different model variables.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 over 3 years ago

LISFLOOD - Bug fix: corrected updating of the reservoirs and lakes volume after water abstraction for anthropogenic use

This release features the following changes:

1. The volume of lakes and reservoirs is correctly updated when water is abstracted for anthropogenic use.

2. The initialization run (or prerun; activated by switching on the "InitLisflood" option) implements a consistent solution throughout the code for the water abstraction module. As in v.4.0.1, the warm start of the initialization run is fully functional.

3. A hot fix to prevent the warm start (i.e. restarting from a previous state) to occasionally fail due to small approximation errors (order of magnitude: 10exp-12) when writing and reading the state files.

The changes (1) and (2) of this release have impacts on the results of LISFLOOD-OS when the following three optional modules are simultaneously activated: reservoirs, lakes, water use.
The magnitude of the impact of the above changes mainly depends on the magnitude of the water demand for energy use, and on the extent of the rice fields within the modelled area.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 almost 4 years ago

LISFLOOD - Waterbalance unit test and warm start in initialization run

• The changes of this release allow the warm start of the initialization run (or prerun; <setoption choice="1" name="InitLisflood"/>).
  More specifically, the changes of this release allow to print the end step of all the state files required to correctly perform the warm start of the initilization run.
  Before this release, the warm start could be correctly performed only for the run (<setoption choice="0" name="InitLisflood"/>).

• Furthermore, a unit test for waterbalance has been added.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 almost 4 years ago

LISFLOOD - Revised variables structure and improved pixel-wise computations for the soil loop

LISFLOOD-OS v4.0.0 includes the following changes:

1. pixel-by-pixel computation of water infiltration into the soil within the soilloop.py module.
2. changes in the allocation of water abstraction.
3. patches in the modules riceirrigation.py and frost.py to avoid non-realistic results in specific conditions.
4. improvements in the computation of the mass balance.
5. bug fixes in the use of the transientlandusechange option, in the use of the option simulatePF, in the writing of the monthly outputs.

Moreover, to ensure compatibility with LISFLOOD-EPIC, modules source code is compatible with XArray variables, and XArray variables will be allocated when EPIC module is ON.
LISFLOOD-EPIC has been developed by JRC-D2. When compared to LISFLOOD v3.2.0, LISFLOOD-EPIC enables a more detailed representation of crop growth and irrigation. References: ASR - Assessing groundwater irrigation sustainability in the Euro-Mediterranean region with an integrated agro-hydrologic model (copernicus.org) , Gelati, E. et al.: Integrated model of irrigation, crop growth and water resources (LISFLOOD-EPIC): description and application in the Euro-Mediterranean area, in preparation, 2020.
Important note: EPIC modules are not yet available from the OS-LISFLOOD repository.

1. computation of water infiltration into the soil within the soilloop.py module.
   LISFLOOD v3.2.0: all the pixels of a land use fraction are allocated to a vector and the number of loops is defined by the most critical pixel. More specifically, all the pixels within the computational domain are allocated the number of iterations that are necessary to allow the numerical stability of the most critical pixel. The number of iterations consequently depends on the specific computational domain. This can lead to different results when modelling sub-catchmennts and entire basins with highly heterogenous soils.
   LISFLOOD v4.0.0: pixel by pixel computation and parallelization of the computations by using the python package numba.
   Benefits of v4.0.0: each pixel is allocated the number of iterations that allow its numerical stability, this guarantees the consistency of results when modelling sub-catchments and the entire basin with highly heterogeneous soils. Decreased computational time due to parallel computations.

2. changes in the allocation of water abstraction.
   LISFLOOD v3.2.0: direct abstraction of the consumptive water use.
   LISFLOOD v4.0.0: abstraction of the demanded water volume for each use (industrial, domestic, energy, livestock); the consumptive water use leaves the system; the unused water volume is returned to the channels.
   Minor note: small differences in the computation of the leakages.

3. patches in the modules riceirrigation.py and frost.py to avoid non-realistic results in specific conditions.
   riceirrigation.py: LISFLOOD v3.2.0: in presence of a very thick third soil layer (10^2 m) the drainage of the rice fields causes non realistic large flow discharge values in the channels.
   LISFLOOD v4.0.0: only soil layers 1a and 1b are used by the rice module (for both the saturation and the drainage phases). By definition, the second soil layer includes the roots depth.
   frost.py: LISFLOOD v3.2.0: there is no upper boundary to the frost index value. Rain events on deeply frozen soils are totally converted into runoff and can cause false alarms.
   LISFLOOD v4.0.0: the maximum frost index value is set to 57.

4. improvements in the computation of the mass balance.
   LISFLOOD v3.2.0: the mass balance is computed by considering the input and output volumes up to the current computational step. Small numerical errors accumulate over time and give the erroneous impression of large mass balance errors.
   LISFLOOD v4.0.0: computation of the mass balance by considering the single computational step.

5. bug fixes in the use of the transientlandusechange option, in the use of the option simulatePF, in the writing of the monthly outputs.
   Correct use of the rice fraction by landusechange.py DYNAMIC; use of the correct soil water content variable for the computation of pF1 (ws1a replaced ws1); correct reporting of the monthly time steps by the optional module indicatorcalc.py.

Lisflood 4.0.0 is the version used for EFAS 5 and GloFAS 4 calibration.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 about 4 years ago

LISFLOOD - New XArray reader

This release includes the following changes:

• a new xarray reader for the water abstraction demand maps;
• a unit test to verify the functioning of the reader for the water abstraction maps;
• bug fixes and improvements to the caching function and the chunking function;
• improvements of the management of latitude and longitude grids to allow maximum precision of the reference system;
• improvements of the definition of the output variables.

All of the changes above improve the code capability to handle input and outputs.
The modelling of the hydrological processes has not been changed.

IMPORTANT NOTE: the results of the unit tests of this release are different from the results of the unit tests of the release 3.1.1. The differences are due exclusively to a different use of of the optional moduels (e.g. groundwatersmooth) within the .xml setttings file of the unit tests. When using the same settings of .xml file, and the same dataset, v3.1.1 and v3.2.0 provide the same results.

Climate Change - Natural Hazard and Storms - Python
Published by doc78 over 4 years ago

LISFLOOD - Faster I/O

NetCDF reader for forcings (or any other temporal data) now based on Xarray, allowing more flexibility and efficiency (dramatic improvement in calibration mode, i.e. without NetCDF outputs).
Two new options in the xml settings file:
NetCDFTimeChunks: chunking size in the time dimension. Recommended value is “auto" but chunking size can be specified manually or set to “-1" to load the whole time series into memory (very fast but expensive in terms of memory).
MapsCaching (True or False): option designed for the lisflood calibration. If set to True, all the static maps and forcings will be stored in a cache so that they don't have to be loaded by each lisflood instance. This option sets the value of NetCDFTimeChunks to "-1", meaning that the whole time series in the NetCDF inputs is loaded into memory.
Several tests have been added: lat/lon domains, inflows, new reader, etc. This version also makes it easier to use lisflood as a library and to run multiple instances of lisflood in a thread-safe environment. This can be done by doing the following:
import lisf1
...
lisf1.main(settings_file, flags)
…

Climate Change - Natural Hazard and Storms - Python
Published by valeriolorini about 5 years ago

LISFLOOD - minor bugfix in loadmap

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo about 6 years ago

LISFLOOD -

• Fix import order in unit tests to avoid name clash: for a new installation, installing an old verison of lisflood-utilities package and its dependencies (GDAL, netCDF)  was making PCRaster libs to segfault during tests.
• Update tox and pytest configurations, add custom tox ini file to gitignore.
• Improve setup script to install correct GDAL python package
• Updated Docker image:

https://hub.docker.com/repository/docker/efas/lisflood

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo over 6 years ago

LISFLOOD -

This release includes:

• fix for High Resolution grids (thanks @agahirpa)
• requirements files with fixed versions to enforce reproducible dev environments
• longer simulation tests (from 1month to 6 months)
• checking TSS output as well

All unit tests passed (py27, py36, py37)
All black box tests passed (see https://github.com/ec-jrc/lisflood-tests for more info)

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo over 6 years ago

LISFLOOD - Increased unit tests coverage

This is a minor release with enriched unit tests suite.

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo over 6 years ago

LISFLOOD - Verified and Tested Python version (no warmstart fixes)

This release is verified and tested with https://github.com/domeniconappo/lisflood-tests test suite.

Test suite:
EFAS daily 2,5 years
EFAS 6hourly 1,5 years
GLOFAS daily 1,5 years

Reference version as agreed:
https://github.com/ec-jrc/lisflood-code/commits/92061c80f2613af89f0d03cd1d2a0ccf3171a1f8

Reference data were executed with Python2.

NOTE: When executing tests with latest versions, we had to re-introduce - in the version under test - a bug that was present in reference version (ie a division that was casted to integer in Python2).
This was necessary to be sure that python3 changes and mass refactoring didn't touch the mathematical model.
Of course, we removed the bug.

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo over 6 years ago

LISFLOOD - copernicus release 2.9.6

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo almost 7 years ago

LISFLOOD - Python2/Python3 with Docker python 3

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo almost 7 years ago

LISFLOOD - Python2/Python3 compatible version

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo almost 7 years ago

LISFLOOD - Python2/Python3 compatible version

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo almost 7 years ago

LISFLOOD - v 2.8.6 Drina tests

Climate Change - Natural Hazard and Storms - Python
Published by domeniconappo over 7 years ago