Recent Releases of OpenQuake

OpenQuake - OpenQuake Engine 3.23.2

[Michele Simionato (@micheles)]

Backported fix to taxonomy mapping with consequences
Backported fix to 64 bit poes critical for multifault sources
Backported fix to workerpool critical for zmq clusters

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 11 days ago

OpenQuake - OpenQuake Engine 3.23.1

[Marco Pagani (@mmpagani)]

Backported fix to Thingbaijam et al. (2017) scaling relation

[Michele Simionato (@micheles)]

Backported fix to logs.dbcmd: querying the calculations caused the DbServer
to break in server installations
Backported fix to oq plot hcurves
Backported fix to Arteta (2021) missing the backarc parameter

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 1 month ago

OpenQuake - OpenQuake Engine 3.23.0

[Michele Simionato (@micheles)]

Fixed contexts.py: we were incorrectly discarding ruptures with magnitude equal
to the minimum magnitude
Hiding spurious "divide by zero" warnings generated by shapely in
get_joyner_boore_distance

[Fatemeh Alishahiha (@FatemehAlsh)]

Added GMM Campbell-Bozorgnia (2003)

[Christopher Brooks (@CB-quakemodel)]

Another small bug fix for NZ 2022 inslab variants of the Kuehn et
al. (2020) GMM - the sigma_mu_model and sigma_mu_epsilon params
were sometimes missing from GSIM object + added a sensitivity test

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Added an occurrence_rate column in median_spectrum_disagg and fixed
the exporter to order the ruptures by ID

[Michele Simionato (@micheles)]

Fixed a bug in classical_damage with sampling reported by Evi Riga
Added utils/fix_consequences to fix consequence.csv files
containing risk_ids rather than taxonomies, now invalid
Implemented quantiles in scenario_risk, visible in aggrisk_tags
Added checks on the consequence functions across taxonomies and perils
Fixed the taxonomy field in consequences.csv to be the exposure
taxonomy and not a fragility function ID
Internal: renamed aristotle -> impact
Changed the sourcewriter to save the rupture_idxs as compressed datasets,
thus reducing the size of some models a lot (i.e. from 400M to 17M)

[Christopher Brooks (@CB-quakemodel)]

Small bug fix for NZ 2022 inslab variants of the Kuehn et al. (2020)
and Parker et al. (2020) GMMs - backarc param was not always in the
req. site params but is needed given included in NZ 2022 site model

[Savvinos Aristeidou (@Savvinos-Aristeidou)]

Added new ANN-based GMM in the gsim library, named aristeidou_2024
Added new IMs, that is the FIV3, Sa_avg2, and Sa_avg3

[Michele Simionato (@micheles)]

Extended rupture_dict to the msr and aspect_ratio parameters
Fixed the instantiation of file-dependent GMPEs from the datastore
Optimized slow tasks both in the preclassical and the classical phase
Reading the site model in calculations with ruptures.hdf5 and making
minimum_intensity mandatory
Added parameter minimum_engine_version

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Storing the reduced asset collection and fixing a bug when exporting
the average losses

[Richard Styron (@cossatot), Michele Simionato (@micheles)]

Extended the preclassical calculator to work in absence of a site collection

[Michele Simionato (@micheles)]

Added a new output "Aggregated Exposure Values" (aggexp_tags) and a new extractor
aggrisk_tags;

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 2 months ago

OpenQuake - OpenQuake Engine 3.22.1

[Michele Simionato (@micheles)]

Fixed HDF5 bug when storing the avg_losses, visible on MacOS with a large
number of assets (say over a million). Fixed the same bug for damages too.

[Christopher Brooks (@CB-quakemodel)]

Added ability to specify period-dependent bias adjustment for USGS
classes of NGAEast GMMs (US 2023 NSHMP). Unit tests also added.
Added ability to specify Chapman and Guo (2021) Coastal Plains site
amplification adjustment for USGS classes of NGAEast GMMs (US 2023
NSHMP). Unit tests also added.
Added sediment depth site param (z_sed) to sites module as required
for Chapman and Guo (2021) Coastal Plains site amp. adjustment. QA
is provided through unit tests and qa_tests_data/classical/case_87.

[Michele Simionato (@micheles)]

Removed the deprecated function openquake.commonlib.datastore.hdf5new
and changed build_dstore_log into create_job_dstore;
Extended the ShakeMap parser to read the intensities associated to SA(0.6)
Fixed a bug while exporting realizations.csv for scenario calculations
Added webapi.calc_timeout configuration parameter
Reduced conditioned_gmfs_gb to 8 GB by default
Parallelized get_mean_covs when conditioning the GMFs

[Christopher Brooks (@CB-quakemodel)]

Added Seattle Basin classes of Kuehn et al. (2020) NGASUB GMM (uses
Cascadia coeffs except for basin term).
Added USGS basin scaling and CyberShake adjustments to the NGAWest2
GMMs + ability to pass base gsim arguments within the NSHMP2014 gsim class.

[Michele Simionato (@micheles)]

Saving the rupture geometries as 32 bit floats to save memory
Internal: added a command oq filter_around lon lat csvfile
Added a command oq plot ebruptures?min_mag=XXX
Added a script utils/build_global_ses
Removed the ability to read GMFs generated with engine <= 3.11
Fixed the filtering of the site collection in event_based with --hc
Internal: writeability check for datadir and scratch_dir (if any)

[Christopher Brooks (@CB-quakemodel)]

Added instantiation-level arguments for contextually applying
the M9 basin term, the CB14 basin term and the USGS basin scaling
model to the AbrahamsonGulerce2020, KuehnEtAl2020, ParkerEtAl2020,
ZhaoEtAl2006 and AtkinsonMacias2009 GMMs as required for the 2023
US NSHM model's subduction interface GMC. Unit tests are also
provided for these GMM adjustments

[Michele Simionato (@micheles)]

Internal: forcing the signature (C, ctx, region, ...) for _get_basin_term
in hazardlib
Extended gmfs_file to multiple .hdf5 files, assuming they correspond to
disjoint sites and events

[Paolo Tormene (@ptormene)]

Added a duplicates_strategy argument to read_df and changed the
approach used while reading station data, to calculate the average values
of stations having the same coordinates instead of raising an error

[Christopher Brooks (@CB-quakemodel)]

Added epistemic uncertainty scaling capabilities to
the Parker et al. (2020) NGASUB GMM.
Added M9 basin term and associated tests to ModifiableGMPE
Added CB14 basin term and associated tests to ModifiableGMPE

[Michele Simionato (@micheles)]

Added parameter with_betw_ratio
Added command oq info peril
Extended consequences to perils
Replaced taxonomy mapping by loss type with taxonomy mapping by peril
Internal: changed the ordering in the composite risk model from
(loss_type, riskid) -> (riskid, loss_type)
Added an exporter for trt_gsim
Internal: added utility function readinput.read_source_models

[Manuela Villani (@ManuelaVillani)]

Improved the "Governing MCE" plot

[Michele Simionato (@micheles)]

Raised an error in case of non-invertible hazard curves, affecting
disaggregation and site-specific hazard spectrum calculations
Changed the ruptures.csv exporter to also export the source IDs
Added support for consequence=losses for liquefaction and landslides
Added a check for missing secondary perils
Added loss types liquefaction and landslide
Removed support for XML consequences, after 3 years of deprecation
Fixed the stored calculation_mode in oq sensitivity_analysis
Added command oq info loss_types

[Paolo Tormene (@ptormene)]

Added the possibility to export the asset collection via command line,
keeping it private from the webui

[Michele Simionato (@micheles)]

Fixed median_spectrum with multiple sites
Extended Aristotle calculation to manage multiple countries at once
Extended taxonomy_mapping.csv with a loss_type field
Modernized the regionalization of Chiou Youngs (2014)
Modernized the regionalization of Campbell Bozorgnia (2014)
Internal: made it possible to override CoeffsTable
Added a memory check in disaggregation calculations
Made scientific_format resilient against encoding errors

[Ilaria Oliveti (@IlariaOliveti)]

Fixed the GMPE Tusa-Langer-Azzaro (2019) table of coefficients, the
IMTs were incorrectly using Hz instead of seconds

[Michele Simionato (@micheles)]

Fixed the assert(losses) error in the view delta_loss in the case of few events

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 3 months ago

OpenQuake - OpenQuake Engine 3.21.0

[Paolo Tormene (@ptormene)]

Added commands oq plot "rupture?" and oq plot "rupture_3d?"

[Michele Simionato (@micheles)]

Extended oq reset to also remove the custom_tmp directory, if any
Added classes CampbellBozorgnia2019, CampbellBozorgnia2019HighQ, CampbellBozorgnia2019LowQ
Reduced the size of the large CSV files in hazardlib and added a check
to forbid files larger than 600k
Added a check on the signatures of get_std_dev_mag and get_median_area
and fixed the signature of StrasserIntraslab.get_std_dev_mag

[Enrico Abcede (@emabcede30), Francis Bernales (@ftbernales)]

Implemented Campbell and Bozorgnia (2019) IA and CAV to campbell_bozorgnia_2014

[Michele Simionato (@micheles)]

Fixed scenario_risk from GMFs when the intensity measure types were
incorrectly identified

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Added a median spectrum post-processor

[Michele Simionato (@micheles)]

Added a parameter mea_tau_phi in the job.ini to save mean, tau and phi
for each rupture, site, gsim and imt in a GMF calculation
Internal: changed oq run to automatically generate the db when possible
Added a check on missing risk files
Reduced the space used by the CollapsedPointSources (2.7x)
Fixed the site model association procedure to work in conditioned
scenario calculations
Used oq engine --run to submit asynchronous jobs to SLURM and
oq run to submit interactive jobs

[Christopher Brooks (@CB-quakemodel)]

Added the Japan region versions of the NGAWest2 relations used to obtain
z1pt0 and z2pt5 from vs30 in prepare sites command

[Michele Simionato (@micheles)]

Fixed oq reduce_sm for calculations with nonparametric sources
Fixed logging the classical time with --sample-sources
Improved the progress log in classical calculations

[Kendra Johnson (@kejohnso)]

Fixed an error invalid literal for int() with base 10 affecting
event based calculations using mutex sources and a nontrivial logic tree

[Michele Simionato (@micheles)]

Optimized the --sample-sources feature
Honored the custom_tmp in classical calculations and saved data transfer
by using TileGetters, then parallelized the saving of the rates

[Lana Todorovic (@LanaTodorovic93)]

Implemented Nowicki Jessee et al. (2018) landslide geospatial model that
computes the areal coverage by landslide occurrence.

[Paolo Tormene (@ptormene)]

Updated extractor for gmf_data for a single event id (used by the IRMT
QGIS plugin), including data for secondary perils

[Michele Simionato (@micheles)]

Refined the tiling calculator (partial tiling, task weighting, saving memory)
Optimized "computing pnes" in classical calculations (3x in the common case)
and fully switched to 32 bit rates, thus saving memory and improving performance

[Nicolas Schmid (@schmidni)]

Fixed a regression in the ShakeMap to_gmfs code, a forgotten sigma^2/2 term

[Marco Pagani (@mmpagani)]

Fixed aliases for Kuehn2020 GMPEs (signalled by Eric Thompson)

[Michele Simionato (@micheles)]

Internal: splitting in groups with homogeneous temporal occurrence model
Optimized the calculation of mean and stdevs in event based calculations, with
a speedup of 13x for the EUR model

[Chris di Caprio (@chrisdicaprio)]

Allowed extrapolation in the Kuehn (2020) GMPEs to solve numeric issues

[Michele Simionato (@micheles)]

Fixed an encoding error when exporting the realizations
Fixed a bug with multifault sources raising a cryptic error
"object has no attribute msparams"
Fixed a critical memory bug causing over 80 GB per core to be needed for
event based calculations with ~6 million sites
Made minimum_intensity mandatory in event based calculations
Reduced the memory consumption in event_based calculations: now even calculations
with 5 million sites can be run with ~2 GB per core
Reduced the memory occupation in gen_poes
Using half the memory in postclassical by using 32 bit arrays
Using half the memory on Windows by using half the threads by default
Fixed a bug in conditional spectrum calculations with a non-contributing TRT

[Savvinos Aristeidou (@Savvinos-Aristeidou)]

Added new GMM in the gsim library, named aristeidou_2023
Added new IM, that is the inelastic spectral displacement, SDi,
which takes as input one additional parameter (i.e., strength ratio, R)

[Fatemeh Alishahiha (@FatemehAlsh)]

Added a new gsim module named zafarani_2018.
Added a new gsim module named ambraseys_2005.

[Kyle Smith (@kslytherin)]

Added a new gsim file sandikkaya_akkar_2017.py to implement the
Arias Intensity and Cumulative Absolute Velocity ground motion
models from Sandikkaya and Akkar (2017).

[Michele Simionato (@micheles)]

Making sure that the tiles contains at least 100 sites in classical tiling
calculations
Rewritten the SLURM algorithm to work around the submission limit
Added uniqueness check for the station coordinates
Internal: removed parameter [dbserver]listen in openquake.cfg
Added command oq submit n job_ini for usage in supercomputers
Reimplemented the sensitivity analysis in a supercomputer-friendly way
Fixed export realizations which was truncating the branch_path
Extended logic tree filtering via smlt_branch
Worked on disaggregation by multifault source

[Michele Simionato (@micheles), Christopher Brooks (@CB-quakemodel)]

Internal: added a function for filtering sites around a rupture

[Michele Simionato (@micheles)]

Internal: added method GsimLogicTree.to_node
Internal: added commands oq compare oqparam and oq plot_sites csvfiles
Fixed exporting the ruptures when imported from a CSV
Fixed the issue of small negative eigenvalues in conditioned GMFs
calculations by adding a correlation_cutoff
Internal: extended Starmap.apply to split DataFrames
Fixed the avg_gmf exporter in presence of a filtered site collection

[Lana Todorovic (@LanaTodorovic93)]

Improved secondary perils documentation

[Michele Simionato (@micheles)]

Fixed the ordering of the realizations to depend on the source model
branch path also in the case of full enumeration: this is crucial to
get consistent results with the two approaches
Saved data transfer in the context makers in classical calculations
Internal: added utility oq info geohash:<lon>,<lat>
Changed the scenario calculators to discard the sites far away from
the rupture, with the effect of generating different GMFs
Cached get_realizations and optimized the memory occupation; now
302_990_625 realizations (i.e. EUR) require only 90 GB

[Anna Rood (@AnnaRood)]

Updates to the AELO workflow: added additional spectral periods ranging
from 0.02 to 10 s and added option to use ASCE7-22. NB: minor changes in
the results of year 1 are expected

[Michele Simionato (@micheles)]

Changed the GMF-based calculators to always set a custom_site_id
Internal: added variable OQ_SITE_DEBUG

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 7 months ago

OpenQuake - OpenQuake Engine 3.20.1

[Paolo Tormene (@ptormene), Antonio Ettorre (@antonioettorre), , Armando Scarpati (@hascar)]

Improved installation documentation

from previous OpenQuake Engine 3.20.0 version:

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Internal: added commands oq compare assetcol and oq compare sitecol

[Michele Simionato (@micheles)]

Internal: added a command oq reducexp exposure_xml site_model_csv

[Paul Henshaw (@pslh), Antonio Ettorre (@antonioettorre), Michele Simionato (@micheles)]

Updated the documentation about installing the engine

[Michele Simionato (@micheles)]

Raised the default max_potential_paths to 1,000,000
Optimized the calculation of mean hazard curves when use_rates=true;
now it is possible to compute exactly mean curves even with millions
of realizations

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Added command oq plot 'gmf_scenario?imt=PGA&kind=rlz-0'

[Fatemeh Alishahiha (@FatemehAlsh)]

Added vertical component to abrahamson_silva_1997 gsim library

[Michele Simionato (@micheles)]

Internal: removed ProbabilityCurve and rename ProbabilityMap->MapArray
Optimized building the CompositeSourceModel for complex logic trees
Optimized "reading rates" in postclassical
Extended the delta_loss warning to scenario_risk calculations
and documented it

[Michele Simionato (@micheles), Marco Pagani (@mmpagani)]

Fixed the multifault rupture indices and the MultiLine class so
that the New Zealand model can run

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Added a new parameter asce_version for AELO calculations

[Michele Simionato (@micheles)]

Saving memory and disk space in conditioned GMFs scenarios, improving
also the performance in many cases
Fixed the passing of parameters to the underlying GMPE in NRCan15SiteTerm
Added another check for missing gsim in scenario calculations
Using custom hcurves and uhs exporters in AELO mode

[Fatemeh Alishahiha (@FatemehAlsh)]

Added a new gsim library named ghasemi_2009 developed for Iran.

[Marco Pagani (@mmpagani)]

Added a new epistemic uncertainty for modelling relative uncertainty on
MMax

[Michele Simionato (@micheles)]

Added a check to forbid individual_rlzs=true and collect_rlzs=true
Fixed a bug in oq extract "ruptures?rup_id=XXX
Raising an early error if the user forgets to specify a site_model_file
when needed (i.e. for parameters region and xvf)

[Marco Pagani (@mmpagani), Christopher Brooks (@CB-quakemodel)]

Refactored Chiou and Youngs (2014) GMM to be adjustable using methodology
of the Boore et al (2022) paper

[Michele Simionato (@micheles)]

Discarded the stations (if any) from the avg_gmf plot
Added an early check for homeless consequences
Improved the check for missing site parameters
Internal: fixed the serialization of logic trees using
simpleFaultGeometryAbsolute, complexFaultGeometryAbsolute or
characteristicFaultGeometryAbsolute
Added a check when importing hazard_curves.csv files: the values must be
probabilities
Internal: functions must be shorter than 100 lines and have less than
16 arguments to enter in the engine codebase
Fixed scenarios with CanadaSHM6 GMPEs and ModifiableGMPE
Fixed avg_losses-stats exporter in the case of a single realization
Managed duplicated multi fault sources in event based for the New
Zealand model
Forbidden characters .:; in the branchID

[Michal Kolaj (@MichalKolaj)]

Fixed a couple bugs in the Canadians GMPEs (can_shm6_inslab) causing wrong
means

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

Extended the engine and the WebUI to accept a rupture_file
in the contexts of the Aristotle project
Extended the engine and the WebUI to run scenario_risk calculations
from USGS ruptures in the contexts of the Aristotle project

[Michele Simionato (@micheles)]

Raised a clear error for GMF-conditioned calculations with too many sites
Rounding lon, lat with numpy in oqparam.sites
Reduced by half the data transfer in classical by using 32 bit rates
Fixed a bug in oq zip which was missing the exposure.csv files
Introduced a parameter config.memory.avg_losses_max
Changed import_gmfs_hdf5 to not use ExternalLinks
Added oq show usgs_rupture:<usgs_id> printing the rupture parameters

[Matteo Nastasi (@nastasi-oq)]

Add helper to check consistency between debian/changelog and
CONTRIBUTORS.txt

[Michele Simionato (@micheles)]

Added an optional flag config.distribution.compress to reduce the
data transfer by compressing pickles larger than 1 MB
Optimized postclassical both for regular and tiling calculations
Saving disk space in classical calculations (~4x) by gzipping the rates
Removed Python version checks in install.py

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 11 months ago

OpenQuake - OpenQuake Engine 3.20.0

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Internal: added commands oq compare assetcol and oq compare sitecol

[Michele Simionato (@micheles)]

Internal: added a command oq reducexp exposure_xml site_model_csv

[Paul Henshaw (@pslh), Antonio Ettorre (@antonioettorre), Michele Simionato (@micheles)]

Updated the documentation about installing the engine

[Michele Simionato (@micheles)]

Raised the default max_potential_paths to 1,000,000
Optimized the calculation of mean hazard curves when use_rates=true;
now it is possible to compute exactly mean curves even with millions
of realizations

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Added command oq plot 'gmf_scenario?imt=PGA&kind=rlz-0'

[Fatemeh Alishahiha (@FatemehAlsh)]

Added vertical component to abrahamson_silva_1997 gsim library

[Michele Simionato (@micheles)]

Internal: removed ProbabilityCurve and rename ProbabilityMap->MapArray
Optimized building the CompositeSourceModel for complex logic trees
Optimized "reading rates" in postclassical
Extended the delta_loss warning to scenario_risk calculations
and documented it

[Michele Simionato (@micheles), Marco Pagani (@mmpagani)]

Fixed the multifault rupture indices and the MultiLine class so
that the New Zealand model can run

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Added a new parameter asce_version for AELO calculations

[Michele Simionato (@micheles)]

Saving memory and disk space in conditioned GMFs scenarios, improving
also the performance in many cases
Fixed the passing of parameters to the underlying GMPE in NRCan15SiteTerm
Added another check for missing gsim in scenario calculations
Using custom hcurves and uhs exporters in AELO mode

[Fatemeh Alishahiha (@FatemehAlsh)]

Added a new gsim library named ghasemi_2009 developed for Iran.

[Marco Pagani (@mmpagani)]

Added a new epistemic uncertainty for modelling relative uncertainty on
MMax

[Michele Simionato (@micheles)]

Added a check to forbid individual_rlzs=true and collect_rlzs=true
Fixed a bug in oq extract "ruptures?rup_id=XXX
Raising an early error if the user forgets to specify a site_model_file
when needed (i.e. for parameters region and xvf)

[Marco Pagani (@mmpagani), Christopher Brooks (@CB-quakemodel)]

Refactored Chiou and Youngs (2014) GMM to be adjustable using methodology
of the Boore et al (2022) paper

[Michele Simionato (@micheles)]

Discarded the stations (if any) from the avg_gmf plot
Added an early check for homeless consequences
Improved the check for missing site parameters
Internal: fixed the serialization of logic trees using
simpleFaultGeometryAbsolute, complexFaultGeometryAbsolute or
characteristicFaultGeometryAbsolute
Added a check when importing hazard_curves.csv files: the values must be
probabilities
Internal: functions must be shorter than 100 lines and have less than
16 arguments to enter in the engine codebase
Fixed scenarios with CanadaSHM6 GMPEs and ModifiableGMPE
Fixed avg_losses-stats exporter in the case of a single realization
Managed duplicated multi fault sources in event based for the New
Zealand model
Forbidden characters .:; in the branchID

[Michal Kolaj (@MichalKolaj)]

Fixed a couple bugs in the Canadians GMPEs (can_shm6_inslab) causing wrong
means

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

Extended the engine and the WebUI to accept a rupture_file
in the contexts of the Aristotle project
Extended the engine and the WebUI to run scenario_risk calculations
from USGS ruptures in the contexts of the Aristotle project

[Michele Simionato (@micheles)]

Raised a clear error for GMF-conditioned calculations with too many sites
Rounding lon, lat with numpy in oqparam.sites
Reduced by half the data transfer in classical by using 32 bit rates
Fixed a bug in oq zip which was missing the exposure.csv files
Introduced a parameter config.memory.avg_losses_max
Changed import_gmfs_hdf5 to not use ExternalLinks
Added oq show usgs_rupture:<usgs_id> printing the rupture parameters

[Matteo Nastasi (@nastasi-oq)]

Add helper to check consistency between debian/changelog and
CONTRIBUTORS.txt

[Michele Simionato (@micheles)]

Added an optional flag config.distribution.compress to reduce the
data transfer by compressing pickles larger than 1 MB
Optimized postclassical both for regular and tiling calculations
Saving disk space in classical calculations (~4x) by gzipping the rates
Removed Python version checks in install.py

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 11 months ago

OpenQuake - OpenQuake Engine 3.19.1

[Antonio Ettorre (@vot4anto)]

Upgrade pyzmq to 26.0.3

[Michele Simionato (@micheles)]

Backported fix to oq extract ruptures?rup_id=XXX
Backported fix to a fake error in case of a nontrivial
maximum_distance together with a minimum_magnitude
Backported fixes to the Canada SHM6 GMPEs

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq 12 months ago

OpenQuake - OpenQuake Engine 3.19.0

[Graeme Weatherill (@g-weatherill)]

Contributed new GMPEs Weatherill2024ESHM20AvgSA,
Weatherill2024ESHM20SlopeGeologyAvgSA, Weatherill2024ESHM20Homokedastic
and GmpeIndirectAvgSA

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Added geometry_file to discard ruptures

[Michele Simionato (@micheles)]

Fixed exporting from calculations with a grand parent
Added an uniqueness check to source IDs in the same branch
Extended the command oq sample to multi fault sources
Fixed the reading of ruptures with a multisurface with meshes
of different lengths
Fixed oq compare: the tolerance parameters were ignored
Fixed a numerical precision error affecting the hazard curves at
very high intensities
Optimized multi fault sources by moving surface calculations in the
preclassical phase
Fixed disaggregation in case of mutex sources

[Paolo Tormene (@ptormene)]

Added the possibility to display/hide announcements on top of all WebUI
pages, setting them up via the Django admin interface

[Michele Simionato (@micheles)]

Fixed ZalachorisRathje2019 (mag > Mb_ -> mag >= Mb_)
Reduced memory consumption in multiFaultSources
Implemented batch processing for AELO calculations

[Anirudh Rao (@raoanirudh)]

Improved the performance of non-parametric liquefaction models

[Michele Simionato (@micheles)]

Renamed 'fpeak' as 'f0' in the HassaniAtkinson2020 GMPE so that the
Manea2021 GMPE can work
Internal: extended the command oq mosaic sample_rups to multiple models
After 3 years of deprecation, forbidden custom methods in GMPE classes
Changed the universal installer to recreate the venv
Reduced the number of tasks generated by disagg_by_src and added
more checks on the naming conventions for the source IDs
Fixed the gmf_data exporter with a filtered site collection
Added missing validation on duplicate source IDs for source models
in format nrml/0.5
The calculation_mode=ebrisk was not honored when starting from GMFs
Fixed an ordering bug in the IMTs causing two peaks in the UHS in
some situations
Fixed bug in the WongEtAl2022 GMPEs, reported by Thomas Bornstein
Added a job.ini parameter smlt_branch to reduce the logic tree to
a specific branch
Internal: extended oq info to shapefiles

[Anirudh Rao (@raoanirudh)]

Fixed the conditioned GMFs calculator in the case all the
stations are filtered away

[Marco Pagani (@mmpagani)]

Fixed the algorithm used to generate kite surfaces

[Michele Simionato (@micheles)]

Changed the weighted quantile algorithm to not depend on numpy.argsort
(since numpy 1.25 argsort produces a different sorting on machines with
AVX-512 enabled processors, causing different quantiles when the weights
are not all equal)
Improved error message for missing fields in the exposure CSV files
Internal: removed dbserver.user in openquake.cfg
Fixed extract_from_zip when managing zip files coming from MACOSX

[Paolo Tormene (@ptormene)]

Internal: extended the MosaicGetter to a GlobalModelGetter class with
the ability to associate points to countries

[Michele Simionato (@micheles)]

Removed the obsolete by_country hack when managing the global exposures
Changed oq sample to use pandas for reading/writing CSVs, thus fixing
the issue of quoted newlines
Fixed oq check_input exposure.xml
Added rupture_dict parameter to be used in scenarios

[Marco Pagani (@mmpagani)]

Fixed a bug in the calculation of Rjb for gridded surfaces and added a
dependency from alpha_shapes

[Chris di Caprio (@chrisdicaprio)]

Contributed many new GMPEs for the New Zealand model

[Michele Simionato (@micheles)]

Fixed wrong mean_rates_by_src in presence of the colon convention
If there are no sources close to the (single) site, do not raise an error
Fixed a bug with --hc taking the parent site collection instead of
the child site collection
Extended the exposure field mapping functionality to map the same
input field to multiple columns in the asset collection
Removed the experimental aftershock calculator; aftershocks can
be computed by simply including a file delta_rates.csv

[Anirudh Rao (@raoanirudh)]

Limited the cross-correlation coefficient of GodaAtkinson2009
to have an upper bound of 1.0

[Paolo Tormene (@ptormene)]

Added commands oq plot mean_hcurves_rtgm,governing_mce,disagg_by_src

[Michele Simionato (@micheles)]

Raised the limit on the product num_assets * num_rlzs when using
full enumeration by a factor of 10
Fixed exporting-reimporting GMFs in hdf5 format
Fixed a bug causing the insured losses to be larger than the
ground losses in some cases
Added command oq info executing
Added extract/relevant_gmfs?threshold= functionality
Added extract/ruptures?threshold= functionality
Disabling the DbServer with dbserver.host = local
Fixed disagg_source for same ID sources
Disambiguated same ID sources by appending the branch ID after
the exclamation mark and fixed disagg_by_src accordingly
Fixed another bug in disaggregation for mutex sources
Fixed a bug in disagg_source for mutex sources: src_mutex
was not passed
Internal: forced the colon convention on mutex sources
Fixed a bug in the disaggregation with mutex sources incorrectly
averaging over the rates and not the poes

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 1 year ago

OpenQuake - OpenQuake Engine 3.16.7

[Antonio Ettorre (@vot4anto)]

Upgraded Django version to version 3.2.21 with a security fix

[Michele Simionato (@micheles)]

In case of a well known IndexError in disaggregation calculations, solved in
newer versions of the engine, a helpful message is printed
Backported fix on sanity check for avg_losses and aggrisk
Backported fix to avg_gmf
Backported a fix to oq reaggregate
Backported fix to event based risk calculations with
aggregate_by(parent) != aggregate_by(child)
Backported fix to the risk_by_event exporter for some calculations
starting from GMFs on a CSV file
Backported fix to import_gmfs_csv, which was breaking sometimes with a
filtered site collection

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 1 year ago

OpenQuake - OpenQuake Engine 3.18.0

[Michele Simionato (@micheles)]

Fixed the command oq extract ruptures producing wrong newlines on
Windows

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Fixed an ordering bug in the tagnames in extract_agg_curves breaking
the QGIS plugin

[Michele Simionato (@micheles)]

Fixed the function get_mean_stddevs_cy14 to recover the same results
for the Canada SHM6 model as in engine 3.11
Required at least Python 3.10.6 on mac and windows

[Claudio Schill (@claudio525)]

Optimized the Kuehn sigma_mu_adjustment calculation by replacing the
multi-step interpolation with a single 2D/3D space interpolation

[Michele Simionato (@micheles)]

Fixed a bug in calculations with a filtered site collection using the
HM2018CorrelationModel
Internal: raised a clear error message when get_composite_source_model is
called without passing a datastore in presence of multifault sources

[Christopher Brooks (@CB-quakemodel)]

Added non-ergodic implementation of Zhao et al. (2016) GMM (ray-tracing
of travel paths through anelastically attenuating volcanic regions and
subsequent adjustment of the path term based on this distance to reduce
the predicted ground-motion).

[Michele Simionato (@micheles)]

Optimized (2x) the generation of ground motion fields
Internal: using a single random number generator inside the GmfComputer
and not one for each gsim, causing an artificial correlation between gsims
Implemented maximum_distance_stations and fixed another site
collection filtering bug in conditioned GMFs
Extended conditioned GMFs to number_of_ground_motion_fields > 1
Disabled SLOW MODE on macOS, since the amount of free memory returned by
psutil cannot be trusted
Speeding up count_ruptures for multifault sources, which gives a huge
in some calculations (like event_based for Dominican Republic)

[Marco Pagani (@mmpagani)]

Fixed a bug in the code that resamples a line, with a small effect
on the hazard of models containing simple fault sources

[Michele Simionato (@micheles)]

Optimized the calculation of quantile hazard maps (7x)
Internal: stored _rates instead of _poes in classical calculations

[Lana Todorovic (@LanaTodorovic93), Anirudh Rao (@raoanirudh)]

Added several regional liquefaction models to the secondary perils module,
including:
ZhuEtAl2017LiquefactionCoastal and ZhuEtAl2017LiquefactionGeneral,
RashidianBaise2020Liquefaction, AllstadtEtAl2022Liquefaction,
AkhlagiEtAl2021LiquefactionA and AkhlagiEtAl2021LiquefactionB
(experimental),
Bozzoni2021LiquefactionEurope, TodorovicSilva2022NonParametric
Added new site parameters required by some of these new models to site.py
Added LiqOccur as a valid IMT, referring to the occurrence or
non-occurrence of liquefaction at a site
Added basic documentation for the secondary perils module
(originally written by @cossatot and extended by @LanaTodorovic93)
Added rock-slope co-seismic failure computation of Grant et al. (2016)

[Michele Simionato (@micheles)]

Fixed sanity check on avg_losses and aggrisk breaking for event_based_damage
calculations following event_based_risk calculations
Fixed a bug in avg_gmf causing the average to be computed incorrectly
Removed the XML exporters for hazard curves, maps and uniform hazard spectra
after nearly 5 years of deprecation

[Paolo Tormene (@ptormene)]

In the WebUI users are not allowed to see other users outputs by default
(the old default can be obtained setting ACL_ON = False)

[Manuela Villani (@mvillani), Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

Produced ASCE-7 and ASCE-41 outputs for the AELO project and hidden
a few internal outputs

[Athanasios Papadopoulos (@thpap)]

Adjusted the Swiss-specific implementations of the GMPEs used
in the Swiss national seismic hazard (SUIhaz15) and risk (ERM-CH23) models.
The new implementation returns all of total, intra- and inter-event sigmas,
rather than just the total one
Extended the ModifiableGMPE class by adding an
apply_swiss_amplification_sa method that is used in ERM-CH23 to
apply site specific adjustments for site effects and intra-event
uncertainty
Added ch_ampl03, ch_ampl06, ch_phis2s03, ch_phis2s06,
ch_phisss03, ch_phis2s06 site parameters to the site.py file

[Paolo Tormene (@ptormene)]

Upgraded requirements: Shapely to version 2.0.1 and Pandas to version 2.0.3

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

Implemented AEP, OEP curves

[Michele Simionato (@micheles)]

Internal: changed events.year to be in the range 1..eff_time
Enhanced oq engine --delete-calculation to remove calculation files
even if the DbServer is on a remote machine
Fixed another site collection filtering bug in conditioned GMFs
Fixed a regression in oq reaggregate
Better error message for logic trees with branchsets exceeding the limit
Fixed a bug with mutex sources and disagg_by_src

[Manuela Villani (@mvillani), Kendra Johnson (@kejohnso), Michele Simionato (@micheles)]

Implemented Risk Targeted Ground motion for the AELO project

[Paolo Tormene (@ptormene)]

Fixed consequence calculations for 'homeless', 'fatality' and 'injury'
with/without a specified 'time_event'
Raise an early error if any station data has zero intensity value

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

Internal: checking encoding and newlines of all CSV files in the repository

[Christopher Brooks (@CB-quakemodel)]

Implemented the Atkinson (2015) GMPE with the alt. eff. depth model

[Michele Simionato (@micheles)]

Fixed a bug in event based risk calculations with
aggregate_by(parent) != aggregate_by(child)
Fixed a bug when exporting risk_by_event in absence of ruptures
Fixed a bug by not exporting event_based_mfd in absence of ruptures
Enabled the pointsource_distance approximation by default at 100 km
Fixed an error with sources below the minimum_magnitude
Introduced support for SLURM (experimental)
Fixed bug in conditioned GMFs, due to the station sites being excluded
from the complete site collection
Reduced slow tasks and memory consumption
in the event_based_risk calculator (3x for Chile)

[Paolo Tormene (@ptormene)]

Raise an error if files specified in site_model_file do not have the same
headers
Give a better error message for risk calculations where the aggregate_by
tag is specified but it doesn't exist in the exposure model

[Michele Simionato (@micheles)]

Improved error message when quantiles are not supported in event based risk
Reduced memory consumption in the master node in post_risk
Reduced slow tasks in the ebrisk calculator and memory consumption
in the master node

[Paolo Tormene (@ptormene)]

Fixed bug #8907: unable to run ClassicalDamage demo on Windows

[Astha Poudel (@asthapoudel), Anirudh Rao (@raoanirudh), Paolo Tormene (@ptormene)]

Updates to the infrastructure risk assessment and connectivity analysis
module to incorporate additional performance metrics including Complete
Connectivity Loss (CCL), Partial Connectivity Loss (PCL), Weighted
Connectivity Loss (WCL), and Efficiency Loss (EL) based on
Poudel et al. (2023).
Implemented the possibility of including both weighted and unweighted
graphs for infrastructure risk.
Implemented the possibility of including simple undirected graphs, simple
directed graphs, multigraphs and multi-directed graphs.
Implemented the Bradley (2012) cross-correlation equations between peak
ground velocity (PGV), peak ground acceleration (PGA), and 5% damped
pseudo-spectral acceleration (SA)

[Michele Simionato (@micheles)]

Moved the generation of the hazard curves from the GMFs in post_execute
Fixed the error "You must supply a list of magnitudes as 2-digit strings"
when starting from ruptures in a model with GMPETables (i.e. Canada)
Replaced sequential tiling with parallel tiling in large classical
calculations
Huge reduction of the memory usage in the event_based calculator; also,
reduced the slow tasks

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 1 year ago

OpenQuake - OpenQuake Engine 3.17.1

[Michele Simionato (@micheles)]

Fixed the associations event_id, rup_id, rlz_id

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 2 years ago

OpenQuake - OpenQuake Engine 3.17.0

[Michele Simionato (@micheles)]

Raise an early error if the user disable both realizations and statistics
Internal: added dbserver.user in openquake.cfg
Solved the hanging of classical calculations due to large zmq packets
and reduced memory occupation

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Added support for the "direct" method in MRD calculations

[Michele Simionato (@micheles)]

Reimplemented the conditional_spectrum calculator as a post-processor

[Paolo Tormene (@ptormene), Michele Simionato (@micheles)]

Raise an error when the same parameter is set in different sections
of the job.ini file

[Michele Simionato (@micheles)]

Fixed tau-phi inversion in LanzanoLuzi2019
Fixed another bug with conditioned GMFs appearing as the error
array[m, n, bad] = exp(mean_covs[0, g, m, n], im) TypeError: list indices must be integers or slices, not tuple

[Claudio Schill (@claudio525)]

Fixed sorting bug in the sigma_mu adjustment factor in
the Kuehn et al. (2020) GMM

[Michele Simionato (@micheles)]

Fixed a subtle filtering bug in conditioned GMFs
Fixed shakemap calculations to take the risk IMTs and not the hazard IMTs
Extended shakemap2gmfs to amplify the GMFs based on user-provided vs30
values
Removed sorting of the sections in the multiFaultSource reader to keep
the rup_idxs consistent with the order in the files
GMFs too small to produce losses now give a warning rather than an error
Fixed bug in get_ry0_distance breaking conditioned GMFs
Made sure lon and lat are rounded to 5 digits in the site collection

[Graeme Wheaterill]

Fixes to bchydro_2016_epistemic, eshm20_craton and lanzano_2019

[Michele Simionato (@micheles)]

Extended sensitity analysis to file parameters
Introduced new loss type residents

[Marco Pagani (@mmpagani)]

Fixed bug in get_profiles_from_simple_fault_data affecting KiteSurfaces

[Michele Simionato (@micheles)]

Fixed the rupture exporter in presence of GMPETables
Optimized the postclassical phase
Fixed a serious issue in the classical calculator with tiles
Vectorized the association assets <-> hazard sites (up to 32x speedup)
Fixed bug in disaggregation calculations associated to an error
in the line [mag] = np.unique(np.round(ctx.mag, 6))
Optimized the calculation of geohashes by using numba

[Paolo Tormene (@ptormene)]

Extended total_losses to include business_interruption

[Michele Simionato (@micheles)]

Recognized occupants_avg field in the exposure
Optimized reading the exposure (both speed and memory)
Optimized the rupture sampling for MultiFaultSources and improved
the parallelization for MultiFaultSources and MultiPointSources
Made the parameter truncation_level mandatory
Fixed the damage state header in the aggrisk outputs
Changed the order of the IMTs to be by period and not lexicographic
Fixed the realizations extractor for scenario calculations reading
ruptures with multiple different tectonic region types

[Kendra Johnson (@kejohnso)]

Fixed a bug in disaggregation by Lon-Lat-TRT when there are multifault
ruptures and multiple sites

[Michele Simionato (@micheles)]

Deprecated the conversion in the exposure
Added new loss types "area" and "number" for usage in risk calculations
Made the contexts immutable and fixed mutability bugs potentially
affecting the modules akkar_bommer_2010, bindi_2011_ipe, can_shm6_inslab,
lanzano_2019, youd_etal_2002, zhang_zhao_2005, cauzzi_faccioli_2008_swiss,
chiou_youngs_2008_swiss campbell_bozorgnia_2014
Changed the disaggregation algorithm (see manual)
Added a view risk_by_rup
Internal: added command oq compare med_gmv <imt>
Extended extract/ruptures to extract a single rupture given the rup_id
Fixed a bug in classical_damage causing a ValueError: could not broadcast
input array from shape (X,Y) into shape (X,)
Added check for missing site parameters

[Paolo Tormene (@ptormene)]

Internal: reduced the size of the gsim column in the view
required_params_per_trt in the .rst report

[Michele Simionato (@micheles)]

Optimized the generation of events
Fixed the GMFs CSV importer so that it can import exported GMFs even in
the presence of a custom_site_id
Implemented sampling for mutex sources with grp_probability
Introduced extreme_gmv parameter in the job.ini
Introduced a new output "Annual Frequency of Events" and refined the command
oq plot event_based_mfd?
Moved the minimum magnitude filtering after the rupture sampling
Changed the internals of rupture sampling and fixed the generated MFD
in the case of a same source appearing in multiple logic tree branches
Fixed a bug in scenario calculations causing a mysterious IndexError in the
line rctx.clon = c.lons[rctx.sids]
Changed the disaggregation exporter to add a column with the IML
associated to the PoE for the mean disaggregation outputs
Internal: added a parameter override_vs30 in the job.ini

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Fixed a bug in the source writer when writing NonParametricSources

[Michele Simionato (@micheles)]

Changed the conditional spectrum storage and fixed the exporter to export
all PoEs

[Marco Pagani (@mmpagani)]

Fixed conditional spectrum with investigation time different from 1 year

[Michele Simionato (@micheles)]

Fixed documentation of the Extractor for hazard curves and maps
Fixed bug in oq zip: when starting from a directory different from the
job.ini directory, .hdf5 files for nonparametric ruptures were not included
Honored minimum_magnitude for MultiFaultSources
Overhauled the experimental postprocessing framework and fixed many
bugs in the compute_mrd postprocessor

[Anirudh Rao (@raoanirudh)]

Added a verification test suite for the Conditioned GMFs calculator

[Michele Simionato (@micheles)]

Added output rates_by_src
Fixed method '/v1/ini_defaults' returning NaNs for the site parameters
Added output mean_disagg_bysrc for single site calculations with use_rates
Added command oq reduce_smlt
Fixed reinsurance calculations when there is a single loss type
Changed the source seed generation to not depend on the source ID suffix
Tracebacks were printed 3 times in the console: this is finally fixed
Forbidden duplicate sources in atomic groups
Forbidden duplicate mutually exclusive sources
Fixed conditional spectrum calculator with multiple sites and implemented
parallelization
Fixed the source ID generation for CollapsedPointSources
Fixed the error message for wrong weights in the gsim logic tree file
Fixed KeyError in calculations using the EasternCan15Mid with PGA
Replaced hard-coded limit to 25 levels in the MRD postprocessor
with a simple warning

[Kendra Johnson (@kejohnso)]

Added a job parameter 'reqv_ignore_sources', which indicates which
sources - specified as a list of thier source ids - should not be collapsed
using the equivalent distance

[Michele Simionato (@micheles)]

Extended (re)insurance calculations to manage insurance deductibles at asset
level
Internal: added a parameter use_rates to compute statistical hazard curves
in terms of rates, not probabilities
Internal: changed disagg_by_src to work in terms of rates
Added command oq runsite <lon>,<lat> for mosaic PSHA calculations
Enhanced --reuse-input to reuse preclassical calculations if possible
Normalized IMTs in the minimum_intensity dictionary (i.e. SA(1.)=>SA(1.0))
Fixed event based calculations running out of memory due to hazard curve
arrays being instantiated without need

[Julián Santiago Montejo Espitia (@SantiMon1417)]

Contributed the GMPE Arteta et al. (2023) for crustal events on northern
South America

[Michele Simionato (@micheles)]

Setting both sites and site_model_file now works also for models
with nonstandard site parameters, like EUR

[Paolo Tormene (@ptormene)]

Added AELO mode for the engine webui, providing a web form to insert input
values and launch a calculation. Input values are validated and the
progress of the calculation can be monitored through the webui. On
completion, the server sends an email to the job owner, providing feedback
about failure or completion, with a link to retrieve a traceback or to
browse the calculation outputs. Corresponding tests were added and the
existing tests for the webui were improved, cleaning up calculations after
they have been checked.

[Michele Simionato (@micheles)]

Changed the default: now all realizations are consider in disaggregation
calculations, not only the one closest to the mean hazard curve
Added feature infer_occur_rates for multiFault sources
Sampling the ruptures was raising a BlockingIOError sometimes

[Kendra Johnson (@kejohnso)]

Made it possible to use extendModel for multiple branchSets in the
source model logic tree

[Michele Simionato (@micheles)]

Added an exporter for disagg_by_src
Implemented mean disaggregation outputs, with the means computed in terms
of rates, not probabilities

[Kendra Johnson (@kejohnso)]

Corrected a bug when using the reqv feature: all sources were
collapsed instead of the ones with the specified tectonic region types

[Michele Simionato (@micheles)]

Changed the lon-lat binning algorithm to produce 4 times less bins
Fixed install.py: it must not set the shared_dir
Optimized disaggregation in performance, memory consumption and data
transfer; also changed the task distribution algorithm

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 2 years ago

OpenQuake - OpenQuake Engine 3.16.5

[Kendra Johnson (@kejohnso)]

Added DEFINED_FOR_INTENSITY_MEASURE_COMPONENT = const.IMC.RotD50 to
the NGAEast and NSHMP2014 GMPEs

[Michele Simionato (@micheles)]

Fixed the hanging of classical calculations due to large zmq packets

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 2 years ago

OpenQuake - OpenQuake Engine 3.16.4

[Antonio Ettorre (@vot4anto)]

Upgraded Django version to version 3.2.19 with some security fixes

[Claudio Schill (@claudio525)]

Fixed sorting bug in the sigma_mu adjustment factor in
the Kuehn et al. (2020) GMM

[Graeme Weatherill (@g-weatherill)]

Fixes to bchydro_2016_epistemic, eshm20_craton and lanzano_2019

[Marco Pagani (@mmpagani)]

Fixed bug in get_profiles_from_simple_fault_data affecting KiteSurfaces

[Michele Simionato (@micheles)]

Fixed tau-phi inversion in LanzanoLuzi2019
Fixed shakemap calculations to take the risk IMTs and not the hazard IMTs
Removed sorting of the sections in the multiFaultSource reader to keep
the rup_idxs consistent with the order in the files
GMFs too small to produce losses now give a warning rather than an error
Fixed a subtle filtering bug breaking conditioned GMFs
Fixed bug in get_ry0_distance breaking conditioned GMFs
Fixed the rupture exporter in presence of GMPETables
Fixed severe bug affecting the classical tiling calculator
Fixed rounding bug in disaggregation (in the line
[mag] = np.unique(np.round(ctx.mag, 6)))
Fixed the error message for wrong weights in the gsim logic tree file

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 2 years ago

OpenQuake - OpenQuake Engine 3.16.3

[Kendra Johnson (@kejohnso)]

Fixed a bug in hazardlib.geo.surface.multi affecting the disaggregation

[Michele Simionato (@micheles)]

Fixed the damage state header in the aggrisk outputs
Fixed the realizations extractor for scenario calculations starting from
ruptures with multiple different tectonic region types
Fixed a subtle mutability bug affecting various GMPEs
Fixed rupture sampling in the case of a same source appearing in multiple
logic tree branches
Fixed a bug in classical_damage causing a ValueError: could not broadcast
input array from shape (X,Y) into shape (X,) in some situations
Fixed the CSV importer so that it can import exported GMFs even in
the presence of a custom_site_id

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 2 years ago

OpenQuake - OpenQuake Engine 3.16.2

[Michele Simionato (@micheles)]

Fixed a bug in extract_agg_curves affecting the QGIS plugin
Fixed a bug in scenario calculations causing a mysterious IndexError in the
line rctx.clon = c.lons[rctx.sids]

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Fixed a bug in the source writer when writing NonParametricSources
Removed the conditional spectrum calculator (it was experimental and
not working properly in many cases)

[Michele Simionato (@micheles)]

Now we log only when the DB schema is really upgraded
Fixed the documentation about extracting hazard curves and maps
Fixed KeyError in calculations using the EasternCan15Mid with PGA
The minimum_magnitude parameter was ignored for multiFault sources
Replaced hard-coded limit to 25 levels in the MRD postprocessor
with a simple warning
Fixed bug in oq zip: when starting from a directory different from the
job.ini directory, .hdf5 files for nonparametric ruptures were not included
Fixed a 32 bit / 64 bit bug in the hinge function of Kuehn et al. (2020)
Forbidden source IDs with dots (".")

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 2 years ago

OpenQuake - OpenQuake Engine 3.16.1

[Michele Simionato (@micheles)]

Fixed event based calculations running out of memory due to hazard curve
arrays being instantiated without need
Sampling the ruptures was raising a BlockingIOError sometimes

[Kendra Johnson (@kejohnso)]

Fixed bug in extendModel with multiple branchsets

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 2 years ago

OpenQuake - OpenQuake Engine 3.16.0

[Matteo Nastasi (@nastasi-oq)]

Bumped oq-libs to version 3.8.0

[Michele Simionato (@micheles), Paolo Tormene (@ptormene)]

Fixed the agg_curves extractor to also extract individual curves, thus
fixing the QGIS plugin visualization

[Michele Simionato (@micheles)]

Optimized the rupture sampling for MultiFaultSources
Supported collect_rlzs together with disagg_by_src to avoid running
out of memory in models with many realizations and many sources

[Marco Pagani (@mmpagani)]

Implemented the correction of Lanzano et al. (2019) as described in
Lanzano et al. (2022)

[Michele Simionato (@micheles)]

Fixed disaggregation calculations when too few tasks were generated

[Anirudh Rao (@raoanirudh), Michele Simionato (@micheles)]

Changed the algorithm used in classical_risk and classical_bcr
calculations to take into account the risk_investigation_time

[Michele Simionato (@micheles)]

Added a warning for pointlike sources with magnitudes >= 8
Optimized the disaggregation by using our own truncnorm_sf function and
not scipy

[Anirudh Rao (@raoanirudh)]

Implemented Conditioned GMFs as defined in Engler et al. (2022)

[Matteo Nastasi (@nastasi-oq)]

Removed python3-engine-master and python3-engine-worker ubuntu packages

[Michele Simionato (@micheles)]

Internal: renamed rup_id -> seed in event based calculations to avoid
confusion
Parallelized the reinsurance calculation

[Marco Pagani (@mmpagani)]

Completed implementation of grp_probability for SourceGroups with mutex
ruptures

[Michele Simionato (@micheles)]

Huge improvement of the task distribution when using zmq; also, used
less memory on the master node
Fixed context collapsing in the multi-site case
Optimised postclassical, both in terms of memory in the master node
and reading performance in the workers
Added parallel.multispawn facility in oq engine --multi --run
Improved oq engine --multi to run at most 5 jobs at the time
Fixed bug introduced in engine 3.9 producing wrong numbers in the case of
mutex ruptures (i.e. the New Madrid cluster in the USA model)
Added --venv option to the universal installer to install in custom places
Bug fix: the avg_losses-rlzs output in classical risk was stored
incorrectly
Speed up preclassical calculations in presence of complex fault sources and
similar; for instance there is a 3x speedup for the SAM model

[Trevor Allen (@treviallen)]

Added class Allen2012_SS14 to Allen (2012) GMM to incorporate Vs30 scaling
based on Seyhan & Stewart (2014) amplification factors
Added classes SomervilleEtAl2009NonCratonic_SS14 and
SomervilleEtAl2009YilgarnCraton_SS14 to Somerville et al (2009) GMM to
incorporate Vs30 scaling based on Seyhan & Stewart (2014) amplification
factors
Added Allen (2022) GMM for Banda Sea earthquakes observed in the North
Australian Craton

[Michele Simionato (@micheles)]

Moved the horiz_comp_to_geom_mean functionality from the gsim logic tree
to the job.ini
Optimized the parsing of XML files (35x for the Alaska model)
Bug fix: ModifiableGMPEs with underlying tables were not receiving a
single magnitude when calling the compute method
Rewritten ModifiableGMPE to avoid a subtle memory leak

[Matteo Nastasi (@nastasi-oq)]

Add WEBUI_PATHPREFIX variable to add a prefix to usual webui web path

[Michele Simionato (@micheles)]

Fixed the exporter of avg_losses-stats when using collect_rlzs=true
Mitigate the issue of ultra-long planar ruptures affecting many models
Forced the usage of collect_rlzs for large exposures when computing
avg_losses with many realizations
Bug fix: min_mag and max_mag were not honored when using a
magnitude-dependent maximum distance
Reduced the memory consumption when using the ps_grid_spacing approximation

[Marco Pagani (@mmpagani)]

Added a method to compute the mean rate density of two IMTs
Added the possibility to define the edges of disaggregation bins
Added a couple of utility methods to the weichert module

[Michele Simionato (@micheles)]

Introduced automatic tiling for extra-large calculations
Added command oq purge failed
Fixed memory leak in the tiling calculator by resetting the ProcessPool
Fixed an indexing error breaking the GMPE AtkinsonBoore2006 with
stress drop adjustment

[C. Bruce Worden]

Extended the AbrahamsonEtAl2014 GMPE to extrapolate the vs30

[Michele Simionato (@micheles)]

Reduced the number of warnings when the horizontal component is not
convertible into geometric mean in ModifiableGMPE

[Guillaume Daniel (@guyomd)]

Added the highest magnitude bin to the Stepp (1972) completeness analysis
and improved the legend in the plot

[Manuela Villani (@mvillani)]

Added the Wong et al. (2022) ground motion model

[Marco Pagani (@mmpagani)]

Added a constant_term attribute to the faultActivityData uncertainty
model, with a default value of 9.1 for backward compatibility

[Michele Simionato (@micheles)]

Removed support for Python < 3.8

[Prajakta Jadhav (@Prajakta-Jadhav-25), Dharma Wijewickreme (@Dharma-Wijewickreme)]

Added GMPE for Zhang and Zhao(2005)

[Michele Simionato (@micheles)]

Added conditional spectrum calculator in hazardlib
Fixed the performance report that failed for NGAEast GMPEs
Reduced the memory consumption in large classical calculations
(in make_contexts)

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Fixed the calculation of conditional spectra

[Michele Simionato (@micheles)]

Corner case bug: in the presence of parent site collection and a child
site model the engine was associating the site model to the parent
Strongly reduced the slow tasks and memory consumption in event_based_risk
calculations and removed the need to keep (a lot of) the gmf_data table
in memory
Fixed a performance issue in event based risk calculations, due to the same
aggregation IDs being computed for each task
Increased sys.recursionlimit to solve a rare pickling issue and made
the engine continue even if _csm is not stored
Added a parameter max_aggregations with a default of 100,000
Changed the risk calculators to reuse the hazard exposure (if any)

[Antonio Ettorre (@vot4anto)]

Bumped h5py to version 3.7.0 and added fiona 1.8.21

[Marco Pagani (@mmpagani)]

Added the get_dip and get_strike methods to the gridded surface

[Michele Simionato (@micheles)]

Added calculation of reinsurance-risk_by_event and reinsurance-aggcurves
given a reinsurance.xml file
Supported absolute values for deductible and insurance_limit
in insurance calculations (before we supported only fractions)
Reduced precision in the header of the UHS csv files
Managed the case with CoVs = 0 for the beta distribution

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 2 years ago

OpenQuake - OpenQuake Engine 3.15.0

[Michele Simionato (@micheles)]

Added a check on sum(srcs_weights) == 1 for mutex sources

[Kendra Johnson (@kejohnso)]

Fixed disaggregation by lon, lat in presence of multiFaultSources

[Michele Simionato (@micheles)]

Replaced command oq download_shakemap with oq shakemap2gmfs
Raised an error for missing required IMTs in ShakeMap grid files
Extended the custom_site_id to 8 characters
Restricted the accepted characters in risk IDs
Extended disagg_by_src to mutually exclusive sources (i.e. Japan)
and managed "colon" sources specially

[Anne Hulsey (@annehulsey)]

Contributed Mag_Dist_TRT and Mag_Dist_TRT_Eps disaggregations

[Michele Simionato (@micheles)]

Internal: added a way to disable the DbServer from openquake.cfg or
by setting OQ_DATABASE=local
Implemented total_losses, even for insurance calculations
Optimized "computing risk" in the event_based_risk calculator (~30% faster)
Changed the magnitude binning formula, thus fixing some disaggregation
calculations (for instance when there is a single magnitude for a TRT)
Changed the aggrisk/aggcurves exporters in presence of insurance losses
Internal: changed how avg_losses, src_loss_table and agg_curves-stats are
stored to simplify the management of secondary losses
Internal: we have now repeatable rupture IDs in classical PSHA

[Pablo Iturrieta (@pabloitu)]

Added support for negative binomial temporal occurrence models

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Added support for disaggregation in case of mutually exclusive sources

[Michele Simionato (@micheles)]

Fixed error message when trying to compute disagg_by_src with too many
sources: in some cases, it contained a misleading reference to point sources
Reorganized the Advanced Manual; changed the theme to be consistent with
the OpenQuake manual
Internal: added command oq db engine_version
Added a check for required site parameters not passed correctly
Fixed ps_grid_spacing approximation when the grid is degenerate
Logging a warning when starting from an old hazard calculation
The extra fields of the site collection were lost when using --hc

[Julián Santiago Montejo Espitia]

Implemented the Arteta et al. (2021) subduction model for Colombia

[Michele Simionato (@micheles)]

Added host field to the job table (dbserver restart required)
--exports=csv was not honored for the realizations output; now it is

[Paul Henshaw (@pslh), Sandra Giacomini]

Ported the OpenQuake manual from latex to reStructuredText format

[Michele Simionato (@micheles)]

Entered automatically in sequential mode if there is not enough memory
Raised an early error for missing risk IDs in the vulnerability files
Changed the definition of aggrisk again to ensure consistency with the
average losses

[Tom Son (@SnowNooDLe)]

Added width and hypo_depth estimation to Campbell and Bozorgnia (2014)

[Michele Simionato (@micheles)]

Improved the precision of the ps_grid_spacing approximation
Added a check for missing mags when using GMPETables
Fixed a bug in upgrade_nrml -m for point sources with different usd/lsd
Automatically discard irrelevant TRTs in disaggregation calculations

[Ashta Poudel, Anirudh Rao (@raoanirudh), Michele Simionato (@micheles)]

Added the ability to run connectivity analysis in event_based_damage
and scenario_damage calculation with an appropriate exposure

[Tom Son (@SnowNooDLe), Michele Simionato (@micheles)]

Added ztor estimation to Campbell and Bozorgnia (2014)

[Michele Simionato (@micheles)]

Internal: removed REQUIRES_COMPUTED_PARAMETERS
Using PointMSR when the reqv approximation is enabled
Internal: changed the rupture storage for classical calculations
Optimized rupture instantiation for point sources
Optimized distance calculations for point sources

[Tom Son (@SnowNooDLe), Claudio Schill]

Simple performance improvement of Kuehn et al. 2020 model

[Michele Simionato (@micheles)]

Changed again the string representation of logic tree paths and added
an utility hazardlib.lt.build to build trees from literal lists
Fixed the field source_info.trti in the datastore to point to the
correct tectonic region type index and not to zero
Added a check for inconsistent IDs across different kinds of risk functions
(i.e. fragility and consequence)
Fixed a logging statement that could run out of memory in large calculations
Optimized iter_ruptures for point sources by vectorizing the generation of
planar surfaces by magnitude, nodal plane and hypocenter

[Tom Son (@SnowNooDLe)]

Implemented a missing piece in Chiou & Youngs (2014) model
Predicted PSA value at T ≤ 0.3s should be set equal to the value of PGA
when it falls below the predicted PGA

[Marco Pagani (@mmpagani)]

Added the possibility of disaggregating in terms of epsilon*
Added a method to compute the cross-correlation matrix
Added Hassani & Atkinson (2018)
Added Hassani & Atkinson (2020)

[Michele Simionato (@micheles)]

Fixed disaggregation returning NaNs in some situations with
nonParametric/multiFaultSources
Bug fix: not storing far away ruptures coming from multiFaultSources
Implemented CScalingMSR
Optimized context collapsing in classical calculations
Setting ps_grid_spacing now sets the pointsource_distance too
Saving memory in preclassical calculations on machines with 8 cores or less
Changed the magnitude-dependent maximum_distance feature to discard
ruptures below minmag and above maxmag
Added the ability to estimate the runtime of a calculation by using
the --sample-sources option
Fixed a wrong formula in modifiable_gmpe.add_between_within_stds
Reduced the stress on the memory in classical calculations, thus
improving the performance
Setting the truncation_level to the empty string is now forbidden;
some GMFs calculations not setting truncation_level can now give
different results since truncation_level=None is now replaced with
truncation_level=99

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 2 years ago

OpenQuake - OpenQuake Engine 3.14.0

Release 3.14.0

[Michele Simionato (@micheles)]

Changed the definition of aggrisk: dividing by the effective time
Internal: removed flag save_disk_space since now it is always on
Slightly changed the collapsing of nodal planes and hypocenters in
presence of the equivalent distance approximation (reqv)
Extended oq reduce_sm to multiFaultSources
Fixed the check on unique section IDs for multiFaultSources
Implemented multi-aggregation with a syntax like
aggregate_by=taxonomy,region;taxonomy;region
Removed the obsolete commands oq to_shapefile and oq from_shapefile
and turned pyshp into an optional dependency
Setting num_rlzs_disagg=0 is now valid and it means considering
all realizations in a disaggregation calculation
Rounded the magnitudes in multiFaultSources to two digits

[Marco Pagani (@mmpagani)]

Extended ModifiableGMPE to work with GMPETable and subclasses

[Michele Simionato (@micheles)]

Upgraded shapely from version 1.7 to version 1.8: this causes slight
changes in the results for most calculations
Removed the not used (and not working) functionality applyToSourceType
Raised an error when the total standard deviation is zero, unless
truncation_level is set to zero

[Tom Son (@SnowNooDLe)]

Fixed a typo and a few bugs within Kuehn et al. (2020) model to include
Z2.5 when the given region is JAPAN

[Michele Simionato (@micheles)]

Changed /extract/events to return events sorted by ID
Changed the default amplification method to "convolution"
Fixed a bug with discard_trts sometimes discarding too much
Raised a helpful error message when ensurepip is missing
Fixed parentdir bug in event_based_damage
Fixed sorting bug in the /v1/calc/run web API
Internal: introduced limited unique rupture IDs in classical calculations
with few sites

[Prajakta Jadhav (@Prajakta-Jadhav-25), Dharma Wijewickreme (@Dharma-Wijewickreme)]

Added GMPE Youd et al. (2002) and the corresponding site parameters

[Michele Simionato (@micheles)]

Fixed the exporter aggrisk-stats in the case of zero losses
Vectorized all GMPEs and forbidden non-vectorized GMPEs
Raised the limit to 94 GMPEs per tectonic region type
Optimized the NBCC2015_AA13 GMPEs
Optimized the GMPETable and the derived NGAEast GMPEs
Fixed a 32/64 bit bug in oq export loss_maps-stats

[Marco Pagani (@mmpagani)]

Added a more flexible version of the GC2 implementation
Added caching of distances in multi fault ruptures
Added the NRCan site term to the modifiable GMPE

[Michele Simionato (@micheles)]

Optimized .get_bounding_box, .polygon and .mesh_size for MultiFaultSources
Fixed bug in presence of mixed vectorized/nonvectorized GMPEs
Extended oq postzip to multiple files and oq abort to multiple jobs
Internal: changed install.py to install the venv in /opt/openquake/venv
Fixed a BOM issue on Windows when reading job.ini files

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 3 years ago

OpenQuake - OpenQuake Engine 3.11.5

Release 3.11.5

[Michele Simionato (@micheles)]

Fixed a 32/64 bit bug in oq export loss_maps-stats

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 3 years ago

OpenQuake - OpenQuake Engine 3.13.0

[Michele Simionato (@micheles)]

Improved the precision of the pointsource_distance approximation
Added command oq show rlz:<no>
Internal: added an environment variable OQ_DATABASE

[Manuela Villani (@mvillani)]

Added a function in the modifiable GMPE to convert ground-motion assuming
different representations of the horizontal component.

[Kendra Johnson (@kejohnso)]

Implemented possibility of assigning the parameters floating_x_step and
floating_y_step for kite fault sources in the job configuration file

[Michele Simionato (@micheles)]

The branchID is now autogenerated in the gsim logic tree files, thus
solving the issue of wrong branch paths for duplicated branchIDs
Added a check for missing gsim information in the job.ini file
Fixed the case of continuous fragility functions with minIML=noDamageLimit

[Miguel Leonardo-Suárez (@mleonardos)]

Added GMPE from Jaimes et al. (2020) for Mexican intraslab earthquakes

[Michele Simionato (@micheles)]

Enforced ps_grid_spacing <= pointsource_distance
Internal: added command oq plot source_data?
The engine is now splitting the MultiFaultSources, thus improving the task
distribution

[Claudia Mascandola (@mascandola)]

Added a new class to the abrahamson_2015 gmm.
Added a new class to the lanzano_luzi_2019 and skarlatoudis_2013 gmms

[Marco Pagani (@mmpagani), Shreyasvi Chandrasekhar (@Shreyasvi91)]

Added GMM from Bora et al. (2019)
Fixed bug in the multifault surface when defined using kite fault surfaces

[Giuseppina Tusa (@gtus23)]

Added a new gsim file tusa_langer_azzaro_2019.py to implement the GMMs
from Tusa et al. (2020).

[Michele Simionato (@micheles)]

Added command oq compare uhs CALC_1 CALC_2
split_sources=false is now honored in disaggregation calculations
Internal: rup/src_id now refers to the row in the source_info table

[Miguel Leonardo-Suárez (@mleonardos)]

Added the GMPE Arroyo et al. (2010) for Mexican subduction interface events

[Marco Pagani (@mmpagani)]

Added a new method to the modifiable GMPE with which is possible to
compute spatially correlated ground-motion fields even when the initial
GMM only provides the total standard deviation.
Fixed a bug in the modify_recompute_mmax
Added a get_coeffs method to the CoeffTable class
Added support for EAS, FAS, DRVT intensitity measure types

[Michele Simionato (@micheles)]

Extended the mag-dependent filtering to the event based calculator
The flag discrete_damage_distribution=true was incorrectly ignored when
computing the consequences
Implemented reaggregate_by feature
Supported the custom_site_id in the GMF exporters
Bug fix: the site collection of the child calculation was ignored when
using the --hazard-calculation-id option
Supported Python 3.9 and deprecated Python 3.6
Extended oq prepare_site_model to support .csv.gz files
Solved the issue of "compute gmfs" slow tasks in event_based and used
the same approach in classical calculations too
Made sure valid.gsim instantiates the GSIM
ShakeMap calculations failing with a nonpositive definite correlation
matrix now point out to the manual for the solution of the problem
Introduced the GodaAtkinson2009 cross correlation between event model
Specifying consequence files without fragility files now raises an error
Fixed a bug in event_based_risk with nontrivial taxonomy mapping producing
NaNs in the event loss table
Internal: added kubernetes support from the WebUI

[Shreyasvi Chandrasekhar (@Shreyasvi91)]

Added a new GMPE for significant duration proposed by Bahrampouri et al
(2021).

[Claudia Mascandola (@mascandola)]

Added the computation of tau and phi stdevs to the sgobba_2020 GMPE
Added a new class to the lanzano_2019 gmm.

[Michele Simionato (@micheles)]

Changed completely the storage of the PoEs and reduced the memory
consumption in classical calculations (plus 4x speedup in "postclassical")
Changed the behavior of sites_slice
Changed custom_site_id to an ASCII string up to 6 characters
Fixed the error raised in presence of a mag-dep distance for a tectonic
region type and a scalar distance for another one

[Yen-Shin Chen (@vup1120)]

Added the Thingbaijam et al. (2017) Magnitude Scaling Law for Strike-slip

[Michele Simionato (@micheles)]

Changed the API of ContextMaker.get_mean_stds
Extended the WebUI to run sensitivity analysis calculations
Changed the string representation of logic tree paths and enforced a
maximum of 64 branches per branchset
Added command oq info disagg
Accepted site models with missing parameters by using the global site
parameters instead
Supported the syntax
source_model_logic_tree_file = ${mosaic}/XXX/in/ssmLT.xml
Fixed a performance bug with ignore_master_seed=true
Added a command oq info cfg to show the configuration file paths
Added a check on the intensity measure levels with --hc is used
Bug fix: pointsource_distance = 0 was not honored
Fixed a small bug of oq zip job_haz.ini -r job_risk.ini: now it works
even if the oqdata directory is empty
Optimized the aggregation of losses in event_based_risk and made it possible
to aggregate by site_id for more than 65,536 sites
Fixed the calculation of average insured losses with a nontrivial taxonomy
mapping: now the insured losses are computed before the average procedure,
not after
Unified scenario_risk with event_based_risk, changing the numbers
when producing discrete damage distributions
Added aggrisk output to event based damage calculation
Added parameter discrete_damage_distribution in scenario damage
calculations and changed the default behavior
Deprecated consequence models in XML format
Event based damage calculations now explicitly require to specify
number_of_logic_tree_samples (before it assumed a default of 1)

[Elena Manea (@manea), Laurentiu Danciu (@danciul)]

Added the GMPE Manea (2021)

[Michele Simionato (@micheles)]

Added a check against duplicated branchset IDs
Improved error checking when reading the taxonomy mapping file
Renamed conversion -> risk_id in the header of the taxonomy mapping file

[Antonio Ettorre (@vot4anto)]

Bumped h5py to version 3.1.0

[Michele Simionato (@micheles)]

Renamed the parameter individual_curves -> individual_rlzs
Reduced the number of disaggregation outputs and removed the long-time
deprecated XML exporters
Fixed the ShakeMap calculator failing with a TypeError:
get_array_usgs_id() got an unexpected keyword argument 'id'
Added conseq_ratio in the aggcurves exporter for event_based_damage
Added a conditional_spectrum calculator
Fixed an array<->scalar bug in abrahamson_gulerce_2020
Restored the classical tiling calculator

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 3 years ago

OpenQuake - OpenQuake Engine 3.12.1

[Michele Simionato (@micheles)]

Fixed a bug in event_based_risk with nontrivial taxonomy mapping producing
NaNs in the event loss table
Bug fix: pointsource_distance = 0 was not honored
Improved the universal installer when installing over a previous
installation from packages
Fixed an error in oq zip job_haz.ini -r job_risk.ini
Fixed the disaggregation Mag_Lon_Lat exporter header mixup
Fixed the ShakeMap calculator failing with a TypeError:
get_array_usgs_id() got an unexpected keyword argument 'id'
Fixed the Abrahamson Gulerce GMPE failing with a scalar<->array error

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 3 years ago

OpenQuake - OpenQuake Engine 3.11.4

[Michele Simionato (@micheles)]

Fixed a bug in the adjustment term in NSHMP2014 breaking the USA model
Fixed the sanity check in event_based_damage giving false warnings
Fixed the corner case when there are zero events per realization in
scenario_damage

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 3 years ago

OpenQuake - OpenQuake Engine 3.12.0

[Marco Pagani (@mmpagani)]

Updated verification tables for Abrahamson et al. (2014) and checked
values with other public resources.

[Michele Simionato (@micheles)]

Added command oq info consequences
Improved error message for area_source_discretization too large
Improved command oq info exports
Internal: changed the signature of hazardlib.calc.hazard_curve.classical
Extended the multi-rupture scenario calculator to multiple TRTs
Removed the experimental feature pointsource_distance=?
Refactored the GMPE tests, with a speedup of 1-14 times
Added a script utils/build_vtable to build verification tables
oq info gsim_logic_tree.xml now displays the logic tree
Fixed a bug in the adjustment term in NSHMP2014 breaking the USA model

[Graeme Weatherill (@g-weatherill)]

Implements Abrahamson & Gulerce (2020) NGA Subduction GMPE

[Nico Kuehn (@nikuehn)/Graeme Weatherill (@g-weatherill)]

Implements Kuehn et al. (2020) NGA Subduction GMPE

[Chung-Han Chan/Jia-Cian Gao]

Implements Lin et al. (2011)

[Graeme Weatherill (@g-weatherill)/Nico Kuehn (@nikuehn)]

Implements Si et al. (2020) NGA Subduction GMPE

[Michele Simionato (@micheles)]

There is now a huge speedup when computing the hazard curve statistics
if numba is available
Made it possible to compute consequences in presence of a taxonomy mapping
Fixed a bug in get_available_gsims: GSIM aliases were not considered
Optimized the single site case by splitting the sources less
Restricted the acceptable methods in GMPE subclasses

[Claudia Mascandola (@mascandola)]

Added the Lanzano et al. (2020) GMPE

[Stanley Sayson (@stansays)]

Added the Stewart et al. (2016) GMPE for V/H
Added the Bozorgnia and Campbell (2016) GMPE for V/H
Added the Gulerce and Abrahamson (2011) GMPE
Corrected Campbell and Bozorgnia (2014) GMPE

[Michele Simionato (@micheles)]

Fixed a subtle bug: in presence of a nontrivial taxonomy mapping, loss
curves could be not computed due to duplicated event IDs in the event
loss table coming from a int->float conversion
Forced a name convention on the coefficient tables (must start with COEFFS)
Replaced IMT classes with factory functions
Changed the minimum_distance from a parameter of the GMPE to a
parameter in the job.ini
Supported consequences split in multiple files

[Claudia Mascandola (@mascandola)]

Added the Sgobba et al. (2020) GMPE

[Michele Simionato (@micheles)]

Improved the warning on non-contributing TRTs and made it visible
for all calculators
Fixed a bug in scenarios from CSV ruptures with wrong TRTs
Added a limit of 12 characters to IMT names
Forbidded multiple inheritance in GMPE hierarchies
Added parameter ignore_encoding_errors to the job.ini
Extended the damage calculators to generic consequences
Renamed cname -> consequence in the CSV input files
Made sure the CSV writer writes in UTF-8

[Graeme Weatherill (@g-weatherill)]

Updates Kotha et al. (2020) slope/geology model coefficients

[Michele Simionato (@micheles)]

Improved post_risk to use all the cores in a cluster, since it
was using the master only
Improved the validation of the investigation_time in event_based_damage
Renamed the losses_by_event CSV exporter to risk_by_event and
made it work consistently for losses, damages and consequences; also
removed the no_damage field

[Marco Pagani (@mmpagani), Michele Simionato (@micheles)]

Implemented MultiFaultSources
Added method for computing rjb to kite surfaces
Added support for new epistemic uncertainties in the SSC LT

[Michele Simionato (@micheles)]

Fixed newlines in the CSV exports on Windows

[Graeme Weatherill (@g-weatherill)]

Added Ameri (2014) GMPE for the Rjb case

[Michele Simionato (@micheles)]

Optimized the slow tasks in event_based calculations
Added an early check for fragility functions in place of vulnerability
functions or viceversa

[Marco Pagani (@mmpagani)]

Numeric fix to the amplification with the convolution method
Implemented the BakerJayaram2008 cross correlation model
Fixed the calculation of distances for kite surfaces with Nan values

[Michele Simionato (@micheles)]

Fixed logic tree bug: MultiMFDs were not modified
Internal: added a view composite_source_model to show the sources by group

[Nicolas Schmid (@schmidni)]

Added possibility to use *.shp files instead of *.xml files when
doing risk calculations from shakemaps.

[Michele Simionato (@micheles)]

Rewritten the event_based_damage calculation to support aggregate_by
Made it possible to run an event based risk calculation starting from a
parent ran by a different user

[Pablo Heresi (@pheresi)]

Implemented Idini et al (2017) GSIM.
Added dynamic site parameter 'soiltype'

[Michele Simionato (@micheles)]

Added support for traditional disaggregation
Removed the global site parameter reference_siteclass and turned
backarc, z1pt0 and z2pt into dynamic site parameters
Internal: storing the SiteCollection in a pandas-friendly way
Added HDF5 exporter/importer for the GMFs
Replaced XML exposures with CSV exposures in the demos

[Claudia Mascandola (@mascandola)]

Fix to LanzanoEtAl2016 in presence of a "bas" term in the site model

[Nicolas Schmid (@schmidni)]

Improve performance for ShakeMap calculations when spatialcorr and crosscorr
are both set to 'no'
Add feature to do ShakeMap calculations for vulnerability models using MMI.

[Michele Simionato (@micheles)]

Added a flag ignore_master_seed (false by default)
Estimated the uncertainty on the losses due to the uncertainty in the
vulnerability functions in event_based_risk and scenario_risk calculations
Supported exposures with generic CSV fields thanks to the exposureFields
mapping
Honored custom_site_id in the hazard curves and UHS CSV exporters
Added a check for the case of aValue=-Inf in the truncatedGR MFD
Extended the engine to read XML ShakeMaps from arbitrary sources (in
particular local path names and web sites different from the USGS site)
Fixed readinput.get_ruptures to be able to read ruptures in engine 3.11
format
scenario_risk calculations starting from ruptures in CSV format now
honor the parameter number_of_ground_motion_fields

[Nicolas Schmid (@schmidni)]

Optimized spatial covariance calculations for ShakeMaps (more than 10x)
Adjusted logic in cross correlation matrix for ShakeMaps; now calculations
are skipped for corr='no'

[Michele Simionato (@micheles)]

Added a cholesky_limit to forbid large Cholesky decompositions in ShakeMap
calculations
Weighted the heavy sources in parallel in event based calculations
Supported zero coefficient of variations with the beta distribution
Internal: changed how the agg_loss_table is stored
Fixed the avg_losses exporter when aggregate_by=id
Fully merged the calculators scenario_risk, event_based_risk and ebrisk and
ensured independency from the number of tasks even for the "BT" and "PM"
distributions
Storing the agg_loss_table as 64 bit floats instead of 32 bit floats
Changed the algorithm used to generate the epsilons to avoid storing the
epsilon matrix

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 3 years ago

OpenQuake - OpenQuake Engine 3.11.3

[Michele Simionato (@micheles)]

Fixed hdf5.dumps that was generating invalid JSON for Windows pathnames,
thus breaking the QGIS plugin on Windows
Fix a bug when reusing a hazard calculation without aggregate_by for a
risk calculation with aggregate_by
Fixed the aggregate curves exporter for aggregate_by=id: it was exporting
b'asset_id' instead of asset_id

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 4 years ago

OpenQuake - OpenQuake Engine 3.11.2

[Matteo Nastasi (@nastasi-oq)]

Fixed setup.py 'pyproj' dependency

[Antonio Ettorre (@vot4anto)]

Fixed docker builder

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 4 years ago

OpenQuake - OpenQuake Engine 3.11.1

[Michele Simionato (@micheles)]

Fixed memory regression in ebrisk calculations

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 4 years ago

OpenQuake - OpenQuake Engine 3.11.0

[Michele Simionato (@micheles)]

Extended the collapse_logic_tree feature to scenarios and event based
calculations
Extended the taxonomy mapping feature to multiple loss types
The error was not stored in the database if the calculation failed
before starting
Made ground_motion_fields=true mandatory in event_based_risk

[Robin Gee (@rcgee)]

Added a check for missing soil_intensities in classical calculations
with site amplification

[Michele Simionato (@micheles)]

Documented all the parameters in a job.ini file, and removed some
obsolete ones
Added a CSV exporter for the output avg_gmf
Fixed reporting in case of CorrelationButNoInterIntraStdDevs errors
Better error message when the rupture is far away from the sitesa
Made the calculation report exportable with --exports rst
The boolean fields vs30measured and backarc where not cast correctly
when read from a CSV field (the engine read them always as True)
Extended oq plot to draw more than 2 plots
Raised an early error for zero probabilities in the hypocenter distribution
or the nodal plane distribution
Extended the autostart zmq distribution logic to celery and dask
Stored the _poes during the classical phase and not after, to save time
Implemented a memory-saving logic in the classical calculator based on
the memory.limit parameter in openquake.cfg;

[Richard Styron (@cossatot)]

Added TaperedGRMFD to hazardlib

[Michele Simionato (@micheles)]

Fixed a wrong check failing in the case of multi-exposures with multiple
cost types
Removed a check causing a false error "Missing vulnerability function
for taxonomy"
Consequence functions associated to a taxonomy missing in the exposure
are now simply discarded, instead of raising an error
Added a warning when there are zero losses for nonzero GMFs
Added a command oq plot avg_gmf?imt=IMT
Internal: stored avg_gmf as a DataFrame
Honored the individual_curves parameter in avg_losses, agg_losses and
and agg_curves (i.e. by default only expose the statistical results)
Refactored the oq commands and removed the redundant oq help since
there is oq --help instead
Support for input URLs associated to an input archive
Introduced deformation_component parameter in the secondary perils
Optimized the storage of the risk model with a speedup of 60x
for a calculation with ~50,000 fragility functions (2 minutes->2seconds)
and a 3x reduction on disk space
Accepted aggregate_by=id in scenario/event based calculations
Accepted aggregate_by=site_id in scenario/event based calculations
Removed the generation of asset loss maps from event_based_risk
Made the "Aggregate Losses" output in scenario_risk consistent with
event_based_risk and scenario_risk and supported aggregate_by
Perform the disaggregation checks before starting the classical part
Changed the "Aggregate Loss Curves" CSV exporter to generate a file for
each realization, for consistency with the other exporters
The ebrisk outputs "Total Losses" and "Total Loss Curves" are now included
in the outputs "Aggregate Losses" and "Aggregate Curves"
Introduced an agg_loss_table dataset and optimized the generation of
aggregate loss curves (up to 100x speedup)
Removed misleading zero losses in agg_losses.csv
Fixed oq recompute_losses and renamed it to oq reaggregate
Bug fix: ignore_covs=true now sets the coefficient of variations to zero

[Anirudh Rao (@raoanirudh)]

Improved error handling of bad or zero coefficients of variation
for the Beta distribution for vulnerability

[Michele Simionato (@micheles)]

Fixed 32 bit rounding issues in scenario_risk: now the total losses and
and the sum of the average losses are much closer
Internal: made the loss type occupants a bit less special
Documented oq to_nrml

[Claudia Mascandola (@mascandola)]

Added the Lanzano et al. (2019) GMPE

[Michele Simionato (@micheles)]

Honored minimum_asset_loss also in the fully aggregated loss table,
not only in the partially aggregated loss tables and average losses
Bug fixed: the log was disappearing in presence of an unrecognized
variable in the job.ini
Implemented minimum_asset_loss in scenario_risk for consistency
with the ebrisk calculator
Added a command oq plot gridded_sources?
Fixed oq recompute_losses to expose the outputs to the database
Fixed oq engine --run --params that was not working for
the pointsource_distance
Changed the meaning of the pointsource_distance approximation

[Marco Pagani (@mmpagani), Michele Simionato (@micheles), Thomas Chartier (@tomchartier)]

Added experimental version of KiteSource and KiteSurface

[Michele Simionato (@micheles)]

Changed the serialization of ruptures to support MultiSurfaces
Fixed a small bug of logic in the WebUI: if the authentication is
turned off, everyone must be able to see all calculations
Fixed a bug in the calculation of averages losses in scenario_risk
calculations in presence of sites with zero hazard
Optimized the prefiltering by using a KDTree
Experimental: implemented gridding of point sources
Reduced slow tasks due to big complex fault sources
Moved the parameter num_cores into openquake.cfg
Internal: introduced the environment variable OQ_REDUCE
Using pandas to export the GMF in CSV format
Internal: required h5py == 2.10.0
Internal: made the classical ruptures pandas-friendly
Internal: made the damage distributions pandas-friendly

[Marco Pagani (@mmpagani)]

Added a new type of undertainty for the seismic source characterisation
logic tree called TruncatedGRFromSlipAbsolute
Added a get_fault_surface_area method to sources

[Michele Simionato (@micheles)]

Changed the source seed algorithm in event based calculations
Added an estimate of the portfolio damage error due to the seed dependency
Stored the damage distributions in a pandas-friendly way and extended
DataStore.read_df to accept multi-indices

[Viktor Polak (@viktor76525)]

Added the Phung et al. (2020) GMPE

[Michele Simionato (@micheles)]

Implemented truncGutenbergRichterMFD from slip rate and rigidity
Fixed bug when computing the damage distributions per asset and event
Simplified/optimized the UCERF filtering

[Viktor Polak (@viktor76525)]

Added the Chao et al. (2020) GMPE

[Michele Simionato (@micheles)]

Introduced an early memory check in classical calculations
Reduced the memory occupation in classical calculations
Implemented AvgPoeGMPE
Forbidded the usage of aggregate_by except in ebrisk calculations
Added a check on valid branch ID names: only letters, digits and
the characters "#:-_." are accepted
Huge performance improvement for very complex logic trees
Shortened the logic tree paths when exporting the realizations

[Graeme Weatherill (@g-weatherill)]

Refactor of the Kotha et al. (2020) GMM and its adjustments for ESHM20

[Michele Simionato (@micheles)]

Huge speedup in models with src_multiplicity > 1
Fixed bug in source model logic tree sampling with more than 2 branchsets
Fixed hazard maps all zeros for individual_curves=true and more than 1 site
Fixed a bug in oq prepare_site_model when sites.csv is
the same as the vs30.csv file and there is a grid spacing
Speeding up the preclassical calculator
Added an entry point /extract/eids_by_gsim for the QGIS plugin
Internal: automatically convert the source IDs into unique IDs
Changed scenario calculations to depend on the ses_seed, not the
random_seed
Added check on the versions of numpy, scipy and pandas between master and
workers
Added a check for large seed dependency in the GMFs and an estimate of the
portfolio error due to the seed dependency

[Viktor Polak (@viktor76525)]

Added fpeak site parameter
Added the Hassani and Atkinson (2020) GMPE

[Marco Pagani (@mmpagani)]

Added a check on DEFINED_FOR_REFERENCE_VELOCITY when using amplification
Added a method to create a TruncatedGRMFD from a value of scalar seismic
moment
Added a method to the modifiable GMPE to add (or subtract) a delta std
Added a method to the modifiable GMPE to set the total std as the sum of
tau plus a delta

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 4 years ago

OpenQuake - OpenQuake Engine 3.10.1

[Matteo Nastasi (@nastasi-oq)]

Add info to doc about OpenQuake manual path for linux and mac installers

[Laurentiu Danciu (@danciul) and Athanasios Papadopoulos]

Implemented intensity prediction equations for use in the Swiss Risk Model.
The new IPEs refer to models obtained from the ECOS (2009), Faccioli and
Cauzzi (2006), Bindi et al. (2011), and Baumont et al. (2018) studies.
Added new float site parameter 'amplfactor'
Extended the ModifiableGMPE class to allow amplification of the
intensity of the parent IPE based on the ‘amplfactor’ site parameter

[Anirudh Rao (@raoanirudh)]

Fixed the glossary in the manual

[Michele Simionato (@micheles)]

Avoided storing too much performance_data when pointsource_distance is on
Fixed performance regression in classical_risk, classical_damage,
classical_bcr
Fixed oq engine --run job.ini for ShakeMap calculations

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 4 years ago

OpenQuake - OpenQuake Engine 3.10.0

[Richard Styron (@cossatot)]

Added secondary perils ZhuLiquefactionGeneral and HazusLateralSpreading,
supplementing HazusLiquefaction and NewmarkDisplacement

[Michele Simionato (@micheles)]

Fixed a bug with site models containing non-float parameters
Raised the limit on the asset ID from 20 to 50 characters
Changed the /extract/events API to extract only the relevant events
Removed the GMF npz exporter
Speed-up risk saving in scenario_risk and scenario_damage

[Antonio Ettorre (@vot4anto)]

Bumped GDAL to version 3.1.2

[Michele Simionato (@micheles)]

Optimized scenario_damage for the case of many sites
Implemented secondary perils
Fixed a 32 bit/64 bit bug in oq prepare_site_model when sites.csv is
the same as the vs30.csv file
Parallelized by GSIM when there is a single rupture

[Francis Bernales (@ftbernales)]

Added the Stewart et al. (2016) GMPE
Added the Bozorgnia & Campbell (2016) GMPE
Added the Gulerce et al. (2017) GMPE

[Michele Simionato (@micheles)]

Unified source model logic tree sampling with gsim logic tree sampling
Added early_latin and late_latin sampling algorithms
Changed the logic tree sampling algorithm and made it possible to use
both early_weights and late_weights
Restored magnitude-dependent maximum distance
Displaying the hazard maps in the WebUI for debugging purposes
Used the hazard map to get the disaggregation IML from the disaggregation
PoE and added a warning for zero hazard
Internal: implemented multi-run functionality (oq engine --multi --run)
Reduced tremendously the data transfer in disaggregation calculations
Internal: introduced compress/decompress utilities
Reduced the memory and disk space occupation in classical calculations with
few sites; also changed slightly the rupture collapsing mechanism
In disaggregation, force poes_disagg == poes
Fixed multi-site disaggregation: ruptures far away were not discarded,
just considered distant 9999 km

[Marco Pagani (@mmpagani)]

Added a prototype implementation of the kernel method

[Michele Simionato (@micheles)]

Added zipcode site parameter
Added command oq renumber_sm ssmLT.xml

[Robin Gee (@rcgee)]

Set DEFINED_FOR_REFERENCE_VELOCITY for GMPEs modified for Switzerland

[Michele Simionato (@micheles)]

Added parameter max_num_loss_curves to the job.ini file
Changed oq engine --reuse-hazard to just reuse the source model, if
possible
Added command oq recompute_losses <calc_id> <aggregate_by>
Fixed noDamageLimit, minIML, maxIML not being honored in continuous
fragility functions
Unified the scenario calculator with the event based one, with
minor differences in the numbers akin to a change of seed
Fixed a bug in event based when a rupture occurs more than 65535 times
Added a demo EventBasedDamage
Fixed bug in event_based_damage: the number of buildings in no damage
state was incorrect
Added commands oq nrml_to csv and oq nrml_to gpkg
Supported year and ses_id >= 65536 in event based

[Graeme Weatherill (@g-weatherill)]

Implements a heteroskedastic standard deviation model for the Kotha et al.
(2020) GMPE

[Michele Simionato (@micheles)]

Called check_complex_fault when serializing the source in XML
Restored scenario_damage with fractional asset number
Added a view oq extract disagg_by_src
Fixed error with large ShakeMap calculations ('events' not found)
Raised an error when using disagg_by_src with too many point sources
The minimum_magnitude parameter was incorrectly ignored in UCERF

[Iason Grigoratos (@jasongrig)]

Implemented the Zalachoris & Rathje (2019) GMM

[Michele Simionato (@micheles)]

Optimized the disaggregation outputs, saving storage time

[Graeme Weatherill (@g-weatherill)]

Adds PGV coefficients to USGS CEUS GMPE tables (where applicable)

[Michele Simionato (@micheles)]

Removed the disagg_by_src exporter
Internal: added filtering features to the datastore
Calculations with a number of levels non-homogenous across IMTs are
now an error
Implemented rupture collapsing in disaggregation (off by default)
Fixed a bug in the dmg_by_event exporter: the damage distributions could
be associated to the wrong GMPE in some cases
Solved a bug with nonparametric ruptures: due to rounding errors,
the disaggregation matrix could contain (small) negative probabilities
Extended the scenario calculators to compute the statistical outputs
if there is more than one GMPE
Fixed the formula used for the avg_damages-rlzs outputs in event based
damage calculations
Raised an error if investigation_time is set in scenario calculations

[Graeme Weatherill (@g-weatherill)]

Fixed a bug in the mixture model application when running multiple GMPEs

[Michele Simionato (@micheles)]

Replaced outputs losses_by_asset with avg_losses-rlzs, and
dmg_by_asset with ``avg_damages-rlzs`, for consistency with the
event based outputs
Extended the /extract/ API to manage JSON and removed the oqparam API
Added a check on ebrisk to avoid generating too many loss curves
Introduced an output "Source Loss Table" for event based risk calculations
Raised an early error when max_sites_disagg is below the number of
sites in disaggregation calculations
Extended the amplification framework to use different intensity levels
for different amplification functions
Optimized the disaggregation in the case of multiple realizations
Fixed bug in GMF amplification without intensity_measure_types_and_levels
Optimized the computation of the disaggregation PMFs by orders of magnitude
by using numpy.prod
Changed the disaggregation calculator to distribute by magnitude bin,
thus reducing a lot the data transfer
Vectorized the disaggregation formula
Do not perform the disaggregation by epsilon when not required
Introduced management of uncertainty in the GMF amplifi
Changed the disaggregation calculator to distribute by IMT, thus reducing
a lot the data transfer in calculations with many IMTs
Changed /extract/disagg_layer to produce a single big layer
Changed the binning algorithm for lon, lat in disaggregation, to make
sure that the number of bins is homogeneous across sites

[Marco Pagani (@mmpagani)]

Fixed a bug in the ParseNDKtoGCMT parser + updated tests.
Ported the method serialise_to_hmtk_csv implemented in the corresponding
class of the catalogue toolkit + added a test into the GCMTCatalogue class.
Added a modifiable GMPE using the site term of CY14.
Added a generalised modificable GMPE. This first version allows the
definition of the epsilon of the within event residual.

[Michele Simionato (@micheles)]

Introduced a mixed XML+HDF5 format for gridded sources
Internal: added a check on gridded sources: the arrays prob_occurs must
have homogeneous length across ruptures
Removed the dependency from PyYAML, replaced the .yml files in the HMTK with
.toml files and added an utility utils/yaml2toml

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 4 years ago

OpenQuake - OpenQuake Engine 3.9.0

[Michele Simionato (@micheles)]

Fixed a type error in the command oq engine --run --params
Restored the flag split_sources for testing purposes
Fixed a BOM bug in CSV exposures
When exporting the loss curves per asset now we also export the loss ratio
and the inverse return period, for consistency with the other exporters
Fixed the exporter of the loss curves per asset: due to an ordering bug
in some cases it was exporting wrong losses
Added a flag save_disk_space to avoid storing the inputs
Changed the logic underlying the pointsource_distance approximation and
added the syntax pointsource_distance=?
Logged a warning when the pointsource_distance is too small

[Graeme Weatherill (@g-weatherill)]

Implemented Pitilakis et al. (2020) Site Amplification Model

[Michele Simionato (@micheles)]

Fixed an export bug with modal_damage_state=true in scenario_damage
calculations
Fixed a bug in calc_hazard_curves with multiple TRTs
Fixed how AvgGMPE was stored and made it applicable with correlation models
if all underlying GMPEs are such

[Paolo Tormene (@ptormene)]

Added a second tectonic region type to the EventBasedPSHA demo

[Michele Simionato (@micheles)]

Fixed an ordering bug in /extract/rupture_info affecting the QGIS plugin
Fixed oq engine --eo output_id output_dir for the Full Report output
Added year and ses_id to the events table
Removed NaNs in the low return period part of the loss curves
Fixed the tot_curves and tot_losses exporters in ebrisk calculations
Reduced the rupture storage in classical calculations by using compression
Improved the task distribution in the classical calculator, avoiding
generating too few or too many tasks
Enhanced oq check_input to check complex fault geometries
Added a warning against magnitude-dependent maximum_distance

[Marco Pagani (@mmpagani)]

Fixed a bug in the coeff table of YEA97

[Graeme Weatherill (@g-weatherill)]

Implemented support for Gaussian Mixture Model approach to characterise
ground motion model uncertainty

[Michele Simionato (@micheles)]

Enhanced oq reduce_sm to read the source models in parallel
Deprecated the usage of a different number of intensity levels per IMT

[Matteo Nastasi (@nastasi-oq)]

Internal: added 'oq-taxonomy' to docker images

[Michele Simionato (@micheles)]

Extended the pointsource_distance approximation to work on single site
calculations, with a spectacular performance benefit in most calculations
Added Bindi2011, Bindi2014 and Cauzzi2014 scaled GMPEs contributed by
the INGV
Added a check on classical calculations which are too large to run
Added a parameter collapse_level and a new collapsing algorithm
Added a check for missing TRTs in the GSIM logic tree file
Reduced the storage required for site specific calculations
with complex logic trees by removing duplicated ruptures
Restored the computation of the mean disaggregation when multiple
realizations are requested
Slightly changed the syntax of oq info (see oq info --help) and added
information about the available IMTs, MFDs and source classes
Optimized get_composite_source_model (in the case of a complex source
specific logic trees a speedup of 80x was measured)
Internal: fixed oq info source_model_logic_tree.xml
Avoided reading multiple times the source models in the case of complex
logic trees
Moved the check on invalid TRTs earlier, before processing the source models
Removed the ucerf_classical calculator (just use the classical one)

[Paolo Tormene (@ptormene)]

Added a warning in oq reduce_sm listing duplicate source IDs

[Michele Simionato (@micheles)]

Improved oq reduce_sm to reduce also duplicated source IDs if they
belong to different source types
Removed the ucerf_hazard calculator (just use the event_based one)
Changed the seed algorithm in all event based calculators including UCERF
Fixed the ShakeMap code to use the formula for the median and not the mean
Added a check on excessive data transfer in disaggregation calculations
Changed back the disaggregation calculator to read the rupture data from
the workers, thus saving a lot of memory and time
Fixed a bug that made it impossible to abort/remove a failed task
Added extendModel feature to the source model logic tree parser

[Graeme Weatherill (@g-weatherill)]

Fixed bug in the HMTK: the bin_width parameter was not passed to
mtkActiveFaultModel.build_fault_model

[Michele Simionato (@micheles)]

Avoided submitting too many tasks in the disaggregation calculator
Added a parameter discard_trts for manual reduction of GSIM logic tree
Fixed a bug in case of duplicated nodal planes affecting the Italy model
Removed dynamic reduction of the GSIM logic tree (i.e. now the
logic tree is known upfront, before calculating the PoES)

[Paolo Tormene (@ptormene)]

Fixed an encoding issue in reading configuration files on Windows

[Michele Simionato (@micheles)]

Internal: started the zmq workers when the DbServer starts
Fixed a bug when reading rupture.txt files
Internal: added an option --calc-id to oq run
Added a check against negative number of cores in openquake.cfg
Raised a clear error message if the enlarged bounding box of the sources
does not contain any site or if it is larger than half the globe

[Kendra Johnson (@kejohnso)]

Correction to catalogue plotting tool in hmtk to include the last bins
in density plots

[Paolo Tormene (@ptormene)]

Added Classical PSHA Non-parametric sources Demo

[Robin Gee (@rcgee)]

Change the header of the exported sigma_epsilon_XX.csv file to indicate
that values correspond to inter event sigma

[Graeme Weatherill (@g-weatherill)]

Adds independent verification tables for the USGS CEUS models and revises
implementation for collapsed epistemic uncertainty on sigma and site
amplification
Enhances SERA adaptation of the Abrahamson et al. (2015) BC Hydro GMPE to
add in a configurable smoothed tapering term on the forearc/backarc scaling

[Michele Simionato (@micheles)]

Added a check on the engine version between master and workers

[Paolo Tormene (@ptormene)]

Removed the multi_node flag, that is not used anymore

[Michele Simionato (@micheles)]

Added a command oq postzip to send small calculations to the WebUI
Added a limit of 1000 sources when disagg_by_src=true
Internal: fixed oq export input -e zip that was flattening the tree
structure of the input files in the exported zip archive
Implemented GMFs amplification
Introduced the flag approx_ddd to support the old algorithm in
scenario_damage calculations; it is automatically used for exposures
with fractional asset numbers

[Paolo Tormene (@ptormene)]

Modified the server views in order to allow using
numpy.load(allow_pickle=False) in the QGIS IRMT plugin
Internal: changed some copy.deepcopy calls into copy.copy in hazardlib

[Michele Simionato (@micheles)]

Removed implicit intensity_measure_types_and_levels
Added a check to forbid case-similar headers in the exposures
Improved the error message in case of CSV exposures with wrong headers
Reduced the slow tasks issue in event_based/ebrisk with many sites
Enhanced oq compare to accept a file with the control sites
Improved the error message for duplicate sites
Speedup of the ebrisk calculator
Extended the minimum_intensity feature to the classical calculator
Solved a memory bug when using the nrcan site term: due to a deepcopy
the engine could run out of memory in the workers for large site collections
Added a check to forbid multiple complexFaultGeometry nodes
Internal: we are now shutting down the ProcessPool explicitly in order
to support Python 3.8
Internal: removed the class hazardlib.gsim.base.IPE
Changed the aggregate loss curves generation to not use the partial
asset loss table, with a huge memory reduction
Extended oq check_input to accept multiple files
Changed the scenario damage calculator to use discrete damage distributions
Forced the "number" attribute in the exposure must be an integer in the
range 1..65535, extrema included

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 5 years ago

OpenQuake - OpenQuake Engine 3.8.1

[Michele Simionato (@micheles)]

Fixed random HDF5 bug in disaggregation calculations
Fixed memory issue in nrcan15_site_term.p
Fixed get_duplicates check in the SiteCollection
Fixed bug in case of MMI (log(imls) -> imls)

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 5 years ago

OpenQuake - OpenQuake Engine 3.8.0

[Graeme Weatherill (@g-weatherill)]

Updates SERA Craton GMPE to incorporate NGA East site response and reflect
changes in CEUS USGS model

[Michele Simionato (@micheles)]

The total loss curves in event_based_risk are now built with pandas
Added an option oq engine --param to override the job.ini parameters
Internal: reduced the number of NGAEastUSGS classes from 39 to 1
Internal: reduced the number of NGAEast classes from 44 to 2
Internal: reduced the 15 NSHMP2014 classes to a single class
Internal: reduced the 22 NBCC2015_AA13 classes to a single class

[Graeme Weatherill (@g-weatherill)]

Adds complete suite of GMPEs for the Central and Eastern US, as adopted
within the 2018 US National Seismic Hazard Map
Implements NGA East site amplification model within NGA East Base class
Implemented site amplification by convolution
Improved the error message if the event_id does not start from zero in
the gmfs.csv files
Changed the rupture exporter to export LINESTRINGs instead of degenerate
POLYGONs
Introduced minimum_loss_fraction functionality in ebrisk
Refined the rupture prefiltering mechanism, possibly changing the numbers
in calculations with nonzero coefficients of variations
Optimized the generation of aggregate loss curves in ebrisk
Introduced an experimental AvgGMPE and used it to implement (optional)
reduction of the gsim logic tree

[Graeme Weatherill (@g-weatherill)]

Implemented Abrahamson et al (2018) update of the BC Hydro GMPE
Added configurable nonergodic sigma option to BC Hydro and SERA GMPEs
Small refactoring and bug fix in average SA GMPE

[Michele Simionato (@micheles)]

Avoided reading multiple times the GSIM logic tree
Changed the GSIM logic tree sampling by ordering the branches by TRT
Ignored IMT-dependent weights when using sampling to make such calculations
possible
Storing (partially) the asset loss table

[Robin Gee (@rcgee)]

Set DEFINED_FOR_REFERENCE_VELOCITY in CampbellBozorgnia2003NSHMP2007

[Graeme Weatherill (@g-weatherill)]

Re-adjustment of SERA Subduction model epistemic scaling factors

[Michele Simionato (@micheles)]

Improved the task distribution in the ebrisk calculator
Fixed a bug in ebrisk with aggregate_by when building the rup_loss_table
Storing the asset loss table in scenario_risk, but only for assets and
events above over a loss_ratio_threshold parameter
Storing the asset damage table in scenario_damage and event based damage,
but only for assets and events above a collapse_threshold parameter
Avoided transferring the GMFs upfront in scenario_damage, scenario_risk
and event_based_damage

[Daniele Viganò (@daniviga)]

Included pandas in the engine distribution

[Michele Simionato (@micheles)]

Avoided reading multiple time the gsim logic tree file and relative files
Added a check for duplicate sites in the site model file
Implemented an event_based_damage calculator
Added an API /v1/calc/ID/extract/gmf_data?event_id=XXX
Added an API /v1/calc/ID/extract/num_events
Fixed the /v1/calc/ID/status endpoint to return an error 404 when needed
Removed the "sites are overdetermined" check, since it now unneeded
Turned the calculation of consequences into a plugin architecture

[Matteo Nastasi (@nastasi-oq)]

Add '/v1/ini_defaults' web api entry point to retrieve all default
values for ini attributes (attrs without a default are not returned)

[Michele Simionato (@micheles)]

Renamed rlzi -> rlzi in the sigma-epsilon dataset and exporter
Renamed id -> asset_id in all the relevant CSV exporters
Renamed rlzi -> rlz_id in the dmg_by_event.csv output
Renamed rupid -> rup_id in the ruptures.csv output
Renamed id -> event_id in the events.csv output and gmfs.csv output
Renamed sid -> site_id in the gmfs.csv output
Renamed ordinal -> rlz_id in the realizations.csv output

[Alberto Chiusole (@bebosudo)]

Changed the way how the available number of CPU cores is computed

[Kendra Johnson (@kejohnso), Robin Gee (@rcgee)]

Added GMPEs for Rietbrock-Edwards (2019) and Yenier-Atkinson (2015)

[Michele Simionato (@micheles)]

Added more check on the IMTs and made it possible to import a GMF.csv
file with more IMTs than needed
Enabled magnitude-dependent pointsource_distance
Removed the syntax for magnitude-dependent maximum distance, since
now it can be automatically determined by the engine
Saving more information in the case of single-site classical hazard
Extended pointsource_distance to generic sources
Removed the boundary information from the CSV rupture exporter
Changed the /extract/rupture/XXX API to returns a TOML that can be
used by a scenario calculator
Added general support for file-reading GMPEs
Made it possible to disaggregate on multiple realizations
with the parameters rlz_index or num_rlzs_disagg
Fixed downloading the ShakeMaps (again)
Better error message in case of too large maximum_distance
Optimized the case of point sources with an hypocenter distribution and
GSIMs independent from it and in general the case of ruptures with
similar distances

[Graeme Weatherill (@g-weatherill)]

Updates SERA craton GMPE to reflect updates to NGA East site response model

[Michele Simionato (@micheles)]

Fixed and HDF5 SWMR issue in large disaggregation calculations
Made rrup the unique acceptable filter_distance
Fixed disaggregation with a parent calculation
Models with duplicated values in the hypocenter and/or nodal plane
distributions are now automatically optimized
Fixed an issue with missing noDamageLimit causing NaN values in
scenario_damage calculations
Added more validations for predefined hazard, like forbidding the site model

[Marco Pagani (@mmpagani)]

Adding the shift_hypo option for distributed seismicity

[Michele Simionato (@micheles)]

Raised an early error for extra-large GMF calculations
Reduced the GMF storage by using 32 bit per event ID instead of 64 bit
Raised an error in case of duplicated sites in the site model
Fixed the case of implicit grid with a site model: sites could be
incorrectly discarded
Fixed the ShakeMap downloader to find also unzipped uncertaintly.xml
files
Fixed the rupture exporters to export the rupture ID and not the
rupture serial
Removed the non-interesting agg_maps outputs
Changed the task distribution in the classical calculator and added
a task_multiplier parameter

[Marco Pagani (@mmpagani)]

Fixed a bug in the GenericGmpeAvgSA

[Michele Simionato (@micheles)]

Added a /v1/calc/validate_zip endpoint to validate input archives
Deprecated inferring the intensity measure levels from the risk functions
Fixed a too strict check on the minimum intensities of parent an child
calculations
Extended the ebrisk calculator to compute at the same time both the
aggregate curves by tag and the total curves

[Marco Pagani (@mmpagani)]

Implemented Morikawa and Fujiwara (2013) GMM

[Michele Simionato (@micheles)]

Changed the seed algorithm in sampling with more than one source model,
thus avoiding using more GMPEs than needed in some cases
If ground_motion_fields=false is set, the GMFs are not stored even
if hazard_curves_from_gmfs=true
oq show job_info now works while the calculation is running
Reduced the sent data transfer in ebrisk calculations
Deprecated the old syntax for the reqv feature
Added short aliases for hazard statistics mean, max and std
Reduced substantially the memory occupation in the task queue
Added an API /extract/sources and an experimental oq plot sources
Added a check on valid input keys in the job.ini
Fixed the check on dependent calculations
Specifying at the same time both a grid and individual sites is an error

[Daniele Viganò (@daniviga)]

Docker containers rebased on CentOS 8
Fixed an issue causing zombie ssh processes
when using zmq as task distribution mechanism
Introduced support for RHEL/CentOS 8

[Michele Simionato (@micheles)]

Added a check for no GMFs in event_based_risk
Avoided transferring the site collection
Storing the sources in TOML format

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 5 years ago

OpenQuake - OpenQuake Engine 3.7.1

[Michele Simionato (@micheles)]

Fixed disaggregation with a parent calculation
Fixed the case of implicit grid with a site model: sites could be
incorrectly discarded
Fixed the ShakeMap downloader to find also unzipped uncertaintly.xml
files
Fixed the rupture exporters to export the rupture ID and not the
rupture serial

[Marco Pagani (@mmpagani)]

Fixed a bug in the GenericGmpeAvgSA

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 5 years ago

OpenQuake - OpenQuake Engine 3.7.0

[Michele Simionato (@micheles)]

Hiding calculations that fail before the pre-execute phase (for instance,
because of missing files); they already give a clear error
Added an early check on truncation_level in presence of correlation model

[Guillaume Daniel (@guyomd)]

Implemented Ameri (2017) GMPE

[Michele Simionato (@micheles)]

Changed the ruptures CSV exporter to use commas instead of tabs
Added a check forbidding aggregate_by for non-ebrisk calculators
Introduced a task queue
Removed the cache_XXX.hdf5 files by using the SWMR mode of h5py

[Kris Vanneste (@krisvanneste)]

Updated the coefficients table for the atkinson_2015 to the actual
values in the paper.

[Michele Simionato (@micheles)]

Added an /extract/agg_curves API to extract both absolute and relative
loss curves from an ebrisk calculation
Changed oq reset --yes to remove oqdata/user only in single-user mode
Now the engine automatically sorts the user-provided intensity_measure_types
Optimized the aggregation by tag
Fixed a bug with the binning when disaggregating around the date line
Fixed a prefiltering bug with complex fault sources: in some cases, blocks
ruptures were incorrectly discarded
Changed the sampling algorithm for the GMPE logic trees: now it does
not require building the full tree in memory
Raised clear errors for geometry files without quotes or with the wrong
header in the multi_risk calculator
Changed the realizations.csv exporter to export '[FromShakeMap]' instead
of '[FromFile]' when needed
Changed the agg_curves exporter to export all realizations in a single file
and all statistics in a single file
Added rlz_id, rup_id and year to the losses_by_event output for ebrisk
Fixed a bug in the ruptures XML exporter: the multiplicity was multiplied
(incorrectly) by the number of realizations
Fixed the pre-header of the CSV outputs to get proper CSV files
Replaced the 64 bit event IDs in event based and scenario calculations
with 32 bit integers, for the happiness of Excel users

[Daniele Viganò (@daniviga)]

Numpy 1.16, Scipy 1.3 and h5py 2.9 are now required

[Michele Simionato (@micheles)]

Changed the ebrisk calculator to read the CompositeRiskModel directly
from the datastore, which means 20x less data transfer for Canada

[Anirudh Rao (@raoanirudh)]

Fixed a bug in the gmf CSV importer: the coordinates were being
sorted and new site_ids assigned even though the user input sites
csv file had site_ids defined

[Michele Simionato (@micheles)]

Fixed a bug in the rupture CSV exporter: the boundaries of a GriddedRupture
were exported with lons and lats inverted
Added some metadata to the CSV risk outputs
Changed the distribution mechanism in ebrisk to reduce the slow tasks

[Graeme Weatherill (@g-weatherill)]

Updates Kotha et al. (2019) GMPE to July 2019 coefficients
Adds subclasses to Kotha et al. (2019) to implement polynomial site
response models and geology+slope site response model
Adds QA test to exercise all of the SERA site response calculators

[Michele Simionato (@micheles)]

Internal: there is not need to call gsim.init() anymore

[Graeme Weatherill (@g-weatherill)]

Adds parametric GMPE for cratonic regions in Europe

[Michele Simionato (@micheles)]

In the agglosses output of scenario_risk the losses were incorrectly
multiplied by the realization weight
Removed the output sourcegroups and added the output events

[Graeme Weatherill (@g-weatherill)]

Adds new meta ground motion models to undertake PSHA using design code
based amplification coefficients (Eurocode 8, Pitilakis et al., 2018)
Adds site amplification model of Sandikkaya & Dinsever (2018)

[Marco Pagani (@mmpagani)]

Added a new rupture-site metric: the azimuth to the closest point on the
rupture

[Michele Simionato (@micheles)]

Fixed a regression in disaggregation with nonparametric sources, which
were effectively discarded
The site amplification has been disabled by default in the ShakeMap
calculator, since it is usually already taken into account by the USGS

[Daniele Viganò (@daniviga)]

Deleted calculations are not removed from the database anymore
Removed the 'oq dbserver restart' command since it was broken

[Richard Styron (@cossatot)]

Fixed YoungsCoppersmith1985MFD.from_total_moment_rate(): due to numeric
errors it was producing incorrect seismicity rates

[Michele Simionato (@micheles)]

Now we generate the output disagg_by_src during disaggregation even in the
case of multiple realizations
Changed the way the random seed is set for BT and PM distributions
The filenames generated by disagg_by_src exporter now contains the site ID
and not longitude and latitude, consistently with the other exporters
Accepted again meanLRs greater than 1 in vulnerability functions of kind LN
Fixed a bug in event based with correlation and a filtered site collection
Fixed the CSV exporter for the realizations in the case of scenarios
with parametric GSIMs
Removed some misleading warnings for calculations with a site model
Added a check for missing risk_investigation_time in ebrisk
Reduced drastically (I measured improvements over 40x) memory occupation,
data transfer and data storage for multi-sites disaggregation
Sites for which the disaggregation PoE cannot be reached are discarded
and a warning is printed, rather than killing the whole computation
oq show performance can be called in the middle of a computation again
Filtered out the far away distances and reduced the time spent in
saving the performance info by orders of magnitude in large disaggregations
Reduced the data transfer by reading the data directly from the
datastore in disaggregation calculations
Reduced the memory consumption sending disaggregation tasks incrementally
Added an extract API disagg_layer
Moved max_sites_disagg from openquake.cfg into the job.ini
Fixed a bug with the --config option: serialize_jobs could not be overridden
Implemented insured losses

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 5 years ago

OpenQuake - OpenQuake Engine 3.6.0

[Michele Simionato (@micheles)]

In some cases applyToSources was giving a fake error about the source
not being in the source model even if it actually was

[Chris Van Houtte (@cvanhoutte)]

Adds the Van Houtte et al. (2018) significant duration model for New
Zealand

[Michele Simionato (@micheles)]

Added a way to compute and plot the MFD coming from an event based
Storing the MFDs in TOML format inside the datastore

[Robin Gee (@rcgee)]

Moves b4 constant into COEFFS table for GMPE Sharma et al., 2009

[Graeme Weatherill (@g-weatherill)]

Adds functionality to Cauzzi et al. (2014) and Derras et al. (2014)
calibrated GMPEs for Germany to use either finite or point source distances

[Michele Simionato (@micheles)]

Restored the ability to associate site model parameters to a grid of sites
Made it possible to set hazard_curves_from_gmfs=true with
ground_motion_fields=false in the event based hazard calculator
Introduced a mechanism to split the tasks based on an estimated duration
Integrated oq plot_memory into oq plot
Removed NaN values for strike and dip when exporting griddedRuptures
Fixed oq reset to work in multi-user mode
Extended the source_id-filtering feature in the job.ini to multiple sources
Supported WKT files for the binary perils in the multi_risk calculator
Added an early check on the coefficients of variation and loss ratios of
vulnerability functions with the Beta distribution
Made sure that oq engine --dc removes the HDF5 cache file too
Removed the flag optimize_same_id_sources because it is useless now
Introduced a soft limit at 65,536 sites for event_based calculations
Fixed a performance regression in ucerf_classical that was filtering
before splitting, thus becoming extra-slow
Improved the progress log, that was delayed for large classical calculations
Exported the ruptures as 3D multi-polygons (instead of 2D ones)
Changed the aggregate_by exports for consistency with the others
Changed the losses_by_event exporter for ebrisk, to make it more
consistent with scenario_risk and event_based_risk
Changed the agglosses and losses_by_event exporters in scenario_risk,
by adding a column with the realization index
Changed the generation of the hazard statistics to consume very little
memory
Fixed a bug with concurrent_tasks being inherited from the parent
calculation instead of using the standard default
Removed the dependency from mock, since it is included in unittest.mock
For scenario, replaced the branch_path with the GSIM representation in
the realizations output
Added a check for suspiciously large source geometries
Deprecated the XML disaggregation exporters in favor of the CSV exporters
Turned the disaggregation calculator into a classical post-calculator
to use the precomputed distances and speedup the computation even more
Fixed the disaggregation calculator by discarding the ruptures outside
the integration distance
Optimized the speed of the disaggregation calculator by moving a statistical
functions outside of the inner loop
Changed the file names of the exported disaggregation outputs
Fixed an export agg_curves issue with pre-imported exposures
Fixed an export agg_curves issue when the hazard statistics are different
from the risk statistics
Removed the disaggregation statistics: now the engine disaggregates only on
a single realization (default: the closest to the mean)
Forbidden disaggregation matrices with more than 1 million elements
Reduced the data transfer when computing the hazard curves
Optimized the reading of large CSV exposures
Fixed the --hc functionality across users
Optimized the reduction of the site collection on the exposure sites
Made more robust the gsim logic tree parser: lines like
<uncertaintyModel gmpe_table="../gm_tables/Woffshore_low_clC.hdf5">
are accepted again
Added a check against duplicated values in nodal plane distributions and
hypocenter depth distributions
Changed the support for zipped exposures and source models: now the
name of the archive must be written explicitly in the job.ini
Added support for numpy 1.16.3, scipy 1.3.0, h5py 2.9.0
Removed the special case for event_based_risk running two calculations

[Graeme Weatherill (@g-weatherill)]

Adds the Tromans et al. (2019) adjustable GMPE for application to PSHA
in the UK

[Michele Simionato (@micheles)]

Optimized src.sample_ruptures for (multi)point sources and are sources
Fixed a mutability bug in the DistancesContext and made all context
arrays read-only: the fix may affect calculations using the GMPEs
berge_thierry_2003, cauzzi_faccioli_2008 and zhao_2006;
Fixed a bug with the minimum_distance feature
Fixed a bug in the exporter of the aggregate loss curves: now the loss
ratios are computed correctly even in presence of occupants
Removed the (long time deprecated) capability to read hazard curves and
ground motion fields from XML files: you must use CSV files instead

[Marco Pagani (@mmpagani)]

Implemented a modified GMPE that add between and within std to GMPEs only
supporting total std

[Michele Simionato (@micheles)]

Added the ability to use a taxonomy_mapping.csv file
Fixed a bug in classical_damage from CSV: for hazard intensity measure
levels different from the fragility levels, the engine was giving incorrect
results
Serialized also the source model logic tree inside the datastore
Added a check on missing intensity_measure_types in event based
Fixed oq prepare_site_model in the case of an empty datadir
Added a comment line with useful metadata to the engine CSV outputs
Removed the long time deprecated event loss table exporter for event based
risk and enhanced the losses_by_event exporter to export the realization ID
Removed the long time deprecated GMF XML exporter for scenario
IMT-dependent weights in the gsim logic tree can be zero, to discard
contributions outside the range of validity of (some of the) GSIMs
Now it is possible to export individual hazard curves from an event
Added a view gmvs_to_hazard

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 6 years ago

OpenQuake - OpenQuake Engine 3.5.2

[Daniele Viganò (@daniviga)]

Fixed packaging issue, the .hdf5 tables for Canada were missing

[Michele Simionato (@micheles)]

Fixed regression in the gsim logic tree parser for the case
of .hdf5 tables

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 6 years ago

OpenQuake - OpenQuake Engine 3.5.1

[Michele Simionato (@micheles)]

Added a rlzi column to to sig_eps.csv output
Accepted GMF CSV files without a rlzi column
Accepted a list-like syntax like return_periods=[30, 60, 120, 240, 480]
in the job.ini, as written in the manual
Fixed a bug in the asset_risk exporter for uppercase tags

[Paul Henshaw (@pslh)]

Fixed an encoding bug while reading XML files on Windows

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 6 years ago

OpenQuake - OpenQuake Engine 3.5.0

[Michele Simionato (@micheles)]

Added a view gmvs_to_hazard

[Giovanni Lanzano (@giovannilanzanoINGV)]

Lanzano and Luzi (2019) GMPE for volcanic zones in Italy

[Michele Simionato (@micheles)]

Now it is possible to export individual hazard curves from an event
based calculation by setting hazard_curves_from_gmfs = true and
`individual_curves = true (before only the statistics were saved)

[Graeme Weatherill (@g-weatherill)]

Adds adaptation of Abrahamson et al. (2016) 'BC Hydro' GMPEs calibrated
to Mediterranean data and with epistemic adjustment factors

[Chris Van Houtte (@cvanhoutte)]

Added new class to bradley_2013b.py for hazard maps
Modified test case_37 to test multiple sites

[Marco Pagani (@mmpagani)]

Fixed a bug in the logic tree parser and added a check to forbid logic
trees with applyToSources without applyToBranches, unless there is a
single source model branch

[Michele Simionato (@micheles)]

Removed the experimental parameter prefilter_sources

[Daniele Viganò (@daniviga)]

Multiple DbServer ZMQ connections are restored to avoid errors under heavy
load and/or on slower machines

[Michele Simionato (@micheles)]

Removed the ugly registration of custom signals at import time: now they
are registered only if engine.run_calc is called
Removed the dependency from rtree
Removed all calls to ProcessPool.shutdown to speed up the tests and to
avoid non-deterministic errors in atexit._run_exitfuncs

[Marco Pagani (@mmpagani)]

Added tabular GMPEs as provided by Michal Kolaj, Natural Resources Canada

[Michele Simionato (@micheles)]

Extended the ebrisk calculator to support coefficients of variations

[Graeme Weatherill (@g-weatherill)]

Adds Kotha et al (2019) shallow crustal GMPE for SERA
Adds 'ExperimentalWarning' to possible GMPE warnings
Adds kwargs to check_gsim function

[Michele Simionato (@micheles)]

Fixed problems like SA(0.7) != SA(0.70) in iml_disagg
Exposed the outputs of the classical calculation in event based
calculations with compare_with_classical=true
Made it possible to serialize together all kind of risk functions,
including consequence functions that before were not HDF5-serializable
Fixed a MemoryError when counting the number of bytes stored in large
HDF5 datasets
Extended asset_hazard_distance to a dictionary for usage with multi_risk
Extended oq prepare_site_model to work with sites.csv files
Optimized the validation of the source model logic tree: now checking
the sources IDs is 5x faster
Went back to the old logic in sampling: the weights are used for the
sampling and the statistics are computed with identical weights
Avoided to transfer the epsilons by storing them in the cache file
and changed the event to epsilons associations
Reduced the data transfer in the computation of the hazard curves, causing
in some time huge speedups (over 100x)
Implemented a flag modal_damage_state to display only the most likely
damage state in the output dmg_by_asset of scenario damage calculations
Reduced substantially the memory occupation in classical calculations
by including the prefiltering phase in the calculation phase

[Daniele Viganò (@daniviga)]

Added a 'serialize_jobs' setting to the openquake.cfg
which limits the maximum number of jobs that can be run in parallel

[Michele Simionato (@micheles)]

Fixed two exporters for the ebrisk calculator (agg_curves-stats and
losses_by_event)
Fixed two subtle bugs when reading site_model.csv files
Added /extract/exposure_metadata and /extract/asset_risk
Introduced an experimental multi_risk calculator for volcanic risk

[Guillaume Daniel (@guyomd)]

Updating of Berge-Thierry (2003) GSIM and addition of several alternatives
for use with Mw

[Michele Simionato (@micheles)]

Changed the classical_risk calculator to use the same loss ratios for all
taxonomies and then optimized all risk calculators
Temporarily removed the insured_losses functionality
Extended oq restore to download from URLs
Removed the column 'gsims' from the output 'realizations'
Better parallelized the source splitting in classical calculations
Added a check for missing hazard in scenario_risk/scenario_damage
Improved the GsimLogicTree parser to get the line number information, a
feature that was lost with the passage to Python 3.5
Added a check against mispellings in the loss type in the risk keys
Changed the aggregation WebAPI from
aggregate_by/taxonomy,occupancy/avg_losses?kind=mean&loss_type=structural to
aggregate/avg_losses?kind=mean&loss_type=structural&tag=taxonomy&tag=occupancy
Do not export the stddevs in scenario_damage in the case of 1 event
Fixed export bug for GMFs imported from a file
Fixed an encoding error when storing a GMPETable
Fixed an error while exporting the hazard curves generated by a GMPETable
Removed the deprecated feature aggregate_by/curves_by_tag

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 6 years ago

OpenQuake - OpenQuake Engine 3.4.0

[Michele Simionato (@micheles)]

Compatibility with 'decorator' version >= 4.2

[Giovanni Lanzano (@giovannilanzanoINGV)]

Contributed a GMPE SkarlatoudisEtAlSSlab2013

[Michele Simionato (@micheles)]

Changed the event loss table exporter to export also rup_id and year
Extended the ebrisk calculator to compute loss curves and maps

[Rodolfo Puglia (@rodolfopuglia)]

Spectral acceleration amplitudes at 2.5, 2.75 and 4 seconds added

[Marco Pagani (@mmpagani)]

Improved the event based calculator to account for cluster-based models

[Michele Simionato (@micheles)]

Removed the now redundant command oq extract hazard/rlzs

[Daniele Viganò (@daniviga)]

Fixed 'oq abort' and always mark killed jobs as 'aborted'

[Michele Simionato (@micheles)]

Made it possible to use in the Starmap tasks without a monitor argument
Stored the sigma and epsilon parameters for each event in event based
and scenario calculations and extended the gmf_data exporter consequently
Fixed the realizations CSV exporter which was truncating the names of the
GSIMs
Deprecated the XML exporters for hcurves, hmaps, uhs
Introduced a sap.script decorator
Used the WebExtractor in oq importcalc
Restored validation of the source_model_logic_tree.xml file
Raised an early error for missing occupants in the exposure
Added a check to forbid duplicate file names in the uncertaintyModel tag
Made it possible to store the asset loss table in the ebrisk calculator
by specifying asset_loss_table=true in the job.ini
Added a flag oq info --parameters to show the job.ini parameters
Removed the source_name column from the disagg by source output

[Rao Anirudh]

Fixed wrong investigation_time in the calculation of loss maps from
loss curves

[Robin Gee (@rcgee)]

Added capability to optionally specify a time_cutoff parameter to
declustering time window

[Michele Simionato (@micheles)]

Merged the commands oq plot_hmaps and oq plot_uhs inside oq plot
Changed the storage of hazard curves and hazard maps to make it consistent
with the risk outputs and Extractor-friendly

[Chris Van Houtte (@cvanhoutte)]

Added necessary gsims to run the Canterbury Seismic Hazard Model
in Gerstenberger et al. (2014)
Added a new gsim file mcverry_2006_chch.py to have the Canterbury-
specific classes.
Added a new gsim file bradley_2013b.py to implement the
Christchurch-specific modifications to the Bradley2013 base model.

[Michele Simionato (@micheles)]

Added a check on the intensity measure types and levels in the job.ini,
to make sure they are ordered by period
Reduced the number of client sockets to the DbServer that was causing
(sporadically) the hanging of calculations on Windows
Extended the WebAPI to be able to extract specific hazard curves, maps
and UHS (i.e. IMT-specific and site specific)
Removed the realization index from the event loss table export, since
is it redundant
Forced all lowercase Python files in the engine codebase
Removed the dependency from nose

[Robin Gee (@rcgee)]

Updated GMPE of Yu et al. (2013)

[Michele Simionato (@micheles)]

Added an Extractor client class leveraging the WebAPI and enhanced
oq plot_hmaps to display remote hazard maps
Added a check when disaggregation is attempted on a source model
with atomic source groups
Implemented serialization/deserialization of GSIM instances to TOML
Added a check against mispelled rupture distance names and fixed
the drouet_alpes_2015 GSIMs
Changed the XML syntax used to define dictionaries IMT -> GSIM
Now GSIM classes have an .init() method to manage notrivial
initializations, i.e. expensive initializations or initializations
requiring access to the filesystem
Fixed a bug in event based that made it impossible to use GMPETables
Associated the events to the realizations even in scenario_risk: this
involved changing the generation of the epsilons in the case of asset
correlation. Now there is a single aggregate losses output for all
realizations
Removed the rlzi column from the GMF CSV export
Introduced a new parameter ebrisk_maxweight in the job.ini
For classical calculations with few sites, store information about the
realization closest to the mean hazard curve for each site
Removed the max_num_sites limit on the event based calculator

[Valerio Poggi (@klunk386)]

Added an AvgSA intensity measure type and a GenericGmpeAvgSA which is
able to use it

[Michele Simionato (@micheles)]

Introduced the ability to launch subtasks from tasks
Stored rupture information in classical calculations with few sites

[Chris Van Houtte (@cvanhoutte)]

Adding conversion from geometric mean to larger horizontal component in
bradley_2013.py

[Michele Simionato (@micheles)]

Fixed a bug in applyToSources for the case of multiple sources
Moved the prefiltering on the workers to save memory
Exported the aggregated loss ratios in avg losses and agg losses
Removed the variables quantile_loss_curves and mean_loss_curves: they
were duplicating quantile_hazard_curves and mean_hazard_curves
Only ruptures boundingbox-close to the site collection are stored

[Marco Pagani (@mmpagani)]

Added cluster model to classical PSHA calculator

[Michele Simionato (@micheles)]

Fixed a bug in scenario_damage from ShakeMap with noDamageLimit=0
Avoided the MemoryError in the controller node by speeding up the saving
of the information about the sources
Turned utils/reduce_sm into a proper command
Fixed a wrong coefficient in the ShakeMap amplification
Fixed a bug in the hazard curves export (the filename did not contain
the period of the IMT thus producing duplicated files)
Parallelized the reading of the exposure

[Marco Pagani (@mmpagani)]

Fixed the implementation on mutex ruptures

[Michele Simionato (@micheles)]

Changed the aggregated loss curves exporter
Added an experimental calculator ebrisk
Changed the ordering of the events (akin to a change of seed in the
asset correlation)

[Robin Gee (@rcgee)]

Fixed bug in tusa_langer_2016.py BA08SE model - authors updated b2 coeff
Fixed bug in tusa_langer_2016.py related to coeffs affecting Repi models

[Michele Simionato (@micheles)]

Added a check to forbid to set ses_per_logic_tree_path = 0
Added an API /extract/event_info/eidx
Splitting the sources in classical calculators and not in event based
Removed max_site_model_distance
Extended the logic used in event_based_risk - read the hazard sites
from the site model, not from the exposure - to all calculators
In classical_bcr calculations with a CSV exposure the retrofitted field
was not read. Now a missing retrofitted value is an error

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 6 years ago

OpenQuake - OpenQuake Engine 3.3.2

[Robin Gee (@rcgee)]

Fixed bug in tusa_langer_2016.py BA08SE model - authors updated b2 coeff

[Michele Simionato (@micheles)]

Fixed a bug in scenario_damage from ShakeMap with noDamageLimit=0
Avoided the MemoryError in the controller node by speeding up the saving
of the information about the sources
Fixed a wrong coefficient in the ShakeMap amplification
Fixed a bug in the hazard curves export (the filename did not contain
the period of the IMT thus producing duplicated files)

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 6 years ago

OpenQuake - OpenQuake Engine 3.3.1

[Michele Simionato (@micheles)]

Fixed the GMF exporter to export the event IDs and not event indices

[Robin Gee (@rcgee)]

Fixed bug in tusa_langer_2016.py related to coeffs affecting Repi models

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 6 years ago

OpenQuake - OpenQuake Engine 3.3.0

[Graeme Weatherill (@g-weatherill)]

Adds GMPE suite for national PSHA for Germany

[Daniele Viganò (@daniviga)]

Added a warning box when an unsupported browser is used to view the WebUI
Updated Docker containers to support a multi-node deployment
with a shared directory
Moved the Docker containers source code from oq-builders
Updated the documentation related to the shared directory
which is now mandatory for multi-node deployments

[Matteo Nastasi (@nastasi-oq)]

Removed tests folders

[Stéphane Drouet (@stephane-on)]

Added Drouet & Cotton (2015) GMPE including 2017 erratum

[Michele Simionato (@micheles)]

Optimized the memory occupation in classical calculations (Context.poe_map)
Fixed a wrong counting of the ruptures in split fault sources with
an hypo_list/slip_list causing the calculation to fail
Made the export of uniform hazard spectra fast
Made the std hazard output properly exportable
Replaced the ~ in the header of the UHS csv files with a -
Restored the individual_curves flag even for the hazard curves
Implemented dGMPE weights per intensity measure type
Extended --reuse-hazard to all calculators
Fixed a bug in event_based_risk from GMFs with coefficients of variations

[Graeme Weatherill (@g-weatherill)]

Adds magnitude scaling relation for Germany

[Michele Simionato (@micheles)]

Used floats for the the GSIM realization weights, not Python Decimals
Added a flag fast_sampling, by default False
Added an API /extract/src_loss_table/<loss_type>
Removed the rupture filtering from sample_ruptures and optimized it in
the RuptureGetter by making use of the bounding box
Raised the limit on ses_per_logic_tree_path from 216 to 232;
Added a parameter max_num_sites to increase the number of sites accepted
by an event based calculation up to 2 ** 32 (the default is still 2 ** 16)
Added a command oq compare to compare hazard curves and maps within
calculations
Extended the engine to read transparently zipped source models and exposures
Restored the check for invalid source IDs in applyToSources
Extended the command oq zip to zip source models and exposures
Parallelized the associations event ID -> realization ID
Improved the message when assets are discarded in scenario calculations
Implemented aggregation by multiple tags, plus a special case for the
country code in event based risk

[Marco Pagani (@mmpagani)]

Added two modified versions of the Bindi et al. (2011) to be used in a
backbone approach to compute hazard in Italy
Added a modified version of Berge-Thierry et al. 2003 supporting Mw

[Michele Simionato (@micheles)]

Changed the way loss curves and loss maps are stored in order to unify
the aggregation logic with the one used for the average losses
Now it is possible to compute the ruptures without specifying the sites
Added an early check for the case of missing intensity measure types
Deprecated the case of exposure, site model and region_grid_spacing all
set at the same time
Implemented multi-exposure functionality in event based risk
Changed the event based calculator to store the ruptures incrementally
without keeping them all in memory
Refactored the UCERF event based calculator to work as much as possible
the regular calculator
Optimized the management and storage of the aggregate losses in the event
based risk calculation; also, reduced the memory consumption
Changed the default for individual_curves to "false", which is the right
default for large calculations
Optimized the saving of the events
Removed the save_ruptures flag in the job.ini since ruptures must be saved
always
Optimized the rupture generation in case of sampling and changed the
algorithm and seeds
Fixed a bug with the IMT SA(1) considered different from SA(1.0)
Removed the long-time deprecated GMF exporter in XML format for event_based
Added a re-use hazard feature in event_based_risk in single-file mode
Made the event ID unique also in scenario calculations with
multiple realizations
Removed the annoying hidden .zip archives littering the export directory
Added an easy way to read the exposure header
Added a way to run Python scripts using the engine libraries via oq shell
Improved the minimum_magnitude feature
Fixed the check on missing hazard IMTs
Reduced substantially the memory occupation in event based risk
Added the option spatial_correlation=no correlation for risk calculations
from ShakeMaps
Removed the experimental calculator ucerf_risk
Optimized the sampling of time-independent sources for the case of
prefilter_sources=no
Changed the algorithm associating events to SESs and made the event based
hazard calculator faster in the case of many SESs
Reduced substantially the memory consumption in event based risk
Made it possible to read multiple site model files in the same calculation
Implemented a smart single job.ini file mode for event based risk
Now warnings for invalid parameters are logged in the database too
Fixed oq export avg_losses-stats for the case of one realization
Added oq export losses_by_tag and oq export curves_by_tag
Extended oq export to work in a multi-user situation
Forbidden event based calculations with more than max_potential_paths
in the case of full enumeration
Saved a large amount of memory in event_based_risk calculations
Added a command oq export losses_by_tag/<tagname> <calc_id>
Extended oq zip to zip the risk files together with the hazard files
Changed the building convention for the event IDs and made them unique
in the event loss table, even in the case of full enumeration
Optimized the splitting of complex fault sources
Fixed the ShakeMap download procedure for uncertainty.zip archives
with an incorrect structure (for instance for ci3031111)
Disabled the spatial correlation in risk-from-ShakeMap by default
Optimized the rupture sampling where there is a large number of SESs
Extended the reqv feature to multiple tectonic region types and
removed the spinning/floating for the TRTs using the feature
Reduced the GMPE logic tree upfront for TRTs missing in the source model
Fixed the ShakeMap downloader to use the USGS GeoJSON feed
Improved the error message when there are more than 65536 distinct tags
in the exposure
Turned vs30measured into an optional parameter

[Chris Van Houtte (@cvanhoutte)]

Added siteclass as a site parameter, and reference_site_class as
a site parameter than can be specified by the user in the ini file
Added new classes to mcverry_2006.py to take siteclass as a predictor
Updated comments in mcverry_2006.py
Added new mcverry_2006 test tables to account for difference in site
parameter
Added qa_test_data classical case_32

[Michele Simionato (@micheles)]

Fixed the rupture exporter for Canada
Extended the oq prepare_site_model to optionally generate the
fields z1pt0, z2pt5 and vs30measured
It is now an error to specify both the sites and the site model in the
job.ini, to avoid confusion with the precedency
Implemented a reader for site models in CSV format
Made the export_dir relative to the input directory
Better error message for ShakeMaps with zero stddev
Added a source_id-filtering feature in the job.ini
Added a check on non-homogeneous tectonic region types in a source group
Fixed the option oq engine --config-file that broke a few releases ago
Replaced nodal_dist_collapsing_distance and
hypo_dist_collapsing_distance with pointsource_distance and made
use of them in the classical and event based calculators

[Graeme Weatherill (@g-weatherill)]

Fixes to hmtk completeness tables for consistent rates and addition of
more special methods to catalogue

[Michele Simionato (@micheles)]

Restricted ChiouYoungs2008SWISS01 to StdDev.TOTAL to avoid a bug
when computing the GMFs with inter/intra stddevs
Raised an error if assets are discarded because too far from the hazard
sites (before it was just a warning)
Added an attribute .srcidx to every event based rupture and stored it
Fixed an issue with the Byte Order Mark (BOM) for CSV exposures prepared
with Microsoft Excel
Reduced the site collection instead of just filtering it; this fixes
a source filtering bug and changes the numbers in case of GMF-correlation
Added a command oq prepare_site_model to prepare a sites.csv file
containing the vs30 and changed the engine to use it
Added a cutoff when storing a PoE=1 from a CSV file, thus avoiding NaNs
in classical_damage calculations
Reduced the data transfer in the risk model by only considering the
taxonomies relevant for the exposure
Extended oq engine --run to accept a list of files
Optimized the saving of the risk results in event based in the case of
many sites and changed the command oq show portfolio_loss to show
mean and standard deviation of the portfolio loss for each loss type

[Marco Pagani (@mmpagani)]

Added a first and preliminary version of the GMM for the Canada model
represented in an analytical form.
Added a modified version of Atkinson and Macias to be used for the
calculation of hazard in NSHMP2014.
Added support for PGA to the Si and Midorikawa (1999).

[Michele Simionato (@micheles)]

Made it possible to run the risk over an hazard calculation of another user
Worked around the OverflowError: cannot serialize a bytes object larger
than 4 GiB in event based calculations
Started using Python 3.6 features
Fixed the check on vulnerability function ID uniqueness for NRML 0.5
Ruptures and GMFs are now computed concurrently, thus mitigating the
issue of slow tasks
Changed the name of the files containing the disaggregation outputs:
instead of longitude and latitude they contain the site ID now
If a worker runs close to out of memory, now a warning appears in the
main log
'lons' and 'lats' are now spelled 'lon' and 'lat' in
the REQUIRES_SITES_PARAMETERS to be consistent with site_model.xml

[Daniele Viganò (@daniviga)]

Fixed a bug about 'The openquake master lost its controlling terminal'
when running with 'nohup' from command line

[Michele Simionato (@micheles)]

The export_dir is now created recursively, i.e. subdirectories are
automatically created if needed
Fixed a bug with the minimum_magnitude feature and extended it to be
tectonic region type dependent
Changed the rupture generation to yield bunches of ruptures, thus avoiding
the 4GB pickle limit
Parallelized the splitting of the sources, thus making the preprocessing
faster

[Marco Pagani (@mmpagani)]

Implemented two additional versions of the Silva et al. 2002 GMPE
Added the possibility of setting rake to 'undefined'
Added first 'modified GMPE' implementing the site term for Canada 2015 model
Fixed a bug in the disaggregation calculation due to wrong binning of magnitudes

[Michele Simionato (@micheles)]

Now the combination uniform_hazard_spectra=true and mean_hazard_curves=false
is accepted again, as requested by Laurentiu Danciu (@danciul)

[Daniele Viganò (@daniviga)]

Support for Ubuntu Trusty is removed
Replaced supervisord with systemd in Ubuntu packages

[Michele Simionato (@micheles)]

Changed the way the rupture geometries are stored to be consistent with
the source geometries
We are now saving information about the source geometries in the datastore
(experimentally)
Fixed a bug in event based with sampling causing incorrect GMFs
Unified all distribution mechanisms to returns the outputs via zmq
Added a check for inconsistent IMTs between hazard and risk
Replaced the forking processpool with a spawning processpool

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 6 years ago

OpenQuake - OpenQuake Engine 3.2.0

[Matteo Nastasi (@nastasi-oq)]

specified 'amd64' as the only architecture supported by ubuntu packages

[Michele Simionato (@micheles)]

Changed the source writer: now the srcs_weights are written in the XML
file only if they are nontrivial
Changed the algorithm assigning the seeds: they are now generated before
the source splitting; also, a seed-related bug in the splitting was fixed
For event based, moved the rupture generation in the prefiltering phase

[Daniele Viganò (@daniviga)]

Fixed a bug with CTRL-C when using the processpool distribution

[Robin Gee (@rcgee)]

Raised the source ID length limit in the validation from 60 to 75 characters
to allow sources with longer IDs

[Michele Simionato (@micheles)]

Introduced a multi_node flag in openquake.cfg and used it to
fully parallelize the prefiltering in a cluster
Used the rupture seed as rupture ID in event based calculations
Changed the deprecation mechanism of GSIMs to use a class attribute
superseded_by=NewGsimClass
Solved the pickling bug in event based hazard by using generator tasks
Improved the distribution of the risk tasks by changing the weight

[Pablo Heresi (@pheresi)]

Contributed the HM2018CorrelationModel

[Michele Simionato (@micheles)]

Restored the individual_curves flag that for the moment is used for the
risk curves
Introduced two experimental new parameters floating_distance and
spinning_distance to reduce hypocenter distributions and nodal plane
distributions of ruptures over the corresponding distances
Optimized the parsing of the logic tree when there is no "applyToSources"
Made the IMT classes extensible in client code
Reduced the hazard maps from 64 to 32 bit, to be consistent with the
hazard curves and to reduce by half the download time

[Graeme Weatherill (@g-weatherill)]

Implements a fix of Montalva et al (2016) for new coefficients (now
Montalva et al. (2017))

[Michele Simionato (@micheles)]

Parallelized the reading of the source models
Optimized oq info --report by not splitting the sources in that case
Speedup the download of the hazard curves, maps and uhs
Honored concurrent_tasks in the prefiltering phase too
It is now legal to compute uniform hazard spectra for a single period
Added command oq plot_memory
Introduced a MultiGMPE concept
Saved the size of the datastore in the database and used it in the WebUI

[Graeme Weatherill (@g-weatherill)]

Adds geotechnical related IMTs

[Michele Simionato (@micheles)]

Renamed /extract/agglosses -> /extract/agg_losses and same for aggdamages
Supported equivalent epicentral distance with a reqv_hdf5 file
Fixed the risk from ShakeMap feature in the case of missing IMTs
Changed the way gmf_data/indices and ruptures are stored
Added experimental support for dask
Added 11 new site parameters for geotechnic hazard
Changed the SiteCollection to store only the parameters required by the
GSIMs

[Robin Gee (@rcgee)]

The number of sites is now an argument in the method _get_stddevs()
in the GMPE of Kanno, 2006

[Michele Simionato (@micheles)]

Changed the serialization of ruptures to HDF5: the geometries are now
stored in a different dataset
Bug fix: the asset->site association was performed even when not needed
Made it possible to serialize to .hdf5 multipoint sources and
nonparametric gridded sources
Added a check on source model logic tree files: the uncertaintyModel
values cannot be repeated in the same branchset
Added a flag std_hazard_curves; by setting it to true the user can
compute the standard deviation of the hazard curves across realizations

[Marco Pagani (@mmpagani)]

Added Thingbaijam et al. (2017) magnitude-scaling relationship

[Michele Simionato (@micheles)]

Added an /extract/ API for event_based_mfd
Fixed a bug in the classical_damage calculators: multiple loss types
were not treated correctly

[Marco Pagani (@mmpagani)]

Adding tests to the method computing decimal time

[Michele Simionato (@micheles)]

Removed the event_based_rupture calculator and three others
Added a field size_mb to the output table in the database and made
it visible in the WebUI as a tooltip
Added a command oq check_input job.ini to check the input files
Made the loss curves and maps outputs from an event based risk calculation
visible to the engine and the WebUI (only the stats)
Added a check on duplicated branchIDs in GMPE logic trees

[Daniele Viganò (@daniviga)]

Fixed a bug when reading exposure with utf8 names on systems with non-utf8
terminals (Windows)
Changed the openquake.cfg file and added a dbserver.listen parameter
Added the hostname in the WebUI page. It can be customize by the user
via the local_settings.py file

[Michele Simionato (@micheles)]

Added a Content-Length to the outputs downloadable from the WebUI
Fixed a bug when extracting gmf_data from a hazard calculation with a
filtered site collection
Stored an attributed events.max_gmf_size
Added a check on exposures with missing loss types
Added a LargeExposureGrid error to protect the user by tricky exposures
(i.e. France with assets in the Antilles)
Changed the event_based_risk calculator to compute the loss curves and
maps directly; removed the asset_loss_table
Changed the event_based_risk calculator to distribute by GMFs always
Optimized the memory consumption in the UCERF classical calculator
Added a parameter minimum_magnitude in the job.ini
Added an utility utils/combine_mean_curves.py

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 6 years ago

OpenQuake - OpenQuake Engine 3.1.0

[Marco Pagani (@mmpagani) and Changlong Li (@mstlgzfdh)]

Added a version of the Yu et al. (2013) GMPE supporting Mw

[Michele Simionato (@micheles)]

Reduced the data transfer in the UCERF calculators
Stored the zipped input files in the datastore for reproducibility
Fixed a regression when reading GMFs from an XML in absence of a sites.csv
file

[Robin Gee (@rcgee)]

Extend oq to_shapefile method to also work with YoungsCoppersmithMFD
and arbitraryMFD MFD typologies.

[Michele Simionato (@micheles)]

Now the hazard statistics can be computed efficiently even in a single
calculation, i.e. without the --hc option
Added a check on the Python version in the oq command
Reduced the data transfer when sending the site collection
Changed the default filter_distance

[Daniele Viganò (@daniviga)]

Fixed a bug where the PID was not saved into the database
when using the command line interface
Made it impossible to fire multiple CTRL-C in sequence
to allow processes teardown and tasks revocation when Celery is used

[Michele Simionato (@micheles)]

Used scipy.spatial.distance.cdist in Mesh.get_min_distance
Prefiltered sites and assets in scenario calculations
Made it possible to specify the filter_distance in the job.ini
Made rtree optional again and disabled it in macOS
Optimized the SiteCollection class and doubled the speed of distance
calculations in most continental scale calculations
Fixed an ordering bug in event based risk from GMFs when using a
vulnerability function with PMF
Replaced Rtree with KDtree except in the source filtering
Parallelized the source prefiltering
Removed the tiling feature from the classical calculator
Undeprecated hazardlib.calc.stochastic.stochastic_event_set and
made its signature right
Removed the source typology from the ruptures and reduced the rupture
hierarchy
Removed the mesh spacing from PlanarSurfaces
Optimized the instantiation of the rtree index
Replaced the old prefiltering mechanism with the new one

[Daniele Viganò (@daniviga)]

Managed the case of a dead controlling terminal (SIGHUP)

[Michele Simionato (@micheles)]

Removed Decimal numbers from the PMF distribution in hazardlib
Fixed another tricky bug with rtree filtering across the international
date line
Added a parameter prefilter_sources with values rtree|numpy|no
Removed the prefiltering on the workers, resulting in a huge speedup
for gridded ruptures at the cost of a larger data transfer
Changed the losses_by_event output to export a single .csv file with
all realizations
Added a cross_correlation parameter used when working with shakemaps
Now sites and exposure can be set at the same time in the job.ini
Introduced a preclassical calculator
Extended the scenario_damage calculator to export dmg_by_event
outputs as well as losses_by_event outputs if there is a consequence
model
Unified region and region_constraint parameters in the job.ini
Added a check to forbid duplicated GSIMs in the logic tree
Introduced some changes to the realizations exporter (renamed field
uid -> branch_path and removed the model field)
Added a command oq celery inspect
Reduced the check on too many realizations to a warning, except for
event based calculations
Improved the hazard exporter to exports only data for the filtered
site collection and not the full site collection
Extended the BCR exporter to export the asset tags

[Catalina Yepes (@CatalinaYepes)]

Revised/enhanced the risk demos

[Michele Simionato (@micheles)]

Added a warning about the option optimize_same_id_sources when the user
should take advantage of it

[Daniele Viganò (@daniviga)]

celery-status script converted into oq celery status command
Removed Django < 1.10 backward compatibility
Updated Python dependices (numpy 1.14, scipy 1.0.1,
Django 1.10+, Celery 4+)

[Michele Simionato (@micheles)]

Implemented scenario_risk/scenario_damage from shakemap calculators
Exported the asset tags in the asset based risk outputs
Fixed a numeric issue for nonparametric sources causing the hazard curves
to saturate at high intensities
Added an utility to download shakemaps
Added an XML exporter for the site model
Slight change to the correlation module to fix a bug in the SMTK
Added a distribution mechanism threadpool

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 7 years ago

OpenQuake - OpenQuake Engine 3.0.1

[Michele Simionato (@micheles)]

Fixed a numeric bug affecting nonparametric sources and causing saturating
hazard curves at high intensity

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 7 years ago

OpenQuake - OpenQuake Engine 3.0.0

[Michele Simionato (@micheles)]

Fixed a bug with newlines in the logic tree path breaking the CSV exporter
for the realizations output
When setting the event year, each stochastic event set is now considered
independent
Fixed a bug in the HMTK plotting libraries and added the ability to
customize the figure size
Fixed bug in the datastore: now we automatically look for the attributes
in the parent dataset, if the dataset is missing in the child datastore
Extended extract_losses_by_asset to the event based risk calculator
Stored in source_info the number of events generated per source
Added a script utils/reduce_sm to reduce the source model of a calculation
by removing all the sources not affecting the hazard
Deprecated openquake.hazardlib.calc.stochastic.stochastic_event_set
Fixed the export of ruptures with a GriddedSurface geometry
Added a check for wrong or missing <occupancyPeriods> in the exposure
Fixed the issue of slow tasks in event_based_risk from precomputed GMFs
for sites without events
Now the engine automatically associates the exposure to a grid if
region_grid_spacing is given and the sites are not specified otherwise
Extracting the site mesh from the exposure before looking at the site model
Added a check on probs_occur summing up to 1 in the SourceWriter
oq show job_info now shows the received data amount while the
calculation is progressing

[Daniele Viganò (@daniviga)]

Removed support for Python 2 in setup.py
Removed files containing Python 2 dependencies
Added support for WebUI groups/permissions on the
export outputs and datastore API endpoints

[Michele Simionato (@micheles)]

Fixed oq show for multiuser with parent calculations
Fixed get_spherical_bounding_box for griddedSurfaces
Implemented disaggregation by source only for the case
of a single realization in the logic tree (experimental)
Replaced celery with celery_zmq as distribution mechanism
Extended oq info to work on source model logic tree files
Added a check against duplicated fields in the exposure CSV
Implemented event based with mutex sources (experimental)
Add an utility to read XML shakemap files in hazardlib
Added a check on IMTs for GMFs read from CSV

[Daniele Viganò (@daniviga)]

Changed the default DbServer port in Linux packages from 1908 to 1907

[Michele Simionato (@micheles)]

Logged rupture floating factor and rupture spinning factor
Added an extract API for losses_by_asset
Added a check against GMF csv files with more than one realization
Fixed the algorithm setting the event year for event based with sampling
Added a command oq importcalc to import a remote calculation in the
local database
Stored avg_losses-stats in event based risk if there are multiple
realizations
Better error message in case of overlapping sites in sites.csv
Added a an investigation time attribute to source models with
nonparametric sources
Bug fix: in some cases the calculator event_based_rupture was generating
too few tasks and the same happened for classical calculation with
`optimize_same_id_sources=true
Changed the ordering of the epsilons in scenario_risk
Added the ability to use a pre-imported risk model
Very small result values in scenario_damage (< 1E-7) are clipped to zero,
to hide numerical artifacts
Removed an obsolete PickleableSequence class
Fixed error in classical_risk when num_statistics > num_realizations
Fixed a TypeError when reading CSV exposures with occupancy periods
Extended the check on duplicated source IDs to models in format NRML 0.5
Added a warning when reading the sources if .count_ruptures() is
suspiciously slow
Changed the splitting logic: now all sources are split upfront
Improved the splitting of complex fault sources
Added a script to renumber source models with non-unique source IDs
Made the datastore of calculations using GMPETables relocatable; in
practice you can run the Canada model on a cluster, copy the .hdf5 on
a laptop and do the postprocessing there, a feat previously impossible.

[Valerio Poggi (@klunk386)]

Included a method to export data directly from the Catalogue() object into
standard HMTK format.

[Michele Simionato (@micheles)]

Now the parameter disagg_outputs is honored, i.e. only the specified
outputs are extracted from the disaggregation matrix and stored
Implemented statistical disaggregation outputs (experimental)
Fixed a small bug: we were reading the source model twice in disaggregation
Added a check to discard results coming from the wrong calculation
for the distribution mode celery_zmq
Removed the long time deprecated commands
oq engine --run-hazard and oq engine --run-risk
Added a distribution mode celery_zmq
Added the ability to use a preimported exposure in risk calculations
Substantial cleanup of the parallelization framework
Fixed a bug with nonparametric sources producing negative probabilities

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 7 years ago

OpenQuake - OpenQuake Engine 2.9.0

[Michele Simionato (@micheles)]

Deprecated the NRML format for the GMFs

[Matteo Nastasi (@nastasi-oq)]

Debian package moved to Python 3.5

[Graeme Weatherill (@g-weatherill)]

Small bug fix for Derras et al (2014) GMPE when Rjb = 0.0

[Michele Simionato (@micheles)]

Improved the .rst reports for classical calculations with tiling
Reduced the data transfer in the event based risk calculator by
reading the event IDs directly from the workers
Integrated the gmf_ebrisk calculator inside the event based calculator
Improved the weighting algorithm for the sources in the event based
rupture calculator
Improved error message for source model files declared as nrml/0.5 when
they actually are nrml/0.4
Optimized the classical_bcr calculator for the case of many realizations
Extended the exposure CSV importer to manage the retrofitted field

[Marco Pagani (@mmpagani), Changlong Li]

Adds the Yu et al. (2013) GMPEs

[Michele Simionato (@micheles)]

Fixed a bug in the hazard outputs: they were displayed in the WebUI even
if they were missing
Implemented splitting of nonparametric sources

[Marco Pagani (@mmpagani)]

Fixed the 'get_closest_points' method for the
openquake.hazardlib.geo.surface.gridded.GriddedSurface class

[Michele Simionato (@micheles)]

Now the source model paths are relative to the source model logic tree file
Fixed an international date line bug when using rtree for prefiltering
Deprecated nrml.parse, it is now called nrml.to_python
Improved the task distribution by splitting the AreaSources upfront
and by improving the weighting algorithm

[Daniele Viganò (@daniviga)]

TMPDIR can be customized via the openquake.cfg file
Updated dependencies compatibility in setup.py

[Michele Simionato (@micheles)]

If the hazard is precomputed, setting the site_model_file,
gsim_logic_tree_file or source_model_logic_tree_file gives a warning
Removed the obsolete API /extract/qgis- and added extract/hcurves,
extract/hmaps, extract/uhs for use with the QGIS plugin
Removed the deprecated GeoJSON exporters
Fixed a bug with oq engine --run job.ini --exports npz
Fixed the ordering of the IMTs in hazardlib
oq engine --delete-calculation now aborts the calculation first
Added some documentation about how to access the datastore
Introduced a minimum_distance for the GSIMs
Fixed several small issues with the UCERF calculators; now ucerf_hazard
can be used as a precalculator of gmf_ebrisk
Initial support for disaggregation by source
Added the ability to import large exposures as CSV (experimental)
Changed the source weights to be proportional to the number of GSIMs
relevant for the tectonic region type, thus improving the task distribution

[Daniele Viganò (@daniviga)]

The RPM python3-oq-engine package replaced python-oq-engine
RPM packages moved to Python 3.5

[Michele Simionato (@micheles)]

Added the ability to dump a specific calculation
Changed the signature of the extract command to oq extract what calc_id,
where what is a path info plus query string;

[Graeme Weatherill (@g-weatherill)]

Implements significant duration GMPEs of Bommer et al. (2009) and Afshari &
Stewart (2016)
Adds significant duration IMT definitions to support IMTs

[Michele Simionato (@micheles)]

Run the DbServer as a detached process
Improved the test coverage for event based with GMF correlation
Optimized the event based risk calculator from ruptures: now the ruptures
are instantiated in the workers and not in the controller if possible
Exported the parameter ses_per_logic_tree_path in the ruptures.csv file
Improved the display names for the risk outputs
Added a /v1/:calc_id/abort route to the engine server and Abort buttons
to the WebUI
Fixed the seeds properly in the case of vulnerability functions with PMFs:
now even if the ground motion fields are all equal, the risk numbers
will be different since there is a different seed per each field
Stored a rupture loss table in event based risk calculations
Made sure that the number of effective ruptures is stored in csm_info
Fixed the HMTK tests to work with numpy from 1.11 to 1.14
Added a command oq shell to open an embedded (I)Python shell
Turned the 'realizations' output into a dynamic output

[Matteo Nastasi (@nastasi-oq)]

Split package from python-oq-engine to python-oq-engine,
python-oq-engine-master and python-oq-engine-worker
Implemented an API /v1/on_same_fs to check filesystem accessibility
between engine and a client application

[Michele Simionato (@micheles)]

Reduced the data transfer when computing the hazard curves in postprocessing
Removed the FilteredSiteCollection class

[Nick Ackerley (@nackerley)]

Some improvements to the plotting routines of the HMTK

[Michele Simionato (@micheles)]

Removed the ability to run oq engine --run job_h.ini,job_r.ini
Forbidden the site model in gmf_ebrisk calculations
When the option --hc is given the ruptures can now be read directly
from the workers, thus saving some startup time
Optimized the storage of the ruptures: the site IDs are not saved anymore
Added a check for missing risk_investigation_time
Reduced the data transfer in the gmf_ebrisk calculator
Now the gmf_ebrisk calculator builds the aggregate loss curves too
Extended the gmf_ebrisk calculator to use the GMFs produced by an event
based hazard calculation, both via CSV and via the --hc option
Fixed a performance bug in the computations of the aggregate loss curves
by reading the full event loss table at once
Fixed oq zip to work with gmf_ebrisk calculations
Fixed a serious bug in the gmf_ebrisk calculator: the results were in most
cases wrong and dependent on the number of spawned tasks
Now the master_seed controls the generation of the epsilons in all
situations (before in event_based_risk without asset_correlation it was
managed by random_seed)
Changed the management of the epsilons in the gmf_ebrisk calculator to
be the same as in the event_based_risk calculator
Added a check on the input files: the listed paths must be relative paths
Fixed a bug when storing the disagg-bins in the case the lenghts of the
arrays are not the same for all sites
In the case of a single tile the prefiltering was applied twice: fixed the
problem and generally improved the filtering/weighting of the sources
Fixed the CSV exporter for disaggregation outputs when iml_disagg is set

[Graeme Weatherill (@g-weatherill)]

Fixed ASK14 GMPE behaviour to remove ValueError
Implements comprehensive suite of NGA East ground motion models for
central and eastern United States
Minor modification of check_gsim functions to permit instantiated classes
to be passed

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 7 years ago

OpenQuake - OpenQuake Engine 2.8.0

[Michele Simionato (@micheles)]

iml_disagg and poes_disagg cannot coexist in the job.ini file
Added a check on conditional_loss_poes in the event_based_risk calculator:
now it requires asset_loss_table to be set

[Anirudh Rao (@raoanirudh)]

Sorted taxonomies before comparison in the BCR calculator

[Michele Simionato (@micheles)]

Optimized the disaggregation calculation by performing the PMF extraction
only once at the end of the calculation and not in the workers
Added an oq zip command
Avoided running an useless classical calculation if iml_disagg is given

[Valerio Poggi (@klunk386)]

Implemented subclasses for ZhaoEtAl2006Asc and AtkinsonBoore2006 to account
for the distance filter used by SGS in their PSHA model for Saudi Arabia.
Distance threshold is hard coded to 5km in this implementation.

[Michele Simionato (@micheles)]

Added a warning if the aggregated probability of exceedence (poe_agg) in
a disaggregation output is very dissimilar from poes_disagg
Removed the flag split_sources
Optimized the operation arrange_data_in_bins in the disaggregation
calculator and reduced the data transfer by the number of tectonic
region types
Improved the sending of the sources to the workers, especially for the
MultiPointSources
Better error message if the user sets a wrong site_id in the sites.csv file
Now the distance and lon lat bins for disaggregation are built directly
from the integration distance
Used uniform bins for disaggregation (before they were potentially
different across realizations / source models)
Improved the error message if the user forgets both sites and sites.csv
in a calculation starting from predetermined GMFs
Improved the error message if the user specifies a non-existing file in
the job.ini
Change the ordering of the TRT bins in disaggregation: now they are
ordered lexicographically
Added more validity checks on the job.ini file for disaggregation
Changed the .hdf5 format generated by oq extract -1 hazard/rlzs; you can
still produce the old format by using oq extract -1 qgis-hazard/rlzs
Optimized the disaggregation calculator by instantiating
scipy.stats.truncnorm only once per task and not once per rupture
Optimized the disaggregation calculator when iml_disagg is specified,
by caching duplicated results
Made sure that oq dbserver stop also stops the zmq workers if the zmq
distribution is enabled
Added a check when disaggregating for a PoE too big for the source model
If iml_disagg is given, forbid intensity_measure_types_and_levels
Fixed the disaggregation outputs when iml_disagg is given: the PoE has
been removed by the name of the output and correct PoE is in the XML file
Fixed oq export loss_curves/rlzs for event_based_risk/case_master
Removed the obsolete parameter loss_curve_resolution
Fixed a Python 3 unicode error with oq engine --run job.zip
Added a command oq abort <calc_id>
Stored the avg_losses in classical risk in the same way as in
event_based_risk and made them exportable with the same format
Removed the outputs losses_by_tag from the event based risk calculators
and changed the default for the avg_losses flag to True
WebUI: now every job runs in its own process and has name oq-job-
Refactored the WebUI tests to use the DbServer and django.test.Client
Added an experimental feature optimize_same_id_sources
Fixed a bug in gmf_ebrisk when there are zero losses and added more
validity checks on the CSV file
The parameter number_of_ground_motion_fields is back to being optional in
scenario calculators reading the GMFs from a file, since it can be inferred
Removed the deprecated risk outputs dmg_by_tag, dmg_total,
losses_by_tag, losses_total
Deprecated the .geojson exporters for hazard curves and maps
We now keep the realization weights in case of logic tree sampling (before
they were rescaled to 1 / R and considered all equals)
Optimized sampling for extra-large logic trees
Added a check on missing source_model_logic_tree
Fix to the gmf_ebrisk calculator: the realization index in the event loss
table was incorrect and too many rows were saved
Added a way to restrict the source logic model tree by setting a sm_lt_path
variable in the job.ini (experimental)
Fixed the precedence order when reading openquake.cfd

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 7 years ago

OpenQuake - OpenQuake Engine 2.7.0

[Michele Simionato (@micheles)]

Fixed the risk exporters for tags containing non-ASCII characters

[Valerio Poggi (@klunk386)]

Implemented the Pankow and Pechmann 2004 GMPE (not verified)

[Graeme Weatherill (@g-weatherill)]

Added Derras et al. (2014) GMPE
Implemented the Zhao et al. (2016) GMPE for active shallow crustal,
subduction interface and subduction slab events
Adds 'rvolc' (volcanic path distance) to the distance context

[Michele Simionato (@micheles)]

The outputs loss_curves-rlzs and loss_curves-stats are now visible again
as engine outputs (before they were hidden)
Added a debug command oq plot_assets and fixed oq plot_sites
Added a flag --multipoint to the command oq upgrade_nrml
Deprecated several outputs: hcurves-rlzs, agg_loss_table, losses_total,
dmg_by_tag, dmg_total, losses_by_tag, losses_by_tag-rlzs
Extended the command oq extract job_id to check the database
Optimized the scenario damage calculator by vectorizing the calculation
of the damage states
Extended the FragilityFunctions to accept sequences/arrays of intensity
levels, as requested by Hyeuk Ryu

[Daniele Viganò (@daniviga)]

Added support for groups in the WebUI/API server

[Michele Simionato (@micheles)]

Added an experimental distribution mode of kind "zmq"
Implemented an API /extract/agglosses/loss_type?tagname1=tagvalue1&...
with the ability to select all tagvalues (*) for a given tagname
Deprecated reading hazard curves from CSV, since it was an experimental
features and nobody is using it
Changed the exporter of the event loss table to export all realizations
into a single file

[Graeme Weatherill (@g-weatherill)]

Adds the Bindi et al. (2017) GMPEs for Joyner-Boore and Hypocentral
Distance

[Michele Simionato (@micheles)]

Made it mandatory to specify the sites for all calculators reading the
GMFs from a CSV file
Tested also the case of calculators requiring a shared_dir
Improved the error checking when parsing vulnerability functions with PMF

[Daniele Viganò (@daniviga)]

Fixed a bug in oq reset command which was not stopping
the DbServer properly

[Michele Simionato (@micheles)]

Implemented an API /extract/aggcurves/loss_type?tagname1=tagvalue1&...
Implemented an API /extract/aggdamages/loss_type?tagname1=tagvalue1&...
Every time a new calculation starts, we check if there is a newer version
of the engine available on GitHub
Changed the search logic for the configuration file openquake.cfg
Implemented an API /extract/agglosses/loss_type?tagname1=tagvalue1&...
Fixed several bugs in the gmf_ebrisk calculator, in particular in presence
of asset correlation and missing values on some sites
Fixed a bug with logging configured a WARN level instead of INFO level
if rtree was missing (affecting only oq run)
Changed the ScenarioDamage demo to use two GSIMs
Added a node <tagNames> in the exposure
Added a web API to extract the attributes of a datastore object
Fixed oq to_shapefile and oq from_shapefile to work with NRML 0.5
(except MultiPointSources)
Added information about the loss units to the agg_curve outputs
oq extract hazard/rlzs now extracts one realization at the time
The rupture filtering is now applied during disaggregation
Changed the /extract wen API to return a compressed .npz file
Fixed a bug with multi-realization disaggregation, celery and no
shared_dir: now the calculation does not hang anymore

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 7 years ago

OpenQuake - OpenQuake Engine 2.6.0

[Michele Simionato (@micheles)]

Fixed the GMF .npz export when the GMFs are extracted from a file
Stored the number of nonzero losses per asset and realization in
event_based_risk calculations with asset_loss_table=True

[Daniele Viganò (@daniviga)]

Fixed 'openquake' user creation in RPM when SELinux is in enforcing mode
Changed the behaviour during RPM upgrades:
the old openquake.cfg configuration file is left untouched and the new one
installed as openquake.cfg.rpmnew

[Michele Simionato (@micheles)]

Added a check on number_of_ground_motion_fields when the GMFs are
extracted from a NRML file
Added a command oq extract able to extract hazard outputs into HDF5 files
Fixed a bug when reading GMFs from a NRML file: the hazard sites were
read from the exposure (incorrectly) and not from the GMFs
Fixed a bug in MultiMFDs of kind arbitraryMFD

[Valerio Poggi (@klunk386)]

Implemented the Atkinson (2010) GMPE as subclass Atkinson2010Hawaii
of BooreAtkinson2008

[Michele Simionato (@micheles)]

Used the new loss curves algorithm for the asset loss curves and loss maps
Added a generic extract functionality to the web API
Fixed a bug when computing the rjb distances with multidimensional meshes
Changed the GMF CSV exporter to export the sites too; unified it with the
event based one

[Daniele Viganò (@daniviga)]

Changed the 'CTRL-C' behaviour to make sure that all children
processes are killed when a calculation in interrupted

[Michele Simionato (@micheles)]

Fixed a bug in the statistical loss curves exporter for classical_risk
Replaced the agg_curve outputs with losses by return period outputs
Turned the DbServer into a multi-threaded server
Used zmq in the DbServer
Fixed correct_complex_sources.py
Fixed oq export hcurves-rlzs -e hdf5
Changed the source weighting algorithm: now it is proportional to the
the number of affected sites
Added a command oq show dupl_sources and enhances oq info job.ini
to display information about the duplicated sources
Added a flag split_sources in the job.ini (default False)
Updated the demos to the format NRML 0.5

[Valerio Poggi (@klunk386)]

Implemented the Munson and Thurber 1997 (Volcanic) GMPE

[Graeme Weatherill (@g-weatherill)]

Adapts CoeffsTable to be instantiated with dictionaries as well as strings

[Daniele Viganò (@daniviga)]

Extended the 'oq reset' command to work on multi user installations

[Michele Simionato (@micheles)]

Fixed a bug: if there are multiple realizations and no hazard stats,
it is an error to set hazard_maps=true or uniform_hazard_spectra=true
Implemented aggregation by asset tag in the risk calculators
Fixed a small bug in the HMTK (in get_depth_pmf)
Extended the demo LogicTreeCase1ClassicalPSHA to two IMTs and points
Added a documentation page oq-commands.md
Removed the automatic gunzip functionality and added an automatic
checksum functionality plus an oq checksum command
Made the demo LogicTreeCase2ClassicalPSHA faster
Fixed the export by realization of the hazard outputs
Changed the generation of loss_maps in event based risk, without the option
--hc: now it is done in parallel, except when reading the loss ratios
Renamed --version-db to --db-version, to avoid
confusions between oq --version and oq engine -version
Fixed bug in the exported outputs: a calculation cannot export the results
of its parent
Extended the sz field in the rupture surface to 2 bytes, making it
possible to use a smaller mesh spacing
Changed the ordering of the fields in the loss curves and loss maps
generated by the event based risk calculator; now the insured fields
are at the end, before they were intermixed with each loss type
Changed the format of array all_loss_ratios/indices
The size in bytes of the GMFs was saved incorrectly
Added an exporter gmf_scenario/rup-XXX working also for event based
First version of the calculator gmf_ebrisk
Implemented risk statistics for the classical_damage calculator
Added a .csv importer for the ground motion fields
Implemented risk statistics for the classical_bcr calculator

[Armando Scarpati (@hascar)]

Show to the user the error message when deleting a calculation
in the WebUI fails

[Michele Simionato (@micheles)]

Better error message when running a risk file in absence of hazard
calculation
Changed the sampling logic in event based calculators
Imported GMFs from external file into the datastore

[Daniele Viganò (@daniviga)]

Added the 'celery-status' script in 'utils' to check the
task distribution in a multi-node celery setup

[Michele Simionato (@micheles)]

Removed an excessive check from the WebUI: now if an output exists,
it can be downloaded even if the calculation was not successful

[Armando Scarpati (@hascar)]

Visualized the calculation_mode in the WebUI

[Michele Simionato (@micheles)]

Made the upgrade_manager transactional again
Changed the storage of the GMFs; as a consequence the exported .csv
has a different format

[Daniele Viganò (@daniviga)]

Fixed a bug introduced by a change in Django 1.10 that was causing
the HTTP requests log to be caught by our logging system and
then saved in the DbServer
Updated requirements to allow installation of Django 1.11 (LTS)

[Michele Simionato (@micheles)]

Added two commands oq dump and oq restore
Added a check that on the number of intensity measure types when
generating uniform hazard spectra (must be > 1)

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 7 years ago

OpenQuake - OpenQuake Engine 2.5.0

[Armando Scarpati (@hascar)]

Added a confirmation dialog when trying to remove a calculation via the
WebUI

[Michele Simionato (@micheles)]

Hazard maps were not exposed to the engine in event based calculations
Fixed the check on the DbServer instance: it was failing in presence
of symbolic links
Optimized MultiMFD objects for the case of homogeneous parameters
Added an .npz exporter for the scenario_damage output dmg_by_asset
Removed the pickled CompositeSourceModel from the datastore
Improved the error message when the rupture mesh spacing is too small
Unified the versions of baselib, hazardlib and engine
Raised a clear error if the user does not set the calculation_mode
Made it is possible to pass the hdf5 full path to the DataStore class
Made it possible to use CELERY_RESULT_BACKEND != 'rpc://'

[Michele Simionato (@micheles), Daniele Viganò (@daniviga)]

Merged the oq-hazardlib repository into oq-engine.
The python-oq-hazardlib package is now provided by python-oq-engine

[Michele Simionato (@micheles)]

Added CSV exports for the agg_curve outputs
Fixed a bug in oq export hcurves-rlzs --exports hdf5
Restored the parameter sites_per_tile with a default of 20,000, i.e.
tiling starts automatically if there are more than 20,000 sites
Better error message for invalid exposures
Removed the deprecated XML outputs of the risk calculators
Added an end point v1/calc/XXX/oqparam to extract the calculation
parameters as a JSON dictionary
Fixed the excessive logic tree reduction in event based calculators
Improved the command oq db
Fixed an encoding bug when logging a filename with a non-ASCII character
Fixed a bug when exporting a GMF with ruptureId=0
Added a parameter disagg_outputs to specify the kind of disaggregation
outputs to export
Raised an early error if the consequence model is missing some taxonomies
Restored the tiling functionality in the classical calculator; to enable
it, set num_tiles in the job.ini file
If there are no statistical hazard curves to compute, do not transfer
anything
Fixed a small bug in oq plot and added a test

[Daniele Viganò (@daniviga)]

Added collectstatic and createsuperuser subcommands to the
oq webui command
Added a local_settings.py.pam template to use PAM as the authentication
provider for API and WebUI
Now the command oq webui start tries to open a browser tab
with the WebUI loaded

Climate Change - Natural Hazard and Storm - Python
Published by daniviga almost 8 years ago

OpenQuake - OpenQuake Engine 2.4.0

[Michele Simionato (@micheles)]

Now the command oq export loss_curves/rlz-XXX works both for the
classical_risk calculator and the event_based_risk calculator

[Daniele Viganò (@daniviga)]

Remove the default 30 day-old view limit in the WebUI calculation list

[Michele Simionato (@micheles)]

Fixed a broken import affecting the command oq upgrade_nrml
Made it possible to specify multiple file names in
in the source_model_logic_tree file
Reduced the data transfer in the object RlzsAssoc and improved the
postprocessing of hazard curves when the option --hc is given
Changed the ruptures.xml exporter to export unique ruptures
Fixed a bug when downloading the outputs from the WebUI on Windows
Made oq info --report fast again by removing the rupture fine filtering
Improved the readibility of the CSV export dmg_total
Removed the column eid from the CSV export ruptures; also
renamed the field serial to rup_id and reordered the fields
Changed the event loss table exporter: now it exports an additional
column with the rup_id
Changed scenario npz export to export also the GMFs outside the maximum
distance
Fixed scenario npz export when there is a single event
Replaced the event tags with numeric event IDs
The mean hazard curves are now generated by default
Improved the help message of the command oq purge
Added a @reader decorator to mark tasks reading directly from the
file system
Removed the .txt exporter for the GMFs, used internally in the tests
Fixed a bug with relative costs which affected master for a long time,
but not the release 2.3. The insured losses were wrong in that case.
Added an .hdf5 exporter for the asset loss table
Loss maps and aggregate losses are computed in parallel or sequentially
depending if the calculation is a postprocessing calculation or not
Deprecated the XML risk exporters
Removed the .ext5 file
Restored the parameter asset_loss_table in the event based calculators
Added a full .hdf5 exporter for hcurves-rlzs
Removed the individual_curves flag: now by default only the statistical
hazard outputs are exported
Saved a lot of memory in the computation of the hazard curves and stats
Renamed the parameter all_losses to asset_loss_table
Added an experimental version of the event based risk calculator which
is able to use GMFs imported from an external file
Added a max_curve functionality to compute the upper limit of the
hazard curves amongst realizations
Raised an error if the user specifies quantile_loss_curves
or conditional_loss_poes in a classical_damage calculation
Added a CSV exporter for the benefit-cost-ratio calculator
The classical_risk calculator now reads directly the probability maps,
not the hazard curves
Turned the loss curves into on-demand outputs
for the event based risk calculator
The loss ratios are now stored in the datastore and not in an
external .ext5 file
The engine outputs are now streamed by the WebUI
Used a temporary export directory in the tests, to avoid conflicts
in multiuser situations
Added an .npz exporter for the loss maps
Raised an error early when using a complex logic tree in scenario
calculations
Changed the CSV exporter for the loss curves: now it exports all the
curves for a given site for the classical_risk calculator
Fixed the save_ruptures procedure when there are more than 256
surfaces in the MultiSurface
Renamed the csq_ outputs of the scenario_damage to losses_
Changed the way scenario_damage are stored internally to be more
consistent with the other calculators
Removed the GSIM from the exported file name of the risk outputs
New CSV exporter for GMFs generated by the event based calculator
The event IDs are now unique and a constraint on the maximum
number of source groups (65,536) has been added
Added an output losses_by_event to the scenario_risk calculator
Changed the output ruptures.csv to avoid duplications
Added an output losses_by_taxon to the scenario_risk calculator
Fixed a performance bug in get_gmfs: now the scenario risk and damage
calculators are orders of magnitude faster for big arrays
Added an export test for the event loss table in the case of multiple TRTs
Removed the experimental rup_data output
Added an .npz export for the output losses_by_asset
Exported the scenario_risk aggregate losses in a nicer format

[Daniele Viganò (@daniviga)]

The 'oq webui' command now works on a multi-user installation
Splitted RPM packages into python-oq-engine (single node) and
python-oq-engine-master/python-oq-engine-worker (multi-node)

[Paolo Tormene (@ptormene)]

The 'Continue' button in the Web UI is now available also for risk
calculations

[Michele Simionato (@micheles)]

Fixed a Python 3 bug in the WebUI when continuing a calculation: the
hazard_calculation_id was passed as a string and not as an integer
Changed to rupture storage to use variable length-arrays, with a speedup
of two orders of magnitude
Avoided storing twice the rupture events
Optimized the serialization of ruptures on HDF5 by using a sids output
Changed the Web UI button from "Run Risk" to "Continue"
The avg field in the loss curves is computed as the integral of the curve
again, and it is not extracted from the avg_losses output anymore
Made the fullreport exportable
Fixed the rup_data export, since the boundary field was broken
Restored the output losses_by_taxon in the event_based_risk calculator
Fixed the calculator event based UCERF so that average losses can
be stored

[Daniele Viganò (@daniviga)]

Added a check to verify that an 'oq' client is talking to the
right DbServer instance
Introduced an optional argument for 'oq dbserver' command line
to be able to override its default interface binding behaviour

[Michele Simionato (@micheles)]

Optimized the event based calculators by reducing the number of calls
to the GmfComputer and by using larger arrays
Added a check on missing vulnerability functions for some loss type
for some taxonomy
Now we save the GMFs on the .ext5 file, not the datastore
Fixed bug in event_based_risk: it was impossible to use vulnerability
functions with "PM" distribution
Fixed bug in event_based_risk: the ebrisk calculator is required as
precalculator of event_based_risk, not others
Fixed bug in scenario_risk: the output all_losses-rlzs was aggregated
incorrectly
Now the ucerf_risk calculators transfer only the events, not the ruptures,
thus reducing the data transfer of several orders of magnitude
Added a view get_available_gsims to the WebUI and fixed the API docs
Introduced a configuration parameter max_site_model_distance with default
of 5 km
Implemented sampling in the UCERF event based hazard calculator

[Daniele Viganò (@daniviga)]

Use threads instead of processes in DbServer because SQLite3
isn't fork-safe on macOS Sierra

[Michele Simionato (@micheles)]

Fixed a TypeError when deleting a calculation from the WebUI
Extended the command oq to_hdf5 to manage source model files too
Improved significantly the performance of the event based calculator
when computing the GMFs and not the hazard curves
Stored information about the mean ground motion in the datastore
Saved the rupture mesh with 32 floats instead of 64 bit floats
Raised the limit on the event IDs from 2^16 to 2^32 per task
Fixed classical_risk: there was an error when computing the statistics
in the case of multiple assets of the same taxonomy on the same site
Changed the UCERF event based calculators to parallelize by SES
Fixed a site model bug: when the sites are extracted from the site model
there is no need to perform geospatial queries to get the parameters
Added a command oq normalize to produce good sites.csv files
Introduced a ses_seed parameter to specify the seed used to generate
the stochastic event sets; random_seed is used for the sampling only
Changed the build_rcurves procedure to read the loss ratios directly from
the workers

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 8 years ago

OpenQuake - OpenQuake Engine 2.3.0

[Michele Simionato (@micheles)]

oq info --report now filters the ruptures and reports the correct
number of effective ruptures even for classical calculators
Stripped the TRT information from the event loss table CSV export
and optimized its performance
Fixed a bug when storing the GMPE logic tree file in the datastore
Added a command oq run_tiles (experimental)
Fixed the event based calculator so that it can run UCERF ruptures
Fixed a bug in the scenario_risk calculator in case of multiple assets
of the same taxonomy on the same site with no insurance losses
Now the event IDs are generated in the workers in the event based calculator
and there is a limit of 65536 tasks with 65536 ruptures each
Changed the UCERF classical calculators to compute one branch at the time
Fixed the header occupants:float32 in the CSV risk exports involving
occupants
Fixed the name of the zipped files downloaded by the Web UI: there
was a spurious dot
Fixed the UCERF classical calculator in the case of sampling
Reduced the size of the event tags in the event based calculators, thus
saving GB of disk space in UCERF calculations
Fixed the name of the files downloaded by the Web UI: they must not
contain slashes
Now deleting a calculation from the Web UI really deletes it, before
if was only hiding it

[Daniele Viganò (@daniviga)]

Moved the OpenQuake Engine manual sources inside doc/manual

[Michele Simionato (@micheles)]

Introduced an experimental classical time dependent UCERF calculator
Added a dynamic output for source group information
Changed the UCERF rupture calculator to fully store the ruptures
Fixed a bug in combine_maps: realizations with zero probability were
discarded, thus breaking the computation of the statistics
Added a command oq reset to reset database and datastores
Reduced the data transfer back and disk space occupation for UCERF
event based risk calculations
Tasks meant to be used with a shared directory are now marked with a
boolean attribute .shared_dir_on
Added a warning when running event based risk calculations with sampling
Made sure that the openquake.cfg file is read only once

[Daniele Viganò (@daniviga)]

Moved the openquake.cfg config file inside the python package
under openquake/engine/openquake.cfg
Removed support to OQ_LOCAL_CFG_PATH and OQ_SITE_CFG_PATH vars;
only the OQ_CONFIG_FILE enviroment variable is read

[Michele Simionato (@micheles)]

If there is a single realization, do not compute the statistics
Changed the separator from comma to tab for the output ruptures
If there are no conditional_loss_poes, the engine does not try to
export the loss maps anymore
Fixed oq engine --make-html-report when using Python 3
Fixed bug when running oq info job.ini with NRML 0.5 source models

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 8 years ago

OpenQuake - OpenQuake Engine 2.2.0

[Michele Simionato (@micheles)]

Fixed an HDF5 bug by not using a vstr array for the asset references
Fixed a wrong error message generated by oq purge
Added information about the rupture in the event loss table exports
Fixed a bug and added a test calculation with nonparametric sources
Fixed the classical UCERF calculator when there is more than one branch
Added .npz exporter for gmf_data for event based calculations

[Daniele Viganò (@daniviga)]

Port WebUI/API server to Django 1.9 and 1.10
Add dependencies to setup.py

[Michele Simionato (@micheles)]

Increased the splitting of ComplexFaultSources
Added a way to reuse the CompositeSourceModel from a previous computation
Turned the loss maps into dynamically generated outputs
Extended the source model writer to serialize the attributes
src_interdep, rup_interdep, srcs_weights
Fixed a bug when exporting the uniform hazard spectra in presence of
IMTs non spectral acceleration
Fixed a bug when computing the loss maps in presence of insurance,
temporarily introduced in master
Made the datastore for event based risk calculations much lighter
by computing the statistical outputs at export time
Now it is possible to post process event based risk outputs with the
--hc option
Added a command oq to_hdf5 to convert .npz files into .hdf5 files
Moved commonlib.parallel into baselib
Merged the experimental calculator ebrisk into event_based_risk and
used correctly the random_seed for generating the GMFs (not the master_seed)
Added a flag ignore_covs to ignore the coefficients of variation
Changed the GMF scenario exporter to avoid generating composite arrays with
a large number of fields
Exporting in .npz format rather than HDF5
Introduced a shared_dir parameter in openquake.cfg
Fixed a serialization bug for planar surfaces
Removed the flag asset_loss_table: the loss ratios are
saved if and only if the loss_ratios dictionary is non-empty
Added a CSV exporter for the GMFs in the event based calculator
Added a CSV exporter for the rup_data output
Added a CSV exporter for the disaggregation output
Stored the disaggregation matrices directly (no pickle)
Turned the CompositeRiskModel into a HDF5-serializable object
Fixed all doctests for Python 3

[Daniele Viganò (@daniviga)]

Removed the 'oq-engine' wrapper (command already deprecated)

[Michele Simionato (@micheles)]

Assigned a year label to each seismic event in the event based calculator
Now the ebrisk calculator supports the case of asset_correlation=1 too
Made it possible to export the losses generated by a specific event
Lowered the limit on the length of source IDs to 60 chars
Fixed excessive strictness when validating consequenceFunction.id
Added an ucerf_rupture calculator able to store seismic events and
rupture data and reduced the data transfer

[Daniele Viganò (@daniviga)]

MANIFEST now includes all files, with any extension located in the
tests folders. It is now possible to run tests from an installation
made with packages

[Michele Simionato (@micheles)]

Improved error message when the user gives a source model file instead of
a source model logic tree file
Fixed the management of negative calculation IDs
Relaxed the tolerance so that the tests pass on Mac OS X
Implemented csv exporter for the ruptures
Optimized the epsilon generation in the ebrisk calculator for
asset_correlation=0
Improved the performance of the scenario risk calculators
Now by default we do not save the ruptures anymore
Fixed a memory leak recently introduced in parallel.py
Simplified classical_risk (the numbers can be slightly different now)
Serialized the ruptures in the HDF5 properly (no pickle)
Introduced a parameter iml_disagg in the disaggregation calculator
Fixed oq reduce to preserve the NRML version
Fixed a bug when splitting the fault sources by magnitude

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 8 years ago

OpenQuake - OpenQuake Engine 2.1.1

[Michele Simionato (@micheles)]

Fixed a bug when splitting the fault sources by magnitude

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 8 years ago

OpenQuake - OpenQuake Engine 2.1.0

[Michele Simionato (@micheles)]

There is now a flag save_ruptures that can be turned off on demand;
by default the ruptures are always saved in the event based calculators
Optimized the memory consumption when using a ProcessPoolExecutor (i.e
fork before reading the source model) by means of a wakeup task
Reduced the splitting of the fault sources
Added a view task_slowest displaying info about the slowest task
(only for classical calculations for the moment)
concurrent_tasks=0 disable the concurrency
Optimized the saving time of the GMFs
Changed the default number of concurrent tasks and increased the
relative weight of point sources and area sources
Fixed the UCERF event loss table export and added a test for it
Optimized the computation of the event loss table
Introduced two new calculators ucerf_risk and ucerf_risk_fast

[Paolo Tormene (@ptormene)]

Added to the engine server the possibility to log in and out
programmatically by means of HTTP POST requests

[Michele Simionato (@micheles)]

Optimized the memory consumption of the event based risk calculators
Extended the oq show command to work in a multi-user environment
Improved the test coverage of the exports in the WebUI
Removed the SourceManager: now the sources are filtered in the workers
and we do not split in tiles anymore
Made the full datastore downloadable from the WebUI
Added a command "oq db" to send commands the engine database
(for internal usage)
By default the WebUI now displays only the last 100 calculations
Added more validity checks to the disaggregation parameters; split the
sources even in the disaggregation phase
Added an optimized event based calculator computing the total losses by
taxonomy and nothing else
Filtered the sources up front when there are few sites (<= 10)
Reduced the number of tasks generated when filter_sources is False
Saved engine_version and hazardlib_version as attributes of the datastore
Avoided saving the ruptures when ground_motion_fields is True
Finalized the HDF5 export for hazard curves, hazard maps and uniform
hazard spectra
Restored a weight of 1 for each rupture in the event based calculator
Removed the MultiHazardCurveXMLWriter
Improved the saving of the ruptures in event based calculations
Reduced the data transfer due to the rlzs_by_gsim parameter
Added an HDF5 export for scenario GMFs
If filter_sources if false, the light sources are not filtered, but the
heavy sources are always filtered
Now the dbserver can be stopped correctly with CTRL-C
Parallelized the splitting of heavy sources
Changed the event loss table exporter: now a single file per realization
is exported, containing all the loss types
Removed the dependency from the Django ORM
Now the WebUI restarts the ProcessPoolExecutor at the end of each job,
to conserve resources
Optimized the computation of hazard curves and statistics, especially
for the memory consumption
Reduced the data transfer due to the rlzs_assoc and oqparam objects
Fixed a bug in the disaggregation calculator when a source group has
been filtered away by the maximum distance criterium
Fixed an encoding error in the reports when the description contains a
non-ASCII character
Changed the distribution framework: celery is supported in a way more
consistent with the other approaches; moreover, ipyparallel is supported
Hazard maps are now a fake output, dynamically generated at export time
Made the number of produced tasks proportional to the number of tiles
Raised an error for event_based_risk producing no GMFs
Added a view for the slow sources
Transmitted the attributes of a SourceGroup to the underlying sources
Fixed the names of exported files for hazard maps in .geojson format
Added an header with metadata to the exported hazard curves and maps
Avoid storing filtered-away probability maps, thus fixing a bug
Restored the precalculation consistency check that was disabled during the
transition to engine 2.0
Fixed a bug with oq engine --delete-calculation
Hazard curves/maps/uniform spectra can now be recomputed
Restored the early check on missing taxonomies
Raise an early error if an user forget the rupture_mesh_spacing parameter
Fixed a bug while deleting jobs from the db in Ubuntu 12.04
Ported the shapefile converter from the nrml_converters
Added source model information in the file realizations.csv
oq engine --run job.ini --exports csv now also exports the realizations
Introduced the format NRML 0.5 for source models
Added a check on the version in case of export errors
Extended oq purge to remove calculations from the database too
Fixed --make-html-report: the view task_info was not registered
Stored several strings as HDF5-variable-length strings
Fixed an export bug for the hazard curves in .geojson format
Removed the array cost_types from the datastore
Taxonomies with chars not in the range a-z0-9 were incorrectly rejected
Improved the XML parsing utilities in speed, memory, portability and
easy of use
Forbidden the reuse of exposure because is was fragile and error prone
Fixed a bug with the realizations array, which in hazard calculations
was empty in the datastore

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 8 years ago

OpenQuake - OpenQuake Engine 2.0.1

[Michele Simionato (@micheles)]

Fixed a bug for tectonic region types filtered away

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 8 years ago

OpenQuake - OpenQuake Engine 2.0

[Michele Simionato (@micheles)]

Quoted the taxonomies in the CSV exports
Fixed a bug in classical_damage and added a master test for it
Fixed the escaping of the taxonomies in the datastore
Fixed the names of the exported risk files
Fixed a segfault in the WebUI when exporting files with h5py >= 2.4
Added a command oq dbserver to start/stop the database server
The engine exports the hazard curves one file per IMT
Exported lon and lat with 5 digits after the decimal point
Added a command oq info --build-reports
Introduced experimental support for exporting .hdf5 files

[Daniele Viganò (@daniviga)]

Reworked substantially the engine documentation: removed obsolete pages,
updated to engine 2.0 and added instructions for Windows and Mac OS X
Remove oq_create_db script, db is created by the DbServer
Move oq_reset_db into utils and clean old code

[Michele Simionato (@micheles)]

Now the DbServer automatically upgrades the database if needed
Renamed oq-lite -> oq and added a subcommand oq engine
Added a CSV reader for the hazard curves
Having time_event=None in the hazard part of a calculation is now valid
Added an exporter for the rupture data, including the occurrence rate
Refactored the CSV exporters
Moved celeryconfig.py; now celery must be started with
celery worker --config openquake.engine.celeryconfig
Added a default location ~/oqdata/dbserver.log for the DbServer log
Added an early check on the SA periods supported by the GSIMs
Now the gsim_logic_tree file is parsed only once
Added a document about the architecture of the engine
The realizations are now exported as a CSV file
Escaped taxonomies in the datastore
The Web UI log tool is now escaping the HTML
Moved openquake.commonlib.commands -> openquake.commands and
openquake.commonlib.valid -> openquake.risklib.valid to have a
linear tower of internal dependencies
Supported all versions of Django >= 1.5
Provided a better error message in the absence of openquake.cfg
Removed the check on the export_dir when using the WebUI
Reduce the data transfer of the realization association object
If uniform_hazard_spectra is true, the UHS curves are generated
even if hazard_maps is false; the hazard maps are not exported
Optimized the filtering of PointSources
Initial work on the UCERF event based hazard calculator
Added a test calculation crossing the International Date Line (Alaska)

[Daniele Viganò (@daniviga)]

Remove the dependency from the python 'pwd' package which is not
available on Windows
Supervisord init scripts are now provided for the dbserver, celery
and the webui. Celery is not started by default, other two are.

[Michele Simionato (@micheles)]

Another export fix: made sure it is run by the current user
Fixed the export: if the export directory does not exists, it is created
Introduced the configuration variable multi_user, false for source
installations and true for package installations
Fixed the WebUI export
Removed the .txt outputs from the WebUI page engine/<output_id>/outputs
(they are useful only internally)
Fixed the export: first the xml exporter is tried and then the csv exporter;
if both are available, only the first is used, not both of them
Optimized the case when the epsilons are not required, i.e. all the
covariance coefficients are zero in the vulnerability functions
Added another test for event based risk (case_miriam)
Revisited the distribution mechanism and refined the weight of the
ruptures in the event based calculators to avoid generating slow tasks
Added an automatic help for the subcommands of oq-lite and managed
--version correctly
The event based risk calculator now use different seeds for different
realizations; also, the performance has been substantially improved
Improved the .rst reports with data transfer information
Removed the RlzsAssoc object from the datastore
Fixed the number of tasks generated by the risk calculators
Refactored the serialization of CompositionInfo instances to HDF5
Used exponential notation with 5 decimal digits in most exported XML files
Refactored the sampling mechanics in the event based calculators
The event_based_risk calculator infers the minimum intensity of the GMFs
from the vulnerability functions (if not specified in the job.ini)
Fixed the avg_losses-stats: they were not generated in absence of
loss curves
Added a command oq-lite info --exports
Added filtering on the mininum intensity also in the event based
hazard calculator; improved the performance and memory occupation
Added a view displaying the calculation times by source typology
Fixed the test of GMPETable after the correction in hazardlib
Optimized the saving of the asset loss table
Optimized the case of multiple assets of the same taxonomy on the
same point and introduced a datastore view assets_by_site
Fixed HDF5 segmentation faults in the tests for Ubuntu 16.04

[Daniele Viganò (@daniviga)]

Add support for Ubuntu 16.04 (xenial) packages
Removed the openquake_worker.cfg file because it is not used anymore

[Michele Simionato (@micheles)]

Replaced PostgreSQL with SQLite
Introduced a dbserver to mediate the interaction with the database
Restored the signal handler to manage properly kill signals so that
the workers are revoked when a process is killed manually
Fixed in a more robust way the duplicated log bug
Made more robust the killing of processes by patching concurrent.futures
Fixed a critical bug with celery not being used even when use_celery
was true.
Improved the validation of NRML files
Added a command oq-engine --show-log <job_id>

[Daniele Viganò (@daniviga)]

Use the 'postgresql' meta package as dependency of the .deb
package to support newer versions of Postgres; this makes
Trusty package installable on Ubuntu 16.04 and Debian 8

[Daniele Viganò (@daniviga), Michele Simionato (@micheles)]

Fixed a bug in oq-engine --export-outputs

[Daniele Viganò (@daniviga), Matteo Nastasi (@nastasi-oq)]

Allow installation of the binary package on Ubuntu derivatives

[Matteo Nastasi (@nastasi-oq)]

Backport of libhdf5 and h5py for ubuntu 'precise' serie

[Michele Simionato (@micheles)]

Removed openquake/engine/settings.py
Made the dependency on celery required only in cluster installations
Integrated the authentication database in the engine server database
Fixed the description in the Web UI (before it was temporarily set to
the string "A job").
Introduced filtering on the minimum intensity of the ground shaking
Solved the issue of serializing large SES collections, over the HDF5 limit
The loss maps and curves XML exporters now export the coordinates
of the assets, not the coordinates of the closest hazard site
Stored the job.ini parameters into a table in the datastore
Added a check on the IMTs coming from the risk models
Changed the aggregate loss table exporter to export the event tags,
not the event IDs
Fixed a bug with the CSV export of the ground motion fields
Fixed a bug with the export of UHS curves with --exports=xml
Reduced substantially the data transfer and the memory occupation
for event based calculations with a large number of assets: we
can run the California exposure with half million assets now
Fixed a bug in the SESCollection exporter
Changed the asset<->epsilons association: before for a given taxonomy the
assets were ordered by asset_ref, now they are ordered by id. This
has a minor impact on the numbers sensitive to the epsilons, akin to a
change of seeds
Added a test on the ordering of the epsilons
Accepted . and | as valid characters for source IDs
Changed the GMF calculator to use a single seed per unique rupture
Changed the SESCollection exporter: now a single file is exported, before
we were exporting one file per source model path per tectonic region model
Changed the event based calculators to avoid duplicating ruptures
occurring more than once
Changed the risk calculators to work in blocks of assets on the same site
Made it possible to set different integration distances for different
tectonic region types
Optimized the aggregation by asset in the event based risk calculator
Reporting the source_id when the filtering fails

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 9 years ago

OpenQuake - OpenQuake Engine 1.9.1

[Michele Simionato (@micheles)]

Fixed a bug in the Web UI when running a risk calculation starting
from a previous calculation

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 9 years ago

OpenQuake - OpenQuake Engine 1.9

[Michele Simionato (@micheles)]

Fixed a bug such that in some circumstances the logging stream handler
was instantiated twice, resulting in duplicated logs
Changed the default job status to 'executing' (was 'pre_executing')
Fixed the ordering of the logs in the Web UI
Removed the dependency from PostGIS
Restored the monitoring which was accidentally removed
Removed the obsolete option --hazard-output-id
Printed the names of the files exported by the engine, even when there
are multiple files for a single output
Introduced four new tables job, output, log, performance: all the other
60+ database tables are not used anymore

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 9 years ago

OpenQuake - OpenQuake Engine 1.8

[Michele Simionato (@micheles)]

Removed two oq-engine switches (--export-stats and --list-inputs)
and fixed --show-view; unified --delete-hazard-calculation and
--delete-risk-calculation into a single --delete-calculation
Updated make_html_report.py to extract the full report from the
datastore
If use_celery is true, use celery to determine a good default for
the parameter concurrent_tasks
Made celery required only in cluster situations
Fixed the duplication of exported result in the classical_damage
calculator when there is more than one realization
Removed several obsolete or deprecated switches from the oq-engine command
Replaced all classical calculators with their lite counterparts
Fixed the site-ordering in the UHS exporter (by lon-lat)

[Paolo Tormene (@ptormene)]

Added API to validate NRML

[Michele Simionato (@micheles)]

The engine can now zip files larger than 2 GB (used in the export)
Now the loss maps and curves are exported with a fixed ordering: first
by lon-lat, then by asset ID
Replaced the old disaggregation calculator with the oq-lite one

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 9 years ago

OpenQuake - OpenQuake Engine 1.7

[Michele Simionato (@micheles)]

Fixed an encoding bug in --lhc
Fixed an export bug: it is now possible to export the outputs generated
by another user, if the read permissions are set correctly

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 9 years ago

OpenQuake - OpenQuake Engine 1.6.0

[Daniele Viganò (@daniviga)]

Added the oq_reset_db script. It removes and recreates the database and
the datastore

[Matteo Nastasi (@nastasi-oq)]

Demos moved to /usr/share/openquake/risklib

[Michele Simionato (@micheles)]

Removed the 'view' button from the Web UI
Removed the epsilon_sampling configuration parameter
Made customizable the display_name of datastore outputs (before it was
identical to the datastore key)
The zip files generated for internal use of the Web UI are now hidden
Made visible to the engine only the exportable outputs of the datastore
Closed explicitly the datastore after each calculation
Replaced the old scenario calculators with the HDF5-based calculators
Fixed a very subtle bug in the association queries: some sites outside
of the region constraint were not discarded in some situations
Removed the self-termination feature terminate_job_when_celery_is_down
Removed the epsilon sampling "feature" from the scenario_risk calculator
Replaced the event based calculators based on Postgres with the new ones
based on the HDF5 technology

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 9 years ago

OpenQuake - OpenQuake Engine 1.5.1

[Michele Simionato (@micheles)]

Fixed a bug affecting exposures with multiple assets on the same site

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 9 years ago

OpenQuake - OpenQuake Engine 1.5.0

[Michele Simionato (@micheles)]

The event based calculators in the engine are now officially deprecated
and they raise a warning when used
Optimization: we do not generate the full epsilon matrix if all
coefficients of variation are zero
Fixed two subtle bugs in the management of the epsilons: it means that
all event based risk calculations with nonzero coefficients of variations
will produce slightly different numbers with respect to before
Removed excessive checking on the exposure attributes 'deductible' and
'insuredLimit' that made it impossible to run legitimate calculations
Changed the meaning of 'average_loss' for the aggregated curves: now it
is the sum of the aggregated losses in the event loss table,
before it was extracted from the aggregated loss curve
Changed the way the average losses (and insured average losses) are
computed by the event based risk calculator: now they are extracted
from the event loss table, before they were extracted from the loss curves
Set to NULL the stddev_losses and stddev_insured_losses for the event based
risk calculator, since they were computed incorrectly
Introduced a new experimental command
'oq-engine --show-view CALCULATION_ID VIEW_NAME'; the only view available
for the moment is 'mean_avg_losses'
Negative calculation IDs are interpreted in a Pythonic way, i.e. -1
means the last calculation, -2 the calculation before the last one, etc.
If a site parameter is more distant than 5 kilometers from its closest
site, a warning is logged
Changed the splitting of fault sources to reduce the number of generated
sources and avoid data transfer failures if rupture_mesh_spacing is too
small
Changed the event loss table export: now the CSV file does not contain
the magnitude and the rows are ordered by rupture tag first and loss second
Removed the calculator EventBasedBCR
Longitude and latitude are now rounded to 5 digits
Fixed a very subtle bug in the vulnerability functions, potentially
affecting calculations with nonzero coefficients of variation and nonzero
minIML; the numbers produced by the engine were incorrect; see
https://bugs.launchpad.net/oq-engine/+bug/1459926
'investigation_time' has been replaced by 'risk_investigation_time' in
risk configuration files
Initial support for Django 1.7

[Daniele Viganò (@daniviga)]

Removed the bin/openquake wrapper: now only bin/oq-engine is
available

[Michele Simionato (@micheles)]

Added parameter parallel_source_splitting in openquake.cfg

[Daniele Viganò (@daniviga)]

setup.py improvements
Added MANIFEST.in
celeryconfig.py moved from /usr/openquake/engine to
/usr/share/openquake/engine

[Matteo Nastasi (@nastasi-oq)]

Packaging system improvement

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq over 9 years ago

OpenQuake - OpenQuake Engine 1.4.0

[Daniele Viganò (@daniviga)]

Fixed debian/control: add missing lsb-release to build deps

[Matteo Nastasi (@nastasi-oq), Daniele Viganò (@daniviga)]

Fixed dependencies version management

[Matteo Nastasi (@nastasi-oq), Daniele Viganò (@daniviga)]

Add binary package support for both Ubuntu 12.04 (Precise)
and Ubuntu 14.04 (Trusty)

[Michele Simionato (@micheles)]

Removed the SiteModel table: now the association between the sites and the
site model is done by using hazardlib.geo.geodetic.min_distance

[Daniele Viganò (@daniviga)]

added authentication support to the 'engineweb' and the 'engineserver'

[Michele Simionato (@micheles)]

the aggregate loss curves can be exported in CSV format

[Matteo Nastasi (@nastasi-oq)]

added 'outtypes' attribute with list of possible output types for
each output item in outputs list API command
added '/v1/calc//status' API command
added 'engineweb' django application as local web client for oq-engine

[Michele Simionato (@micheles)]

Renamed the maximum_distance parameter of the risk calculators to
asset_hazard_distance, to avoid confusion with the maximum_distance
parameter of the hazard calculators, which has a different meaning;
is it an error to set the maximum_distance in a job_risk.ini file
Added to the API an URL /v1/calc/:calc_id/remove to hide jobs
A new key is_running is added to the list of dictionaries returned by
the URL /v1/calc/list
Replaced the mock tests for the engine server with real functional tests
Added a resource /v1/calc/:calc_id/traceback to get the traceback of a
failed calculation
Now the logs are stored also in the database, both for the controller node
and the worker nodes
Bypassed Django when deleting calculations from the database: this avoids
running out of memory for large calculations
Fixed an issue in the scenario calculator: the GMFs were not filtered
according to the distance to the rupture
Now critical errors appear in the log file
Added a --run command to run hazard and risk together
Fixed bug in the event based calculator; in the case
number_of_logic_tree_samples > 0 it was generating incorrect hazard curves.
Also improved (a lot) the performance in this case.
Fixed a tricky bug happening when some tectonic region type are filtered
away.
The event based risk calculator now save only the non-zero losses in
the table event_loss_asset.
Added a CSV exporter for the Stochastic Event Sets, for debugging purposes.
The GMF CSV exporter now sorts the output by rupture tag.

[Matteo Nastasi (@nastasi-oq)]

Each pull request must be accompanied by an update of the debian
changelog now.

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq almost 10 years ago

OpenQuake - OpenQuake Engine 1.3.0

[Matteo Nastasi (@nastasi-oq)]

gunzip xml demos files after copied into /usr/openquake/engine directory

[Michele Simionato (@micheles)]

Updated python-django dependency >= 1.6.1, (our repository already
includes a backported version for Ubuntu 'precise' 12.04); this change
makes unnecessary "standard_conforming_strings" postgresql configuration
variable setting
The event based risk calculator is able to disaggregate the event loss
table per asset. To enable this feature, just list the assets you are
interested in in the job.ini file: "specific_assets = a1 a2 a3"
We have a new hazard calculator, which can be invoked by setting in the
job.ini file: "calculation_mode = classical_tiling"
This calculators is the same as the classical calculator (i.e. you will
get the same numbers) but instead of considering all the hazard sites at
once, it splits them in tiles and compute the hazard curves for each tile
sequentially. The intended usage is for very large calculations that
exceed the available memory. It is especially convenient when you have
very large logic trees and you are interested only in the statistics (i.e.
mean curves and quantile curves). In that case you should use it with the
option individual_curves=false. Notice that this calculator is still in
an experimental stage and at the moment is does not support UHS curves.
Hazard maps and hazard curves are supported.
We have a new risk calculator, which can be invoked by setting in the
job.ini file: "calculation_mode = classical_damage"
This calculator is able to compute the damage distribution for each asset
starting from the hazard curves produced by the classical
(or classical_tiling) calculator and a set of fragility functions. Also
this calculator should be considered in experimental stage.
A significant change has been made when the parameter
number_of_logic_tree_samples is set to a non-zero value. Now, if a branch
of the source model logic tree is sampled twice we will generate the
ruptures twice; before the ruptures were generated once and counted twice.
For the classical calculator there is no effect on the numbers (sampling
the same branch twice will produce two copies of identical ruptures);
however, for the event based calculator, sampling the same branch twice
will produce different ruptures. For instance, in the case of a simple
source model with a single tectonic region type, before we would have
generated a single file with the stochastic event sets, now we generate
number_of_logic_tree_samples files with different stochastic event sets.
The previous behavior was an optimization-induced bug.
Better validation of the input files (fragility models, job.ini)
The ability to extract the sites from the site_model.xml file
Several missing QA tests have been added
The export mechanism has been enhanced and more outputs are being exported
in CSV format
New parameter complex_fault_mesh_spacing
Some error messages have been improved
A lot of functionality has been ported from the engine to oq-lite,
i.e. a lite version of the engine that does not depend on
PostgreSQL/PostGIS/Django nor from RabbitMQ/Celery. This version is
much easier to install than the regular engine and it is meant for
small/medium computation that do not require a cluster. The engine
demos, have been moved to the oq-risklib repository, so that they can
be run via the oq-lite command without installing the full engine.
Currently the following calculators have been ported (all are to be
intended as experimental): classical hazard, classical tiling, event
based hazard, scenario hazard, classical risk, scenario damage,
classical damage.

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 10 years ago

OpenQuake - OpenQuake Engine 1.3.1

[Michele Simionato (@micheles)]

Fixed a tricky bug happening when some tectonic region types are filtered
away.

Climate Change - Natural Hazard and Storm - Python
Published by nastasi-oq about 10 years ago

OpenQuake - OpenQuake Engine 1.2.2

The OpenQuake Engine release 1.2

See: https://github.com/gem/oq-engine/wiki#release-12-current-stable

Changes since 1.2.1

Refined version management

Climate Change - Natural Hazard and Storm - Python
Published by daniviga over 10 years ago

OpenQuake - OpenQuake Engine 1.0

The OpenQuake Engine release 1.0

Climate Change - Natural Hazard and Storm - Python
Published by daniviga over 10 years ago