Releases | Open Sustainable Technology

RT1 - RT1 v2.0

new

HDF5 containers can now be used instead of pickle-dumps to store Fits and MultiFits objects
RTprocess is now directly appending results to an HDF5-container during parallel processing
colored log-messages can be used (requires colorama package)
there's now a proper container for defdict to simplify get/set of variable definitions
indicate fit-results in fit.plot.analyzemodel

changes

processfunc is now private (e.g. _processfunc) -> use run_processing instead!

fixes

several bugfixes along the way

Atmosphere - Radiative Transfer - Python
Published by raphaelquast over 1 year ago

RT1 - RT1 v1.5.2

new

there's now a quick and easy data-export function RTprocess.export_data()
- extract parameters and performance-metrics from a folder of dump-files
there's now an RTresults.dump_fits - generator that directly returns Fits-objects
RTprocess object can now be initialized without a config-file
option to pass a list of dump-files to rtprocess.run_finalout() added

fixes

speedup for indexing of folders with large amounts of dump-files
some minor logging updates

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 4 years ago

RT1 - RT1 v1.5.1

new

add possibility to provide numeric V & SRF functions for faster evaluation if no interaction-term is calculated
there is now a property fit.config_name that indicates the config-name for fits that belong to a MultiFits object
datasets provided to rtfits.Fits are now checked for sorting (and automatically sorted if necessary)

fixes

make ncoefs a propery for "Rayleigh" and "Isotropic" phase-functions
improved logging messages
- don't log each successful progress step, only errors and warnings
- don't create a single file for each error (log traceback to log-file directly)
keep minimal info from fit.fit_output when dumping fits to disc
improvements on caching and pre-computation steps
- pre-evaluate fn-coefficients during finalout-generation
- cache lambda-functions for V.p and SRF.brdf
fix sorting of fractional contributions in rtplots.printsig0analysis()
some updates on rtplots.polarplot() for symbolic parameter specification
make sure copying of config-files and modules is only attempted from the main process

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 4 years ago

RT1 - RT1 v1.5

new

add possibility to provide numeric V & SRF functions for faster evaluation if no interaction-term is calculated
there is now a property fit.config_name that indicates the config-name for fits that belong to a MultiFits object
datasets provided to rtfits.Fits are now checked for sorting (and automatically sorted if necessary)

fixes

make ncoefs a propery for "Rayleigh" and "Isotropic" phase-functions
improved logging messages
- don't log each successful progress step, only errors and warnings
- don't create a single file for each error (log traceback to log-file directly)
keep minimal info from fit.fit_output when dumping fits to disc
improvements on caching and pre-computation steps
- pre-evaluate fn-coefficients during finalout-generation
- cache lambda-functions for V.p and SRF.brdf
fix sorting of fractional contributions in rtplots.printsig0analysis()
some updates on rtplots.polarplot() for symbolic parameter specification
make sure copying of config-files and modules is only attempted from the main process

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 4 years ago

RT1 - RT1 v1.4

new

add possibility to call finalizer after rt1.rtprocess.run_processing() finished
- to "clean-up" after processing is done (or the process is terminated)
add option to save dumps in rtprocess.run_processing() even if postprocess is not called via
rt1.rtprocess.run_processing(..., dump_fit=True)

Multiconfig Fits

add possibility to specify multiple configs in a single .ini file
- processing of multiple configs involves only a single call to read()!
- you can specify as many configs as you like in a single .ini file!
- to specify multiconfigs, add the config-specific changes to sections starting with [#config <NAME> <SECTION>]
  for example:

[defdict]
--- the default defdict ---

[#config NAME defdict]
--- all specifications that differ from the default defdict ---

Multiconfig fits will be handled with a rt1.rtfits.MultiFits object
- use rt1.rtfits.MultiFits.accessor to access Fits propeties of all configs
all processed configurations can be saved in a single .dump-file!
- this avoids duplicating data that is shared between the configs

fixes

fix issue in the default "domainfuncs" of rt1.general_functions.scale()
make sure rt1.rtfits.reinit_object() only copies relevant properties
speed up rtprocess.RTresults initialization for folders with many files

Atmosphere - Radiative Transfer - Python
Published by raphaelquast about 4 years ago

RT1 - v1.3

new

logging implemented
incorporate basic NetCDF export functionality via xarray (rt1.processing_config.postprocess_xarray())
new rtmetrics class to provide easy access to common metrics calculation
add option to disable postprocessing va RTprocess.run_processing(postprocess=False)
- add option to run processing and NetCDF export in 2 separate runs
  - e.g. via run_processing() and run_finalout()
- add dump_fit option to run_processing() so fits are dumped even if postprocess=False

changes

copy is now an argument of the __init__ of RTprocess instead of RTprocess.run_processing()

unittests

switch tests to github actions
- use conda .yml file for requirements
switch coverage reports to codecov
reactivate stickler & flake8
use github actions for PyPi upload

fixes

fix processing and logging with ncpu=1
enure that the processing_config class and additional modules are loaded from the correct folder
fix future warnings in sympy and matplotlib
make RTresults.load_fit() find fits faster
- allow passing integers to load the nth result
fix sorting in printsig0analysis()
fix parsing error in param_dyn_dict
use pickleable decoration of initializer function
proper detection of submodules in setup.py
incorporate pending changes in scipy > v1.6.3 (e.g. adjust call signature of sympy.Lambdify)

Atmosphere - Radiative Transfer - Python
Published by raphaelquast about 4 years ago

RT1 - v1.2

changes

the reader-function in rt1.rtprocess is now called without dict-unpacking!!
(e.g. via reader(reader_args) instead of reader(**reader_args))
! this might require modifications in existing scripts !

fixes

correct parsing of None in rt1.rtparse
multiprocessing pool initializer also executed in single core processing (for debug)
rt1.rtprocess now uses warnings.warn instead of print for general messages

new

convenience-method interpolate_to_index() added to rt1.genreal_functions

Atmosphere - Radiative Transfer - Python
Published by raphaelquast over 4 years ago

RT1 - v1.1.9

changes

rtfits.preprocess() is now called prior to parallel-processing
- a check if files already exist should now be done within the reader-function!
copying of modules is shifted from the rtparse to rtprocess to avoid copying files when config is read
the configparser now has methods get_module('modulename') as well as get_all_modules() to programmatically import modules
all parallel-processing functions have been shifted to the dedicated rtprocess module

fixes

corrections to the treatment of integer parameter-dynamics specified in fit.defdict
the cache is now only cleared at the beginning of a fit.performfit() call
correct final message in processfunc() progress-bar

new

a dedicated class for parallel processing (rt1.rtprocess) has been introduced
- rt1.rtprocess.RTprocess can be used to setup and run parallel jobs
- rt1.rtprocess.RTresults can be used to provide easy-access to processing-results
some updates on rtplots.printsig0analysis()
- now one can select additional timeseries from both fit.res_df and fit.dataset
- ... and the styles can be set using the printparamstyles - dict

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 5 years ago

RT1 - v1.1.8

changes

manual parameter dynamics are no longer automatically used if a column 'param_dyn' is provided
... to use manual dynamics, you now must use one of the following:
- explicitly specify parameter-dynamics as 'manual' in fit.defdict
- to combine a pandas datetime-offset freq with a manual-dyn use 'freq + manual'

new

the model_definition text has been updated to include more details
the property fit._model_definition is added to allow accessing the model_definition string
(useful for example to write it to a file)
RT1_configparser now provides a function .get_module(modulename) to import a specific module from a file

fixes

fix import-location for copied modules in rtparse (now modules are imported from the copy-path)

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 5 years ago

RT1 - v1.1.7

new

define parameter-grouping with respect to number of observations within a group

fixes

fix dynamic startvalue-assignment
better implementation of rectangularize function

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 5 years ago

RT1 - v1.1.6

added possibility to provide arbitrary start-values via the dataset
- dataset -columns "parameter_start" are now used to assign start-values for the parameters
there is now a load() function in rt1.rtfits to load fit-configurations dumped via fit.dump()

fixes

cache mask during fit
interpolation now works also with unsorted groupings
-> now it's possible to fit interpolated seasonalities etc.
... some speed-ups

Atmosphere - Radiative Transfer - Python
Published by raphaelquast almost 5 years ago

RT1 - v1.1.4

add possibility for parsing (dtype-converted) lists in rtparse.get_process_specs()