Recent Releases of E3SM Diagnostics Package
E3SM Diagnostics Package - v3.0.0 Modernization with Xarray and xCDAT
What's Changed
v3.0.0 marks a major milestone after nearly two years of work by the core development team. This release introduces a completely new back-end, replacing CDAT with Xarray and xCDAT. Due to the significant scale of code changes, this has been incremented as a major release. The user-facing API for running E3SM Diagnostics remains backward-compatible between v2 and v3.
The modernization improves performance, usability, and maintainability, paving the way for future enhancements to E3SM development. The refactored codebase is now more robust and extensively covered by unit tests, setting a solid foundation for ongoing testing and development.
This release also includes several new enhancements including support for more EAMxx variables, standardization of regridding with xESMF's conservative_normed
method, and improving diagnostic run performance by skipping unnecessary regridding of datasets with the same lat-lon grid.
We are excited to share this release with the E3SM community and remain committed to continuously improving the software!
Enhancements
- Add support for more EAMxx variables by @chengzhuzhang, @tomvothecoder, and @czender in https://github.com/E3SM-Project/e3sm_diags/pull/880
- Standardize regridding to
conservative_normed
by @tomvothecoder and @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/940 - Improve diagnostic run performance by skipping regridding for two datasets on same lat-lon grids by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/959
- Revamp logging to improve readability and output E3SM diagnostic configuration, prevent duplicate log messages, and save complete console output to log file in
prov
directory by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/924 - Make viewer accessible at the top-level of results directory by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/927
Bug Fixes
- Fix prov files not printing with links in
prov
index.html by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/937
Refactor and testing
- Merge v3.0.0 development branch (
cdat-migration-fy24
) intomain
with the new Xarray/xCDAT backend by @tomvothecoder, @chengzhuzhang, @forsyth2, and @xylar in https://github.com/E3SM-Project/e3sm_diags/pull/902
DevOps
- Replace deprecated SciPy cwt with PyWavelets cwt by @tomvothecoder, @chengzhuzhang, and @whannah1 in https://github.com/E3SM-Project/e3sm_diags/pull/913
- Make
scipy
a base dependency inci.yml
anddev.yml
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/905 - Constrain
dask !=2024.12.0,!=2024.12.1
for performance by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/926 - Replace data downloads from LCRC webserver to avoid throttling by using data containers by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/901
- Adopt
ruff
as the central tool for linting and formatting by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/920
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.12.1...v3.0.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder about 1 month ago

E3SM Diagnostics Package - v3.0.0rc4
Refer to the v3.0.0 changelog for more information.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder about 2 months ago

E3SM Diagnostics Package - v3.0.0rc3
Refer to the v3.0.0 changelog for more information.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder 3 months ago

E3SM Diagnostics Package - v3.0.0rc2
Refer to the v3.0.0 changelog for more information.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder 3 months ago

E3SM Diagnostics Package - v3.0.0rc1
Refer to the v3.0.0 changelog for more information.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder 3 months ago

E3SM Diagnostics Package - v2.12.1 A Fond Farewell to CDAT
As we release version v2.12.1 of E3SM Diags, we would like to take a moment to acknowledge and express our gratitude to the developers and contributors from the CDAT (Climate Data Analysis Tools) community. Over the years, CDAT has been an invaluable tool for our development and analysis processes. This release marks a milestone, as it will be the final version of E3SM Diags that includes a dependency on CDAT.
Looking ahead, E3SM Diags will transition to utilizing xarray/xCDAT tools, which align with our future goals and enhances the performance and usability of our software. We are committed to ensuring a smooth transition for our users and will provide backward compatibility in upcoming releases.
User-facing changes
- add Qbo wavelet analysis by @justin-richling @whannah1 in https://github.com/E3SM-Project/e3sm_diags/pull/836
- add H2OLNZ in zonal mean 2d stratosphere by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/868
- make --closedcontourcmd in TC detection resolution-specific by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/824
- Fixing African easterly wave density plots in TC analysis by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/851
- Convert H2OLNZ units to ppm by volume by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/874
- add_TCO_60S60N to default lat-lon variables by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/879
Other Update
- Updates for tutorial2024 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/811
- Update tutorial by @forsyth2 in https://github.com/E3SM-Project/e3sm_diags/pull/812
- Fix a variable name displayed in viewer by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/820
- fix save_ncfiles for annual_cycle_zonal_mean by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/822
- Update cfg files by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/830
- Fix integration test download script by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/835
- Drop support for python 3.8 by @altheaden in https://github.com/E3SM-Project/e3sm_diags/pull/844
- Tweak behaviors of build and release CI workflows by @altheaden in https://github.com/E3SM-Project/e3sm_diags/pull/847
- Release 2.12.1 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/893
New Contributors
- @altheaden made their first contribution in https://github.com/E3SM-Project/e3sm_diags/pull/844
- @justin-richling made their first contribution in https://github.com/E3SM-Project/e3sm_diags/pull/836
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.12.0...v2.12.1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang 5 months ago

E3SM Diagnostics Package - v2.12.0
New Feature
- Add Wheeler and Kiladis diagram for tropical subseasonal diags - Wheeler and Kiladis by @jjbenedict @brianpm @chengzhuzhang @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/732
Updates and bug fixes
- Add intermediate output in json for ARM Diags by @yunpengshan2014 in https://github.com/E3SM-Project/e3sm_diags/pull/806
- Edit in-place modified derived variables for fixing aerosol budget table by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/805
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.11.0...v2.12.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder about 1 year ago

E3SM Diagnostics Package - v2.12.0rc1
New Feature
- Add Wheeler and Kiladis diagram for tropical subseasonal diags - Wheeler and Kiladis by @jjbenedict @brianpm @chengzhuzhang @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/732
Updates and bug fixes
- Add intermediate output in json for ARM Diags by @yunpengshan2014 in https://github.com/E3SM-Project/e3sm_diags/pull/806
- Edit in-place modified derived variables for fixing aerosol budget table by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/805
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.11.0...v2.12.0rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang about 1 year ago

E3SM Diagnostics Package - v2.11.0
What's Changed
New Features
- Initial support variable naming convention for EAMxx by @golaz in https://github.com/E3SM-Project/e3sm_diags/pull/712
- Support comparing monthly mean climatology (in addition to seasonal and annual mean) @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/712
Documentation Updates
- Update zppy cfg for perlmutter by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/783
- Update inter-comparison of CMIP models workflow by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/786
Bug fixes
- Fix
cosp_histogram_driver.py
not writing mean variables to netCDF by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/782
Others
- Bump/v2.11.0rc1 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/789
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.10.1...v2.11.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder about 1 year ago

E3SM Diagnostics Package - v2.11.0rc1
What's Changed
New Features
- Initial support variable naming convention for EAMxx by @golaz in https://github.com/E3SM-Project/e3sm_diags/pull/712
- Support comparing monthly mean climatology (in addition to seasonal and annual mean) @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/712
Documentation Updates
- Update zppy cfg for perlmutter by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/783
- Update inter-comparison of CMIP models workflow by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/786
Bug fixes
- Fix
cosp_histogram_driver.py
not writing mean variables to netCDF by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/782
Others
- Bump/v2.11.0rc1 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/789
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.10.1...v2.11.0rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang about 1 year ago

E3SM Diagnostics Package - v2.10.1
What's Changed
Bug Fixes
- Fix type conditional check and
UnboundLocalError
forparams_results
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/770 - Fix default diagnostics attrs not being copied to parent CoreParameter by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/778
Refactor
- [Refactor]: Refactor integration tests and add
use_cfg
flag torun_diags()
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/747 - Port over
cdp_viewer.OutputViewer
and removecdp
dependency by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/773
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.10.0...v2.10.1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder over 1 year ago

E3SM Diagnostics Package - v2.10.1rc1
What's Changed
- [Refactor]: Refactor integration tests and add
use_cfg
flag torun_diags()
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/747 - Fix type conditional check and
UnboundLocalError
forparams_results
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/770
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.10.0...v2.10.1rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder over 1 year ago

E3SM Diagnostics Package - v2.10.0
What's Changed
Features
- Add more aerosol metrics by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/763
Bug Fixes
- Fix typo in zonal mean figure titles by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/753
Refactor
- [Refactor]: Update
e3sm_diags_driver.run_diags
asCoreParameter
method by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/742 - Replace
parser.readfp()
withparser.read_file()
for Python 3.12 support by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/765
Documentation
- [Doc]: Add GitHub issues and pull request templates by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/741
DevOps
- Update DevOps configs by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/755
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.9.0...v2.10.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder over 1 year ago

E3SM Diagnostics Package - v2.9.0
New Features
- Add new aci and aerosol activation metrics from ARM Diags v3 by @chengzhuzhang (collaborating with ARM infrastructure group with contribution from Xiaojian Zheng @lannyzxj, @EmmyChengTao and Shaocheng Xie. )in https://github.com/E3SM-Project/e3sm_diags/pull/679
- Mixed phase partition LCF diagram by @chengzhuzhang with @zyuying, @yunpengshan2014 and @Jiwengithub in https://github.com/E3SM-Project/e3sm_diags/pull/675
- Add aerosol global metrics benchmarks file developed by E3SM Aerosol Working Group @susburrows and @kaizhangpnl , and implemented by @chengzhuzhang and @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/683
- Use total (interstitial and cloud-borne) aerosols directly from 2D burdens by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/682
- Add GPCP3.2 precipiation data and replace v2.2 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/690
- Add QREFHT and U10 comparison with ERA5 variables by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/688
- Support relative difference plot in zonal mean 2d set by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/651
- Additional variables to QBO json output files by @golaz in https://github.com/E3SM-Project/e3sm_diags/pull/646
CDAT Migration:
- CDAT Migration Phase 1: Replace
cdp.cdp_run
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/641 - CDAT Migration Phase 1 - Replace
CDPParameter
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/638 - CDAT Migration Phase 1 - Replace
cdp_parser.CDPParser
by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/650
Other Changes:
- Update Acknowledgement and Features Contribution in README by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/692
- Bump dependencies in conda env yml files by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/644
- Increment CACHE_NUMBER to fix conda envs by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/645
- fix typo in aerosol budget printout by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/676
- Update quick guide for perlmutter; update diagnostics paths from
mache
by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/691 - Fix
numpy.int
-->int
by @xylar in https://github.com/E3SM-Project/e3sm_diags/pull/694 - Change workflow to mamba by @forsyth2 in https://github.com/E3SM-Project/e3sm_diags/pull/693
- Simplify the conda env yml files by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/696
- Fix release workflow not copying docs for latest tags by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/698
- Fix dir ref in release workflow by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/699
- Fix segfault issue with newest shapely and esmpy by @xylar in https://github.com/E3SM-Project/e3sm_diags/pull/715
- Import shapely but not esmpy in main package by @xylar in https://github.com/E3SM-Project/e3sm_diags/pull/722
- update enviroment variable by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/725
- Add e3sm_diags import by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/727
New Contributors
- @mahf708 made their first contribution in https://github.com/E3SM-Project/e3sm_diags/pull/676
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.8.0...v2.9.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 1 year ago

E3SM Diagnostics Package - v2.9.0rc5
What's Changed
- update enviroment variable by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/725
- Add e3sm_diags import by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/727
- Bump to 2.9.0rc5 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/730
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.9.0rc4...v2.9.0rc5
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 1 year ago

E3SM Diagnostics Package - v2.9.0rc4
What's Changed
- Import shapely but not esmpy in main package by @xylar in https://github.com/E3SM-Project/e3sm_diags/pull/722
- Bump to 2.9.0rc4 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/724
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.9.0rc3...v2.9.0rc4
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 1 year ago

E3SM Diagnostics Package - v2.9.0rc3
What's Changed
- Fix segfault issue with newest shapely and esmpy by @xylar in https://github.com/E3SM-Project/e3sm_diags/pull/715
- Bump to 2.9.0rc3 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/717
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.9.0rc2...v2.9.0rc3
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 1 year ago

E3SM Diagnostics Package - v2.9.0rc2
What's Changed
- Fix dir ref in release workflow by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/699
- Use total (interstitial and cloud-borne) aerosols directly from 2D burdens by @mahf708 in https://github.com/E3SM-Project/e3sm_diags/pull/682
- Bump to v2.9.0rc2 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/710
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.9.0rc1...v2.9.0rc2
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 2 years ago

E3SM Diagnostics Package - v2.9.0rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder almost 2 years ago

E3SM Diagnostics Package - v2.8.0
New Features
- Add MERRA2 aerosol data for AOD diagnostics by @chengzhuzhang (with help from @yfenganl and @mingxuanwupnnl )in https://github.com/E3SM-Project/e3sm_diags/pull/624
- Add Aerosol Budget Tables (set name =
aerosol_budget
) by @chengzhuzhang @tomvothecoder (with help from @kaizhangpnl, @susburrows, @TaufiqHassan, @xuezhengllnl, @keziming )in https://github.com/E3SM-Project/e3sm_diags/pull/625 - add example running with zppy @chengzhuzhang https://github.com/E3SM-Project/e3sm_diags/pull/622
Other Changes
- Fix matplotlib rotation bug by @tomvothecoder https://github.com/E3SM-Project/e3sm_diags/pull/622
- Update GitHub Actions for changing to
main
branch by @forsyth2 @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/619 and https://github.com/E3SM-Project/e3sm_diags/pull/621 - Updated examples by @forsyth2 in https://github.com/E3SM-Project/e3sm_diags/pull/618
- Remove
cdtime
as a dependency by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/627
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.7.0...v2.8.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder over 2 years ago

E3SM Diagnostics Package - v2.8.0rc1 Aerosols
New Features
- Add MERRA2 aerosol data for AOD diagnostics by @chengzhuzhang (with help from @yfenganl and @mingxuanwupnnl )in https://github.com/E3SM-Project/e3sm_diags/pull/624
- Add Aerosol Budget Tables by @chengzhuzhang @tomvothecoder (with help from @kaizhangpnl, @susburrows, @TaufiqHassan, @xuezhengllnl, @keziming )in https://github.com/E3SM-Project/e3sm_diags/pull/625
- add example running with zppy @chengzhuzhang https://github.com/E3SM-Project/e3sm_diags/pull/622
Other Changes
- Fix matplotlib rotation bug by @tomvothecoder https://github.com/E3SM-Project/e3sm_diags/pull/622
- Update GitHub Actions for changing to
main
branch by @forsyth2 @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/619 and https://github.com/E3SM-Project/e3sm_diags/pull/621 - Updated examples by @forsyth2 in https://github.com/E3SM-Project/e3sm_diags/pull/618
- Remove
cdtime
as a dependency by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/627
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.7.0...v2.8.0rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 2 years ago

E3SM Diagnostics Package - v2.7.0
New Features:
- Added scatter plots for comparing annual mean long-term Absorption Optical Depth (AOD) Aerosol Absorption Optical Depth (AAOD) at 550 nm from AERONET measurements. Scripts and data contributed by @yfenganl
- Add cmip6 comparison box whisker plot. Scripts contributed by @golaz
- Add CMIP6 models + E3SM v1 v2, AMIP and HISTORICAL runs as baseline for box whisker plots and Taylor Diagrams
- Support Model only analysis in lat-lon set
- Add lat_lon_land annd lat_lon_river drivers and viewers supported in the main viewer.
- Integrate in ELM land variable pre-processing and MOSART river variables for lat-lon model vs model default run. (contributed by @darincomeau @jenniferholm and @hydrotian )
- Add a standalone script to generate a summary plot of E3SM diagnostics. by @ambrad
- Included a Metrics checker to check consistency between two lat-lon run.
Enhanced Features:
- Add AODDUST to lat_lon plots (model only)
- Add Q to zonal_mean_2d plots
- Add TREFMNAV/tasmin TREFMXAV/tasmax to core set
- Replace ERA Interim with ERA5 in core sets
- Update to MACv2 for AOD at 550nm diags
Major Fixes:
- Set regrid method to bilinear for a subset of variables with masks or missing data
- Fix and Enhancement for Diurnal Cycle of Precipitation
- Fix extreme values of Albedo from obs values over high latitudes by masking high latitude in lat-lon plot
Other Updates:
- Update DevOps tooling; Update
black
to fixclick
ImportError
- Add
xarray
andmache
to conda env dependencies and removedask
- Update input data path; nco invocation in pre-processing
- Fix
FileNotFoundError
for log file
Contributors
- @chengzhuzhang @tomvothecoder @golaz @forsyth2 @ambrad @yfenganl @darincomeau @jenniferholm and @hydrotian
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.6.1...v2.7.0
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 3 years ago

E3SM Diagnostics Package - v2.7.0rc2
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 3 years ago

E3SM Diagnostics Package - v2.7.0rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 3 years ago

E3SM Diagnostics Package - v2.6.1
What's Changed
- Optimize nco invocation by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/540
- Document change from acme_diags by @forsyth2 in https://github.com/E3SM-Project/e3sm_diags/pull/541
- Fix missing “CLDLOW_TAU*_MISR in lat-lon by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/542
- Replace print statements with logger and execute integration tests explicitly over implicitly by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/544
- Drop container/Docker support and update README badges and docs by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/547
- Remove xerr=0 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/553
- Bump to 2.6.1rc1 by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/554
- Add e3sm_diags for CMIP example workflow by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/555
- enable short test name to be printed for TC and ARM figures by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/558
- Bump to 2.6.1 by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/559
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.6.0...v2.6.1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder about 3 years ago

E3SM Diagnostics Package - v2.6.1rc1
What's Changed
- Optimize nco invocation by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/540
- Document change from acme_diags by @forsyth2 in https://github.com/E3SM-Project/e3sm_diags/pull/541
- Fix missing “CLDLOW_TAU*_MISR in lat-lon by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/542
- Replace print statements with logger and execute integration tests explicitly over implicitly by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/544
- Drop container/Docker support and update README badges and docs by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/547
- Remove xerr=0 by @chengzhuzhang in https://github.com/E3SM-Project/e3sm_diags/pull/553
- Bump to 2.6.1rc1 by @tomvothecoder in https://github.com/E3SM-Project/e3sm_diags/pull/554
Full Changelog: https://github.com/E3SM-Project/e3sm_diags/compare/v2.6.0...v2.6.1rc1
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder over 3 years ago

E3SM Diagnostics Package - v2.6.0
From v2.6.0, Module name acme_diags is renamed to e3sm_diags, users should update their Python run scripts to use import e3sm_diags
instead.
Feature
- Add pressure-latitude set focusing on Stratosphere #535
- Refine plevs to include stratosphere #525
- Plot improvements for TC #521
- Use cartopy_offlinedata #524
Bug Fixes
- Fix post-processing induced warm bias over South Polar area #523
- Fix flipped lat of ozone obs #491
- Modify LH formula for NET_FLUX_SRF #519
- Update remaining acme_diags/ refs to e3sm_diags/ #530
- Add xml support for get_static_var #516
Cleanup
- Rename dir acme_diags to e3sm_diags #495
- Consolidate pre-processing scripts #514
Docs
- Update load e3sm unified paths #489
- Update README #507
- Update dev install docs #508
- Update unified paths #510
CI/CD
- Enable ability to manually update docs on
gh-pages
#497 - Update CI/CD workflows to only regenerate master docs #498
- Update publish-docs job in release_workflow.yml #500
- Add tbump to simplify package versioning #494
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 3 years ago

E3SM Diagnostics Package - v2.6.0rc1
Bug Fixes
- Fix flipped lat of ozone obs #491
- Modify LH formula for NET_FLUX_SRF #519
- Update remaining acme_diags/ refs to e3sm_diags/ #530
Features
- Add xml support for get_static_var #516
- Refine plevs to include stratosphere #525
- Plot improvements for TC #521
- Use cartopy_offlinedata #524
Docs
- Update load e3sm unified paths #489
- Update README #507
- Update dev install docs #508
- Update unified paths #510
CI/CD
- Enable ability to manually update docs on
gh-pages
#497 - Update CI/CD workflows to only regenerate master docs #498
- Update publish-docs job in release_workflow.yml #500
- Add tbump to simplify package versioning #494
Cleanup
- Rename dir acme_diags to e3sm_diags #495
- Consolidate pre-processing scripts #514
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by tomvothecoder over 3 years ago

E3SM Diagnostics Package - TC Analysis and Column Ozone
New features include:
- TC analysis based on: Balaguru at al. 2020: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019MS002024
- Zonal Mean Annual Cycle contour plots (including Stratospheric and Tropospheric column Ozone)
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 4 years ago

E3SM Diagnostics Package - v2.5.0rc4
New features include:
- TC analysis based on: Balaguru at al. 2020: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019MS002024
- Zonal Mean Annual Cycle contour plots
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by forsyth2 almost 4 years ago

E3SM Diagnostics Package - v2.5.0rc3
New features include:
- TC analysis based on: Balaguru at al. 2020: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019MS002024
- Zonal Mean Annual Cycle contour plots
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 4 years ago

E3SM Diagnostics Package - v2.5.0rc2
New features include:
- TC analysis based on: Balaguru at al. 2020: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019MS002024
- Zonal Mean Annual Cycle contour plots
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by forsyth2 almost 4 years ago

E3SM Diagnostics Package - TC Analysis and Zonal Mean Annual Cycle
New features include:
- TC analysis based on:Balaguru at al. 2020: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019MS002024
- Zonal Mean Annual Cycle contour plots
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 4 years ago

E3SM Diagnostics Package - ARM Diags
This is a major release.
New Features:
-
Incorporation of ARM-diags which uses in-situ measurements from DOE's ARM (Atmospheric Radiation Measurement) Program to compare with point grid model output. Diagnostics added include: seasonal mean, annual cycle cycle of a set of atmospheric variables, including (surface temperature, precipitation, radiation fluxes, surface turbulent fluxes...); diurnal cycle of precipitation and cloud; convection onset metrics.
-
MERRA 2 datasets are updated to include monthly main daily maximum and minimum surface air temperature.
Other major enhancement
- Add a test suite for E3SM-Diags on Chrysalis/Anvil
- Image comparison tests added in unit tests and a complete run
- Integrate GitHub Actions CI/CD workflow
- Add and apply formatters and linters to codebase
Minor enhancement
- Code repo clean up by moving test data to E3SM data server and providing data download script.
- Simplify Conda environments
Documentation
- Include quick guide for Chrysalis/Anvil
- Update post-processing and run scripts examples
- Update development guide
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang about 4 years ago

E3SM Diagnostics Package - v2.3.0
- New set: streamflow diags including seasonality maps, annual mean maps and scatter plots based on gauge station data from GSIM
- New set: diurnal cycle of precipitation diagnostics based on 3hourly TRMM data.
- Updated documentation and bug fixes
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 4 years ago

E3SM Diagnostics Package - v2.2.0
- Add 2020 tutorial examples
- Add ERA5 and 1 deg GPCP data
- Removed vtk as a dependency
- Fixes for plotting issues 330 326
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 4 years ago

E3SM Diagnostics Package - v2.1.1
A minor release to address an issue likely stems from cartopy0.18
#318
As a workaround, polar plots are now in square shape.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 5 years ago

E3SM Diagnostics Package - v2.1.0
New Features:
-
Two components of ENSO-diags:
maps of regression coefficient of atmospheric fields over SST anomaly;
scatter plots of atmospheric feedback on SST anomaly. -
Quasi-biennial Oscillation (QBO) analysis
-
Initial support for lat-lon map for Regional Refined Model (RRM) data.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 5 years ago

E3SM Diagnostics Package - v2.0.0
Some major changes in this new release:
- Refactored the code structure to make the code base more flexible to incorporate new diagnostics sets.
- Introduced a new way of running: run with API (i.e. python run_e3sm_diags.py). The old way (e3sm_diags -p) is backward compatible for the core diagnostics sets.
- Added time-series diagnostics for annual mean trend for selected variables (can be ran only with the new way)
- Variables P-E from (GPCP and OAflux), and from Core v2 fluxes are added.
- Dataset class is refactored to be cleaner and more flexible.
- Documentation is updated to reflect the new changes.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang over 5 years ago

E3SM Diagnostics Package - v1.7.1
- Add OMEGA to lat lon diagnostics set
- Fix Dockerfile
- Other bug fixes
We will have a big release for a new way of running the package, including adding set-specific parameters (i.g., plevs
for the zonal_mean_2d
diags set).
Please stay tuned!
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 6 years ago

E3SM Diagnostics Package - v1.7.0
- Python 3 is now supported in lieu of Python 2.
- Pressure-Longitude meridional mean contour plots added.
- Added quick guide for using Docker.
- Added quick guide for ANL ALCF Cooley using Singularity.
- Added support for Singularity.
- Other small issues fixed.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang almost 6 years ago

E3SM Diagnostics Package - v1.6.1
- No longer pinning specific versions of CDAT.
- So using
>=
instead of=
.
- So using
- Builds are now
noarch
.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 6 years ago

E3SM Diagnostics Package - v1.6.0
- In addition to climatology files, e3sm_diags now supports time series input (monthly data of multiple years saved in one file per each variable) to have climatology computed during runtime.
- Users are able to specify the time slices for model vs model and model vs obs diagnostics runs.
- An example is here, with the diags results here.
- Both NCO-generated time series files and CMIP-compliant time series files can be used as input files.
- For examples with CMIP data, see here for a model vs obs comparison or here for a model vs model comparison.
- Users are able to specify the time slices for model vs model and model vs obs diagnostics runs.
- New quick start guide for NERSC Cori.
- Major code refactor for reading in data and getting variables.
- Scripts added for creating the time series and climatology files for the observational data.
- User-facing instances of "acme" are replaced with "e3sm".
- Please don't change the project name again.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 6 years ago

E3SM Diagnostics Package - v1.5.0
e3sm_diags
can now be ran as a container, in addition to being manually installed and ran that way.- View how to do so on Edison at NERSC via Shifter here.
- The auxiliary script to run a container,
e3sm_diags_container.py
, supports Docker and Shifter. - Singularity support will be added in the next release or so.
- A quick guide will be added for Docker and Singularity soon as well.
- Scripts added for creating the time-series observational data.
- Minor additions to the viewer.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 6 years ago

E3SM Diagnostics Package - v1.4.0
- Added
selectors
parameter.- View the documentation here.
- Major bug-fix regarding issue #156, where too many threads were used.
- Added scripts to create climo and timeseries data for the observational data. Located in the
analysis_data_preprocess
folder.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 6 years ago

E3SM Diagnostics Package - v.1.3.4
- Fixed major issue where NetCDF-4 files would not open on Edison compute nodes.
- Added
output_format_subplot
parameter to save individual panels when using matplotlib for the plotsets:- lat_lon
- polar
- zonal_mean_xy
- zonal_mean_2d
- cosp_histogram
- Expanded provenance. Now for each run, the command used, the
py
file, and thecfg
file used are save in{results_dir}/prov
. Ayml
file of the environment is also saved there as well.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 6 years ago

E3SM Diagnostics Package - v1.3.3
- Finally renamed package to
e3sm_diags
.- But
acme_diags
is still supported for those who update and don't read the changelogs.
- But
- Removed
mesalib
as a dependency. - Other very minor fixes.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 7 years ago

E3SM Diagnostics Package - v1.3.2
- Python 2 and 3 support.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 7 years ago

E3SM Diagnostics Package - v1.3.1
- CDAT 8 support.
- Use
conservative
regridding by default. - Expanded provenance:
- Dump of which dependencies are used, stored in
results_dir/env_prov.txt
. - All parameters work via the command line.
--no_viewer
parameter to not create the viewer.
- Dump of which dependencies are used, stored in
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 7 years ago

E3SM Diagnostics Package - v1.3.0
- Stronger support for model vs model run with a new parameter
run_type
- Cleaner code directory with all default configuration files converted to
.cfg
- Add AOD550, surface wind stress and surface air temperature as default variables
- Several bug fixes
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 7 years ago

E3SM Diagnostics Package - v1.2.1
- The provenance works better. The command should only create one plot.
- Running with the mpl backend on a headless machine should work.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen about 7 years ago

E3SM Diagnostics Package - v1.2.0
- Added Taylor Diagrams for spatial variability for annual and seasonal means.
- Added provenance: a single run can be recreated with a single command.
- A dependency,
cdp
, was updated to support the provenance feature.- If using a development environment, explained here, create a new environment by following the instructions again.
- If using a regular environment, just run
conda update acme_diags
to getv1.2.0
ofacme_diags
.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by chengzhuzhang about 7 years ago

E3SM Diagnostics Package - v1.1.1
- Implemented a workaround for multiprocess running with VCS as the backend
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 7 years ago

E3SM Diagnostics Package - v1.1.0
- Added table to summarize metrics for each season, as part of the lat-lon diagnostics set
- Added new variables from observational and reanalysis datasets, including:
- NetCF (CERES-EBAF TOA, surface)
- SHFLX, netSW surface, netLW surface, net flux sfc (ERA-Interim)
- LHFLX, SHFLX, netSW surface, netLW surface, net flux sfc (MERRA)
- Sea level pressure
- Improved subtitles for lat-lon maps to reflect the years the climatology was averaged over
- Added a "short_test_name" parameter for user-defined model names
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 7 years ago

E3SM Diagnostics Package - v1.0.1
- When a variable isn't present or there's an error in the code for a given diagnostics run, that run is skipped and the other diagnostics runs are ran.
- Expanded on the zonal mean line plots, latitude-longitude contour maps, and polar contour maps default diagnostics.
- Added system tests in
tests/all_sets.cfg
, useacme_diags_driver.py -d all_sets.cfg
to generate one of each plot type in under 1 minute to test that an environment is working correctly.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 7 years ago

E3SM Diagnostics Package - v1.0.0 Documentation
- View the documentation for acme_diags v1.0.0 here.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 7 years ago

E3SM Diagnostics Package - v1.0.0
- Support for diagnostics based on seasonal or annual climatology data, including:
- Latitude-Longitude contour maps (AMWG set 5)
- Polar contour maps (AMWG set 7)
- Zonal mean line plots (AMWG set 3)
- Pressure-Latitude zonal mean contour plots (AMWG set 4)
- CloudTopHeight-Tau joint histograms (AMWG set 13).
- Diagnostics for model vs obs, obs vs obs, and model vs model.
- Updated observational datasets available on LLNL ACME1/AIMS4 and NERSC.
- Two graphical backends: VCS and Matplotlib with cartopy.
- User–addable diagnostics during runtime.
- Diagnostics can be run in serial, or in parallel with multiprocessing or distributedly.
- Documentation including quick start guides for LLNL ACME1/AIMS4 and NERSC, detailed user’s guide, and examples.
Climate Change - Earth and Climate Modeling
- Jupyter Notebook
Published by zshaheen over 7 years ago
