PCMDI Metrics Package
Open-source Python package for Systematic Evaluation of Climate and Earth System Models.
https://github.com/pcmdi/pcmdi_metrics
Category: Climate Change
Sub Category: Earth and Climate Modeling
Keywords
climate climate-analysis climate-data climate-model climate-model-evaluation climate-models climate-science climate-variability python
Keywords from Contributors
vtk cdat earthscience e3sm cmip6 esgf cmor3 prepare snl-applications earth-system-model
Last synced: about 22 hours ago
JSON representation
Repository metadata
Open-source Python package for Collective and Systematic Evaluation of Climate and Earth System Models
- Host: GitHub
- URL: https://github.com/pcmdi/pcmdi_metrics
- Owner: PCMDI
- License: bsd-3-clause
- Created: 2013-05-28T20:48:38.000Z (over 12 years ago)
- Default Branch: main
- Last Pushed: 2025-10-30T00:07:45.000Z (about 2 months ago)
- Last Synced: 2025-10-30T04:34:56.053Z (about 2 months ago)
- Topics: climate, climate-analysis, climate-data, climate-model, climate-model-evaluation, climate-models, climate-science, climate-variability, python
- Language: Python
- Homepage: http://pcmdi.github.io/pcmdi_metrics/
- Size: 245 MB
- Stars: 120
- Watchers: 9
- Forks: 48
- Open Issues: 43
- Releases: 52
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.rst
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Support: docs/supporting-data.rst
- Notice: NOTICE
README.md
PCMDI Metrics Package (PMP)
Conda-forge (CURRENT, recommended):
PCMDI Conda Channel (old, deprecated):
The PCMDI Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle, tropical and extra-tropical modes of variability, ENSO, MJO, regional monsoons, high frequency characteristics of simulated precipitation, and cloud feedback.
PCMDI uses the PMP to produce quick-look simulation summaries across generations of CMIP.
The metrics package consists of the following parts:
- Analysis software
- Observation-based reference database of global (or near global, land or ocean) time series and climatologies
- Package documentation and interactive jupyter notebook demos
- Database of performance metrics computed for CMIP models
The package expects model data to be CF-compliant. To successfully use the package some input data "conditioning" may be required. We provide several demo scripts within the package.
Documentation
Getting Started
-
Installation requirements and instructions are available on the Install page
-
Users will need to contact the PMP developers (pcmdi-metrics@llnl.gov) to obtain supporting datasets and get started using the package.
-
An overview for using the package and template scripts are detailed on the Using-the-package page
References
Latest:
- Lee, J., Gleckler, P. J., Ahn, M.-S., Ordonez, A., Ullrich, P. A., Sperber, K. R., Taylor, K. E., Planton, Y. Y., Guilyardi, E., Durack, P., Bonfils, C., Zelinka, M. D., Chao, L.-W., Dong, B., Doutriaux, C., Zhang, C., Vo, T., Boutte, J., Wehner, M. F., Pendergrass, A. G., Kim, D., Xue, Z., Wittenberg, A. T., and Krasting, J.: Systematic and objective evaluation of Earth system models: PCMDI Metrics Package (PMP) version 3, Geosci. Model Dev., 17, 3919–3948, https://doi.org/10.5194/gmd-17-3919-2024, 2024.
Earlier versions:
-
Gleckler, P. J., Doutriaux, C., Durack, P. J., Taylor, K. E., Zhang, Y., Williams, D. N., Mason, E., and Servonnat, J.: A more powerful reality test for climate models, Eos T. Am. Geophys. Un., 97, https://doi.org/10.1029/2016eo051663, 2016.
-
Gleckler, P. J., Taylor, K. E., and Doutriaux, C.: Performance metrics for climate models, J. Geophys. Res., 113, D06104, https://doi.org/10.1029/2007jd008972, 2008.
Contact
Some installation support for CMIP participating modeling groups is available: pcmdi-metrics@llnl.gov
Acknowledgement
Content in this repository is developed by climate and computer scientists from the Program for Climate Model Diagnosis and Intercomparison (PCMDI) at Lawrence Livermore National Laboratory (LLNL). This work is sponsored by the Regional and Global Model Analysis (RGMA) program, of the Earth and Environmental Systems Sciences Division (EESSD) in the Office of Biological and Environmental Research (BER) within the Department of Energy's Office of Science. The work is performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
LLNL-CODE-2004137
DOE CODE ID: #153383
License
Distributed under the BSD 3-Clause License. See LICENSE for more information.
Release Notes and History
| Versions | Update summary |
|---|---|
| v4.0.1 | Technical update |
| v4.0.0 | Newerly support higher Python versions, >= 3.10, < 3.14 |
| Versions | Update summary |
|---|---|
| v4.0.0rc1 | Pre-release of development version: Newerly support higher Python versions, >= 3.10, < 3.14 |
| ------------- | ------------------------------------- |
| v3.9.3 | Technical update |
| v3.9.2 | Technical update |
| v3.9.1 | New capability (new modes for modes of variability metrics: EA, SCA) and technical update |
| v3.9 | New capability (Decision-Relevant metrics, Database access API) and new demo notebooks |
| v3.8.2 | Technical update |
| v3.8.1 | Technical update with new figure (modes of variability multi-panel plot) |
| v3.8 | New capability (figure generation for ENSO, xCDAT migration completed for Monsoon Wang with figure generation), major dependency update (numpy >= 2.0) |
| v3.7.2 | Technical update |
| v3.7.1 | Technical update with documentation improvements |
| v3.7 | New capability (figure generation for mean climate) and technical update |
| v3.6.1 | Technical update, additional QC repair functions |
| v3.6 | New capability (regional application of precip variability) and technical update |
| v3.5.2 | New capability (QC, new modes for modes of variability metrics: PSA1, PSA2) and technical update |
| v3.5.1 | Technical update |
| v3.5 | Technical update: MJO and Monsoon Sperber xCDAT conversion |
| v3.4.1 | Technical update |
| v3.4 | Technical update: Modes of variability xCDAT conversion |
| v3.3.4 | Technical update |
| v3.3.3 | Technical update |
| v3.3.2 | Technical update |
| v3.3.1 | Technical update |
| v3.3 | New metric added: Sea-Ice |
| v3.2 | New metric added: Extremes |
| v3.1.2 | Technical update |
| v3.1.1 | Technical and documentation update |
| v3.1 | New metric added: Precipitation Benchmarking -- distribution bimodality |
| v3.0.2 | Minor patch and more documentation added |
| v3.0.1 | Minor technical patch |
| v3.0.0 | New metric added: Cloud feedback metric by @mzelinka. xCDAT implemented for mean climate metrics |
| ------------- | ------------------------------------- |
| v2.5.1 | Technical update |
| v2.5.0 | New metric added: Precipitation Benchmarking -- distribution. Graphics updated |
| v2.4.0 | New metric added: AMO in variability modes |
| v2.3.2 | CMEC interface updates |
| v2.3.1 | Technical update |
| v2.3 | New graphics using archived PMP results |
| v2.2.2 | Technical update |
| v2.2.1 | Minor update |
| v2.2 | New metric implemented: precipitation variability across time scale |
| v2.1.2 | Minor update |
| v2.1.1 | Simplified dependent libraries and CI process |
| v2.1.0 | CMEC driver interfaced added. |
| v2.0 | New capabilities: ENSO metrics, demos, and documentations. |
| ------------- | ------------------------------------- |
| v1.2 | Tied to CDAT 8.0. Extensive regression testing added. New metrics: Diurnal cycle and intermittency of precipitation, sample monsoon metrics. |
| v1.1.2 | Now managed through Anaconda, and tied to UV-CDAT 2.10. Weights on bias statistic added. Extensive provenance information incorporated into json files. |
| v1.1 | First public release, emphasizing climatological statistics, with development branches for ENSO and regional monsoon precipitation indices |
| v1.0 | Prototype version of the PMP |
Current Core Development Team
- Jiwoo Lee (LLNL, PMP Lead)
- Kristin Chang (LLNL)
- Peter Gleckler (LLNL)
- Paul Ullrich (LLNL, PCMDI Project PI)
- Shixuan Zhang (PNNL)
All Contributors
Thanks goes to these wonderful people (emoji key):
This project follows the all-contributors specification.
Citation (CITATION.cff)
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!
cff-version: 1.2.0
title: PCMDI Metrics Package
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Jiwoo
family-names: Lee
email: lee1043@llnl.gov
affiliation: Lawrence Livermore National Laboratory
orcid: 'https://orcid.org/0000-0002-0016-7199'
- affiliation: Lawrence Livermore National Laboratory
given-names: Kristin
family-names: Chang
- family-names: Gleckler
given-names: Peter
affiliation: Lawrence Livermore National Laboratory
orcid: 'https://orcid.org/0000-0003-2816-6224'
- given-names: Paul
family-names: Ullrich
affiliation: Lawrence Livermore National Laboratory and UC Davis
orcid: 'https://orcid.org/0000-0003-4118-4590'
identifiers:
- type: doi
value: 10.5281/zenodo.592790
description: Software
repository-code: 'https://github.com/PCMDI/pcmdi_metrics'
url: 'http://pcmdi.github.io/pcmdi_metrics/'
abstract: ' Open-source Python package for Systematic Evaluation of Climate and Earth System Models '
keywords:
- climate
- climate modeling
- climate model evaluation
- Earth System Model evaluation
- climatology
- climate variability
license: BSD-3-Clause
version: '4.0.1'
date-released: '2025-10-13'
Owner metadata
- Name: Program for Climate Model Diagnosis and Intercomparison
- Login: PCMDI
- Email: pcmdi-cmip@llnl.gov
- Kind: organization
- Description: Established in 1989, PCMDI aims to develop advanced methods and tools for diagnosis and intercomparison to improve Earth System and Climate Models
- Website: http://pcmdi.llnl.gov
- Location: Lawrence Livermore National Lab
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/6098305?v=4
- Repositories: 45
- Last ynced at: 2024-03-26T10:12:39.209Z
- Profile URL: https://github.com/PCMDI
GitHub Events
Total
- Create event: 116
- Commit comment event: 2
- Release event: 10
- Delete event: 107
- Member event: 2
- Pull request event: 217
- Fork event: 8
- Issues event: 88
- Watch event: 17
- Issue comment event: 250
- Push event: 719
- Gollum event: 7
- Pull request review comment event: 19
- Pull request review event: 55
Last Year
- Create event: 111
- Commit comment event: 2
- Release event: 10
- Delete event: 102
- Member event: 2
- Pull request event: 206
- Fork event: 7
- Issues event: 83
- Watch event: 15
- Issue comment event: 230
- Push event: 667
- Gollum event: 7
- Pull request review comment event: 19
- Pull request review event: 53
Committers metadata
Last synced: about 2 months ago
Total Commits: 4,124
Total Committers: 42
Avg Commits per committer: 98.19
Development Distribution Score (DDS): 0.522
Commits in past year: 496
Committers in past year: 11
Avg Commits per committer in past year: 45.091
Development Distribution Score (DDS) in past year: 0.234
| Name | Commits | |
|---|---|---|
| Jiwoo Lee | l****3@l****v | 1972 |
| Charles Doutriaux | d****1@l****v | 576 |
| Ana Ordonez | o****4@l****v | 381 |
| Ana Ordonez | a****z@c****t | 368 |
| Peter Gleckler | g****1@l****v | 177 |
| Paul J. Durack | d****1@l****v | 172 |
| zshaheen | z****3@g****m | 112 |
| Min-Seop Ahn | a****6@l****v | 63 |
| Kristin Chang | c****1@l****v | 35 |
| Peter Gleckler | g****1@g****v | 31 |
| muryanto1 | m****1@l****v | 25 |
| Bo Dong postdoc | d****n@h****m | 24 |
| Tom Vo | t****r@g****m | 22 |
| Min-Seop Ahn | 4****n | 20 |
| Jason Boutte | b****n@g****m | 20 |
| Xylar Asay-Davis | x****m@g****m | 19 |
| Muryanto | m****1@m****v | 14 |
| Muryanto | m****1@M****v | 12 |
| ShixuanZhang | s****g@p****v | 10 |
| allcontributors[bot] | 4****] | 10 |
| Peter Gleckler | g****1@c****v | 10 |
| mzelinka | z****1@l****v | 7 |
| omahs | 7****s | 7 |
| Jeff Painter | p****1@l****v | 6 |
| Peter Gleckler | g****1@o****v | 5 |
| Jeff Painter | p****1@o****v | 4 |
| Paul Durack | p****d@p****) | 3 |
| jeffrey Painter | p****1@p****v | 3 |
| mfwehner | m****r@l****v | 2 |
| Lina Muryanto | 3****1 | 2 |
| and 12 more... | ||
Committer domains:
- llnl.gov: 10
- oceanonly.llnl.gov: 4
- comcast.net: 1
- gates.llnl.gov: 1
- md-9776485.llnl.gov: 1
- ml-9516678.llnl.gov: 1
- pnnl.gov: 1
- crunchy.llnl.gov: 1
- public1.(none): 1
- pcmdi11.llnl.gov: 1
- lbl.gov: 1
- noaa.gov: 1
- login20.chn.perlmutter.nersc.gov: 1
- durack1m1.llnl.gov: 1
- durack1ml.llnl.gov: 1
- quartz380.llnl.gov: 1
- ciclad-ng.private.ipsl.fr: 1
- loftair-108.lan: 1
- loftair-85.llnl.gov: 1
- jared.kiwi.nz: 1
- lanl.gov: 1
Issue and Pull Request metadata
Last synced: about 2 months ago
Total issues: 148
Total pull requests: 547
Average time to close issues: 10 months
Average time to close pull requests: 20 days
Total issue authors: 18
Total pull request authors: 18
Average comments per issue: 1.54
Average comments per pull request: 1.62
Merged pull request: 465
Bot issues: 0
Bot pull requests: 8
Past year issues: 43
Past year pull requests: 209
Past year average time to close issues: 11 days
Past year average time to close pull requests: 3 days
Past year issue authors: 12
Past year pull request authors: 8
Past year average comments per issue: 0.67
Past year average comments per pull request: 0.8
Past year merged pull request: 168
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- lee1043 (86)
- gleckler1 (14)
- acordonez (11)
- ShihengDuan (6)
- msahn (6)
- durack1 (5)
- zhangshixuan1987 (4)
- kristinchang3 (4)
- doutriaux1 (2)
- bosup (2)
- xylar (1)
- rbeucher (1)
- tomvothecoder (1)
- dnadeau4 (1)
- andrewdnolan (1)
Top Pull Request Authors
- lee1043 (387)
- acordonez (88)
- durack1 (10)
- zhangshixuan1987 (10)
- kristinchang3 (9)
- allcontributors[bot] (8)
- msahn (7)
- gleckler1 (6)
- xylar (5)
- jasonb5 (4)
- bosup (4)
- mzelinka (2)
- ajonko (2)
- omahs (1)
- tomvothecoder (1)
Top Issue Labels
- enhancement (24)
- New feature (15)
- bug (14)
- doc (10)
- graphic (4)
- feature request (3)
- demo (2)
- build (1)
- duplicate (1)
- QC (1)
- clean up (1)
Top Pull Request Labels
- doc (96)
- enhancement (79)
- bug (42)
- demo (39)
- minor (34)
- New feature (27)
- graphic (23)
- clean up (15)
- build (14)
- feature request (10)
- workflow (6)
- CRITICAL (5)
- QC (3)
- TOPPRIORITY (2)
- DecisionNeededNow (2)
Package metadata
- Total packages: 3
- Total downloads: unknown
- Total dependent packages: 0 (may contain duplicates)
- Total dependent repositories: 0 (may contain duplicates)
- Total versions: 79
proxy.golang.org: github.com/pcmdi/pcmdi_metrics
- Homepage:
- Documentation: https://pkg.go.dev/github.com/pcmdi/pcmdi_metrics#section-documentation
- Licenses: bsd-3-clause
- Latest release: v4.0.1+incompatible (published 2 months ago)
- Last Synced: 2025-10-29T20:11:48.592Z (about 2 months ago)
- Versions: 36
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent packages count: 5.395%
- Average: 5.576%
- Dependent repos count: 5.758%
proxy.golang.org: github.com/PCMDI/pcmdi_metrics
- Homepage:
- Documentation: https://pkg.go.dev/github.com/PCMDI/pcmdi_metrics#section-documentation
- Licenses: bsd-3-clause
- Latest release: v4.0.1+incompatible (published 2 months ago)
- Last Synced: 2025-10-29T20:11:48.598Z (about 2 months ago)
- Versions: 36
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent packages count: 5.395%
- Average: 5.576%
- Dependent repos count: 5.758%
conda-forge.org: pcmdi_metrics
model metrics tools
- Homepage: http://github.com/PCMDI/pcmdi_metrics
- Licenses: BSD-3-Clause
- Latest release: 2.5.1 (published about 3 years ago)
- Last Synced: 2025-10-01T07:56:35.779Z (3 months ago)
- Versions: 7
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Forks count: 28.545%
- Dependent repos count: 34.025%
- Stargazers count: 35.643%
- Average: 37.347%
- Dependent packages count: 51.175%
Dependencies
- actions/cache v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- conda-incubator/setup-miniconda v2 composite
- fkirc/skip-duplicate-actions master composite
- pre-commit/action v2.0.3 composite
Score: -Infinity