perun

Calculates the energy consumption of Python scripts by sampling usage statistics from your hardware components.
https://github.com/helmholtz-ai-energy/perun

Category: Consumption
Sub Category: Computation and Communication

Keywords

benchmarking command-line-tool energy energy-monitor hpc mpi python

Keywords from Contributors

parallelism transformers profiling archived observability polars measur routing feature-engine battery

Last synced: about 4 hours ago
JSON representation

Repository metadata

Perun is a Python package that measures the energy consumption of you applications.

README.rst

          .. image:: https://raw.githubusercontent.com/Helmholtz-AI-Energy/perun/main/docs/images/full_logo.svg

| |fair-software| |openssf| |zenodo| |license| |docs|
| |pypi-version| |python-version| |pypi-downloads| |black| |codecov|

.. |fair-software| image:: https://img.shields.io/badge/fair--software.eu-%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F-green
   :target: https://fair-software.eu
.. |openssf| image:: https://bestpractices.coreinfrastructure.org/projects/7253/badge
   :target: https://bestpractices.coreinfrastructure.org/projects/7253
.. |zenodo| image:: https://zenodo.org/badge/523363424.svg
   :target: https://zenodo.org/badge/latestdoi/523363424
.. |pypi-version| image:: https://img.shields.io/pypi/v/perun
.. |pypi-downloads| image:: https://img.shields.io/pypi/dm/perun
.. |black| image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :target: https://github.com/psf/black
.. |codecov| image:: https://codecov.io/gh/Helmholtz-AI-Energy/perun/graph/badge.svg?token=9O6FSJ6I3G
   :target: https://codecov.io/gh/Helmholtz-AI-Energy/perun
.. |python-version| image:: https://img.shields.io/badge/Python-3.9+-blue.svg
   :target: https://www.python.org/downloads/
.. |license| image:: https://img.shields.io/badge/License-BSD_3--Clause-blue.svg
   :target: https://opensource.org/licenses/BSD-3-Clause
.. |docs| image:: https://readthedocs.org/projects/perun/badge/?version=latest
   :target: https://perun.readthedocs.io/en/latest/?badge=latest

===============================================================

*perun* is a Python package that calculates the energy consumption of Python scripts by sampling usage statistics from your Intel, Nvidia or AMD hardware components. It can handle MPI applications, gather data from hundreds of nodes, and accumulate it efficiently. *perun* can be used as a command-line tool or as a function decorator in Python scripts.

Check out the `docs `_ or a working `example `_!

Key Features
------------

- Measures energy consumption of Python scripts and binaries, supporting different hardware configurations
- Capable of handling MPI applications, gathering data from hundreds of nodes efficiently
- Monitor individual functions using decorators
- Tracks energy usage of the application over multiple executions
- Easy to benchmark applications and functions
- Experimental!: Can monitor any non-distributed command line application

-----------------------------------------------------------------


Quick Start
-----------

Installation
^^^^^^^^^^^^

From PyPI:

.. code:: console

    $ pip install perun

Extra dependencies like *nvidia-smi*, *rocm-smi* and *mpi4py* can be installed using pip as well:

.. code:: console

    $ pip install perun[nvidia, rocm, mpi]

From Github:

.. code:: console

    $ pip install git+https://github.com/Helmholtz-AI-Energy/perun

Command Line
^^^^^^^^^^^^

To use *perun* as a command-line tool:

.. code:: console

    $ perun monitor path/to/your/script.py [args]

*perun* will output two files, an HDF5 style containing all the raw data that was gathered, and a text file with a summary of the results.

.. code:: text

    PERUN REPORT

    App name: finetune_qa_accelerate
    First run: 2023-08-15T18:56:11.202060
    Last run: 2023-08-17T13:29:29.969779

    RUN ID: 2023-08-17T13:29:29.969779

    +-----------+------------------------+-----------+-------------+--------------+-------------+-------------+-------------+---------------+-------------+
    | Round #   | Host                   | RUNTIME   | ENERGY      | CPU_POWER    | CPU_UTIL    | GPU_POWER   | GPU_MEM     | DRAM_POWER    | MEM_UTIL    |
    +===========+========================+===========+=============+==============+=============+=============+=============+===============+=============+
    | 0         | hkn0432.localdomain    | 995.967 s | 960.506 kJ  | 231.819 W    | 3.240 %     | 702.327 W   | 55.258 GB   | 29.315 W      | 0.062 %     |
    | 0         | hkn0436.localdomain    | 994.847 s | 960.469 kJ  | 235.162 W    | 3.239 %     | 701.588 W   | 56.934 GB   | 27.830 W      | 0.061 %     |
    | 0         | All                    | 995.967 s | 1.921 MJ    | 466.981 W    | 3.240 %     | 1.404 kW    | 112.192 GB  | 57.145 W      | 0.061 %     |

    The application has been run 7 times. In total, it has used 3.128 kWh, released a total of 1.307 kgCO2e into the atmosphere, and you paid 1.02 € in electricity for it.


Binary support (experimental)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

*perun* is capable of monitoring simple applications written in other languages:

.. code:: console

    $ perun monitor --binary path/to/your/executable [args]

Function Monitoring
^^^^^^^^^^^^^^^^^^^

Using a function decorator

.. code:: python

    import time
    from perun import monitor

    @monitor()
    def main(n: int):
        time.sleep(n)

After running with ``perun monitor``, the report will contain:

.. code:: text

    Monitored Functions

    +-----------+----------------------------+---------------------+------------------+--------------------+------------------+-----------------------+
    | Round #   | Function                   | Avg Calls / Rank    | Avg Runtime      | Avg Power          | Avg CPU Util     | Avg GPU Mem Util      |
    +===========+============================+=====================+==================+====================+==================+=======================+
    | 0         | main                       | 1                   | 993.323±0.587 s  | 964.732±0.499 W    | 3.244±0.003 %    | 35.091±0.526 %        |
    | 0         | prepare_train_features     | 88                  | 0.383±0.048 s    | 262.305±19.251 W   | 4.541±0.320 %    | 3.937±0.013 %         |
    | 0         | prepare_validation_features| 11                  | 0.372±0.079 s    | 272.161±19.404 W   | 4.524±0.225 %    | 4.490±0.907 %         |


MPI
^^^

*perun* is compatible with MPI applications using ``mpi4py``:

.. code:: console

    $ mpirun -n 8 perun monitor path/to/your/script.py

Docs
----

See the `documentation `_ or `examples `_ for more details.

Citing perun
------------

If you found *perun* useful, please cite the conference paper:

::

    Gutiérrez Hermosillo Muriedas, J.P., Flügel, K., Debus, C., Obermaier, H., Streit, A., Götz, M.:
    perun: Benchmarking Energy Consumption of High-Performance Computing Applications.
    In: Cano, J., Dikaiakos, M.D., Papadopoulos, G.A., Pericàs, M., and Sakellariou, R. (eds.)
    Euro-Par 2023: Parallel Processing. pp. 17–31. Springer Nature Switzerland, Cham (2023).
    https://doi.org/10.1007/978-3-031-39698-4_2

.. code-block:: bibtex

   @InProceedings{10.1007/978-3-031-39698-4_2,
     author="Guti{\'e}rrez Hermosillo Muriedas, Juan Pedro
     and Fl{\"u}gel, Katharina
     and Debus, Charlotte
     and Obermaier, Holger
     and Streit, Achim
     and G{\"o}tz, Markus",
     editor="Cano, Jos{\'e}
     and Dikaiakos, Marios D.
     and Papadopoulos, George A.
     and Peric{\`a}s, Miquel
     and Sakellariou, Rizos",
     title="perun: Benchmarking Energy Consumption of High-Performance Computing Applications",
     booktitle="Euro-Par 2023: Parallel Processing",
     year="2023",
     publisher="Springer Nature Switzerland",
     address="Cham",
     pages="17--31",
     isbn="978-3-031-39698-4"
   }

        

Citation (CITATION.cff)

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: perun
message: 'If you use this software, please cite the paper.'
type: software
authors:
  - given-names: Juan Pedro
    family-names: Gutiérrez Hermosillo Muriedas
    email: [email protected]
    affiliation: >-
      Scientific Computing Center, Karlsruhe Institute of Technology
    orcid: 'https://orcid.org/0000-0001-8439-7145'
repository-code: 'https://github.com/Helmholtz-AI-Energy/perun'
repository: 'https://perun.readthedocs.io/en/latest/?badge=latest'
keywords:
  - Python
  - Energy
  - Benchmarking
  - HPC
  - MPI
license: BSD-3-Clause
preferred-citation:
  type: conference-paper
  authors:
    - given-names: Juan Pedro
      family-names: Gutiérrez Hermosillo Muriedas
      email: [email protected]
      affiliation: Karlsruhe Institute of Technology
      orcid: 'https://orcid.org/0000-0001-8439-7145'
    - given-names: Katharina
      family-names: Flügel
    - given-names: Charlotte
      family-names: Debus
    - given-names: Holger
      family-names: Obermaier
    - given-names: Achim
      family-names: Streit
    - given-names: Markus
      family-names: Götz
  title: >-
    perun: Benchmarking Energy Consumption of High-Performance Computing
    Applications
  year: 2023
  collection-title: 'Euro-Par 2023: Parallel Processing'
  collection-doi: 10.1007/978-3-031-39698-4
  doi: 10.1007/978-3-031-39698-4_2
  conference:
    name: >-
      29th International European Conference on Parallel and Distributed
      Computing
    date-start: "2023-08-28"
    date-end: "2017-09-01"

Owner metadata


GitHub Events

Total
Last Year

Committers metadata

Last synced: about 8 hours ago

Total Commits: 203
Total Committers: 9
Avg Commits per committer: 22.556
Development Distribution Score (DDS): 0.296

Commits in past year: 44
Committers in past year: 5
Avg Commits per committer in past year: 8.8
Development Distribution Score (DDS) in past year: 0.364

Name Email Commits
Gutiérrez Hermosillo Muriedas, Juan Pedro j****m@g****m 143
github-actions a****n@g****m 16
pre-commit-ci[bot] 6****] 14
github-actions g****s@g****m 11
semantic-release s****e 10
dependabot[bot] 4****] 6
Markus Goetz m****n@g****m 1
Andreas Fehlner f****r@a****e 1
[email protected] i****7@u****n 1

Committer domains:


Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 76
Total pull requests: 113
Average time to close issues: 4 months
Average time to close pull requests: 11 days
Total issue authors: 10
Total pull request authors: 4
Average comments per issue: 0.45
Average comments per pull request: 0.26
Merged pull request: 103
Bot issues: 0
Bot pull requests: 45

Past year issues: 26
Past year pull requests: 30
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 11 days
Past year issue authors: 5
Past year pull request authors: 4
Past year average comments per issue: 0.54
Past year average comments per pull request: 0.97
Past year merged pull request: 26
Past year bot issues: 0
Past year bot pull requests: 12

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/helmholtz-ai-energy/perun

Top Issue Authors

  • JuanPedroGHM (63)
  • Markus-Goetz (4)
  • yasintha91 (2)
  • aweissen1 (1)
  • diligent-man (1)
  • bom-bahadur (1)
  • janEbert (1)
  • psteinb (1)
  • TomTheBear (1)
  • pfackeldey (1)

Top Pull Request Authors

  • JuanPedroGHM (67)
  • pre-commit-ci[bot] (29)
  • dependabot[bot] (16)
  • andife (1)

Top Issue Labels

  • enhancement (14)
  • cx (8)
  • bug (7)
  • documentation (5)
  • dependencies (4)
  • Frontend:IO (3)
  • Data Model (2)
  • good first issue (1)
  • Processing (1)

Top Pull Request Labels

  • dependencies (16)
  • documentation (1)
  • cx (1)

Package metadata

proxy.golang.org: github.com/helmholtz-ai-energy/perun

proxy.golang.org: github.com/Helmholtz-AI-Energy/perun

pypi.org: perun

Measure the energy used by your MPI+Python applications.

  • Homepage: https://github.com/Helmholtz-AI-Energy/perun
  • Documentation: https://perun.readthedocs.io
  • Licenses: BSD 3-Clause License Copyright (c) 2022, Helmholtz AI Energy All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  • Latest release: 0.9.0 (published 19 days ago)
  • Last Synced: 2025-06-07T10:07:00.661Z (1 day ago)
  • Versions: 55
  • Dependent Packages: 1
  • Dependent Repositories: 1
  • Downloads: 408 Last month
  • Rankings:
    • Dependent packages count: 4.744%
    • Stargazers count: 11.253%
    • Average: 14.963%
    • Downloads: 18.072%
    • Forks count: 19.104%
    • Dependent repos count: 21.642%
  • Maintainers (1)

Dependencies

pyproject.toml pypi
  • black ^22.6.0 develop
  • flake8 ^5.0.4 develop
  • mypy ^0.971 develop
  • pre-commit ^2.20.0 develop
  • pytest ^5.2 develop
  • PyYAML ^6.0
  • click ^8.1.3
  • h5py ^3.7.0
  • mpi4py ^3.1.3
  • py-cpuinfo ^8.0.0
  • pyRAPL ^0.2.3
  • pynvml ^11.4.1
  • python ^3.9
  • python-dotenv ^0.20.0
.github/workflows/semantic_release.yml actions
  • actions/checkout v2 composite
  • relekang/python-semantic-release master composite
.github/workflows/run_tests.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
docs/requirements.txt pypi
  • sphinx-autoapi ==2.1.0
  • sphinx-rtd-theme ==1.2.0
examples/torch_mnist/requirements.txt pypi
  • Jinja2 ==3.1.2
  • MarkupSafe ==2.1.3
  • Pillow ==10.0.0
  • certifi ==2023.5.7
  • charset-normalizer ==3.1.0
  • click ==8.1.3
  • cmake ==3.26.4
  • filelock ==3.12.2
  • h5py ==3.9.0
  • idna ==3.4
  • lit ==16.0.6
  • mpmath ==1.3.0
  • networkx ==3.1
  • numpy ==1.24.4
  • nvidia-cublas-cu11 ==11.10.3.66
  • nvidia-cuda-cupti-cu11 ==11.7.101
  • nvidia-cuda-nvrtc-cu11 ==11.7.99
  • nvidia-cuda-runtime-cu11 ==11.7.99
  • nvidia-cudnn-cu11 ==8.5.0.96
  • nvidia-cufft-cu11 ==10.9.0.58
  • nvidia-curand-cu11 ==10.2.10.91
  • nvidia-cusolver-cu11 ==11.4.0.1
  • nvidia-cusparse-cu11 ==11.7.4.91
  • nvidia-nccl-cu11 ==2.14.3
  • nvidia-nvtx-cu11 ==11.7.91
  • pandas ==2.0.3
  • perun ==0.4.0
  • psutil ==5.9.5
  • py-cpuinfo ==5.0.0
  • pynvml ==11.5.0
  • python-dateutil ==2.8.2
  • pytz ==2023.3
  • requests ==2.31.0
  • six ==1.16.0
  • sympy ==1.12
  • torch ==2.0.1
  • torchvision ==0.15.2
  • triton ==2.0.0
  • typing_extensions ==4.7.1
  • tzdata ==2023.3
  • urllib3 ==2.0.3

Score: 12.67016508808596