Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Kepler

Uses eBPF to probe energy related system stats and exports as Prometheus metrics.
https://github.com/sustainable-computing-io/kepler

Category: Consumption
Sub Category: Computation and Communication

Keywords

cloud-native ebpf energy-consumption energy-efficiency energy-monitor kubernetes machine-learning prometheus prometheus-exporter sustainability

Keywords from Contributors

measur operator transforms online-machine-learning model-server alerting stream-processing tensorflow2 observation tensorflow-io

Last synced: about 6 hours ago
JSON representation

Repository metadata

Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe performance counters and other system stats, use ML models to estimate workload energy consumption based on these stats, and exports them as Prometheus metrics

README.md

GitHub Workflow Status (event)

codecov
OpenSSF Best PracticesOpenSSF Scorecard

License License
License

Twitter URL

Kepler

Kepler (Kubernetes Efficient Power Level Exporter) uses eBPF to probe
energy-related system stats and exports them as Prometheus metrics.

As a CNCF Sandbox project, Kepler uses
CNCF Code of Conduct

Architecture

Kepler Exporter exposes a variety of
metrics about the energy
consumption of Kubernetes components such as Pods and Nodes.

flowchart BT
    classDef kernel fill:#e6f3ff,stroke:#4a90e2,color:#000
    classDef collector fill:#f0fff0,stroke:#2ecc71,color:#000
    classDef hardware fill:#fff0f5,stroke:#e74c3c,color:#000
    classDef estimator fill:#fff5e6,stroke:#f39c12,color:#000
    classDef mapping fill:#f5f0ff,stroke:#9b59b6,color:#000
    classDef calculator fill:#f0f5ff,stroke:#3498db,color:#000
    classDef attribution fill:#fff0f0,stroke:#e74c3c,color:#000
    classDef export fill:#f5fff0,stroke:#27ae60,color:#000

    classDef kernelLevel fill:#e6f3ff,stroke:#999,color:#000
    classDef userSpace fill:#f5f5f5,stroke:#999,color:#000
    classDef resourceCollection fill:#f0fff0,stroke:#999,color:#000
    classDef hardwareMetrics fill:#fff0f5,stroke:#999,color:#000
    classDef estimatorMetrics fill:#fff5e6,stroke:#999,color:#000
    classDef powerModel fill:#f0f0ff,stroke:#999,color:#000

    subgraph KL[Kernel Level]
        direction BT
        TP[Kernel Tracepoint]:::kernel --> EBPF[Kepler eBPF Program]:::kernel
        EBPF --> |Performance Counter Stats|OM[Output Map]:::kernel
    end

    subgraph UP[Userspace Program]
        direction BT
        subgraph RC[Resource Info Collection]
            direction BT
            P1[Process Info Collector]:::collector --> |PID, Names|INFO[Process/Container/VM Info]:::collector
            C1[Container Info Collector]:::collector --> |Container/Pod ID, Namespace|INFO
            V1[VM Info Collector]:::collector --> |VM ID|INFO
        end

        subgraph HM[Hardware Metrics]
            direction BT
            H1[RAPL or hwmon]:::hardware --> |CPU/DRAM/Package Power|PWR[Hardware Power Readings]:::hardware
            H2[NVIDIA/Intel GPU API]:::hardware --> |GPU Power|PWR
            H3[Redfish or ACPI Power Meter]:::hardware --> |Platform Power|PWR
        end

        subgraph EM[Estimator Metrics]
            direction BT
            E1[ML Features: CPU Time]:::estimator --> |CPU/DRAM/Package Power|PWR
            E2[ML Features: CPU Time]:::estimator --> |Platform Power|PWR
        end

        OM --> |Read Map Data|MAP[Activity Mapping]:::mapping
        INFO --> MAP
        MAP --> |Map via PID/cgroup ID|CALC[Energy Calculator]:::calculator
        PWR --> CALC
    end

    subgraph PM[Power Model]
        direction BT
        CALC --> |Process Activity Ratio|ATTR[Idle and Dynamic Energy Attribution]:::attribution
        ATTR --> |Per Process/Container/VM|EXP[Energy Metrics]:::attribution
    end

    EXP --> PROM[Prometheus Export]:::export

    class KL kernelLevel
    class UP userSpace
    class RC resourceCollection
    class HM hardwareMetrics
    class EM estimatorMetrics
    class PM powerModel

Install Kepler

Instructions to install Kepler can be found in the
Kepler docs.

Visualise Kepler metrics with Grafana

To visualise the power consumption metrics made available by the Kepler
Exporter, import the pre-generated
Kepler Dashboard into Grafana:
Sample Grafana dashboard

Contribute to Kepler

Interested in contributing to Kepler? Follow the
Contributing Guide to get started!

Talks & Demos

A full list of talks and demos about Kepler can be found
here.

Community Meetings

Please join the biweekly community meetings. The meeting calendar and agenda can
be found
here

License

With the exception of eBPF code, everything is distributed under the terms of
the Apache License (version 2.0).

eBPF

All eBPF code is distributed under either:

The exact license text varies by file. Please see the SPDX-License-Identifier
header in each file for details.

Files that originate from the authors of kepler use (GPL-2.0-only OR
BSD-2-Clause). Files generated from the Linux kernel i.e vmlinux.h use
GPL-2.0-only.

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in this project by you, as defined in the GPL-2 license, shall be
dual licensed as above, without any additional terms or conditions.

Star History

Star History Chart

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: 4 days ago

Total Commits: 2,133
Total Committers: 65
Avg Commits per committer: 32.815
Development Distribution Score (DDS): 0.65

Commits in past year: 259
Committers in past year: 21
Avg Commits per committer in past year: 12.333
Development Distribution Score (DDS) in past year: 0.826

Name Email Commits
sustainable-computing-bot b****t@s****o 747
Huamin Chen h****n@r****m 310
Marcelo Amaral m****1@i****m 138
Sam Yuan y****9@1****m 120
ji chen j****c@c****m 110
dependabot[bot] 4****] 102
Sunil Thaha s****a@r****m 97
Sunyanan Choochotkaew s****1@i****m 89
Vibhu Prashar v****9@g****m 78
Dave Tucker d****e@d****k 50
Maryam Tahhan m****n@r****m 43
Vimal Kumar v****8@g****m 39
Kaiyi k****1@g****m 34
Parul Singh p****h@r****m 14
Jie Ren j****n@i****m 13
William Caban w****n@g****m 11
Sally O'Malley s****y@r****m 9
Filip Skirtun f****n@i****m 9
huoqifeng h****f@c****m 8
sustainabile-computing-bot i****o@s****o 7
Chen Wang c****n@r****m 7
Takuya Iwatsuka t****a 6
mcalman 6****n 6
Yasumasa Suenaga s****a@o****m 6
root r****t@b****d 5
Arthur Savage a****7@g****m 5
Lu Ken k****u@i****m 5
github-actions[bot] 4****] 5
omahs 7****s 5
Anthony Harivel a****l@r****m 4
and 35 more...

Committer domains:

Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 654
Total pull requests: 1,703
Average time to close issues: 2 months
Average time to close pull requests: 6 days
Total issue authors: 119
Total pull request authors: 62
Average comments per issue: 4.07
Average comments per pull request: 1.64
Merged pull request: 1,367
Bot issues: 0
Bot pull requests: 213

Past year issues: 154
Past year pull requests: 576
Past year average time to close issues: 18 days
Past year average time to close pull requests: 4 days
Past year issue authors: 34
Past year pull request authors: 21
Past year average comments per issue: 1.19
Past year average comments per pull request: 1.39
Past year merged pull request: 431
Past year bot issues: 0
Past year bot pull requests: 90

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/sustainable-computing-io/kepler

Top Issue Authors

  • rootfs (92)
  • jichenjc (57)
  • SamYuan1990 (53)
  • vprashar2929 (52)
  • sunya-ch (44)
  • sthaha (43)
  • dave-tucker (40)
  • marceloamaral (14)
  • KaiyiLiu1234 (13)
  • jharriga (13)
  • vimalk78 (13)
  • jiere (12)
  • husky-parul (9)
  • nikimanoledaki (8)
  • Feelas (7)

Top Pull Request Authors

  • rootfs (251)
  • dependabot[bot] (207)
  • vprashar2929 (199)
  • sthaha (187)
  • SamYuan1990 (153)
  • jichenjc (117)
  • marceloamaral (95)
  • sunya-ch (91)
  • vimalk78 (91)
  • dave-tucker (65)
  • maryamtahhan (57)
  • KaiyiLiu1234 (34)
  • husky-parul (19)
  • jiere (11)
  • sallyom (8)

Top Issue Labels

  • kind/bug (205)
  • kind/feature (134)
  • wontfix (127)
  • kind/documentation (20)
  • enhancement (18)
  • tech-debt (15)
  • help wanted (12)
  • to be closed (9)
  • bug (9)
  • discussion (7)
  • high-priority (5)
  • pinned (5)
  • non-k8s environment (4)
  • good first issue (4)
  • urgent-priority (4)
  • documentation (3)
  • low-priority (3)
  • medium (2)
  • metal-ci (2)
  • github_actions (1)
  • model-server (1)
  • model-db (1)
  • gh-action (1)

Top Pull Request Labels

  • dependencies (212)
  • go (108)
  • github_actions (94)
  • chore (56)
  • ci (34)
  • feat (29)
  • wontfix (21)
  • fix (19)
  • enhancement (10)
  • refactor (9)
  • bug (9)
  • docs (8)
  • kind/feature (6)
  • test (6)
  • requires-manual-test (5)
  • do-not-merge (4)
  • high-priority (2)
  • urgent-priority (2)
  • documentation (1)
  • breaking-change (1)

Package metadata

proxy.golang.org: github.com/sustainable-computing-io/kepler

  • Homepage: https://github.com/sustainable-computing-io/kepler
  • Documentation: https://pkg.go.dev/github.com/sustainable-computing-io/kepler#section-documentation
  • Licenses: Apache-2.0
  • Latest release: v0.7.12 (published 9 months ago)
  • Last Synced: 2024-12-14T10:50:24.449Z (7 months ago)
  • Versions: 64
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Docker Downloads: 361
  • Rankings:
    • Docker downloads count: 1.984%
    • Stargazers count: 3.177%
    • Forks count: 3.353%
    • Average: 4.352%
    • Dependent repos count: 4.79%
    • Dependent packages count: 8.456%

Dependencies

.github/workflows/commitMsg.yml actions
  • gsactions/commit-message-checker v2 composite
.github/workflows/image.yml actions
  • actions/checkout v3 composite
  • actions/setup-go main composite
  • docker/login-action v1 composite
.github/workflows/integration_test.yml actions
  • actions/checkout main composite
  • actions/setup-go main composite
  • sustainable-computing-io/kepler-action main composite
.github/workflows/release.yml actions
  • actions/checkout v2 composite
  • actions/checkout v3 composite
  • actions/create-release v1 composite
  • actions/github-script v5 composite
  • docker/login-action v1 composite
  • peterjgrainger/action-create-branch v2.2.0 composite
.github/workflows/unit_test.yml actions
  • actions/checkout v2 composite
  • actions/setup-go main composite
  • ad-m/github-push-action master composite
  • tj-actions/coverage-badge-go v2 composite
  • tj-actions/verify-changed-files v11.1 composite
build/Dockerfile docker
  • quay.io/sustainable_computing_io/kepler_base cuda-11.8.0-base-ubi8-bcc-0.24 build
  • quay.io/sustainable_computing_io/kepler_builder ubi-8.6-bcc-0.24-go1.18 build
go.mod go
  • github.com/NVIDIA/go-nvml v0.11.6-0
  • github.com/iovisor/gobpf v0.2.0
  • github.com/jszwec/csvutil v1.6.0
  • github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369
  • github.com/onsi/ginkgo v1.16.5
  • github.com/onsi/gomega v1.19.0
  • github.com/prometheus/client_golang v1.12.1
  • github.com/prometheus/client_model v0.2.0
  • github.com/prometheus/common v0.34.0
  • github.com/stretchr/testify v1.7.1
  • golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6
  • k8s.io/api v0.24.0
go.sum go
  • 576 dependencies
.github/workflows/daily.yml actions
.github/workflows/developer_local.yml actions
  • actions/checkout v4 composite
  • actions/setup-go v3 composite
.github/workflows/golang.yml actions
  • actions/checkout v4 composite
  • actions/setup-go v3 composite
  • actions/setup-go v2 composite
  • golangci/golangci-lint-action v3 composite
.github/workflows/gosec.yml actions
  • actions/checkout v4 composite
  • github/codeql-action/upload-sarif v1 composite
  • securego/gosec master composite
.github/workflows/image_base.yml actions
  • actions/checkout v4 composite
  • docker/build-push-action v3 composite
  • docker/login-action v1 composite
  • docker/setup-buildx-action v2 composite
  • docker/setup-qemu-action v2 composite
.github/workflows/image_pr.yml actions
  • actions/checkout v4 composite
  • docker/build-push-action v4 composite
  • docker/login-action v1 composite
  • docker/setup-buildx-action v2 composite
  • docker/setup-qemu-action v2 composite
.github/workflows/integration_test_libbpf.yml actions
  • actions/checkout v4 composite
  • actions/download-artifact v3 composite
  • actions/setup-go v3 composite
  • actions/upload-artifact v3 composite
  • sustainable-computing-io/kepler-action v0.0.1 composite
.github/workflows/platform-validation.yml actions
  • actions/checkout v4 composite
  • actions/download-artifact v3 composite
  • actions/setup-go v3 composite
  • actions/upload-artifact v3 composite
  • docker/build-push-action v4 composite
  • docker/login-action v1 composite
  • docker/setup-buildx-action v2 composite
  • docker/setup-qemu-action v2 composite
  • sustainable-computing-io/kepler-action v0.0.1 composite
.github/workflows/pull_request.yml actions
.github/workflows/push.yml actions
.github/workflows/scorecard.yml actions
  • actions/checkout 93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 composite
  • actions/upload-artifact 3cea5372237819ed00197afe530f5a7ea3e805c8 composite
  • github/codeql-action/upload-sarif 17573ee1cc1b9d061760f3a006fc4aac4f944fd5 composite
  • ossf/scorecard-action e38b1902ae4f44df626f11ba0734b14fb91f8f86 composite

Score: 17.393568780479978