Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Recent Releases of Eco-CI

Eco-CI - v4.5 Backfilling for undersampling, better overhead display macOS hotfix

Eco CI has arrived in the spring season with a lovely new release!

  • We had a bug in the macOS estimations. The hotfix was already on main for a couple of days. Now coming to the v4 release. All previous data of macOS should be considered unreliable! Please re-run your pipelines
  • Eco CI now backfills data. To introduce not too much overhead we sample at 1s. However when steps are very short (2-3s) undersampling hits really hard and you might "miss" 50% of your energy. This is not relevant for many users which run typically 20 minute runs, but we have some users that run ~5s runs for hundrets of times a day. With backfilling we estimate now the last missing part of a second and thus reduce missing data.
  • Overhead of Eco CI is now displayed nicer and more clearly in the table. See this:

Happy CI/CD runs everybody!

What's Changed

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v4...v4.5

Consumption - Computation and Communication - Shell
Published by ArneTR 17 days ago

Eco-CI - v4.4.1 Hotfix for Linux

In some circumstances Linux energy values where absurdly high (kW). this hotfix patches the issue

What's Changed

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v4...v4.4.1

Consumption - Computation and Communication - Shell
Published by ArneTR 3 months ago

Eco-CI - macOS support

macOS is now supported for the standard GitHub shared runners:

  • macos-13
  • macos-14

Find infos here on the assumptions made and parameters set for the energy estimations:

Thanks to @michaellilltokiwa for testing it out and opening the issue that led to the feature!

What's Changed

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v4...v4.4

Consumption - Computation and Communication - Shell
Published by ArneTR 3 months ago

Eco-CI - v4.3 - Jenkins support and macOS beta

Hooray, Jenkins support finally has a documentation!

It was available before, but now it has an official place in the documentation. For anyone trying: We love your feedback on it! And also happy to take PRs if you want to contribute to the documentation.

We also added first macOS support. It should work, but is still in beta. Let us know how it works with your runners!

What's Changed

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v4...v4.3.0

Consumption - Computation and Communication - Shell
Published by ArneTR 3 months ago

Eco-CI - v4.2- Totals Badges in PR view

Eco-CI will now show Summarized information on PR Comments.

This means that if you push multiple commits to the PR it will not only show the last carbon and energy cost, but the cumulated one.

Thanks to @Zeitsperre for the idea!

Furthermore there are also 30-Day averages, Last-30-Day totals and more! See this PR discussion for details: https://github.com/green-coding-solutions/eco-ci-energy-estimation/issues/107

What's Changed

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v4...v4.1.0

Consumption - Computation and Communication - Shell
Published by ArneTR 4 months ago

Eco-CI - v4 Release

Important: First of all thanks to our many contributors for this new version ❤️

New features

  • Eco-CI is now greatly reduced in overhead. Most of the changes we describe in the pre-release notes.

    • The footrpint is greatly reduced from > 200 MB to < 1 MB
    • The overhead is reduced from ~ 20 seconds to almost zero due to pre-calculation of the power profiles

  • Eco-CI is now compatible with the new GMT endpoint format /v2/ci/measurement/add

    • This is mostly an internal change, but allows you to use CarbonDB 2.0

What's Changed

New Contributors

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v3.1...v4.0.0

Consumption - Computation and Communication - Shell
Published by ArneTR 6 months ago

Eco-CI - Minor bug fixing release

This release fixes a bug that if the labels include invalid (as in shell specific) characters it would fail to display the results.

What's Changed

Consumption - Computation and Communication - Shell
Published by ribalba 9 months ago

Eco-CI - v4.0-rc1 Pre-Calculated, low dependency & SCI Embodied Carbon

The functionality of Eco-CI has been rewritten down to it's core.

Although the general concept is still the same (we use Cloud Energy to generate power data based in CPU utilization) we dropped a humongous amount of dependencies and shaved off almost 100x in terms of overhead.

It looks pretty stable in our initial tests on different GitHub machines, still we are releasing this as a release candidate for now

Key facts and architectural changes:

  • Cloud Energy power curves are now pre-calculated in bash variable file instead of dynamically inferenced
  • python3 dependecy dropped. We are using pure bash now
  • bc dependency dropped. We are using awk now, which was a depency beforehand already
  • go dependency dropped. The graph functionality has simply been removed. It was a nice gimmick but provided very few actionable insights
  • Caching dropped. There is simply nothing anymore to cache :)
  • Support of ARM runners added implicitly
  • Support for Ubuntu 20.04 added implicitly
  • Overhead reduced from 23s for initial download and ML training step to 200ms for bash inferencing on pre-calculated files
  • Embodied carbon has been added to truly output an SCI score. Before it was using a fixed constant

We also changed the API with the intent to simplify it. Please note that these changes might be breaking changes, depending on the complexity of your integration.

GitHub breaking Changes

  • json-output in step display-results must be actively set to 'true' to enable creating the JSON files
  • branch must only be submitted one in start-measurement and not in every call to get-measurement
  • send-data must only be submitted one in start-measurement and not in every call to get-measurement
  • show-carbon has been renamed to calculate-co2 to better reflect that actual compute work is done
  • api-base has been renamed to gh-api-base to not confuse with the dashboard api base if you want to host a data drain yourself
  • machine-uuid is now a MUST to be specified. It will not be auto generated anymore
  • machine-uuid, company-uuid and project-uuid must only be submitted one in start-measurement and not in every call to get-measurement

GitLab changes

  • ECO_CI_JSON_OUTPUT environment variable must be exported with value 'true' to enable creating JSON files
  • initialize_energy_estimator is not existent anymore. The functionality has been merged into start_measurement to simplify the API
  • ECO_CI_MACHINE_UUID is now a MUST to be specified. It will not be auto generated anymore

What's Changed

New Contributors

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v3.1...v4.0-rc1

Consumption - Computation and Communication - Shell
Published by ArneTR 11 months ago

Eco-CI - v3.1 SCI remarks and styling

No new functionality. Just the CO2eq output is now marked as SCI.

Also links to https://greensoftware.foundation/articles/use-case-automated-sci-measurements-with-green-metrics-tool and to electricitymaps for clarification that we use location based grid intensity

Consumption - Computation and Communication - Shell
Published by ArneTR about 1 year ago

Eco-CI - v3.0 - CO2 and grid location intensity data

A long awaited feature: Eco-CI can now show Carbon Grid Intensity Data and also then show the CO2e for the run.

See an example of a Pull Request here:

The data is also transmitted to the GMT dashboard and can there be nicely viewed in an overview

Consumption - Computation and Communication - Shell
Published by ArneTR about 1 year ago

Eco-CI - v2.10

What's Changed

Full Changelog: https://github.com/green-coding-solutions/eco-ci-energy-estimation/compare/v2...v2.10

Consumption - Computation and Communication - Shell
Published by dan-mm about 1 year ago

Eco-CI - v2.9

New Features

  • added support for Intel Core i9-9880H (thanks @nstruharova !)

Tweaks

  • if vhost-ratio is not set, model will run without this input parameter
  • changed references from green-coding-berlin -> green-coding-solutions

Consumption - Computation and Communication - Shell
Published by dan-mm about 1 year ago

Eco-CI - Updated references from green-coding.berlin to green-coding.io

All references and endpoints have been updated from .berlin -> .io

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - Configurable github api + better PR formatting

  • you can now set a custom api url, for all github enterprise users. By default, this is set to 'api.github.com', but you can set it to something different for github api calls

  • cleaner looking formatting in minimized comments in PR's

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - New Github CPU

Github has introduced a new CPU to their VM's - the AMD EPYC 7763. Eco-CI now detects this cpu and can give estimation based on its specs

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - Collapsible Comments

  • Visual Clarity QOL: When Eco-CI posts comments on a PR thread with the newest on-push energy reading, it will now edit and collapse old comments in the thread so that only the newest energy reading is displayed.

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - Guard Clause Bugfix

  • Fixed guard clause in case one calls display-results without a make-measurement first

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - Minor Improvements

  • Workflow names will now be sent to Eco-CI front-end
  • if send-data is false, no api calls will be made to get the workflow-id, as its not needed

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - v2.2 - sending CPU utilization stat to Eco-CI front end

Eco-CI will now also send the average cpu utilization stat to our front end view, which has been updated with a more in-depth stats table so you can analyze your pipeline runs :-)

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - Revised view for PR comments

Minor update - changed the view on the PR comments to match the github step summary view, and include the energy graph.

Consumption - Computation and Communication - Shell
Published by dan-mm over 1 year ago

Eco-CI - v2.1 Github PR Issue Comments

Added the ability for Eco-CI to post a comment on Github pull requests with the energy results.
The workflow must be triggered by a PR, and you must call the display-results task with a new optional flag, pr-comment: True

Eco-CI will now also detect if your machine has a suitable python version available on it, and installs python3.10 if it does not.

Consumption - Computation and Communication - Shell
Published by dan-mm almost 2 years ago

Eco-CI - Gitlab Integration

Eco-CI is now available for Gitlab pipelines. Instructions on how to use with Gitlab projects can be found in the readme.

Improved front end here: https://metrics.green-coding.berlin/ci-index.html

Bugfixes:

  • Corrected RAM values for the XGBoost Model (they did not account for vhost ratio). If you notice different (likely higher) energy values from your pipelines than what you had before, this is likely the reason why. They should be more accurate now.
  • Correctly reset venv for python projects in pipelines when running XGBoost model, if such a virtual environment was present.

Consumption - Computation and Communication - Shell
Published by dan-mm almost 2 years ago

Eco-CI - v1.4 CPU & Commit-Hash & Duration added incl. a guard clause

Eco-CI now keeps track of the CPU, the duration and the commit hash of the pipeline you are running.

It will display this in the table, but also on https://metrics.green-coding.berlin if you want to send it.

What's Changed

Full Changelog: https://github.com/green-coding-berlin/eco-ci-energy-estimation/compare/v1.3.1...v1.4

Consumption - Computation and Communication - Shell
Published by ArneTR almost 2 years ago

Eco-CI - v1.3.1 Test update release

Just some minor test fixes and introducing v1 as the standard release tag for the current API spec.

Full Changelog: https://github.com/green-coding-berlin/eco-ci-energy-estimation/compare/v1.3...v1

Consumption - Computation and Communication - Shell
Published by ArneTR about 2 years ago

Eco-CI - v1.3 QoL and Feature Release

The main action is unchanged, but we squashed some bugs, made a nicer table view and had some lovely contributions for JSON display, Readme Fixes, Tests, optional flags and much more!

Big thx to @rajbos and @jreimone ❤️

What's Changed

New Contributors

Full Changelog: https://github.com/green-coding-berlin/eco-ci-energy-estimation/compare/v1.2.0...v1.3

Consumption - Computation and Communication - Shell
Published by ArneTR about 2 years ago

Eco-CI - v1.2.0

  • Badge can now be displayed
  • Option to drill down data in separate chart display

What's Changed

New Contributors

Full Changelog: https://github.com/green-coding-berlin/eco-ci-energy-estimation/compare/v1.1...v1.2.0

Consumption - Computation and Communication - Shell
Published by ArneTR about 2 years ago

Eco-CI - v1.1

  • Model now respects CPU info
  • venv is now used
  • Cache key was not specific enough

Consumption - Computation and Communication - Shell
Published by ArneTR over 2 years ago

Eco-CI - v1.0

See details on https://www.green-coding.berlin/projects/eco-ci

Consumption - Computation and Communication - Shell
Published by ArneTR over 2 years ago