Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

A curated list of open technology projects to sustain a stable climate, energy supply, biodiversity and natural resources.

Recent Releases of Green Metrics Tool

Green Metrics Tool - v1.4.0 - WattHours default + QoL + depedency updates

If you look at the release logs for this month you might be thinking that only dependency updates have happened. But don't be fooled, a lot of long wanted and interesting features have made it to this months release:

  • WattHours are now the default display (instead of Joules) as the unit of energy. You can still switch back to Joules in the settings
  • QoL features in the dashboard:
    • Unselect button when comparing runs
    • href removed for non links for screen readers
    • Badges now have average function
  • GMT can now isolate containers from network by setting --internal
  • GMT is now microjoules under the hood to allow higher resolutions
  • Averages are now weighted averages for CPU utilization and others
  • Sample resolution is now shown in Dashboard and you can compare if you can actually sample at the target rate

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v1.3.0...v1.4.0

Consumption - Computation and Communication - Python
Published by ArneTR 3 months ago

Green Metrics Tool - v1.3.0 Codespaces + CarbonDB 2.0 + QoL / Tests / Package updates

Whoo, what a christmas release!!

GMT comes along for the holidays with an unusual packed feature release and also some more extensive test coverage now even including the frontend.

The higlights are:

  • We now have a default Codespaces setup to try at your convenience. Just click on the Codespaces button in the Readme and try out GMT pre-installed and pre-configured with our trusty Cloud Energy reporter active for cloud measurements
  • The integrated Cloud Energy now accounts vor vhost ratio of your machine so you can get reliable GMT values even in VMs
  • Phase Stats are now also saved if runs fails. So no data gets lost if you want to account for the carbon consumption of your measurement cluster
  • CarbonDB 2.0 is now ready for use. (Please note that this is an enterprise only feature and distributed in a separate repository - See https://www.green-coding.io/products/carbondb/)
  • Clickable Comparsion Charts + Display Missing Values by @ArneTR in https://github.com/green-coding-solutions/green-metrics-tool/pull/1017 So nice!!!!!

Thanks again to our contributors this month:

  • @davidkopp ❤️

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v1.2.0...v1.3.0

Consumption - Computation and Communication - Python
Published by ArneTR 4 months ago

Green Metrics Tool - v1.2.0 STDDEV changes and display tweaks

STDDEV in GMT is now exclusively calculated as SAMPLE-STDDEV and not POP-STDDEV anymore.

This is for two reasons:

  • It is required by certifications like the Blue Angel for Software
  • We got many debates that in cases where the average is only estimated through measurements and is not absolute one MUST use the sample STDDEV. Still one could argue that one does not want to characterize the measured software but rather the measurement setup it is safer to use the sample STDDEV as it is always higher

Furthermore some display changed where made to have the AVG bar optional and quick-access for the timeline was added on overview

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v1.1.1...v1.1.2

Consumption - Computation and Communication - Python
Published by ArneTR 7 months ago

Green Metrics Tool - v1.1.1 SCI / Embodied Carbon Patch & dependencies update

  • (fix) SCI values were not correctly reported anymore.
  • (update) Using the release to also update some depedencies

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v1.1...v1.1.1

Consumption - Computation and Communication - Python
Published by ArneTR 7 months ago

Green Metrics Tool - v.1.1.0 - Patch Release / Feature Release

v1.0 is just a week out, and here it is: The first patch / feature release :)

We encourage everybody to update as one reporter was reporting broken data.

Fixes

  • Some API calls in the HOG are now optional as we have seen machines without disks
  • GMT uses for reporiting the SI norms. That means that 1 KB is 1000 Bytes, not 1024 (as this would be kiB). We fixed one occurence where this was not correctly reported but should be negligeable to most users.
  • network_io_cgroup_container reporter was reporting grossly false high values. Here an accumulation instead of an assign was happening. (Note that this was introduced in v1.0 and was fine in versions before. Blast radius should thus be small)

Features addons

  • SSL is now possible in the installation process. It should be pretty straight forward but documentation is to follow.
  • Cron dir is now refactored and all cron jobs have error handling
  • Added a backup script to import backups to the database (Can handle 100 GB+ imports !)

Important info

CarbonDB 2.0 is actually not part of this release. Due to some branch merging it looks this way on GitHub. It will be released later next week with v1.2

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v1.0...v1.0.1

Consumption - Computation and Communication - Python
Published by ArneTR 7 months ago

Green Metrics Tool - v1.0

The 1.0 release !!!! 🎉

While this release mostly contains updates and tests to asset the stability of the measurements it has two main perks:

  • GMT now has an authentication so it can be used with multiple users to implement ACLs and data retention
  • GMT is now fully compliant with the measurement requirements for the new Blue Angel for Software

We opted for the 1.0 release now as we have used it to capture more than 1 TB of measurement data of 100+ users.

GMT has been featured on many conferences and papers

We believe it is now ready for 1.0. Enjoy!

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v0.25...v1.0

Consumption - Computation and Communication - Python
Published by ArneTR 7 months ago

Green Metrics Tool - v0.25 Security Release! + Feature drop

Security

When install GMT in an unsecured multi-user environment a malicious actor that has already penetrated the system could replace a binary in an allowed sudoers path to get root access rights.

This security vulnerability is to be seen as relatively low, as the system is not open from the outside. It is a local privilege escalation.

We recommend updating to this release as soon as possible. Fix can be applied by just running the install script (install_linux.sh / install_mac.sh) again.

Features

  • Power per Container is now displayed in the dashboard. This feature is based on a linear estimation of the container CPU usage and the total machine power
  • Multi-Stage-Build Docker Images Support
  • Global Timeout to abort measurement when a timer is expired
  • Weekly projects for the timeline projects to run a project only once a week
  • RAPL energy filtering guard clause is now checked

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v0.24...v0.24.1

Consumption - Computation and Communication - Python
Published by ArneTR 9 months ago

Green Metrics Tool - v0.24 - Optimizations, Diffing, Intel PSYS, Performance, CarbonDB & QoL

Oh my, what a release!

The most important: Green Metrics Tool now has optimizations!

When you do a run you will be informed about what to tweak and optimize in order to make your software greener. We are releasing a first bunch of battle tested optimizations into open source. More to follow!

See details on the bottom of the GMT page here: https://www.green-coding.io/projects/green-metrics-tool/

Other very prominent changes are:

What's Changed

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v0.23...v0.24

Consumption - Computation and Communication - Python
Published by ArneTR about 1 year ago

Green Metrics Tool - v0.23 Cluster, DEV & Accuracy Tooling

Key additions

  • Updated our values for the average grid intensity and the embodied carbon values.
    • If you are running on our Cluster you will see now different CO2 values.
  • Cluster status page greatly updated. Give it a try on our status page
  • DEV mode tooling greatly improved. Faster usage scenario development and testing runs!
    • See docs page for details
      • --dev-no-sleeps
      • --dev-flow-timetravel
      • --dev-no-build
  • calibrate.py script to calculate the measurement overhead
  • Package updates and small visual updates and fixes.

Thank you @alexzurbonsen for your contributions! ❤️

What's Changed

New Contributors

Full Changelog: https://github.com/green-coding-solutions/green-metrics-tool/compare/v0.22.1...v0.23

Consumption - Computation and Communication - Python
Published by ArneTR about 1 year ago

Green Metrics Tool - v0.22.1 - Massive Feature Drop 2024 - Hotfix #1

  • Fix for validate time calculation not respecting machine_id

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.22 - Massive Feature Drop 2024

The year starts very welcoming with a massive feature release for the Green Metrics Tool :)

  • New Dev switches added for faster usage scenario development.
  • Cluster is now temperature controlled and has control workloads. Documentation
  • Frontend display improvements
  • Energy Timeline now shows Standard Deviation
  • Lockstep sampling added
  • Faster container boots / shutdowns for GMT
  • Faster unit test runs (2x-6x !)
  • Improved status page
  • GMT color via own commit hash
  • Various improvements and refactorings

Big Thanks to @davidkopp for many contributions! <3

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.21...v0.22

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.21 PSU AC Power MCP39F511N, QoL & Security upgrades

In this release we finally brought the support for our new high resolution AC PSU provider the MCP39F511N.

Also we have some QoL updates like showing the run link directly after a run, refactorings and a security upgrade (a file that we set in the sudoers file is now put to read only by all).

Also we welcome a new contributor: @davidkopp ❤️

Happy measuring y'all :)

What's Changed

New Contributors

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.20.2...v0.21

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.20.2 (Bugfix) HOG Support 🐷 + Docker Root support + QoL + Code Quality Latest

  • NetworkIoCgroupContainerProvider had a segementation error when using multiple container IDs

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.20 - HOG Support 🐷 + Docker Root support + QoL + Code Quality

HOG Suport!!1 ❤️ 🐷

QoL:

  • venv is now mandatory and keeps your Python system packages clean
  • Docker root is now supported! We still recommend it for production, but for dev you now do not need to modify your docker client anymore 🥳
  • Warnings have been added for many providers when they actually cannot run. No more (well ... less :) ) cryptic errors!
  • Docker build logs are now shown in the Logs pane in the Dashboard

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.19.1...v0.20

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.19.1 Timeline View & Visual Improvements in Dashboard & macOS providers

Versin v0.19 adds the Timeline View!

Example for Overview: https://metrics.green-coding.berlin/energy-timeline.html
Example for concrete Timeline: https://metrics.green-coding.berlin/timeline.html?uri=https://github.com/green-coding-berlin/bakerydemo-gold-benchmark&filename=usage_scenario_warm.yml&branch=&machine_id=7

Timeline View is our first step to create a historic view of a software product over many commits. Here you can see how a software changes over time given a certain use case and potentially find when a energy hungry feature was introduced and optimize on it!

Main Features

  • Timeline View added
  • macOS improvements for powermetrics. The provider is now gracefully waited for and measurements do not fail with plain error.
  • Eco-CI statistical data added to dashboard
  • More infos about the running cluster machine added like installed packages, docker daemon stats, etc.
  • Frontend has been greatly refactored to handle differences for machines better, display more warnings in case of inconsistency and rework how graphs are created

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.18...v0.19

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.18 SCI preview & PR energy for Eco-CI

In version 18 we are introducing the capability to output the Green Software Foundation's SCI with our tool

Please read up how to leverage that functionality in the docs

The other major upgrade is that Dashboard has been revamped and is now fully customizable. You can see any chart at any position you want by specifying which metrics shall be included.

Also if you are comparing different machines with different reporters you will get a warning if no matching reporter was found and no comparison can be made for a specific metric. Documentation for customization

Read the full changelog will all QoL, Bugfixes and Style improvements below.

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.17.1...v0.18

Consumption - Computation and Communication - Python
Published by ArneTR over 1 year ago

Green Metrics Tool - v0.17.1 Small Bugfix in Network-IO Display

The data for the network-IO were correctly measured, but displayed wrongly for the [Remove] phase.

For any new measurement with the tool the data will now show correctly. If you also want to bring your already stored measurements up to the correct display please:

  • Run the migration python3 tools/import_data.py migrations/2023_07_10_indices.sql.
  • Rebuild the phase_stats with the command: python3 tools/rebuild_phase_stats.py

Happy measuring with now correct Network-IO in the Remove phase :)

Consumption - Computation and Communication - Python
Published by ArneTR almost 2 years ago

Green Metrics Tool - v0.17 Performance-, QoL-Updates and Bugfixes

This release changed the port-number of the Postgres DB and also some loading code for the OS filesystem mounts.

Please re-run the install script to bring these changes into effect.

Also some indices have been changed, so please run the migration 2023_07_08_indices.sql

For details see: https://docs.green-coding.berlin/docs/installation/updating/

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.16...v0.17

Consumption - Computation and Communication - Python
Published by ArneTR almost 2 years ago

Green Metrics Tool - v0.16 Eco-CI better stats & Gitlab integration & NOP Linux Measurement Cluster + many small updates

Eco-CI has gotten some major updates and a Gitlab integration and also our NOP Linux measurement cluster is now ready!

  • NOP Linux Measurement Cluster installation added
  • Performance has been greatly increased by using copy_from to insert phase_stats
  • Schema of usage_scenario is now checked before run for bette errors and reproducibility
  • API now uses HTTP 204 Status codes to allow proper caching
  • Eco-CI frontend allows now for greater drilldown in metrics
  • Stdout Logs display in Dashboard
  • Runtime phase now supports sub-phases through multiple flows
  • Various minor bugfixes

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.15...v0.16

Consumption - Computation and Communication - Python
Published by ArneTR almost 2 years ago

Green Metrics Tool - v0.15 🔥 Compare Feature & Eco-CI Upgrades

This is a BIG release for the Green Metrics Tool 🔥

We have adapted the measurement workflow to incorporate phases (Baseline, Idle, Installation, Boot, Runtime, Remove) which makes our tool compatible with the requirements specified by the Blue Angel for Software.

Also it is our first step towards a tooling for Software Lifecycle Analysis.

See the comparison in action here, where we compare the influence of changing the database in a Nextcloud installation (swapping MariaDB for SQLite)

  1. Also our Eco-CI Tooling has been greatly upgraded and allows now for drilldown towards different CPUs, Labels and timeframes. Check out our Eco-CI feature demo.

  2. Contributors
    Big thx also to @saiteja13427 and @shekhar-chauhan for making contributions to the Green Metrics Tool ❤️

What's Changed

New Contributors

Consumption - Computation and Communication - Python
Published by ArneTR almost 2 years ago

Green Metrics Tool - v0.14 macOS support + IPMI + configurable Domain + QOL

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.13...v0.14

Consumption - Computation and Communication - Python
Published by ArneTR about 2 years ago

Green Metrics Tool - v0.13 OpenEnergyBadge & Eco-CI + QOL improvements

What has Changed?

  • OpenEnergyBadge
  • All energy based metric-providers now report in mJ
  • Newest XGBoost Model integrated which uses HardwareAvailability and CPUThreads as input variables for more accuarcy
  • API now reports referrer if available
  • Tests use Eco CI energy estimation in VMs and als eco-ci activity checker Eco CI
  • Removed unused fonts and IE icons
  • GMT can now ignore errors from processes if specified
  • Filename can now be changed from usage_scenario.yml
  • GMT can now clone only the branch if requested
  • New tests integrated
  • Non-Compliant measurements can now be marked
  • UI can now move charts and also expand them to full-width
  • All python libraries are now installed via the requirements.txt file(s)

Bugfixes

  • CO2 calculation contained an error Commit
  • Process listing in hardware_info contained only user processes.
  • API now dumps request object correctly in error mail
  • LmSensors metric provider had typo in class name
  • Powerspy2 was not calculating energy correctly in certain cases

Auto Release Notes

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.12...v0.13

Consumption - Computation and Communication - Python
Published by ArneTR about 2 years ago

Green Metrics Tool - v0.12 - Powerspy2, Liniting and Verbose Mode

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.11.1...v0.12

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.11.1 - Quality Release

  • Text transform for SI-Units deactivated kg now shows not anymore as KG etc. @ArneTR
  • Processes are now killed if SIGTERM does not work in 5s time @ribalba
  • Requirements are now installed via requirements.txt @dan-mm
  • Job mechanism now writes email in case of error to client. @ArneTR
  • Testing now integrated in daily runs is triggered manually too. @dan-mm
    Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.10.1...v0.11.1

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.10.1 - Temp & Fan Sensors, Dynamic Dashboard and more ...

This release is a big one!

  • lm_sensors library added by @ribalba
  • Dashboard now dynamically creates containers based on database entries, rather than having a fixed lookup table in the frontend. This allows for a stronger de-coupling and makes the Dashboard reusable in other open-source energy related projects
  • XGBoost submodule updated to use new interpolating reporter
  • Bugfixes
    • Cronjobs could get stuck
  • New SDIA Formula reporter
  • Tests improved on Github VMs @dan-mm
  • config.yml improved markers for testing
  • Dashboard now displays at least basic data if test runs fail @djesic
  • Dashboard can now show data in Watts and Joules

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.9...v0.10.1

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.9 Integration Tests running

Integration Tests added by @dan-mm

What's Changed

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.8.1...v0.9

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - DC Reporter released (incl. bugfix)

  • DC Reporter added
  • When job has no data a flame icon is shown in overview 🔥

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.8...v0.8.1

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.7 - XGBoost Reporter

  • XGBoost reporter included for estimation of AC power

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.6...v0.7

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.6

  • AC Power from PSU as reporter added
  • AC Power reporting now in Dashboard as single and compound metric

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.5...v0.6

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.5 - Procfs Sytem Time Reporter + DC Update

  • DC Measurements updated with 60ms sampling and Averaging instead of Last measurement
  • No /etc/sudoers file needed anymore. Using setuid now
  • Added cpu_utilization_procfs_system_provider

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.4...v0.5

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.4 feat. DC Measurement

Our first release with a DC measurement!

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.3.1...v0.4

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.3.1 - Quality Improvement Release

  • Stderr is now checked on every boot of the metric providers stopping run early in case of errors
  • Lingering static-binary processes are now properly closed in case of manual abort (CTRL+C) or error in measurement process

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - Version 0.3

This is the first release version of the Green Metrics Tool.

We have officially passed the beta stage, as all the relevant reporters are now present in the tool.

Release notes:

  • DRAM Metrics Reporter
  • Network Metrics Reporter
  • Time Reporter
  • Documentation complete and online at https://docs.green-coding.org
  • Docker compose format for usage_scenario.yml

Full Changelog: https://github.com/green-coding-berlin/green-metrics-tool/compare/v0.2.1-beta...v0.3

Consumption - Computation and Communication - Python
Published by ArneTR over 2 years ago

Green Metrics Tool - v0.2.1-beta

  • Co2 calculation is only CPU Package atm. Textual clarification for that
  • Moved Showcases to Company Website and out of Green Metrics Tool
  • Removed old CSS and some style polishing

Consumption - Computation and Communication - Python
Published by ArneTR almost 3 years ago

Green Metrics Tool - v0.2-beta

  • new Charting Library that can handle +200k datapoints https://echarts.apache.org
  • new Interface based in Fomantic-UI (https://www.fomantic-ui.com) to provide cleaner HTML
  • CO2 figures in charts
  • API endpoint for notes
  • Install.sh script
  • Docker containers now use memory filesystem for tmp directory
  • New OO architecture for Metric reporters to make inheritance and extending easy
  • Cronjob system through database queue (jobs.py)

Consumption - Computation and Communication - Python
Published by ArneTR almost 3 years ago

Green Metrics Tool - v0.1-beta

v0.1-beta is a pre-production and beta release.

Pre-Production means: The tool is not yet final in terms of its planned featureset, but is working stable in its current form.

Beta means:

  • Metric Providers are currently only working on Ubuntu 22.04 as they are statically linked. If you need them working on your machine please run make in their directory.
  • Metric providers are not yet falsified. This version is in particular release to give other open source communities to falsifiy our measurement accurary and reproduciblity

Consumption - Computation and Communication - Python
Published by ArneTR almost 3 years ago