Shyft

A cross-platform framework built around high-performancetime-series storage and analysis, providing the computational foundation for hydrological forecasting and energy-market modelling.
https://gitlab.com/shyft-os/shyft

Category: Energy Systems
Sub Category: Energy Markets

Keywords

C++ Hydrology energy-market forecasting optimization python time-series

Last synced: about 5 hours ago
JSON representation

Repository metadata

Source code for shyft. * Time-series for python and c++, including distributed storage and calculations * Hydrologic Forecasting Toolbox, high-performance flexible stacks, including calibration * Energy-market models and micro services

• Host: gitlab.com
• URL: https://gitlab.com/shyft-os/shyft
• Owner: shyft-os
• License: other
• Created: 2019-04-13T16:04:16.075Z (almost 7 years ago)
• Default Branch: master
• Last Synced: 2025-11-27T21:18:41.444Z (5 months ago)
• Topics: C++, Hydrology, energy-market, forecasting, optimization, python, time-series
• Stars: 32
• Forks: 20
• Open Issues: 116
• Releases: 0

https://gitlab.com/shyft-os/shyft/blob/master/

          # Shyft – Framework for Energy-Market Modelling

High-performance framework for **time-series storage, analysis, and modelling**
of **hydrological and energy-market systems**.

[![Documentation](https://img.shields.io/badge/docs-shyft--doc-blue)](https://shyft-os.gitlab.io/shyft-doc/index.html)
[![Discussion](https://img.shields.io/badge/discussion-google--group-blue)](https://groups.google.com/forum/#!forum/shyft)
[![License](https://img.shields.io/badge/license-LGPLv3-blue.svg)](LICENSE)

---

## Overview

**Shyft** is a cross-platform framework built around **high-performance
time-series storage and analysis**, providing the computational foundation
for **hydrological forecasting and energy-market modelling**.


The project originated at **Statkraft** in collaboration with the
**University of Oslo – Department of Geosciences**, and is today used in
**operational energy-sector environments**, including **24×7 systems** within
Statkraft’s Energy Management division.


Shyft combines:

- a **high-performance C++ core**
- a **Python orchestration layer**
- **scalable compute and cache microservices**
- **enterprise-grade time-series and model storage**
- **model-driven system architecture**

The framework was created through close interdisciplinary collaboration
and continues to enable cooperation between:

- domain experts
- researchers
- software engineers
- operational IT teams

while maintaining the performance and robustness required for operational energy systems.

---

## Architecture

![Energy Market Model](./doc/image/shyft_energy_market_model.jpg)

*Figure crafted by Christine Schei Liland*

Shyft provides a modular platform composed of interoperable components
that can be deployed together or separately depending on the operational need.

These components include:

- high-performance **time-series services**
- distributed **DTSS compute services**
- **hydrological forecasting models**
- **energy-market models**
- Python-based orchestration and analysis tools

This modular structure supports **scalable microservice deployments**
and enables packaging strategies that minimize dependencies and image sizes.

---

## Key Capabilities

### Model-Centric Time-Series Platform

Shyft follows a **model-centric architecture** where domain models
(such as hydrology systems or energy-market models) are decorated with
time-series attributes and symbolic expressions.

The time-series engine provides:

- strongly typed time-series objects
- symbolic expressions
- lazy evaluation
- distributed execution
- persistent time-series storage

### Distributed Time-Series Services (DTSS)

The **Distributed Time-Series Service (DTSS)** provides scalable
server-side evaluation of time-series expressions, including:

- distributed time-series evaluation
- high-performance in-memory caching
- websocket-based APIs
- real-time subscriptions
- replication between deployments

### Scalable Algorithm Execution

Shyft separates **domain models** from the **execution of algorithms**,
allowing simulations, forecasts, and analytical workflows to scale
independently from the modelling layer.

This architecture enables **multiple algorithms to operate on the same
domain models**, allowing different simulation, forecasting, or
optimization methods to be applied **without coupling the models to a
specific algorithm implementation**.

Distributed compute services execute algorithms on shared time-series
data and domain models, with results attached back to the models for
analysis and decision support.

### Hydrological Forecasting

Shyft contains hydrological modelling tools designed for operational
forecasting environments, supporting distributed hydrological models
and uncertainty analysis.

### Energy-Market Modelling

The energy-market framework enables modelling of:

- power grid topology
- hydropower systems
- production units
- contracts and portfolios
- operational constraints

The modelling framework allows researchers, vendors, and operators to
integrate proprietary optimization and simulation algorithms.

---

## Documentation

Full documentation is available at:

https://shyft-os.gitlab.io/shyft-doc/

The documentation includes:

- system architecture
- installation guides
- Python API reference
- time-series engine documentation
- hydrology modelling
- energy-market modelling
- developer guides
- release and contribution workflow

---

## Supported Platforms

Shyft packages are automatically built and tested through the GitLab CI pipeline.

Primary development and CI testing platforms:

- **Fedora**
- **Arch Linux**

Additional platform support:

- **Windows (MSYS / MinGW)** *(legacy support for existing deployments)*

Shyft can generally be built on **any modern Linux distribution** with an up-to-date toolchain and the required dependencies (C++ compiler, CMake, and libraries).

Packages and build artifacts are published through the GitLab package registry.

Refer to the documentation for installation instructions:
https://shyft-os.gitlab.io/shyft-doc/

---

## Security and Supply-Chain Transparency

Shyft emphasizes **software supply-chain security** and operational reliability.

The project uses practices including:

- verified GPG-signed commits
- signed release artifacts
- provenance metadata
- transparent dependency management
- preference for distro-maintained packages
- reproducible build pipelines

These practices support safe deployment in operational environments.

For background on secure open-source development practices, the following resources provide useful guidance:

- **Linux Foundation — Developing and Evaluating Secure Open Source Software (OSS)**  
  https://events.linuxfoundation.org/wp-content/uploads/2021/07/Developing-Secure-Open-Source-Software-OSS.pdf

- **Open Source Security Foundation (OpenSSF)**  
  https://openssf.org/

- **SLSA — Supply-chain Levels for Software Artifacts**  
  https://slsa.dev/

---

## Contributing

Shyft development follows a structured workflow involving:

- **Owners**
- **Maintainers**
- **Contributors**

Development happens on the `next-shyft-version` branch and follows a documented release process.

Contribution guidelines, development setup instructions, and workflow details are available in:

Project wiki  
https://gitlab.com/shyft-os/shyft/-/wikis

and specific https://gitlab.com/shyft-os/shyft/-/wikis/How-to/Contribute

---

## Container Images

The CI pipeline produces container images used for building and testing Shyft.

These images are available in the GitLab container registry:

https://gitlab.com/shyft-os/shyft/container_registry

---

## Authors and Project Stewardship

Shyft was initially developed by **Statkraft** in cooperation with the  
**University of Oslo – Department of Geosciences (UiO)**.

The project originated from a small, highly experienced interdisciplinary team
combining expertise in applied mathematics, hydrological science,
energy-system modelling, and software engineering.

The overall **architectural design and C++ core implementation** were primarily developed by:

- **Sigbjørn Helset** 
- **Ola Skavhaug** 

The initial **Python orchestration layer, data pipelines, and modelling integration**
were primarily developed by:

- **John F. Burkhart** 
- **Yisak Sultan Abdella** 

The hydrological modelling algorithms and their composition into operational
modelling stacks were strongly influenced by the hydrological research
experience of **John F. Burkhart** and **Yisak Sultan Abdella**, and by the
research collaboration with the **University of Oslo Department of Geosciences**.

Through this collaboration, UiO researchers have contributed important
hydrological algorithms and modelling components used within Shyft.

Notable contributions include the **SnowTiles algorithm** developed by
**Felix Matt**, and work by **Olga Silantyeva** on **TIN-based hydrological
modelling research** and the integration of the **FSM2 snow model**.
UiO researchers have also actively used Shyft in research and operational
studies, providing valuable feedback that helped shape the evolving
hydrology orchestration framework built on the high-performance C++
core and RocksDB-based services.

This complementary combination of **software architecture, operational
engineering, and hydrological science** formed the foundation of the Shyft
framework.

The project is currently maintained by a group of contributors and maintainers.


Project stewardship and long-term continuity are ensured by:

- **Sigbjørn Helset**, who acts as the **active owner** of the project and maintains the core development and build infrastructure.

Significant development, research collaboration, and operational deployment of Shyft has been supported by:

- **Statkraft AS**
- **University of Oslo – Department of Geosciences (UiO)**
- **Expert Analytics (XAL)** and its contributors



A complete list of contributors can be found in the **Contributors** section
below and in the Git history.

Copyright (C)

Sigbjørn Helset (SiH)  
John F. Burkhart (JFB)  
Ola Skavhaug (OS)  
Yisak Sultan Abdella (YAS)  
Statkraft AS

---

## Contributors

Project contributors include:

- Sigbjørn Helset 
- Ola Skavhaug 
- John Burkhart 
- Yisak Sultan Abdella 
- Felix Matt 
- Olga Silantyeva 
- Francesc Alted 
- Ludovic Pochon-Guérin 
- Eivind Lycke Melvær 
- Cecilie Granerød 
- Andreas Slyngstad 
- Bernardo de Olivera 
- Diako Darian 
- Eivind Aarnæs 
- Vinzenz Gregor Eck 
- Mindaugas Pivoras 
- Roar Emaus 
- Trygve Bærland 
- Albert O. Nybø 
- Gry M. Tveten 
- Magne Nordaas 
- Christine Schei Liland 
- Jens Askgaard 
- Ibrahim Rahmani 
- Anders Kjeldsen 
- Alexander Becker 
- Sarah Dahmen 
- Fabio Zeiser 
- Stian Angelsen 
- Joseph Bradshaw 
- Ole Andreas Grønn Ramsdal 
- John Eivind Rømma Helset 
- Benedikt Reinartz 
- Paul Christoph Bätzing-Rosenvinge 
- Eilidh Troup 
- Petar Kutlesic 
- Christian Vaugelade-Kilaas 
- Jakob Boye Løland Ørbæk 
- Jiyoung Kim 
- Lars Skodje 
- Ana Budisa 
---

## License

Shyft is released under the **LGPL v3 license**.

See the `LICENSE` file.

---

## Citation

If you use Shyft in hydrology-related research, please cite:

Burkhart, J. F., Matt, F. N., Helset, S., Sultan Abdella, Y., Skavhaug, O., and Silantyeva, O.

*Shyft v4.8: a framework for uncertainty assessment and distributed hydrologic modeling for operational hydrology*

Geosci. Model Dev., 14, 821–842, 2021  
https://doi.org/10.5194/gmd-14-821-2021
        

---

Package metadata

• Total packages: 2
• Total downloads:
  • pypi: 995 last-month
• Total dependent packages: 0 (may contain duplicates)
• Total dependent repositories: 1 (may contain duplicates)
• Total versions: 437
• Total maintainers: 2

Score: 11.902965550075578