Power Station Dictionary
A power station dictionary that enables mapping between various naming conventions and associated plant metadata.
https://github.com/osuked/power-station-dictionary
Category: Energy Systems
Sub Category: Energy Data Accessibility and Integration
Keywords
energy-data power-stations
Last synced: about 7 hours ago
JSON representation
Repository metadata
A power station dictionary that enables mapping between various naming conventions and associated plant metadata
- Host: GitHub
- URL: https://github.com/osuked/power-station-dictionary
- Owner: OSUKED
- License: mit
- Created: 2020-12-17T20:36:14.000Z (over 4 years ago)
- Default Branch: shiro
- Last Pushed: 2025-02-17T22:10:31.000Z (2 months ago)
- Last Synced: 2025-04-17T22:07:12.509Z (13 days ago)
- Topics: energy-data, power-stations
- Language: Jupyter Notebook
- Homepage: https://osuked.github.io/Power-Station-Dictionary/
- Size: 33.4 MB
- Stars: 27
- Watchers: 1
- Forks: 9
- Open Issues: 22
- Releases: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
README.md
Power Station Dictionary
The Power Station Dictionary is a site that enables mapping between various power plant ids and automatically extracts data relating to those plants from Frictionless Data packages.
N.b. This project is currently in active development.
Any and all contributions are very welcome. If you are less comfortable editing via GitHub please contribute via this Google Sheet (GitHub is still preferred if possible).
Motivation
Existing work into increasing the visibility of energy data has focused on improving the ability of humans to find datasets, which has historically been a key issue within a highly fragmented energy data landscape. Groups such as the Energy Data Taskforce have prompted a new wave of metadata standardisation and data cataloguing initiatives which have gone a long way to solving this issue, opening up new opportunities such as the creation of digital twins of the power grid. However, these new opportunities bring new challenges. To enable a digital twin of the energy system we need to be able to "describe relationships between assets and datasets", requiring two core extensions to our existing toolset:
- Field-level metadata that describes the contents of individual columns in a dataset
- "Mapping" datasets that are able to express the relationships between other datasets
These additions enable us to move from improving the ability of humans to discover datasets to making it easier for machines to automatically find and extract relevant data - a need that will only increase as the number and size of datasets continues to grow. The benefits extend beyond our digital colleagues though - by pivoting data exploration to be about finding objects/assets, which then reveal the datasets (and attributes) they are linked to, we can create a more intuitive search experience. Similar to Google’s move from searching for "Things not Strings", the data dictionary lets us search for "Assets not Datasets".
To illustrate the benefits of such a framework we are building a pilot dictionary focused on improving the discoverability, linkage, and automated extraction of data relating to power stations on the GB system. Power stations were chosen due to the high number of datasets they relate to, the wide range of ids used to describe them, and the current duplication in efforts to link them across industry and academia. We will then demonstrate how the dictionary can be used for analysis with two case studies: one researching the carbon intensity of individual generators by matching power output and carbon emission datasets, the second linking wholesale price and renewable subsidy data to help explain why wind subsidies have fallen below the average market price.
Dictionary Framework
Dictionary Schema & Core Dataset
The dictionary is composed of two files, a csv containing ids that relate to different power stations and a json containing metadata written as an extension to the Frictionless Data Tabular schema. "Frictionless Data (FD) is an open-source toolkit that brings simplicity to the data experience" through an open-source standard that defines a specification for describing metadata relating to different types of datasets. Once a dataset has been described using the specification it then becomes incredibly easy to load it using different programming languages as well as export it into a wide range of different formats. What makes FD different to most other specifications is that they provide a comprehensive way to describe individual columns within a dataset, including their formats and constraints.
The majority of the schema is the same as the Tabular Schema published by FD. The core change is the use of foreignKeys
to link to external datasets that use ids specified in the dictionary, a separate attributes
entry then describes the columns which should be extracted from the dataset. The hierarchy
attribute for each column then describes whether the ids in that column have a same-as
or part-of
relationship with the asset they’re linked to. A further url_format
entry then provides a way to convert specific IDs into urls (e.g. with wikidata ids).
The datasets linked to the dictionary must be described using the FD tabular schema, however, the metadata does not need to be stored adjacent to the raw source and could be generated by a third party rather than the original data provider. Data-providers from within the energy sector already using this format include Public Utility Data Library and Open Power System Data. As well as being able to link into the dictionary by publishing your datasets using this standard you can make use of a wider ecosystem of data tools.
Building the Knowledge Graph/Website
Once the dictionary has been created a Python library then uses it to programmatically identify the different assets it contains, and then extract data relating to those assets from the datasets linked to the dictionary. The generation steps are as follows:
- Each row of the dictionary is iterated over with the associated ids extracted for each asset
- The datasets linked to the dictionary which contain an id relating to the current asset are identified
- The relevant attributes for each asset which are contained in the linked datasets are then extracted
- For each asset the outputted linked ids, datasets, and attributes are then used to populate a markdown template which forms the basis of a webpage within the dictionary site
Installation
To install the powerdict
library please run:
pip install powerdict
Development Set-Up
To set-up a new environment you can run the following from the batch_scripts directory
setup_env
Alternatively you can run these commands
call conda env create -f environment.yml
call conda activate PowerDict
call ipython kernel install --user --name=PowerDict
Publishing to PyPi
To publish the powerdict
module to PyPi simply run the following from the batch_scripts directory
pypi_publish
or follow these commands
call conda activate PowerDict
call python setup.py sdist bdist_wheel
call twine upload --skip-existing dist/*
When prompted you should enter your PyPi username and password
After this you will be able to install the latest version of powerdict using pip install powerdict
Owner metadata
- Name: Open Source UK Energy Data
- Login: OSUKED
- Email:
- Kind: organization
- Description: OSUKED is a combination of API wrappers, git-scrapers and Python tools for the UK energy sector
- Website: https://osuked.com/
- Location:
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/75696139?v=4
- Repositories: 8
- Last ynced at: 2023-03-04T14:43:56.649Z
- Profile URL: https://github.com/OSUKED
GitHub Events
Total
- Watch event: 4
- Delete event: 1
- Pull request event: 2
- Fork event: 2
Last Year
- Watch event: 4
- Delete event: 1
- Pull request event: 2
- Fork event: 2
Committers metadata
Last synced: 1 day ago
Total Commits: 212
Total Committers: 3
Avg Commits per committer: 70.667
Development Distribution Score (DDS): 0.024
Commits in past year: 0
Committers in past year: 0
Avg Commits per committer in past year: 0.0
Development Distribution Score (DDS) in past year: 0.0
Name | Commits | |
---|---|---|
AyrtonB | A****n@O****m | 207 |
Aiden Kerr | A****r@e****k | 3 |
jsteinemann | j****n@e****k | 2 |
Committer domains:
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 25
Total pull requests: 7
Average time to close issues: 6 months
Average time to close pull requests: 8 months
Total issue authors: 3
Total pull request authors: 5
Average comments per issue: 0.24
Average comments per pull request: 0.71
Merged pull request: 3
Bot issues: 0
Bot pull requests: 0
Past year issues: 0
Past year pull requests: 1
Past year average time to close issues: N/A
Past year average time to close pull requests: N/A
Past year issue authors: 0
Past year pull request authors: 1
Past year average comments per issue: 0
Past year average comments per pull request: 0.0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- AyrtonB (23)
- Jess-St (1)
- Rabscuttler (1)
Top Pull Request Authors
- Jess-St (3)
- axsy99 (1)
- ClemAtt (1)
- AidenJK (1)
- AyrtonB (1)
Top Issue Labels
- enhancement (5)
- good first issue (2)
Top Pull Request Labels
Package metadata
- Total packages: 1
-
Total downloads:
- pypi: 265 last-month
- Total dependent packages: 0
- Total dependent repositories: 1
- Total versions: 6
- Total maintainers: 1
pypi.org: powerdict
A power station dictionary that enables mapping between various naming conventions and associated plant metadata
- Homepage: https://github.com/OSUKED/Power-Station-Dictionary
- Documentation: https://powerdict.readthedocs.io/
- Licenses: mit
- Latest release: 1.3.0 (published over 4 years ago)
- Last Synced: 2025-04-29T14:30:22.315Z (1 day ago)
- Versions: 6
- Dependent Packages: 0
- Dependent Repositories: 1
- Downloads: 265 Last month
-
Rankings:
- Dependent packages count: 7.382%
- Forks count: 13.359%
- Stargazers count: 14.874%
- Average: 19.886%
- Dependent repos count: 22.248%
- Downloads: 41.565%
- Maintainers (1)
Dependencies
- actions/checkout master composite
- actions/setup-python v1 composite
- marvinpinto/action-automatic-releases latest composite
- pypa/gh-action-pypi-publish master composite
- actions/checkout v2 composite
- actions/setup-python v1 composite
- stefanzweifel/git-auto-commit-action v4 composite
- actions/checkout v2 composite
- actions/setup-python v1 composite
- stefanzweifel/git-auto-commit-action v4 composite
- SwaggerPlugin *
- configparser *
- feautils *
- junix *
- mkdocs-material-extensions *
- mkdocstrings *
- nbdev *
Score: 10.577681245178987