dsgrid
Enables the compilation of high-resolution load datasets suitable for forward-looking power system and other analyses.
https://github.com/dsgrid/dsgrid
Category: Energy Systems
Sub Category: Load and Demand Forecasting
Keywords
apache-spark electricity-load energy-data energy-demand energy-demand-forecasting python
Last synced: about 18 hours ago
JSON representation
Repository metadata
Python package for working with demand-side grid projects, datasets and queries
- Host: GitHub
- URL: https://github.com/dsgrid/dsgrid
- Owner: dsgrid
- License: bsd-3-clause
- Created: 2020-12-09T23:51:36.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2025-04-11T22:03:01.000Z (16 days ago)
- Last Synced: 2025-04-18T00:07:55.802Z (10 days ago)
- Topics: apache-spark, electricity-load, energy-data, energy-demand, energy-demand-forecasting, python
- Language: Python
- Homepage: https://dsgrid.github.io/dsgrid/
- Size: 8.82 MB
- Stars: 27
- Watchers: 5
- Forks: 4
- Open Issues: 55
- Releases: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
README.md
dsgrid
Python API for contributing to and accessing demand-side grid model (dsgrid) projects and datasets.
⚠️ dsgrid is under active development and does not yet have a formal package release. Details listed here are subject to change. Please reach out to the dsgrid coordination team with any questions or other feedback. ⚠️
Install
Virtual environment | Dependencies | from PIPY/pip | from pip+git | from cloned repository
Virtual environment
Create a virtual environment in which to install dsgrid. Anaconda or miniconda is recommended.
conda create -n dsgrid python=3.10
conda activate dsgrid
Dependencies
dsgrid uses Apache Spark to manage big data. There are no separate installation steps for Apache Spark beyond installing the dsgrid package, because the pyspark Python dependency handles it. However, you should be aware that Apache Spark's Microsoft Windows support is poor and essentially limited to local mode. That is, if you use dsgrid on a Windows machine you should not attempt to install a full version of Spark nor expect to run on a Spark cluster. As such, we recommend limiting dsgrid use on Windows to browsing the registry, registering and submitting small- to medium-sized datasets, or development work with small test projects. Full dsgrid functionality with large projects requires additional computational resources, e.g., high performance or cloud computing, typically on a Linux operating system.
Additional Notes
- If pyspark complains about not finding Python, you may need to locate your python executable file (python.exe on Windows), copy it, and rename the copy to python3 (python3.exe on Windows)
Spark requires Java 8 or later with the JAVA_HOME environment variable set to the Java installation directory.
On Linux you can install OpenJDK with conda:
conda install openjdk
Windows install instructions are below.
Windows
To install Apache Spark on Windows, follow these instructions.
From PIPY/pip
Not yet available
From pip+git
With ssh keys:
pip install git+ssh://[email protected]/dsgrid/dsgrid.git@main
# or
pip install git+ssh://[email protected]/dsgrid/dsgrid.git@develop
From http:
pip install git+https://github.com/dsgrid/dsgrid.git@main
# or
pip install git+https://github.com/dsgrid/dsgrid.git@develop
From Cloned Repository
First, clone the repository and change into the dsgrid directory. For example:
cd ~ # or other directory where you put repositories
git clone [email protected]:dsgrid/dsgrid.git # or the http address
cd dsgrid
Then install the pacakge using the pip -e flag to directly use the files in the
cloned repository.
Users:
pip install -e .
Developers:
pip install -e '.[dev]'
Usage
dsgrid is primarily a command-line interface (CLI) tool. To see the available commands:
dsgrid --help
Uninstall
pip uninstall dsgrid
If you are using a conda environment
conda deactivate
Software Record
dsgrid is developed under NREL Software Record SWR-21-52, "demand-side grid model".
Owner metadata
- Name: NREL-dsgrid
- Login: dsgrid
- Email:
- Kind: organization
- Description:
- Website:
- Location:
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/31113330?v=4
- Repositories: 4
- Last ynced at: 2023-03-04T04:30:37.941Z
- Profile URL: https://github.com/dsgrid
GitHub Events
Total
- Create event: 17
- Issues event: 5
- Release event: 1
- Watch event: 3
- Delete event: 14
- Member event: 3
- Issue comment event: 21
- Push event: 129
- Pull request event: 24
- Pull request review comment event: 102
- Pull request review event: 72
- Fork event: 1
Last Year
- Create event: 17
- Issues event: 5
- Release event: 1
- Watch event: 3
- Delete event: 14
- Member event: 3
- Issue comment event: 21
- Push event: 129
- Pull request event: 24
- Pull request review comment event: 102
- Pull request review event: 72
- Fork event: 1
Committers metadata
Last synced: 7 days ago
Total Commits: 1,424
Total Committers: 6
Avg Commits per committer: 237.333
Development Distribution Score (DDS): 0.363
Commits in past year: 130
Committers in past year: 4
Avg Commits per committer in past year: 32.5
Development Distribution Score (DDS) in past year: 0.323
| Name | Commits | |
|---|---|---|
| Daniel Thom | d****m@n****v | 907 |
| lixiliu | 3****u | 257 |
| mooneyme | m****y@n****v | 177 |
| Elaine Hale | e****e@n****v | 69 |
| zack jensen | z****n@n****v | 7 |
| roliveir | r****a@n****v | 7 |
Committer domains:
- nrel.gov: 5
Issue and Pull Request metadata
Last synced: 2 days ago
Total issues: 133
Total pull requests: 152
Average time to close issues: 9 months
Average time to close pull requests: 16 days
Total issue authors: 5
Total pull request authors: 5
Average comments per issue: 0.83
Average comments per pull request: 1.24
Merged pull request: 143
Bot issues: 0
Bot pull requests: 0
Past year issues: 45
Past year pull requests: 70
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 9 days
Past year issue authors: 5
Past year pull request authors: 4
Past year average comments per issue: 0.38
Past year average comments per pull request: 1.07
Past year merged pull request: 69
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- daniel-thom (78)
- elainethale (30)
- lixiliu (18)
- ashreeta (4)
- jfhawkin (3)
Top Pull Request Authors
- daniel-thom (134)
- lixiliu (10)
- elainethale (6)
- JensZack (1)
- mooneyme (1)
Top Issue Labels
- enhancement (21)
- bug (12)
- limitation (6)
- AWS (4)
- low priority (4)
- future (3)
- documentation (2)
- missing_feature (1)
Top Pull Request Labels
Dependencies
- actions/checkout v2 composite
- actions/setup-python v2 composite
- codecov/codecov-action v3 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- peaceiris/actions-gh-pages v3.6.1 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- psf/black stable composite
- py-actions/flake8 v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- codecov/codecov-action v3 composite
- python 3.10-slim build
Score: 6.198478716492309