Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

dsgrid

Enables the compilation of high-resolution load datasets suitable for forward-looking power system and other analyses.
https://github.com/dsgrid/dsgrid

Category: Energy Systems
Sub Category: Load and Demand Forecasting

Keywords

apache-spark electricity-load energy-data energy-demand energy-demand-forecasting python

Last synced: about 2 hours ago
JSON representation

Repository metadata

Python package for working with demand-side grid projects, datasets and queries

README.md

dsgrid

Documentation
codecov

Python API for contributing to and accessing demand-side grid model (dsgrid) projects and datasets.

⚠️ dsgrid is under active development and does not yet have a formal package release. Details listed here are subject to change. Please reach out to the dsgrid coordination team with any questions or other feedback. ⚠️

Install | Usage | Uninstall

Install

Virtual environment | Dependencies | from PIPY/pip | from pip+git | from cloned repository

Virtual environment

Create a virtual environment in which to install dsgrid. Anaconda or miniconda is recommended.

conda create -n dsgrid python=3.11
conda activate dsgrid

Dependencies

dsgrid uses Apache Spark to manage big data. There are no separate installation steps for Apache Spark beyond installing the dsgrid package and installing:

pip install "dsgrid-toolkit[spark]"

Otherwise installing the pyspark Python dependency handles it.

However, you should be aware that Apache Spark's Microsoft Windows support is poor and essentially limited to local mode. That is, if you use dsgrid on a Windows machine you should not attempt to install a full version of Spark nor expect to run on a Spark cluster. As such, we recommend limiting dsgrid use on Windows to browsing the registry, registering and submitting small- to medium-sized datasets, or development work with small test projects. Full dsgrid functionality with large projects requires additional computational resources, e.g., high performance or cloud computing, typically on a Linux operating system.

Additional Notes

  • If pyspark complains about not finding Python, you may need to locate your python executable file (python.exe on Windows), copy it, and rename the copy to python3 (python3.exe on Windows)

Spark requires Java 8 or later with the JAVA_HOME environment variable set to the Java installation directory.

On Linux you can install OpenJDK with conda:

conda install openjdk

Windows install instructions are below.

Windows

To install Apache Spark on Windows, follow these instructions.

From PIPY/pip

pip install dsgrid-toolkit

or

pip install "dsgrid-toolkit[spark]"

From pip+git

With ssh keys:

pip install git+ssh://git@github.com/dsgrid/dsgrid.git@main

# or

pip install git+ssh://git@github.com/dsgrid/dsgrid.git@develop

From http:

pip install git+https://github.com/dsgrid/dsgrid.git@main

# or

pip install git+https://github.com/dsgrid/dsgrid.git@develop

From Cloned Repository

First, clone the repository and change into the dsgrid directory. For example:

cd ~                                       # or other directory where you put repositories
git clone git@github.com:dsgrid/dsgrid.git # or the http address
cd dsgrid

Then install the pacakge using the pip -e flag to directly use the files in the
cloned repository.

Users:

pip install -e .

Developers:

pip install -e '.[dev,spark]'

Usage

dsgrid is primarily a command-line interface (CLI) tool. To see the available commands:

dsgrid --help

Uninstall

pip uninstall dsgrid

If you are using a conda environment

conda deactivate

Software Record

dsgrid is developed under NREL Software Record SWR-21-52, "demand-side grid model".

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: 17 days ago

Total Commits: 1,608
Total Committers: 6
Avg Commits per committer: 268.0
Development Distribution Score (DDS): 0.341

Commits in past year: 182
Committers in past year: 3
Avg Commits per committer in past year: 60.667
Development Distribution Score (DDS) in past year: 0.17

Name Email Commits
Daniel Thom d****m@n****v 1060
lixiliu 3****u 270
mooneyme m****y@n****v 177
Elaine Hale e****e@n****v 87
zack jensen z****n@n****v 7
roliveir r****a@n****v 7

Committer domains:

Issue and Pull Request metadata

Last synced: 9 days ago

Total issues: 107
Total pull requests: 167
Average time to close issues: 9 months
Average time to close pull requests: about 1 month
Total issue authors: 5
Total pull request authors: 5
Average comments per issue: 0.83
Average comments per pull request: 1.29
Merged pull request: 145
Bot issues: 0
Bot pull requests: 0

Past year issues: 15
Past year pull requests: 35
Past year average time to close issues: N/A
Past year average time to close pull requests: 9 days
Past year issue authors: 2
Past year pull request authors: 3
Past year average comments per issue: 0.07
Past year average comments per pull request: 1.31
Past year merged pull request: 25
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/dsgrid/dsgrid

Top Issue Authors

  • daniel-thom (68)
  • elainethale (28)
  • lixiliu (8)
  • ashreeta (2)
  • jfhawkin (1)

Top Pull Request Authors

  • daniel-thom (142)
  • lixiliu (14)
  • elainethale (8)
  • JensZack (2)
  • mooneyme (1)

Top Issue Labels

  • enhancement (14)
  • bug (9)
  • limitation (6)
  • low priority (4)
  • future (3)
  • AWS (2)
  • documentation (2)
  • missing_feature (1)

Top Pull Request Labels

Package metadata

proxy.golang.org: github.com/dsgrid/dsgrid

  • Homepage:
  • Documentation: https://pkg.go.dev/github.com/dsgrid/dsgrid#section-documentation
  • Licenses: bsd-3-clause
  • Latest release: v0.2.0 (published 2 months ago)
  • Last Synced: 2025-12-21T22:10:16.149Z (4 days ago)
  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent packages count: 5.401%
    • Average: 5.583%
    • Dependent repos count: 5.764%
pypi.org: dsgrid-toolkit

Python API for accessing demand-side grid model (dsgrid) datasets

  • Homepage: https://www.nrel.gov/analysis/dsgrid
  • Documentation: https://dsgrid.github.io/dsgrid/
  • Licenses: BSD 3-Clause License Copyright (c) 2021, Alliance for Sustainable Energy, LLC All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  • Latest release: 0.2.0 (published 2 months ago)
  • Last Synced: 2025-12-21T22:10:15.529Z (4 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 112 Last month
  • Rankings:
    • Dependent packages count: 8.431%
    • Stargazers count: 13.002%
    • Forks count: 15.474%
    • Average: 21.142%
    • Dependent repos count: 47.66%
  • Maintainers (1)

Dependencies

.github/workflows/codecov.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • codecov/codecov-action v3 composite
.github/workflows/gh-pages.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • peaceiris/actions-gh-pages v3.6.1 composite
.github/workflows/lint.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • psf/black stable composite
  • py-actions/flake8 v2 composite
.github/workflows/pull_request_tests.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
  • codecov/codecov-action v3 composite
docker/spark/Dockerfile docker
  • python 3.10-slim build
pyproject.toml pypi
emr/environment.yml pypi

Score: 11.007783657672535