Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

A curated list of open technology projects to sustain a stable climate, energy supply, biodiversity and natural resources.

HydroShare

A collaborative website for better access to data and models in the hydrologic sciences.
https://github.com/hydroshare/hydroshare

Category: Hydrosphere
Sub Category: Freshwater and Hydrology

Keywords

collaboration django django-rest-framework docker hydro hydrologic-database hydrologic-modeling hydrologic-networks hydrologic-sciences hydrology hydrology-stormwater-analysis hydroshare irods javascript nginx postgresql python solr

Keywords from Contributors

mezzanine transforms measur archiving compose animals optimize humanities generic conversion

Last synced: about 16 hours ago
JSON representation

Repository metadata

HydroShare is a collaborative website for better access to data and models in the hydrologic sciences.

README.md

HydroShare (hydroshare)

HydroShare is a website and hydrologic information system for sharing hydrologic data and models aimed at giving users the cyberinfrastructure needed to innovate and collaborate in research to solve water problems. HydroShare is designed to advance hydrologic science by enabling the scientific community to more easily and freely share products resulting from their research, not just the scientific publication summarizing a study, but also the data and models used to create the scientific publication. With HydroShare users can: (1) share data and models with colleagues; (2) manage who has access to shared content; (3) share, access, visualize and manipulate a broad set of hydrologic data types and models; (4) use the web services API to program automated and client access; (5) publish data and models to meet the requirements of research project data management plans; (6) discover and access data and models published by others; and (7) use web apps to visualize, analyze, and run models on data in HydroShare.

Install

Prerequisites

Supported OS (developer laptops): macOS 10.12+, Win10+ Pro, Ent, Edu, Acad Pro, Acad Ent, CentOS 7 and Ubuntu/Lubuntu 18+ LTS

We got some troubles with Lubuntu 16.04 LTS so probably Ubuntu 16.04 LTS also does not work

Familiarity with docker and git are required to work with HydroShare

Some VM skills such as network settings (Bridge/NAT/Host only) and file sharing are needed if you work with a virtual machine.

For Windows, this link is required to proceed - https://docs.google.com/document/d/1wIQEYq3OkWmzPTHeyGyjXLZWrinEXojJPBTJq7fczL8/edit#heading=h.mfmd8m9mxvsl

One-Time Install

Tables are provided (in Courier font) throughout this wiki for copy-paste of entire blocks.

  1. Open a terminal (macOS, Linux) or command prompt (Windows)
    Navigate to where you will store the source code, for example /Users/yourname/repo/

Typically you will find it under this directory:

cd ~/repo
  1. Get code

Note it should have a default branch set to the develop branch

git clone https://github.com/hydroshare/hydroshare.git

git checkout <branch>

To get current solr revision fixes:

  a. git pull

cd hydroshare

b. docker exec -u hydro-service -ti hydroshare python manage.py solr_update

It’s very important that please DO NOT change the directory name after cloned. Let it be “hydroshare”
If you are running inside a virtual machine such as HydroDev Ubuntu 18.04 from here, you need to:

  1. Log into Docker via application and command line.
    Command line:

    docker login

    You will be asked to enter your username and password

  2. Launch the stack

     ./local-dev-first-start-only.sh

Following the screen instruction to continue.

Run the following command on completion to launch Hydroshare:

docker-compose -f local-dev.yml up

  1. Sanity Checks

    Some WARNINGs are normal.

    HydroShare is available in your browser at https://localhost

    The default admin page is https://localhost/admin

    The default admin account is admin:default

    Swagger API docs https://localhost/hsapi/

  2. Start & Stop & Log

To start HydroShare, only need to open a windows shell, change to HydroShare code directory then run

docker-compose -f local-dev.yml (up | down) [-d] [--build]

Note bracketed -d for daemon is optional and you don’t paste in the brackets

Use -d option in case you want to type new command on this windows or don’t want to see real-time output log.

Use --build option in case docker keeps image in cache and does not update correctly while modifying the Dockerfile and working with PyCharm

CREATE NEW ACCOUNT - This is the same as it's always been in HydroShare. Ask a teammate or hack at it. Basically open a hydroshare console window then use the UI to sign up for a new account and watch the hydroshare container console (docker logs hydroshare) for a verification link and paste that into your browser and save the new account in the UI.

To stop HydroShare, only need to close the running windows or open a new windows then run

docker-compose -f local-dev.yml down

All data is persisted for the next start.

To see the logs in case you start with -d option, open a windows then run

docker-compose -f local-dev.yml logs

Or

docker logs <container name>

Branching
When you activate a new branch, just bring the stack down and up again. Sometimes you can get away with a warm restart of the stack or even relying on the Django debug mode (doing nothing but waiting).

Usage

For all intents and purposes, Hydroshare is a large Python/Django application with some extra features and technologies added on:

  • SOLR for searching
  • Redis for caching
  • RabbitMQ for concurrency and serialization
  • Minio for a S3 file system
  • PostgreSQL for the database backend

The hsctl Script

The hsctl script is your primary tool in interacting with and running tasks against your Hydroshare install. It has the syntax ./hsccl [command] where [command] is one of:

  • loaddb: Deletes existing database and reloads the database specified in the hydroshare-config.yaml file.
  • managepy [args]: Executes a python manage.py [args] call on the running hydroshare container.
  • rebuild: Stops, removes and deletes only the hydroshare docker containers and images while retaining the database contents on the subsequent build as defined in the hydroshare-config.yaml file
  • rebuild --db: Fully stops, removes and deletes any prior hydroshare docker containers, images and database contents prior to installing a clean copy of the hydroshare codebase as defined in the hydroshare-config.yaml file.
  • rebuild_index: Rebuilds the solr/haystack index in a non-interactive way.
  • restart: Restarts the django server only.
  • start: Starts all containers as defined in the docker-compose.yml file.
  • stop: Stops all containers as defined in the docker-compose.yml file.
  • update_index: Updates the solr/haystack index in a non-interactive way.

Testing and Debugging

Testing

Tests are run via normal Django tools and conventions. However, you should use the hsctl script mentioned abouve with the managepy command. For example: ./hsctl managepy test hs_core.tests.api.rest.test_resmap --keepdb.

There are currently over 600 tests in the system, so it is highly recommended that you run the test suites separately from one another.

Debugging

You can debug via PyCharm by following the instructions here.

Other Configuration Options

Contribute

There are many ways to contribute to Hydroshare. Review Contributing guidelines and github practices for information on

  1. Opening issues for any bugs you find or suggestions you may have
  2. Developing code to contribute to HydroShare
  3. Developing a HydroShare App
  4. Submiting pull requests with code changes for review

License

Hydroshare is released under the BSD 3-Clause License. This means that you can do what you want, so long as you don't mess with the trademark, and as long as you keep the license with the source code.

©2017 CUAHSI. This material is based upon work supported by the National Science Foundation (NSF) under awards 1148453, 1148090, 1664061, 1664018, 1664119, 1338606, and 1849458. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: 4 days ago

Total Commits: 15,168
Total Committers: 70
Avg Commits per committer: 216.686
Development Distribution Score (DDS): 0.772

Commits in past year: 822
Committers in past year: 10
Avg Commits per committer in past year: 82.2
Development Distribution Score (DDS) in past year: 0.477

Name Email Commits
pkdash p****a@h****m 3451
mauriel133 m****z@g****m 1901
Devin Cowan d****n@c****g 1776
Scott Black s****k@u****u 1259
hyi h****i@r****g 1190
jeff j****2@g****m 984
Alva Couch a****h@c****g 911
Mark OBrien m****y@n****m 360
zhiyuli z****4@g****m 355
Michael J. Stealey m****y@g****m 346
Tony Castronova c****y@g****m 282
Brian Miles s****b@g****m 281
mohamedmorsy m****r@g****m 262
xuezhaokun h****k@g****m 254
Mark Henderson h****k@g****m 191
gantian127 g****o@y****n 180
Mark Henderson m****n@c****g 172
Mark O'Brien m****n@c****g 156
Calvin Spealman c****n@c****m 110
gantian127 j****7@f****m 106
MSDrao m****5@g****m 81
Alex Lemann l****l@c****m 46
Jeff Sadler J****r 43
Michael J. Stealey s****y@d****g 42
dependabot[bot] 4****] 42
jamy g****7@g****m 38
engineSound h****g@c****g 33
Ken Lippold k****d@i****m 32
castronova t****a@u****u 28
Shawn Crawley s****y@g****m 27
and 40 more...

Committer domains:

Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 3,223
Total pull requests: 2,774
Average time to close issues: 9 months
Average time to close pull requests: 14 days
Total issue authors: 103
Total pull request authors: 49
Average comments per issue: 3.88
Average comments per pull request: 3.01
Merged pull request: 2,225
Bot issues: 0
Bot pull requests: 68

Past year issues: 219
Past year pull requests: 290
Past year average time to close issues: about 2 months
Past year average time to close pull requests: 11 days
Past year issue authors: 13
Past year pull request authors: 10
Past year average comments per issue: 1.56
Past year average comments per pull request: 1.07
Past year merged pull request: 161
Past year bot issues: 0
Past year bot pull requests: 29

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/hydroshare/hydroshare

Top Issue Authors

  • dtarb (445)
  • devincowan (339)
  • pkdash (234)
  • hydrocheck (202)
  • sblack-usu (193)
  • alvacouch (189)
  • horsburgh (129)
  • mjstealey (116)
  • Castronova (103)
  • zhiyuli (99)
  • hyi (97)
  • ndebuhr (74)
  • ghost (74)
  • Maurier (69)
  • gantian127 (62)

Top Pull Request Authors

  • devincowan (624)
  • sblack-usu (399)
  • pkdash (352)
  • hyi (243)
  • Maurier (212)
  • mjstealey (153)
  • alvacouch (122)
  • aphelionz (102)
  • dependabot[bot] (68)
  • ghost (65)
  • zhiyuli (56)
  • gantian127 (51)
  • jsadler2 (35)
  • engineSound (29)
  • selimnairb (26)

Top Issue Labels

  • bug (885)
  • enhancement (284)
  • Discover (214)
  • Resource Landing Page (185)
  • Pages and General HS Components (159)
  • Resource Model (137)
  • REST API (131)
  • design phase needed (114)
  • Web App Connector (108)
  • Performance (79)
  • Model Aggregation (78)
  • iRODS (76)
  • good-first-issue (73)
  • DevOps (70)
  • Access Control (65)
  • Cosmetic (63)
  • File System (61)
  • Publishing (59)
  • Groups (59)
  • Documentation (56)
  • Release Checklist (53)
  • HS Core - Backend Services (44)
  • Code Quality (40)
  • User Profile (38)
  • security (38)
  • Reporting and Metrics (35)
  • Asynchronous tasks (35)
  • Collaboration (35)
  • Critical (31)
  • cloud (27)

Top Pull Request Labels

  • dependencies (60)
  • bug (51)
  • Discover (29)
  • Access Control (20)
  • iRODS (19)
  • Communities (17)
  • Performance (10)
  • Groups (9)
  • REST API (9)
  • DevOps (6)
  • File System (6)
  • Collaboration (4)
  • Reporting and Metrics (3)
  • Resource Model (3)
  • Resource Landing Page (3)
  • Asynchronous tasks (2)
  • 3rd Party File Exchange (2)
  • Code Quality (2)
  • Data Discovery (2)
  • Indexing (2)
  • hotfix (1)
  • javascript (1)
  • Model Aggregation (1)
  • Web App Connector (1)
  • Documentation (1)
  • Cosmetic (1)
  • Publishing (1)
  • enhancement (1)

Package metadata

pypi.org: hydroshare

HydroShare is an online, collaborative system for open sharing of hydrologic data, analytical tools, and models.

  • Homepage: https://www.hydroshare.org/
  • Documentation: https://hydroshare.readthedocs.io/
  • Licenses: BSD 3-Clause License
  • Latest release: 1.9.6 (published over 1 year ago)
  • Last Synced: 2025-04-25T14:36:30.887Z (1 day ago)
  • Versions: 3
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Rankings:
    • Dependent packages count: 9.995%
    • Average: 15.861%
    • Dependent repos count: 21.728%
  • Maintainers (1)

Dependencies

Dockerfile docker
  • hydroshare/hs_docker_base 2.2.10 build

Score: 11.066325813503509