Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

HydroShare

A collaborative website for better access to data and models in the hydrologic sciences.
https://github.com/hydroshare/hydroshare

Category: Hydrosphere
Sub Category: Freshwater and Hydrology

Keywords

collaboration django django-rest-framework docker hydro hydrologic-database hydrologic-modeling hydrologic-networks hydrologic-sciences hydrology hydrology-stormwater-analysis hydroshare irods javascript nginx postgresql python solr

Keywords from Contributors

mezzanine oceanography

Last synced: about 22 hours ago
JSON representation

Repository metadata

HydroShare is a collaborative website for better access to data and models in the hydrologic sciences.

README.md

HydroShare (hydroshare)

HydroShare is a website and hydrologic information system for sharing hydrologic data and models aimed at giving users the cyberinfrastructure needed to innovate and collaborate in research to solve water problems. HydroShare is designed to advance hydrologic science by enabling the scientific community to more easily and freely share products resulting from their research, not just the scientific publication summarizing a study, but also the data and models used to create the scientific publication. With HydroShare users can: (1) share data and models with colleagues; (2) manage who has access to shared content; (3) share, access, visualize and manipulate a broad set of hydrologic data types and models; (4) use the web services API to program automated and client access; (5) publish data and models to meet the requirements of research project data management plans; (6) discover and access data and models published by others; and (7) use web apps to visualize, analyze, and run models on data in HydroShare.

Install

Prerequisites:

  • Docker
  • Node.js

Supported OS (developer laptops): macOS 10.12+, Win10+ Pro, Ent, Edu, Acad Pro, Acad Ent, CentOS 7 and Ubuntu/Lubuntu 18+ LTS

We got some troubles with Lubuntu 16.04 LTS so probably Ubuntu 16.04 LTS also does not work

Familiarity with docker and git are required to work with HydroShare

disovery-atlas (search) frontend requires Node version specified in that module's package.json.
It's recommended to use Node Version Manager to switch between different versions of node in your local environment.

Some VM skills such as network settings (Bridge/NAT/Host only) and file sharing are needed if you work with a virtual machine.

For Windows, this link is required to proceed - https://docs.google.com/document/d/1wIQEYq3OkWmzPTHeyGyjXLZWrinEXojJPBTJq7fczL8/edit#heading=h.mfmd8m9mxvsl

Getting the source code

  1. Open a terminal (macOS, Linux) or command prompt (Windows)
    Navigate to where you will store the source code, for example /Users/yourname/repo/

  2. Clone repository

Note: the default branch for hydroshare is develop

git clone https://github.com/hydroshare/hydroshare.git

Or if you are using ssh:

git clone git@github.com:hydroshare/hydroshare.git
cd hydroshare

One-time local development setup

  1. Log into Docker:
docker login

(You will be asked to authenticate with Docker.)

  1. Launch the stack
./local-dev-first-start-only.sh --seed-dev-resources

This runs a script that will:

  • Delete all containers, images, and volumes for a clean start
  • Update config files with your user/group IDs
  • Install dependencies (npm, pm2) and build frontend assets
  • Recreate Docker containers and database
  • Run migrations and set up search indexes
  • Seed some Hydroshare resources for the seeded user with username asdf (exclude --seed-dev-resources from the command above to skip this). You can also seed data after the fact by running docker exec hydroshare python manage.py seed_dev_resources --username asdf.
  • Starts the stack defined in the Docker Compose file local-dev.yml

The local-dev-first-start-only.sh will spin up all docker containers in the local-dev.yml. It does NOT spin up a container for Discover -- instead, the script uses PM2 to run the Vite dev server to take advantage of HMR.

Alternatively, to run Discover as a static build inside a local Docker container you can:

  • Uncomment the discovery-atlas service in local-dev.yml
  • Uncomment the line in the nginx service in local-dev.yml to have nginx wait for discovery-atlas to be up
  • Change the location /discover/ proxy_pass entry in nginx/nginx-local-dev.conf to http://discovery-atlas:80/discover/
  1. Sanity Checks and where to view the app and documentation:
  • Some WARNINGs are normal.
  • HydroShare is available in your browser at https://localhost
  • The default admin page is https://localhost/admin
  • The default admin account is admin:default
  • Swagger API docs https://localhost/hsapi/
  1. Start & Stop & Log

To start HydroShare, only need to open a shell, change to HydroShare code directory then run

docker-compose -f local-dev.yml (up | down) [-d] [--build]

Note bracketed -d (run in detached mode) is optional and you don’t paste in the brackets.
Use -d option if you want to run your containers in the background and not see live logs.
Use --build option in case docker keeps image in cache and does not update correctly while modifying the Dockerfile.

To stop HydroShare, only need to close the running windows or open a new windows then run

docker-compose -f local-dev.yml down

All data is persisted for the next start.

To see the logs in case you start with -d option, run

docker-compose -f local-dev.yml logs

Or

docker logs <container name>
  1. Logging in / creating an account

The locally-running app will be populated with a couple accounts:

  • admin (pw: default)
  • asdf (pw: asdf) -- if you ran ./local-dev-first-start-only.sh --seed-dev-resources this user will have some Hydroshare resources created

Or use the following process to create a new account:

Open Hydroshare in your browser and visit the sign-up page. Use the UI to sign up for a new account, then view the hydroshare container logs with

docker logs hydroshare

to get a verification link. Look for

Welcome to HydroShare. This email address was used to request an account on www.hydroshare.org.
If you originated the request, please use the link below to verify your email address and activate your account.

and get the link below that text, paste it into your browser and save the new account in the UI.

Usage

For all intents and purposes, Hydroshare is a large Python/Django application with some extra features and technologies added on:

  • A Vue app for searching, backed by MongoDB Atlas
  • Redis for caching
  • RedPanda for concurrency and serialization
  • Minio for a S3 file system
  • PostgreSQL for the database backend

Makefile Commands

The repository includes a small set of convenience targets in the Makefile.
Run them from the repository root with make <target>.

  • generate-resource-edit-schema-json: Regenerates resource edit schema used in resource landing page for resource metadata editing
    hs_cloudnative_schemas/schema/json_schemas/resource_edit_schema.json
    by running
    hs_cloudnative_schemas/schema/scripts/generate_resource_edit_schema_json.py
    inside the hydroshare container.
  • up-discover: Starts the Discover frontend development server with PM2.
  • down-discover: Stops the Discover frontend PM2 process.
  • logs-discover: Streams logs for the Discover frontend PM2 process.

Testing and Debugging

Testing

Tests are run via normal Django tools and conventions. However, you should use the hsctl script mentioned abouve with the managepy command. For example: ./hsctl managepy test hs_core.tests.api.rest.test_resmap --keepdb.

There are currently over 600 tests in the system, so it is highly recommended that you run the test suites separately from one another.

Debugging

You can debug via PyCharm by following the instructions here.

Other Configuration Options

Contribute

There are many ways to contribute to Hydroshare. Review Contributing guidelines and github practices for information on

  1. Opening issues for any bugs you find or suggestions you may have
  2. Developing code to contribute to HydroShare
  3. Developing a HydroShare App
  4. Submiting pull requests with code changes for review

License

Hydroshare is released under the BSD 3-Clause License. This means that you can do what you want, so long as you don't mess with the trademark, and as long as you keep the license with the source code.

©2017 CUAHSI. This material is based upon work supported by the National Science Foundation (NSF) under awards 1148453, 1148090, 1664061, 1664018, 1664119, 1338606, and 1849458. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: 2 days ago

Total Commits: 16,071
Total Committers: 72
Avg Commits per committer: 223.208
Development Distribution Score (DDS): 0.773

Commits in past year: 647
Committers in past year: 9
Avg Commits per committer in past year: 71.889
Development Distribution Score (DDS) in past year: 0.722

Name Email Commits
pkdash p****a@h****m 3651
Devin Cowan d****n@c****g 2017
Maurier m****z@g****m 1934
Scott Black s****k@u****u 1495
hyi h****i@r****g 1190
jeff j****2@g****m 984
Alva Couch a****h@c****g 911
Mark OBrien m****y@n****m 360
zhiyuli z****4@g****m 355
Michael J. Stealey m****y@g****m 346
Tony Castronova c****y@g****m 282
Brian Miles s****b@g****m 281
mohamedmorsy m****r@g****m 262
xuezhaokun h****k@g****m 254
MSDrao m****5@g****m 216
Mark Henderson h****k@g****m 191
gantian127 g****o@y****n 180
Mark Henderson m****n@c****g 172
Mark O'Brien m****n@c****g 156
Calvin Spealman c****n@c****m 110
gantian127 j****7@f****m 106
dependabot[bot] 4****] 48
Alex Lemann l****l@c****m 46
Jeff Sadler J****r 43
Michael J. Stealey s****y@d****g 42
Callie Porter-Borden c****n@c****g 39
jamy g****7@g****m 38
engineSound h****g@c****g 33
Ken Lippold k****d@i****m 32
castronova t****a@u****u 28
and 42 more...

Committer domains:

Issue and Pull Request metadata

Last synced: 2 days ago

Total issues: 3,288
Total pull requests: 3,213
Average time to close issues: 9 months
Average time to close pull requests: 14 days
Total issue authors: 108
Total pull request authors: 52
Average comments per issue: 3.73
Average comments per pull request: 2.66
Merged pull request: 2,423
Bot issues: 0
Bot pull requests: 105

Past year issues: 92
Past year pull requests: 103
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 9 days
Past year issue authors: 10
Past year pull request authors: 9
Past year average comments per issue: 0.57
Past year average comments per pull request: 0.45
Past year merged pull request: 30
Past year bot issues: 0
Past year bot pull requests: 20

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/hydroshare/hydroshare

Top Issue Authors

  • dtarb (445)
  • devincowan (321)
  • pkdash (236)
  • sblack-usu (222)
  • hydrocheck (204)
  • alvacouch (189)
  • horsburgh (126)
  • mjstealey (116)
  • Castronova (102)
  • zhiyuli (99)
  • hyi (97)
  • ghost (74)
  • ndebuhr (74)
  • Maurier (69)
  • gantian127 (62)

Top Pull Request Authors

  • devincowan (862)
  • sblack-usu (485)
  • pkdash (380)
  • hyi (242)
  • Maurier (211)
  • mjstealey (153)
  • alvacouch (123)
  • dependabot[bot] (105)
  • aphelionz (102)
  • ghost (65)
  • zhiyuli (56)
  • gantian127 (51)
  • MSDrao (48)
  • jsadler2 (35)
  • engineSound (29)

Top Issue Labels

  • bug (868)
  • enhancement (277)
  • Discover (212)
  • Resource Landing Page (182)
  • Pages and General HS Components (158)
  • Resource Model (135)
  • REST API (132)
  • design phase needed (113)
  • Web App Connector (108)
  • Model Aggregation (78)
  • Performance (75)
  • iRODS (68)
  • good-first-issue (65)
  • Access Control (65)
  • Cosmetic (59)
  • DevOps (59)
  • Release Checklist (57)
  • File System (57)
  • Groups (56)
  • Publishing (55)
  • Documentation (54)
  • HS Core - Backend Services (42)
  • Code Quality (39)
  • User Profile (35)
  • security (35)
  • Collaboration (33)
  • Critical (30)
  • Reporting and Metrics (30)
  • Asynchronous tasks (28)
  • Linked Data Services (24)

Top Pull Request Labels

  • dependencies (97)
  • bug (53)
  • Discover (29)
  • javascript (28)
  • Access Control (20)
  • iRODS (19)
  • Communities (17)
  • Performance (11)
  • REST API (9)
  • Groups (9)
  • File System (7)
  • DevOps (6)
  • Resource Landing Page (4)
  • Collaboration (4)
  • Resource Model (3)
  • 3rd Party File Exchange (3)
  • Reporting and Metrics (3)
  • Asynchronous tasks (2)
  • Code Quality (2)
  • Indexing (2)
  • Data Discovery (2)
  • Cosmetic (1)
  • hotfix (1)
  • enhancement (1)
  • Documentation (1)
  • Web App Connector (1)
  • Model Aggregation (1)
  • Publishing (1)

Package metadata

pypi.org: hydroshare

HydroShare is an online, collaborative system for open sharing of hydrologic data, analytical tools, and models.

  • Homepage: https://www.hydroshare.org/
  • Documentation: https://hydroshare.readthedocs.io/
  • Licenses: BSD 3-Clause License
  • Latest release: 1.9.6 (published almost 3 years ago)
  • Last Synced: 2026-08-12T13:05:49.748Z (2 days ago)
  • Versions: 3
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Rankings:
    • Dependent packages count: 9.995%
    • Average: 15.861%
    • Dependent repos count: 21.728%
  • Maintainers (1)

Dependencies

Dockerfile docker
  • hydroshare/hs_docker_base 2.2.10 build

Score: 11.213980200239737