msc_pygeoapi
The Meteorological Service of Canada Python API provides up-to-date information on past, present and future weather conditions, climate, water and environmental information over Canada and the world.
https://github.com/eccc-msc/msc-pygeoapi
Category: Sustainable Development
Sub Category: Data Catalogs and Interfaces
Keywords from Contributors
downscale weather wms cmip5 climate cansips cangrd ahccd ogc mapserver
Last synced: about 5 hours ago
JSON representation
Repository metadata
MSC OGC API pygeoapi deployment
- Host: GitHub
- URL: https://github.com/eccc-msc/msc-pygeoapi
- Owner: ECCC-MSC
- License: mit
- Created: 2020-04-20T16:00:52.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2025-04-22T14:45:27.000Z (8 days ago)
- Last Synced: 2025-04-24T13:40:24.074Z (6 days ago)
- Language: Python
- Size: 1.74 MB
- Stars: 19
- Watchers: 10
- Forks: 27
- Open Issues: 3
- Releases: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
README.md
msc_pygeoapi
Overview
MSC GeoMet pygeoapi server configuration and utilities
Installation
Requirements
- Python 3
- virtualenv
Dependencies
Dependencies are listed in requirements.txt. Dependencies
are automatically installed during msc-pygeoapi installation.
Dependencies of note:
- Elasticsearch (5 or above)
- i.e.
sudo echo
deb https://artifacts.elastic.co/packages/5.x/apt stable main> /etc/apt/sources.list.d/elastic.list
- pygeoapi
Installing msc-pygeoapi
# setup virtualenv
python3 -m venv --system-site-packages msc-pygeoapi
cd msc-pygeoapi
source bin/activate
# clone codebase
git clone https://github.com/ECCC-MSC/msc-pygeoapi.git
cd msc-pygeoapi
# add GCWeb theme files
curl -L https://github.com/wet-boew/GCWeb/releases/download/v14.6.0/themes-dist-14.6.0-gcweb.1.zip -o ./themes-gcweb.zip
unzip -o ./themes-gcweb.zip "*/GCWeb/*" -d theme/static
unzip -o ./themes-gcweb.zip "*/wet-boew/*" -d theme/static
mv ./theme/static/themes-dist-14.6.0-gcweb ./theme/static/themes-gcweb
rm -f ./themes-gcweb.zip
# install codebase
python setup.py build
python setup.py install
# configure environment
cp msc-pygeoapi.env dev.env
vi dev.env # edit paths accordingly
. dev.env
# serve API
pygeoapi serve
Running
msc-pygeoapi --version
Server will be located at http://localhost/features
Sample Queries
Hydrometric features (Water Level and Flow)
# all collections
http://localhost/features/collections
# hydrometric daily mean
http://localhost/features/collections/hydrometric-daily-mean
# filter by time
http://localhost/features/collections/hydrometric-daily-mean/items?time=2011-11-11/2012-11-11
# filter by bbox
http://localhost/features/collections/hydrometric-daily-mean/items?bbox=-80,45,-50,55
# filter by station number
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066
# filter by bbox
http://localhost/features/collections/hydrometric-daily-mean/items?bbox=-80,40,-50,54
# filter by station number and time
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066&time=2011-01-01/2012-12-31
# filter by station number and time, limit results
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066&time=2011-01-01/2012-12-31&limit=100
# filter by station number and time, limit and page results
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066&time=2011-01-01/2012-12-31&limit=100&offset=100
# HTML respsonses
# filter by active stations in Nunavut
http://localhost/features/collections/hydrometric-stations/items?STATUS_EN=Active&limit=5000&f=html&PROV_TERR_STATE_LOC=NU
Running the loaders
pip install -r requirements-oracle.txt
# view all data loaders available
msc-pygeoapi data
# examples for some loaders
msc-pygeoapi data hydat <rest of flags/parameters>
msc-pygeoapi data climate-archive <rest of flags/parameters>
msc-pygeoapi data ahccd_cmip5 <rest of flags/parameters>
msc-pygeoapi data marine-weather add -d <path_to_directory of XML files>
# bulletins - delete index
msc-pygeoapi data bulletins_realtime delete-index # use --yes flag to bypass prompt
# realtime - standard workflow
msc-pygeoapi data hydrometric-realtime cache-stations # download stations list to $MSC_PYGEOAPI_CACHEDIR location
sr_subscribe start deploy/default/sarracenia/hydrometric_realtime.conf # begin realtime update process
msc-pygeoapi data hydrometric-realtime clean-indexes --days 30 # use --yes flag to bypass prompt (usually in crontab)
Running processes
# run the CCCS Raster drill process (returns GeoJSON by default)
msc-pygeoapi process cccs execute raster-drill --y=45 --x=-75 --layer=CMIP5.SFCWIND.HISTO.WINTER.ABS_PCTL95
# run the CCCS Raster drill process returning CSV
msc-pygeoapi process cccs execute raster-drill --y=45 --x=-75 --layer=CMIP5.SFCWIND.HISTO.WINTER.ABS_PCTL95 --format=CSV
Development
Running Tests
# install dev requirements
pip install -r requirements-dev.txt
# API tests run against http://localhost:5000
# use --url to override
# run all tests
pytest
# run one test file
pytest test/test_hydat.py
# override endpoint
pytest test/test_hydat.py --url https://example.org/dev
# skip API tests (run only unit tests)
pytest -k 'not api'
Multilingual Updates
# Extract from latest code the keys to be translated
pybabel extract -F babel-mapping.ini -o locale/messages.pot ./
# Update the existing .po language file with new/updated keys:
pybabel update -d locale -l fr -i locale/messages.pot
# Open the relevant .po file and contribute your translations
vi locale/fr/LC_MESSAGES/messages.po
# Then compile a .mo file to be used by the application
pybabel compile -d locale -l fr
Releasing
python setup.py sdist bdist_wheel --universal
twine upload dist/*
Code Conventions
Bugs and Issues
All bugs, enhancements and issues are managed on GitHub.
Contact
Owner metadata
- Name: Meteorological Service of Canada
- Login: ECCC-MSC
- Email:
- Kind: organization
- Description:
- Website: https://www.ec.gc.ca/meteo-weather
- Location: Canada
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/18532679?v=4
- Repositories: 12
- Last ynced at: 2023-03-01T04:15:20.833Z
- Profile URL: https://github.com/ECCC-MSC
GitHub Events
Total
- Watch event: 3
- Delete event: 12
- Issue comment event: 15
- Push event: 75
- Pull request event: 93
- Pull request review event: 21
- Pull request review comment event: 31
- Fork event: 2
- Create event: 18
Last Year
- Watch event: 3
- Delete event: 12
- Issue comment event: 15
- Push event: 75
- Pull request event: 93
- Pull request review event: 21
- Pull request review comment event: 31
- Fork event: 2
- Create event: 18
Committers metadata
Last synced: 1 day ago
Total Commits: 634
Total Committers: 27
Avg Commits per committer: 23.481
Development Distribution Score (DDS): 0.516
Commits in past year: 87
Committers in past year: 12
Avg Commits per committer in past year: 7.25
Development Distribution Score (DDS) in past year: 0.69
Name | Commits | |
---|---|---|
Tom Kralidis | t****s@g****m | 307 |
Louis-Philippe Rousseau Lambert | 1****P | 103 |
Etienne Pelletier | e****r@c****a | 71 |
Kevin Ngai | k****i | 44 |
Louis-Philippe Rousseau Lambert | l****t@c****a | 23 |
adanb13 | 8****3 | 14 |
Alex Hurka | a****a@c****a | 12 |
Bob Du | d****b@g****a | 10 |
felixlaframboise | f****e@c****a | 5 |
kngai | a****n@g****a | 5 |
Geomet user for nightly docker builds | g****y@g****a | 4 |
westhav3 | 3****t | 4 |
PhilippeTh | 9****h | 4 |
Ines Bosch-Alfonso | 6****h | 4 |
Pelletier | E****r@e****a | 4 |
Adam Capparelli | c****a@g****a | 3 |
cpomer10 | 4****0 | 3 |
BrunoFang | b****g@e****a | 2 |
Alexandre Leroux | a****x@c****a | 2 |
Mzafar2 | m****r@e****a | 2 |
tom-cooney | 3****y | 2 |
Samanta-A. Emes-Lapointe | s****e@e****a | 1 |
Bob Du | b****u@c****a | 1 |
Kevin Ngai | k****i@u****b | 1 |
Kevin Ngai | n****k@g****a | 1 |
Munim Adil | a****m@i****a | 1 |
Alexandre Leroux | a****x@e****a | 1 |
Committer domains:
- canada.ca: 6
- ec.gc.ca: 5
- geomet-dev-02.cmc.ec.gc.ca: 2
- infopool-dev1.cmc.ec.gc.ca: 1
- geomet-dev-22.cmc.ec.gc.ca: 1
- users.noreply.github: 1
- geomet-dev-31.edc-mtl.ec.gc.ca: 1
- geomet-dev-03.cmc.ec.gc.ca: 1
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 5
Total pull requests: 236
Average time to close issues: 9 months
Average time to close pull requests: 12 days
Total issue authors: 4
Total pull request authors: 12
Average comments per issue: 2.4
Average comments per pull request: 0.2
Merged pull request: 222
Bot issues: 0
Bot pull requests: 0
Past year issues: 0
Past year pull requests: 84
Past year average time to close issues: N/A
Past year average time to close pull requests: 7 days
Past year issue authors: 0
Past year pull request authors: 9
Past year average comments per issue: 0
Past year average comments per pull request: 0.29
Past year merged pull request: 77
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- BrunoFang (2)
- tristan-morris (1)
- tomkralidis (1)
- bo-lu (1)
Top Pull Request Authors
- Dukestep (78)
- RousseauLambertLP (77)
- kngai (29)
- tomkralidis (27)
- adanb13 (15)
- BrunoFang (2)
- ndiondegodez (2)
- Mzafar2 (2)
- gdecourval (1)
- saemeslapointe (1)
- moonscape09 (1)
- PhilippeTh (1)
Top Issue Labels
- enhancement (1)
- questions (1)
Top Pull Request Labels
- backport required (4)
- bug (2)
- on hold (2)
Dependencies
- flake8 *
- pytest *
- cx_Oracle *
- click <8
- elasticsearch <8
- gdal *
- lxml *
- parse *
- pyproj *
- python-slugify *
- pyyaml *
- rasterio *
- requests *
- sqlalchemy *
- unicodecsv *
- xarray ==0.10.2
- actions/checkout v2 composite
- actions/setup-python v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- ubuntu focal build
- eccc-msc/msc-pygeoapi nightly
Score: 6.386879319362645