msc_pygeoapi

The Meteorological Service of Canada Python API provides up-to-date information on past, present and future weather conditions, climate, water and environmental information over Canada and the world.
https://github.com/eccc-msc/msc-pygeoapi

Category: Sustainable Development
Sub Category: Data Catalogs and Interfaces

Keywords from Contributors

downscale weather wms cmip5 climate cansips cangrd ahccd ogc mapserver

Last synced: about 5 hours ago
JSON representation

Repository metadata

MSC OGC API pygeoapi deployment

README.md

Build Status
Build Status

msc_pygeoapi

Overview

MSC GeoMet pygeoapi server configuration and utilities

Installation

Requirements

Dependencies

Dependencies are listed in requirements.txt. Dependencies
are automatically installed during msc-pygeoapi installation.

Dependencies of note:

Installing msc-pygeoapi

# setup virtualenv
python3 -m venv --system-site-packages msc-pygeoapi
cd msc-pygeoapi
source bin/activate

# clone codebase
git clone https://github.com/ECCC-MSC/msc-pygeoapi.git
cd msc-pygeoapi

# add GCWeb theme files
curl -L https://github.com/wet-boew/GCWeb/releases/download/v14.6.0/themes-dist-14.6.0-gcweb.1.zip -o ./themes-gcweb.zip 
unzip -o ./themes-gcweb.zip "*/GCWeb/*" -d theme/static
unzip -o ./themes-gcweb.zip "*/wet-boew/*" -d theme/static
mv ./theme/static/themes-dist-14.6.0-gcweb ./theme/static/themes-gcweb
rm -f ./themes-gcweb.zip

# install codebase
python setup.py build
python setup.py install

# configure environment
cp msc-pygeoapi.env dev.env
vi dev.env # edit paths accordingly
. dev.env

# serve API
pygeoapi serve

Running

msc-pygeoapi --version

Server will be located at http://localhost/features

Sample Queries

Hydrometric features (Water Level and Flow)


# all collections
http://localhost/features/collections

# hydrometric daily mean
http://localhost/features/collections/hydrometric-daily-mean

# filter by time
http://localhost/features/collections/hydrometric-daily-mean/items?time=2011-11-11/2012-11-11

# filter by bbox
http://localhost/features/collections/hydrometric-daily-mean/items?bbox=-80,45,-50,55

# filter by station number
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066

# filter by bbox
http://localhost/features/collections/hydrometric-daily-mean/items?bbox=-80,40,-50,54

# filter by station number and time
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066&time=2011-01-01/2012-12-31

# filter by station number and time, limit results
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066&time=2011-01-01/2012-12-31&limit=100

# filter by station number and time, limit and page results
http://localhost/features/collections/hydrometric-daily-mean/items?STATION_NUMBER=02RH066&time=2011-01-01/2012-12-31&limit=100&offset=100

# HTML respsonses
# filter by active stations in Nunavut
http://localhost/features/collections/hydrometric-stations/items?STATUS_EN=Active&limit=5000&f=html&PROV_TERR_STATE_LOC=NU

Running the loaders

pip install -r requirements-oracle.txt

# view all data loaders available
msc-pygeoapi data

# examples for some loaders
msc-pygeoapi data hydat <rest of flags/parameters>
msc-pygeoapi data climate-archive <rest of flags/parameters>
msc-pygeoapi data ahccd_cmip5 <rest of flags/parameters>
msc-pygeoapi data marine-weather add -d <path_to_directory of XML files>

# bulletins - delete index
msc-pygeoapi data bulletins_realtime delete-index  # use --yes flag to bypass prompt

# realtime - standard workflow
msc-pygeoapi data hydrometric-realtime cache-stations  # download stations list to $MSC_PYGEOAPI_CACHEDIR location

sr_subscribe start deploy/default/sarracenia/hydrometric_realtime.conf  # begin realtime update process

msc-pygeoapi data hydrometric-realtime clean-indexes --days 30  # use --yes flag to bypass prompt (usually in crontab)

Running processes


# run the CCCS Raster drill process (returns GeoJSON by default)
msc-pygeoapi process cccs execute raster-drill --y=45 --x=-75 --layer=CMIP5.SFCWIND.HISTO.WINTER.ABS_PCTL95

# run the CCCS Raster drill process returning CSV
msc-pygeoapi process cccs execute raster-drill --y=45 --x=-75 --layer=CMIP5.SFCWIND.HISTO.WINTER.ABS_PCTL95 --format=CSV

Development

Running Tests

# install dev requirements
pip install -r requirements-dev.txt

# API tests run against http://localhost:5000
# use --url to override

# run all tests
pytest

# run one test file
pytest test/test_hydat.py

# override endpoint
pytest test/test_hydat.py --url https://example.org/dev

# skip API tests (run only unit tests)
pytest -k 'not api'

Multilingual Updates

# Extract from latest code the keys to be translated
pybabel extract -F babel-mapping.ini -o locale/messages.pot ./

# Update the existing .po language file with new/updated keys:
pybabel update -d locale -l fr -i locale/messages.pot

# Open the relevant .po file and contribute your translations
vi locale/fr/LC_MESSAGES/messages.po

# Then compile a .mo file to be used by the application
pybabel compile -d locale -l fr

Releasing

python setup.py sdist bdist_wheel --universal
twine upload dist/*

Code Conventions

Bugs and Issues

All bugs, enhancements and issues are managed on GitHub.

Contact


Owner metadata


GitHub Events

Total
Last Year

Committers metadata

Last synced: 1 day ago

Total Commits: 634
Total Committers: 27
Avg Commits per committer: 23.481
Development Distribution Score (DDS): 0.516

Commits in past year: 87
Committers in past year: 12
Avg Commits per committer in past year: 7.25
Development Distribution Score (DDS) in past year: 0.69

Name Email Commits
Tom Kralidis t****s@g****m 307
Louis-Philippe Rousseau Lambert 1****P 103
Etienne Pelletier e****r@c****a 71
Kevin Ngai k****i 44
Louis-Philippe Rousseau Lambert l****t@c****a 23
adanb13 8****3 14
Alex Hurka a****a@c****a 12
Bob Du d****b@g****a 10
felixlaframboise f****e@c****a 5
kngai a****n@g****a 5
Geomet user for nightly docker builds g****y@g****a 4
westhav3 3****t 4
PhilippeTh 9****h 4
Ines Bosch-Alfonso 6****h 4
Pelletier E****r@e****a 4
Adam Capparelli c****a@g****a 3
cpomer10 4****0 3
BrunoFang b****g@e****a 2
Alexandre Leroux a****x@c****a 2
Mzafar2 m****r@e****a 2
tom-cooney 3****y 2
Samanta-A. Emes-Lapointe s****e@e****a 1
Bob Du b****u@c****a 1
Kevin Ngai k****i@u****b 1
Kevin Ngai n****k@g****a 1
Munim Adil a****m@i****a 1
Alexandre Leroux a****x@e****a 1

Committer domains:


Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 5
Total pull requests: 236
Average time to close issues: 9 months
Average time to close pull requests: 12 days
Total issue authors: 4
Total pull request authors: 12
Average comments per issue: 2.4
Average comments per pull request: 0.2
Merged pull request: 222
Bot issues: 0
Bot pull requests: 0

Past year issues: 0
Past year pull requests: 84
Past year average time to close issues: N/A
Past year average time to close pull requests: 7 days
Past year issue authors: 0
Past year pull request authors: 9
Past year average comments per issue: 0
Past year average comments per pull request: 0.29
Past year merged pull request: 77
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/eccc-msc/msc-pygeoapi

Top Issue Authors

  • BrunoFang (2)
  • tristan-morris (1)
  • tomkralidis (1)
  • bo-lu (1)

Top Pull Request Authors

  • Dukestep (78)
  • RousseauLambertLP (77)
  • kngai (29)
  • tomkralidis (27)
  • adanb13 (15)
  • BrunoFang (2)
  • ndiondegodez (2)
  • Mzafar2 (2)
  • gdecourval (1)
  • saemeslapointe (1)
  • moonscape09 (1)
  • PhilippeTh (1)

Top Issue Labels

  • enhancement (1)
  • questions (1)

Top Pull Request Labels

  • backport required (4)
  • bug (2)
  • on hold (2)

Dependencies

requirements-dev.txt pypi
  • flake8 *
  • pytest *
requirements-oracle.txt pypi
  • cx_Oracle *
requirements.txt pypi
  • click <8
  • elasticsearch <8
  • gdal *
  • lxml *
  • parse *
  • pyproj *
  • python-slugify *
  • pyyaml *
  • rasterio *
  • requests *
  • sqlalchemy *
  • unicodecsv *
  • xarray ==0.10.2
.github/workflows/flake8.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
.github/workflows/main.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
Dockerfile docker
  • ubuntu focal build
docker/docker-compose.override.yml docker
docker/docker-compose.yml docker
  • eccc-msc/msc-pygeoapi nightly
setup.py pypi

Score: 6.386879319362645