{"id":303797,"name":"WOUDC Data Registry","description":"A platform that manages Ozone and Ultraviolet Radiation data in support of the World Ozone and Ultraviolet Radiation Data Centre (WOUDC), one of six World Data Centres as part of the Global Atmosphere Watch programme of the WMO.","url":"https://github.com/woudc/woudc-data-registry","last_synced_at":"2026-04-16T12:30:19.650Z","repository":{"id":23183130,"uuid":"98365844","full_name":"woudc/woudc-data-registry","owner":"woudc","description":"WOUDC Data Registry is a platform that manages Ozone and Ultraviolet Radiation data in support of the World Ozone and Ultraviolet Radiation Data Centre (WOUDC), one of six World Data Centres as part of the Global Atmosphere Watch programme of the WMO.","archived":false,"fork":false,"pushed_at":"2026-04-02T18:54:19.000Z","size":1094,"stargazers_count":4,"open_issues_count":1,"forks_count":10,"subscribers_count":2,"default_branch":"master","last_synced_at":"2026-04-03T12:46:36.971Z","etag":null,"topics":["gaw","ozone","ozonesonde","spectral","totalozone","ultraviolet","umkehr","uv","wmo"],"latest_commit_sha":null,"homepage":"https://woudc.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/woudc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"docs/support.rst","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-07-26T01:36:59.000Z","updated_at":"2026-04-02T18:54:29.000Z","dependencies_parsed_at":"2026-02-21T10:07:10.494Z","dependency_job_id":null,"html_url":"https://github.com/woudc/woudc-data-registry","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/woudc/woudc-data-registry","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/woudc","download_url":"https://codeload.github.com/woudc/woudc-data-registry/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31427386,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T02:22:46.605Z","status":"ssl_error","status_checked_at":"2026-04-05T02:22:33.263Z","response_time":75,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"owner":{"login":"woudc","name":"World Ozone and Ultraviolet Radiation Data Centre ","uuid":"10194891","kind":"organization","description":"Collaborative software, issue tracker and wiki for WOUDC, one of six World Data Centres as part of the Global Atmosphere Watch programme of the WMO.","email":null,"website":"https://woudc.org","location":"Canada","twitter":null,"company":null,"icon_url":"https://avatars.githubusercontent.com/u/10194891?v=4","repositories_count":10,"last_synced_at":"2024-03-26T09:32:21.289Z","metadata":{"has_sponsors_listing":false},"html_url":"https://github.com/woudc","funding_links":[],"total_stars":19,"followers":0,"following":0,"created_at":"2022-11-03T14:11:16.719Z","updated_at":"2024-03-26T09:32:22.603Z","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/woudc","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/woudc/repositories"},"packages":[],"commits":{"id":5580292,"full_name":"woudc/woudc-data-registry","default_branch":"master","total_commits":334,"total_committers":15,"total_bot_commits":0,"total_bot_committers":0,"mean_commits":22.266666666666666,"dds":0.655688622754491,"past_year_total_commits":64,"past_year_total_committers":6,"past_year_total_bot_commits":0,"past_year_total_bot_committers":0,"past_year_mean_commits":10.666666666666666,"past_year_dds":0.515625,"last_synced_at":"2026-04-15T09:37:40.430Z","last_synced_commit":"a7a35d6d85c7be371f87d0f9cf0267071b271840","created_at":"2024-11-14T00:10:47.421Z","updated_at":"2026-04-15T09:37:29.077Z","committers":[{"name":"Alex Hurka","email":"alexander.hurka@canada.ca","login":null,"count":115},{"name":"Tom Kralidis","email":"tomkralidis@gmail.com","login":"tomkralidis","count":54},{"name":"Kevin Ngai","email":"kngai","login":"kngai","count":50},{"name":"Victoria Rose Spada","email":"83600460+victoriarspada","login":"victoriarspada","count":22},{"name":"danielwaiforssell","email":"51061073+danielwaiforssell","login":"danielwaiforssell","count":22},{"name":"Simran Mattu","email":"mattus@woudc-px-dev3.cmc.ec.gc.ca","login":null,"count":20},{"name":"ahurka","email":"34560347+ahurka","login":"ahurka","count":18},{"name":"Noor","email":"Noor.Al-Duhaidahawi@ec.gc.ca","login":"nalduu","count":12},{"name":"Kevin Ngai","email":"ngaik@woudc-px-dev3.cmc.ec.gc.ca","login":null,"count":10},{"name":"Simran Mattu","email":"77361584+simranmattu14","login":"simranmattu14","count":4},{"name":"Bob Du","email":"bob.du@canada.ca","login":null,"count":3},{"name":"Victoria Spada","email":"victoria.spada@ec.gc.ca","login":null,"count":1},{"name":"Noor Al-Duhaidahawi","email":"alduhaidahawin@woudc-px-dev3.cmc.ec.gc.ca","login":null,"count":1},{"name":"Kevin Ngai","email":"ngaik@woudc-geo-dev4.edc-mtl.ec.gc.ca","login":null,"count":1},{"name":"BobMDu","email":"newbspree@gmail.com","login":null,"count":1}],"past_year_committers":[{"name":"Kevin Ngai","email":"kngai","login":"kngai","count":31},{"name":"Noor","email":"Noor.Al-Duhaidahawi@ec.gc.ca","login":"nalduu","count":12},{"name":"Simran Mattu","email":"mattus@woudc-px-dev3.cmc.ec.gc.ca","login":null,"count":9},{"name":"Tom Kralidis","email":"tomkralidis@gmail.com","login":"tomkralidis","count":7},{"name":"Simran Mattu","email":"77361584+simranmattu14","login":"simranmattu14","count":4},{"name":"Noor Al-Duhaidahawi","email":"alduhaidahawin@woudc-px-dev3.cmc.ec.gc.ca","login":null,"count":1}],"commits_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry/commits","host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-04-15T00:00:09.512Z","repositories_count":6213516,"commits_count":900031220,"contributors_count":34923580,"owners_count":1144558,"icon_url":"https://github.com/github.png","host_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories"}},"issues_stats":{"full_name":"woudc/woudc-data-registry","html_url":"https://github.com/woudc/woudc-data-registry","last_synced_at":"2026-04-03T06:02:20.918Z","status":"active","issues_count":1,"pull_requests_count":152,"avg_time_to_close_issue":16738.0,"avg_time_to_close_pull_request":795399.985915493,"issues_closed_count":1,"pull_requests_closed_count":142,"pull_request_authors_count":8,"issue_authors_count":1,"avg_comments_per_issue":0.0,"avg_comments_per_pull_request":0.4934210526315789,"merged_pull_requests_count":134,"bot_issues_count":0,"bot_pull_requests_count":0,"past_year_issues_count":1,"past_year_pull_requests_count":39,"past_year_avg_time_to_close_issue":16738.0,"past_year_avg_time_to_close_pull_request":643795.8275862068,"past_year_issues_closed_count":1,"past_year_pull_requests_closed_count":29,"past_year_pull_request_authors_count":4,"past_year_issue_authors_count":1,"past_year_avg_comments_per_issue":0.0,"past_year_avg_comments_per_pull_request":0.358974358974359,"past_year_bot_issues_count":0,"past_year_bot_pull_requests_count":0,"past_year_merged_pull_requests_count":29,"created_at":"2024-11-14T00:10:48.989Z","updated_at":"2026-04-03T06:02:20.919Z","repository_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry","issues_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/woudc%2Fwoudc-data-registry/issues","issue_labels_count":{},"pull_request_labels_count":{"enhancement":3,"bug":2},"issue_author_associations_count":{"CONTRIBUTOR":1},"pull_request_author_associations_count":{"CONTRIBUTOR":109,"MEMBER":24,"NONE":19},"issue_authors":{"nalduu":1},"pull_request_authors":{"simranmattu14":27,"victoriarspada":25,"tomkralidis":24,"danielwaiforssell":22,"ahurka":22,"nalduu":16,"kngai":10,"BobMDu":6},"host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-04-05T00:00:08.668Z","repositories_count":14081816,"issues_count":34460311,"pull_requests_count":112427296,"authors_count":11225901,"icon_url":"https://github.com/github.png","host_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories","owners_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/owners","authors_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors"},"past_year_issue_labels_count":{},"past_year_pull_request_labels_count":{"enhancement":3,"bug":2},"past_year_issue_author_associations_count":{"CONTRIBUTOR":1},"past_year_pull_request_author_associations_count":{"CONTRIBUTOR":30,"NONE":5,"MEMBER":4},"past_year_issue_authors":{"nalduu":1},"past_year_pull_request_authors":{"nalduu":16,"simranmattu14":13,"kngai":6,"tomkralidis":4},"maintainers":[{"login":"tomkralidis","count":24,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/tomkralidis"}],"active_maintainers":[{"login":"tomkralidis","count":4,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/tomkralidis"}]},"events":{"total":{"DeleteEvent":1,"PullRequestEvent":42,"ForkEvent":1,"WatchEvent":1,"IssueCommentEvent":13,"PushEvent":57,"PullRequestReviewEvent":15,"PullRequestReviewCommentEvent":31,"CreateEvent":2},"last_year":{"DeleteEvent":1,"PullRequestEvent":26,"ForkEvent":1,"IssueCommentEvent":6,"PushEvent":31,"PullRequestReviewCommentEvent":22,"PullRequestReviewEvent":13,"CreateEvent":2}},"keywords":["gaw","ozone","ozonesonde","spectral","totalozone","ultraviolet","umkehr","uv","wmo"],"dependencies":[{"ecosystem":"pypi","filepath":"requirements-dev.txt","sha":null,"kind":"manifest","created_at":"2022-09-07T14:42:02.000Z","updated_at":"2022-09-07T14:42:02.000Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/requirements-dev.txt","dependencies":[{"id":4113238742,"package_name":"alembic","ecosystem":"pypi","requirements":"*","direct":true,"kind":"development","optional":false},{"id":4113238743,"package_name":"coverage","ecosystem":"pypi","requirements":"*","direct":true,"kind":"development","optional":false},{"id":4113238744,"package_name":"flake8","ecosystem":"pypi","requirements":"*","direct":true,"kind":"development","optional":false},{"id":4113238745,"package_name":"sphinx","ecosystem":"pypi","requirements":"*","direct":true,"kind":"development","optional":false},{"id":4113238746,"package_name":"wheel","ecosystem":"pypi","requirements":"*","direct":true,"kind":"development","optional":false}]},{"ecosystem":"pypi","filepath":"requirements-docs.txt","sha":null,"kind":"manifest","created_at":"2022-09-07T14:42:02.152Z","updated_at":"2022-09-07T14:42:02.152Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/requirements-docs.txt","dependencies":[{"id":4113241607,"package_name":"sphinx","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":4113241608,"package_name":"sphinx-click","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false}]},{"ecosystem":"pypi","filepath":"requirements-pg.txt","sha":null,"kind":"manifest","created_at":"2022-09-07T14:42:02.269Z","updated_at":"2022-09-07T14:42:02.269Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/requirements-pg.txt","dependencies":[{"id":4113242748,"package_name":"psycopg2","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false}]},{"ecosystem":"pypi","filepath":"requirements.txt","sha":null,"kind":"manifest","created_at":"2022-09-07T14:42:02.398Z","updated_at":"2022-09-07T14:42:02.398Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/requirements.txt","dependencies":[{"id":4113242778,"package_name":"click","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":4113242779,"package_name":"elasticsearch","ecosystem":"pypi","requirements":"\u003c8","direct":true,"kind":"runtime","optional":false},{"id":4113242780,"package_name":"jsonschema","ecosystem":"pypi","requirements":"\u003c4.4.0","direct":true,"kind":"runtime","optional":false},{"id":4113242781,"package_name":"pyyaml","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":4113242782,"package_name":"requests","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":4113242783,"package_name":"sqlalchemy","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":4113242784,"package_name":"woudc-extcsv","ecosystem":"pypi","requirements":"\u003e=0.5.0","direct":true,"kind":"runtime","optional":false}]},{"ecosystem":"actions","filepath":".github/workflows/main.yml","sha":null,"kind":"manifest","created_at":"2023-01-13T22:53:50.080Z","updated_at":"2023-01-13T22:53:50.080Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/.github/workflows/main.yml","dependencies":[{"id":6891676300,"package_name":"actions/checkout","ecosystem":"actions","requirements":"v2","direct":true,"kind":"composite","optional":false},{"id":6891676301,"package_name":"actions/setup-python","ecosystem":"actions","requirements":"v2","direct":true,"kind":"composite","optional":false}]},{"ecosystem":"pypi","filepath":"setup.py","sha":null,"kind":"manifest","created_at":"2024-04-02T11:33:38.454Z","updated_at":"2024-04-02T11:33:38.454Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/setup.py","dependencies":[]},{"ecosystem":"deb","filepath":"debian/control","sha":null,"kind":"manifest","created_at":"2026-02-21T10:07:10.182Z","updated_at":"2026-02-21T10:07:10.182Z","repository_link":"https://github.com/woudc/woudc-data-registry/blob/master/debian/control","dependencies":[{"id":25822961293,"package_name":"debhelper","ecosystem":"deb","requirements":"\u003e= 9","direct":true,"kind":"build","optional":false},{"id":25822961294,"package_name":"python3","ecosystem":"deb","requirements":"\u003e= 3.8","direct":true,"kind":"build","optional":false},{"id":25822961296,"package_name":"dh-python","ecosystem":"deb","requirements":"*","direct":true,"kind":"build","optional":false},{"id":25822961297,"package_name":"python3-all","ecosystem":"deb","requirements":"*","direct":true,"kind":"build","optional":false},{"id":25822961298,"package_name":"python3-setuptools","ecosystem":"deb","requirements":"*","direct":true,"kind":"build","optional":false},{"id":25822961305,"package_name":"elasticsearch","ecosystem":"deb","requirements":"\u003e= 8.12.0","direct":true,"kind":"runtime","optional":false},{"id":25822961306,"package_name":"postgresql","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":25822961307,"package_name":"python3-click","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":25822961308,"package_name":"python3-elasticsearch","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":25822961309,"package_name":"python3-psycopg2","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":25822961310,"package_name":"python3-requests","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":25822961311,"package_name":"python3-sqlalchemy","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":25822961312,"package_name":"woudc-extcsv","ecosystem":"deb","requirements":"*","direct":true,"kind":"runtime","optional":false}]}],"score":4.31748811353631,"created_at":"2024-11-14T00:10:42.966Z","updated_at":"2026-04-16T12:30:19.652Z","avatar_url":"https://github.com/woudc.png","language":"Python","category":"Atmosphere","sub_category":"Radiative Transfer","monthly_downloads":0,"total_dependent_repos":0,"total_dependent_packages":0,"readme":"# WOUDC Data Registry\n\n[![Build Status](https://github.com/woudc/woudc-data-registry/workflows/build%20%E2%9A%99%EF%B8%8F/badge.svg)](https://github.com/woudc/woudc-data-registry/actions)\n[![Coverage Status](https://coveralls.io/repos/github/woudc/woudc-data-registry/badge.svg?branch=master)](https://coveralls.io/github/woudc/woudc-data-registry?branch=master)\n[![Documentation](https://readthedocs.org/projects/woudc-data-registry/badge/)](https://woudc-data-registry.readthedocs.org)\n\n## Overview\n\nWOUDC Data Registry is a platform that manages ozone and ultraviolet\nradiation data in support of the [World Ozone and Ultraviolet Radiation Data\nCentre (WOUDC)](https://woudc.org), one of six World Data Centres as part of\nthe [Global Atmosphere Watch](https://community.wmo.int/activity-areas/gaw) programme of the\n[WMO](https://www.wmo.int).\n\n## Installation\n\n### Requirements\n- [Python](https://python.org) 3 and above\n- [virtualenv](https://virtualenv.pypa.io/)\n- [Elasticsearch](https://www.elastic.co/products/elasticsearch) (5.5.0 and above)\n- [woudc-extcsv](https://github.com/woudc/woudc-extcsv)\n\n### Dependencies\nDependencies are listed in [requirements.txt](requirements.txt). Dependencies\nare automatically installed during installation.\n\n### Installing woudc-data-registry\n\n```bash\n# setup virtualenv\npython3 -m venv woudc-data-registry_env\ncd woudc-data-registry_env\nsource bin/activate\n\n# clone woudc-extcsv and install\ngit clone https://github.com/woudc/woudc-extcsv.git\ncd woudc-extcsv\npip install -r requirements.txt\npip install .\ncd ..\n\n# clone codebase and install\ngit clone https://github.com/woudc/woudc-data-registry.git\ncd woudc-data-registry\npip install .\n# optional: for PostgreSQL backends\npip install -r requirements-pg.txt\n\n\n# set system environment variables\ncp default.env foo.env\nvi foo.env  # edit database connection parameters, etc.\n. foo.env\n```\n\n\n### Initializing the Database\n\n```bash\n\n# NOTE: -v/--verbosity option applies to all CLI commands\n\n# create database\nmake ENV=foo.env createdb\n\n# drop database\nmake ENV=foo.env dropdb\n\n# show configuration\nwoudc-data-registry admin config\n\n# show configuration and set output verbosity\nwoudc-data-registry admin config --verbosity DEBUG\n\n# initialize model (database tables)\nwoudc-data-registry admin registry setup\n# optional: initialize specific models only in comma seperated format. e.g.\nwoudc-data-registry admin registry setup -m \"DiscoveryMetadata,DataRecord\"\n\n# initialize search engine\nwoudc-data-registry admin search setup\n# optional: initialize specific indexes only in comma seperated format. e.g.\nwoudc-data-registry admin search setup -i \"data_records,stations\"\n\n# load core metadata\nwoudc-data-registry admin init -d data/\n\n# backup core metadata\nwoudc-data-registry admin backup -d data/\n\n# cleanups\n\n# re-initialize model (database tables)\nwoudc-data-registry admin registry teardown\nwoudc-data-registry admin registry setup\n\n# re-initialize search engine\nwoudc-data-registry admin search teardown\nwoudc-data-registry admin search setup\n\n# optional: teardown specific tables in comma seperated format. e.g.\nwoudc-data-registry admin registry teardown -m \"DataRecord,OzoneSonde\"\n\n# optional teardown specific indexes in comma seperated format. e.g.\nwoudc-data-registry admin search teardown -i \"data_records,stations\"\n\n# If required reinitialized StationDobsonCorrections table and index\nwoudc-data-registry admin setup-dobson-correction -d data/\n```\n\n### Running woudc-data-registry\n\nTIP: autocompletion can be made available in some shells via:\n\n```bash\neval \"$(_WOUDC_DATA_REGISTRY_COMPLETE=source woudc-data-registry)\"\n```\n\n#### Core Metadata Management\n\n```bash\n# list all instances of foo (where foo is one of:\n#  project|dataset|contributor|country|station|instrument|deployment)\nwoudc-data-registry \u003cfoo\u003e list\n# e.g.\nwoudc-data-registry contributor list\n\n# show a specific instance of foo with a given registry identifier\nwoudc-data-registry \u003cfoo\u003e show \u003cidentifier\u003e\n# e.g.\nwoudc-data-registry station show 023\nwoudc-data-registry instrument show ECC:2Z:4052:002:OzoneSonde\n\n# add a new instance of foo (contributor|country|station|instrument|deployment)\nwoudc-data-registry \u003cfoo\u003e add \u003coptions\u003e\n# e.g.\nwoudc-data-registry contributor add -id foo -n \"Contributor name\" -c \"CAN\" -w IV -u https://example.org -e you@example.org -f foouser -g -75,45\nwoudc-data-registry station add --identifier \"foo\" --name \"Station name\" --gaw_id \"\" --country \"CAN\" --wmo-region \"IV\" --start-date \"YYYY-MM-DD\" --geometry \"lat,lon,elevation\"\nwoudc-data-registry deployment add -s \"001\" -c \"MSC:WOUDC\"\n\n# update an existing instance of foo with a given registry identifier\nwoudc-data-registry \u003cfoo\u003e update -id \u003cidentifier\u003e \u003coptions\u003e\n# e.g.\nwoudc-data-registry station update -n \"New station name\"\nwoudc-data-registry deployment update --end-date 'Deployment end date'\n\n# delete an instance of foo with a given registry identifier\nwoudc-data-registry \u003cfoo\u003e delete \u003cidentifier\u003e\n# e.g.\nwoudc-data-registry deployment delete 018:MSC:WOUDC\n\n# for more information about options on operation (add|update):\nwoudc-data-registry \u003cfoo\u003e \u003coperation\u003e --help\n# e.g.\nwoudc-data-registry instrument update --help\n```\n\n#### Data Processing\n\n```bash\n# Gather the files from the ftp account\nwoudc-data-registry data gather /path/to/dir\n\n# ingest directory of files (walks directory recursively)\nwoudc-data-registry data ingest /path/to/dir\n\n# ingest single file\nwoudc-data-registry data ingest foo.dat\n\n# ingest without asking permission checks\nwoudc-data-registry data ingest foo.dat -y\n\n# verify directory of files (walks directory recursively)\nwoudc-data-registry data verify /path/to/dir\n\n# verify single file\nwoudc-data-registry data verify foo.dat\n\n# verify core metadata only\nwoudc-data-registry data verify foo.dat -l\n\n# ingest with only core metadata checks\nwoudc-data-registry data ingest /path/to/dir -l\n\n# update date ranges of all relevant tables with dates gathered from data_records\nwoudc-data-registry data update-date-ranges\n# optional: update specific supported tables in comma seperated format. e.g.\nwoudc-data-registry data update-date-ranges -m \"Contributor,Deployment,Station,Instrument\"\n```\n\n#### Dobson Section Corrections\n```bash\n# Corrects both AD and CD data from TotalOzone Dobson Data\nwoudc-data-registry correction dobson-correction /path/to/dir --mode [test|ops]\n\n# --code gives to option to choose to correct a specific code \nwoudc-data-registry correction dobson-correction /path/to/dir --code [AD|CD] --mode [test|ops]\n\n# --weeklyingest outputs the files in a specific folder structure, similar to incoming folders\nwoudc-data-registry correction dobson-correction /path/to/dir --mode [test|ops] --weeklyingest\n```\n\n#### Search Index Generation\n\n```bash\n# sync all data and metadata tables (except data product tables) to ElasticSearch\nwoudc-data-registry admin search sync\n# optional: sync only specific tables to ES in comma seperated format. e.g.\nwoudc-data-registry admin search sync -m \"DiscoveryMetadata,DataRecord\"\n\n# sync the data product tables (uv_index_hourly, totalozone, and ozonesonde) to ElasticSearch\nwoudc-data-registry admin search product-sync\n# optional: sync only specific product tables to ES in comma seperated format. e.g.\nwoudc-data-registry admin search product-sync -m \"TotalOzone,UVIndex\"\n```\n\n#### UV Index Generation\n\n```bash\n# Teardown and generate entire uv_index_hourly table\nwoudc-data-registry product uv-index generate /path/to/archive/root\n\n\n# Only generate uv_index_hourly records within year range\nwoudc-data-registry product uv-index update -sy start-year -ey end-year /path/to/archive/root\n```\n\n#### Total Ozone Generation\n\n```bash\n# Teardown and generate entire totalozone table\nwoudc-data-registry product totalozone generate /path/to/archive/root\n```\n\n#### OzoneSonde Generation\n\n```bash\n# Teardown and generate entire ozonesonde table\nwoudc-data-registry product ozonesonde generate /path/to/archive/root\n```\n#### Report Generation\n\nThe `woudc-data-registry data ingest` command accepts a `-r/--report` flag, which is a path pointing to a directory.\nWhen that flag is provided, an operator report and a run report are automatically written to that directory\nwhile the files are being processing.\n\n`woudc-data-registry data ingest /path/to/dir -r /path/to/reports/location`\n\nThe run report has a filename `run_report`. The file contains a series of blocks,\none per contributor in a processing run, of the following format:\n\n```\n\u003ccontributor acronym\u003e\n\u003cstatus\u003e: \u003cfilepath\u003e\n\u003cstatus\u003e: \u003cfilepath\u003e\n\u003cstatus\u003e: \u003cfilepath\u003e\n...\n```\n\nWhere `\u003cstatus\u003e` is either `Pass` or `Fail`, depending on how the file reported in that line fared in processing.\n\nThe operator report is a more in-depth error log in CSV format, with a filename like `operator-report-\u003cdate\u003e.csv`.\nOperator reports contain one line per error or warning that happened during the processing run. The operator report\nis meant to be a human-readable log which makes specific errors easy to find and diagnose.\n\n#### Sending Emails to Contributors\n\nTo generate emails for contributors:\n\n```bash\nwoudc-data-registry data generate-emails /path/to/dir\n```\n\n#### Sending Reminder Emails to Contributors\n\n```bash\npython3 woudc_data_registry/reminders/contributor_reminder.py --mode ops\n# For testing:\npython3 woudc_data_registry/reminders/contributor_reminder.py --mode test\n```\n\n#### Publishing Notifications to MQTT Server\n```bash\nwoudc-data-registry publish publish-notification --hours number_of_hours\n```\n\n#### Delete Record\n\n```bash\nwoudc-data-registry data delete-record /path/to/bad/file/\n```\n\nIf a bad file was previously ingested, it can be removed using this command. This removes the file from the registry and the WAF.\n\n### Development\n\n```bash\n# install dev requirements\npip install -r requirements-dev.txt\n```\n\n#### Building the Documentation\n\n```bash\n# build local copy of https://woudc.github.io/woudc-data-registry\ncd docs\nmake html\npython3 -m http.server  # view on http://localhost:8000/\n```\n\n#### Running Tests\n\n```bash\n# run tests like this:\ncd woudc_data_registry/tests\npython3 test_data_registry.py\npython3 test_report_generation.py\n\n# or this:\npython3 setup.py test\n\n# measure code coverage\ncoverage run --source=woudc_data_registry -m unittest woudc_data_registry.tests.test_data_registry\ncoverage report -m\n```\n\n#### Code Conventions\n\n* [PEP8](https://www.python.org/dev/peps/pep-0008)\n\n### Bugs and Issues\n\nAll bugs, enhancements and issues are managed on [GitHub](https://github.com/woudc/woudc-data-registry/issues).\n\n## Contact\n\n* [Tom Kralidis](https://github.com/tomkralidis)\n\n","funding_links":[],"readme_doi_urls":[],"works":{},"citation_counts":{},"total_citations":0,"keywords_from_contributors":["downscaled","ahccd","cangrd","cansips","cccs","climate","cmip5"],"project_url":"https://ost.ecosyste.ms/api/v1/projects/303797","html_url":"https://ost.ecosyste.ms/projects/303797"}