Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Smart Citizen Data

A python package for analyzing environmental sensor's data.
https://github.com/fablabbcn/smartcitizen-data

Category: Natural Resources
Sub Category: Air Quality

Keywords

air-quality data-analysis data-science environmental-monitoring low-cost-sensor machine-learning python sensor-data sensors smartcitizen

Last synced: about 4 hours ago
JSON representation

Repository metadata

A python package for analyzing environmental sensor's data

README.md

Smart Citizen Data

DOI
Binder
PyPI version
Python application

Welcome to SmartCitizen Data. This is a data analysis framework for working with sensor data in different ways:

  • Interacting with several sensors APIs
  • Clean data, export and calculate metrics
  • Model sensor data and calibrate sensors
  • Generate data visualisations - matplotlib, plotly or uplot
  • Generate analysis reports in html or pdf and upload them to Zenodo

Installation

You can check it out in the Binder before installing if you want. Works with Python 3.* (tested between 3.9 and 3.11).

You can just run:

pip install scdata

Work on the source code

Simply clone the repository with:

git clone https://github.com/fablabbcn/smartcitizen-data.git
cd smartcitizen-data

Install scdata package with requirements:

python setup.py install

Or if you want to edit:

cd scdata
pip install --editable .

Tokens and config

A configuration file is available at ~/.config/scdata/config.yaml, which contains a set of configurable variables to allow or not the local storage of relevant data in the data folder, normally in ~/.cache/scdata:

data:
  cached_data_margin: 2
  load_cached_api: true
  reload_metadata: true
  store_cached_api: true
paths:
  config: /Users/username/.config/scdata
  data: /Users/username/.cache/scdata
  export: /Users/username/.cache/scdata/export
  interim: /Users/username/.cache/scdata/interim
  inventory: ''
  models: /Users/username/.cache/scdata/models
  processed: /Users/username/.cache/scdata/processed
  raw: /Users/username/.cache/scdata/raw
  reports: /Users/username/.cache/scdata/reports
  uploads: /Users/username/.cache/scdata/uploads
zenodo_real_base_url: https://zenodo.org
zenodo_sandbox_base_url: http://sandbox.zenodo.org

Also, .env files will be picked from ~/.cache/scdata. An env.example is provided. If you want to upload data to Zenodo, you will need to fill set an environment variable called ZENODO_TOKEN in your environment. The .env file is not mandatory for scdata to work. It only serves for authentication if you want to POST or GET data without throttling. You can still do GET requests but they will be throttled if not authenticated/authorized by the platform. See user roles for more info.

Using with jupyter lab (optional)

It can also be used with jupyter lab or jupyter. For this install juypterlab.

Contribute

Issues and PR more than welcome!

Funding

This work has received funding from the European Union's Horizon 2020 research and innovation program under the grant agreement No. 689954

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: 8 days ago

Total Commits: 1,047
Total Committers: 10
Avg Commits per committer: 104.7
Development Distribution Score (DDS): 0.053

Commits in past year: 101
Committers in past year: 3
Avg Commits per committer in past year: 33.667
Development Distribution Score (DDS) in past year: 0.277

Name Email Commits
Oscar Gonzalez o****r@g****m 992
Daniel Mateos d****l@m****o 26
Pral2a g****m@p****m 16
vico v****r@s****e 3
dependabot[bot] 4****] 2
Viktor Smari v****i@g****m 2
Stephan Hohn s****n@g****m 2
Stephan Hohn s****n@p****e 2
Dwaipayan Munshi 5****5 1
Anna Molter 3****r 1

Committer domains:

Issue and Pull Request metadata

Last synced: about 1 month ago

Total issues: 12
Total pull requests: 14
Average time to close issues: 3 months
Average time to close pull requests: 8 months
Total issue authors: 4
Total pull request authors: 5
Average comments per issue: 0.75
Average comments per pull request: 0.64
Merged pull request: 6
Bot issues: 0
Bot pull requests: 6

Past year issues: 1
Past year pull requests: 0
Past year average time to close issues: N/A
Past year average time to close pull requests: N/A
Past year issue authors: 1
Past year pull request authors: 0
Past year average comments per issue: 0.0
Past year average comments per pull request: 0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/fablabbcn/smartcitizen-data

Top Issue Authors

  • oscgonfer (8)
  • Maryam159 (2)
  • jnow (1)
  • staffhorn (1)

Top Pull Request Authors

  • dependabot[bot] (6)
  • oscgonfer (5)
  • mcsage (1)
  • dwaipayan05 (1)
  • anmolter (1)

Top Issue Labels

  • bug (1)

Top Pull Request Labels

  • dependencies (6)

Package metadata

pypi.org: scdata

Analysis of sensors and time series data

  • Homepage: https://github.com/fablabbcn/smartcitizen-data
  • Documentation: https://docs.smartcitizen.me/
  • Licenses: GNU-GPL3.0
  • Latest release: 1.4.0 (published about 1 month ago)
  • Last Synced: 2026-04-18T12:11:58.067Z (15 days ago)
  • Versions: 60
  • Dependent Packages: 0
  • Dependent Repositories: 2
  • Downloads: 241 Last month
  • Rankings:
    • Dependent packages count: 10.103%
    • Forks count: 11.411%
    • Dependent repos count: 11.558%
    • Average: 12.636%
    • Downloads: 14.035%
    • Stargazers count: 16.074%
  • Maintainers (1)

Dependencies

setup.py pypi
  • Flask *
  • Jinja2 ==2.11.2
  • PyYAML ==5.3.1
  • branca ==0.4.0
  • folium ==0.12.1
  • geopy ==1.21.0
  • matplotlib ==3.2.1
  • numpy *
  • pandas >=1.3.0
  • pdfrw ==0.4
  • plotly *
  • reportlab ==3.5.
  • requests ==2.23.0
  • scikit-learn *
  • scipy *
  • seaborn ==0.10.1
  • termcolor ==1.1.0
  • tqdm *
  • tzwhere ==3.0.3
  • urllib3 ==1.25.9
.devcontainer/Dockerfile docker
  • python 3.8 build
.github/workflows/python-latest.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
.github/workflows/python-publish.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
  • pypa/gh-action-pypi-publish 27b31702a0e7fc50959f5ad993c78deac1bdfc29 composite
requirements.txt pypi
  • Flask *
  • Jinja2 *
  • PyYAML *
  • awswrangler *
  • boto3 *
  • branca *
  • folium *
  • geopy *
  • matplotlib *
  • pandas *
  • pydantic *
  • pytest *
  • requests *
  • scikit-learn *
  • scipy *
  • seaborn *
  • smartcitizen-connector *
  • termcolor ==1.1.0
  • timezonefinder *
  • tqdm *
  • urllib3 *
.github/workflows/python-multiple-versions.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite

Score: 11.018629143155449