Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Terraso

Develop an open source technology platform to help local community leaders practice Integrated Landscape Management.
https://github.com/techmatters/terraso-backend

Category: Natural Resources
Sub Category: Soil and Land

Keywords from Contributors

optimize archiving measur transforms observation conversion animals compose projection generic

Last synced: about 8 hours ago
JSON representation

Repository metadata

A Django project that powers the backend of Terraso platform.

README.md

Terraso Backend

Terraso backend is a Django project that powers the backend of the Terraso
platform.

Requirements

  • Docker: version 24 or better
  • Python: 3.12 or better

Running locally with Docker

Set up your environment file

$ cp .env.sample .env

In the .env file

  • set values for GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRETbased on what you have set up in console.cloud.google.com > APIs & Services > Credentials.

  • set values for APPLE_CLIENT_ID, APPLE_KEY_ID, APPLE_TEAM_ID, APPLE_PRIVATE_KEY and based on what you have set up on developer.apple.com > Certificates, Identifiers & Profiles > Keys.

  • set values for MICROSOFT_CLIENT_ID and either MICROSOFT_CLIENT_SECRET (less secure) or both MICROSOFT_PRIVATE_KEY and MICROSOFT_CERTIFICATE_THUMBPRINT based on what you have set up on portal.azure.com > App Registrations > [App Name] > Certificates & Secrets

Start building the Docker images (make sure there's requirements.txt
file created before building the images):

$ make build

Run the database migrations before the first run:

$ make migrate

Than the run command can be executed:

$ make run

If you want to have a user to access the admin panel, you need to create
it:

$ make bash
# (inside the web container)
$ python terraso_backend/manage.py createsuperuser
$ exit

Debugging locally with Docker

To debug while running tests, just use regular Python breakpoint() and
run the tests normally. Pytest will stop properly on break point giving
you access to the Python debugger.

To debug while using the application locally, it's also possible to use
Python breakpoint(). To have access to the Python debugger, you'll
need to attach to the application running container.

So, assuming that the application is running with make run:

List the running containers

$ docker ps

Get the id of the web container before next step

$ docker attach <web-container-id>

This will give you access to the web running container

Make the application request call that will pass on breakpoint, like
calling an API or clicking in some button. As soon as the process get to
the break point, the attached shell should open the Python debugger. To
continue the application request processing, just release the debugger.

Interacting with the database

Connect to the database

While the database is running (e.g. via make run), use:

make connect_db

Run or rollback a specific migration

You can tell the database to migrate a specific app to a specific number with:

APP_MIGRATION_NAME="{app_name} {migration_number}" make migrate

So to e.g. rollback migration 0014 of the soil_id app, you could run:

APP_MIGRATION_NAME="soil_id 0014" make migrate

Loading sample data

Import sample landscape data (names, descriptions, links):

$ python terraso_backend/manage.py loaddata sampledata.json

Import landscape boundaries geodata:

$ python terraso_backend/manage.py load_landscapes_geojson --airtable_api_key xxxxx

Download Soil ID data:

$ make download-soil-data

Reset the database

You can reset the database back to its default state:

$ python terraso_backend/manage.py flush

Backup the database

The contents of the database can be dumped to a JSON file for backup. Optionally, they can also be uploaded to a S3 bucket. This can be activated to be triggered from a button in the Django admin console. See the relevant README.md for more details.

Print migration SQL

You can view the SQL commands that a migration will issue by running

$ python terraso_backend/manage.py sqlmigrate {app_name} {migration_name}

or in Docker using the make command

$ APP_MIGRATION_NAME="{app_name} {migration_name}" make print_migration_sql

For example:

$ APP_MIGRATION_NAME="story_map 0001_initial" make print_migration_sql

Contributing

Before contributing to the project, it's recommended that you set up
your local git running the following command:

$ make setup-git-hooks

This will activate two git hooks to automatically check Python code
style and commit message structure before each commit.

Dealing with dependencies

It is possible to lock the dependencies to run the project with the
following command

$ make lock

It is also possible to lock development dependencies with:

$ make lock-dev

The lock process creates/updates the files requirements.txt and
requirements-dev.txt. With these files in place it's possible to
install dependencies running:

$ make install

To install development dependencies run:

$ make install-dev

Local development

Your local machine needs:

  • Docker
  • Python

make lock requires GDAL and Cython:

$ brew install gdal

$ apt install gdal

cython

$ pip3 install cython

Generating GraphQL public documentation

The API docs are generated by
SpectaQL. So, make sure you
followed their README to have it installed before proceding. Run the
following command to generate the documentation:

$ make api_doc

Enjoy! ;-)

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: about 1 month ago

Total Commits: 1,424
Total Committers: 17
Avg Commits per committer: 83.765
Development Distribution Score (DDS): 0.382

Commits in past year: 265
Committers in past year: 7
Avg Commits per committer in past year: 37.857
Development Distribution Score (DDS) in past year: 0.223

Name Email Commits
dependabot[bot] 4****] 880
Paul Schreiber p****l@t****g 231
Caio Carrara c****a 104
Jose Buitron j****n@g****m 62
David Code Howard d****d@t****g 50
garo (they/them) 3****m 42
Jodie Zhu 8****u 14
Carissa Knipe c****a@t****g 14
garo (they/them) 3****k 5
Jeancarlo Barrios J****s 5
tm-ruxandra r****a@t****g 4
ajabeckett 3****t 4
Paul Schreiber p****l@t****g 2
Andrew Carter 6****e 2
Caio Carrara c****o@t****g 2
Johannes 8****y 2
Robert Bo Davis bo@i****g 1

Committer domains:

Issue and Pull Request metadata

Last synced: 17 days ago

Total issues: 243
Total pull requests: 2,094
Average time to close issues: 3 months
Average time to close pull requests: 4 days
Total issue authors: 14
Total pull request authors: 15
Average comments per issue: 0.81
Average comments per pull request: 0.24
Merged pull request: 1,623
Bot issues: 11
Bot pull requests: 1,373

Past year issues: 11
Past year pull requests: 367
Past year average time to close issues: 2 months
Past year average time to close pull requests: 4 days
Past year issue authors: 5
Past year pull request authors: 7
Past year average comments per issue: 1.73
Past year average comments per pull request: 0.12
Past year merged pull request: 210
Past year bot issues: 2
Past year bot pull requests: 280

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/techmatters/terraso-backend

Top Issue Authors

  • david-code (64)
  • paulschreiber (40)
  • shrouxm (39)
  • ltseng (28)
  • caiocarrara (20)
  • josebui (13)
  • DerekCaelin (12)
  • dependabot[bot] (11)
  • garobrik (5)
  • knipec (4)
  • JeancarloBarrios (3)
  • Hanal-code (2)
  • tm-ruxandra (1)
  • robert-bo-davis (1)

Top Pull Request Authors

  • dependabot[bot] (1,373)
  • paulschreiber (323)
  • caiocarrara (114)
  • josebui (83)
  • shrouxm (66)
  • david-code (59)
  • knipec (26)
  • jodiezhuu (12)
  • JeancarloBarrios (9)
  • tm-ruxandra (7)
  • garobrik (6)
  • ajabeckett (6)
  • johannesparty (4)
  • robert-bo-davis (4)
  • plentydone (2)

Top Issue Labels

  • landpks (58)
  • Engineering (46)
  • tech-debt (31)
  • slim MVP (24)
  • High (17)
  • enhancement (12)
  • dependencies (11)
  • python (11)
  • site creation (10)
  • Medium (10)
  • project management (7)
  • good first issue (6)
  • profiles (6)
  • 1000L (6)
  • bug (5)
  • audit log (5)
  • Low (3)
  • blocked (1)
  • Research (1)
  • Capri MVP (1)
  • stale (1)
  • needs triage (1)
  • testing (1)
  • kobotoolbox (1)

Top Pull Request Labels

  • dependencies (1,373)
  • python (1,239)
  • blocked (71)
  • github_actions (12)
  • Engineering (1)

Dependencies

.github/workflows/build.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/check-commits.yml actions
  • actions/checkout v3 composite
  • agenthunt/conventional-commit-checker-action v1.0.0 composite
Dockerfile docker
  • python 3.11.1-slim-bullseye build
requirements/base.in pypi
  • boto3 *
  • dj-database-url *
  • django *
  • django-cors-headers *
  • django-dirtyfields *
  • django-filter *
  • django-oauth-toolkit *
  • django-safedelete *
  • django-storages *
  • django-structlog *
  • geopandas *
  • graphene-django *
  • httpx *
  • prettyconf *
  • psycopg2 *
  • pyjwt *
  • pyproj *
  • python-magic *
  • rules *
  • shapely *
requirements/deploy.in pypi
  • ddtrace *
  • gunicorn *
requirements/dev.in pypi
  • black * development
  • flake8 * development
  • freezegun * development
  • ipdb * development
  • isort * development
  • mixer * development
  • moto * development
  • pytest * development
  • pytest-cov * development
  • pytest-django * development
  • respx * development
requirements-dev.txt pypi
  • anyio ==3.6.2 development
  • appnope ==0.1.3 development
  • asttokens ==2.2.1 development
  • attrs ==22.2.0 development
  • backcall ==0.2.0 development
  • black ==23.1.0 development
  • boto3 ==1.26.64 development
  • botocore ==1.29.69 development
  • certifi ==2022.12.7 development
  • cffi ==1.15.1 development
  • charset-normalizer ==3.0.1 development
  • click ==8.1.3 development
  • coverage ==7.1.0 development
  • cryptography ==39.0.1 development
  • decorator ==5.1.1 development
  • exceptiongroup ==1.1.0 development
  • executing ==1.2.0 development
  • faker ==12.0.1 development
  • flake8 ==6.0.0 development
  • freezegun ==1.2.2 development
  • h11 ==0.14.0 development
  • httpcore ==0.16.3 development
  • httpx ==0.23.3 development
  • idna ==3.4 development
  • iniconfig ==2.0.0 development
  • ipdb ==0.13.11 development
  • ipython ==8.10.0 development
  • isort ==5.12.0 development
  • jedi ==0.18.2 development
  • jinja2 ==3.1.2 development
  • jmespath ==1.0.1 development
  • markupsafe ==2.1.2 development
  • matplotlib-inline ==0.1.6 development
  • mccabe ==0.7.0 development
  • mixer ==7.2.2 development
  • moto ==4.1.2 development
  • mypy-extensions ==1.0.0 development
  • packaging ==23.0 development
  • parso ==0.8.3 development
  • pathspec ==0.11.0 development
  • pexpect ==4.8.0 development
  • pickleshare ==0.7.5 development
  • platformdirs ==3.0.0 development
  • pluggy ==1.0.0 development
  • prompt-toolkit ==3.0.36 development
  • ptyprocess ==0.7.0 development
  • pure-eval ==0.2.2 development
  • pycodestyle ==2.10.0 development
  • pycparser ==2.21 development
  • pyflakes ==3.0.1 development
  • pygments ==2.14.0 development
  • pytest ==7.2.1 development
  • pytest-cov ==4.0.0 development
  • pytest-django ==4.5.2 development
  • python-dateutil ==2.8.2 development
  • requests ==2.28.2 development
  • responses ==0.22.0 development
  • respx ==0.20.1 development
  • rfc3986 ==1.5.0 development
  • s3transfer ==0.6.0 development
  • six ==1.16.0 development
  • sniffio ==1.3.0 development
  • stack-data ==0.6.2 development
  • toml ==0.10.2 development
  • tomli ==2.0.1 development
  • traitlets ==5.9.0 development
  • types-toml ==0.10.8.3 development
  • urllib3 ==1.26.14 development
  • wcwidth ==0.2.6 development
  • werkzeug ==2.2.3 development
  • xmltodict ==0.13.0 development
requirements.txt pypi
  • aniso8601 ==9.0.1
  • anyio ==3.6.2
  • asgiref ==3.6.0
  • attrs ==22.2.0
  • boto3 ==1.26.64
  • botocore ==1.29.69
  • bytecode ==0.14.0
  • cattrs ==22.2.0
  • certifi ==2022.12.7
  • cffi ==1.15.1
  • charset-normalizer ==3.0.1
  • click ==8.1.3
  • click-plugins ==1.1.1
  • cligj ==0.7.2
  • cryptography ==39.0.1
  • ddsketch ==2.0.4
  • ddtrace ==1.7.5
  • deprecated ==1.2.13
  • dj-database-url ==1.2.0
  • django ==4.1.7
  • django-cors-headers ==3.13.0
  • django-dirtyfields ==1.9.1
  • django-filter ==22.1
  • django-ipware ==4.0.2
  • django-oauth-toolkit ==2.2.0
  • django-safedelete ==1.3.1
  • django-storages ==1.13.2
  • django-structlog ==4.1.0
  • envier ==0.4.0
  • exceptiongroup ==1.1.0
  • fiona ==1.9.0
  • geopandas ==0.12.2
  • graphene ==3.2.1
  • graphene-django ==3.0.0
  • graphql-core ==3.2.3
  • graphql-relay ==3.2.0
  • gunicorn ==20.1.0
  • h11 ==0.14.0
  • httpcore ==0.16.3
  • httpx ==0.23.3
  • idna ==3.4
  • jmespath ==1.0.1
  • jsonschema ==4.17.3
  • jwcrypto ==1.4.2
  • munch ==2.5.0
  • numpy ==1.24.2
  • oauthlib ==3.2.2
  • packaging ==23.0
  • pandas ==1.5.3
  • prettyconf ==2.2.1
  • promise ==2.3
  • protobuf ==4.21.12
  • psycopg2 ==2.9.5
  • pycparser ==2.21
  • pyjwt ==2.6.0
  • pyproj ==3.4.1
  • pyrsistent ==0.19.3
  • python-dateutil ==2.8.2
  • python-magic ==0.4.27
  • pytz ==2022.7.1
  • requests ==2.28.2
  • rfc3986 ==1.5.0
  • rules ==3.3
  • s3transfer ==0.6.0
  • shapely ==2.0.1
  • six ==1.16.0
  • sniffio ==1.3.0
  • sqlparse ==0.4.3
  • structlog ==22.3.0
  • tenacity ==8.1.0
  • text-unidecode ==1.3
  • typing-extensions ==4.4.0
  • urllib3 ==1.26.14
  • wrapt ==1.14.1
  • xmltodict ==0.13.0
.github/workflows/codeql.yml actions
  • actions/checkout v4 composite
  • github/codeql-action/analyze v2 composite
  • github/codeql-action/autobuild v2 composite
  • github/codeql-action/init v2 composite
docker-compose.base.yml docker
  • postgis/postgis 14-master
  • techmatters/terraso_backend latest
docker-compose.ci.yml docker
docker-compose.dev.yml docker
package.json npm
pyproject.toml pypi
.github/workflows/auto-approve.yml actions
  • hmarr/auto-approve-action v3 composite

Score: 7.428333194190806