{"id":535,"name":"marine_debris_ML","description":"Marine debris detection with commercial satellite imagery and deep learning.","url":"https://github.com/NASA-IMPACT/marine_debris_ML","last_synced_at":"2026-05-23T17:30:32.162Z","repository":{"id":45740885,"uuid":"344885779","full_name":"NASA-IMPACT/marine_debris_ML","owner":"NASA-IMPACT","description":"Marine debris detection with commercial satellite imagery and deep learning.","archived":false,"fork":false,"pushed_at":"2021-11-05T15:51:35.000Z","size":25351,"stargazers_count":110,"open_issues_count":2,"forks_count":30,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-04-30T06:20:22.624Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NASA-IMPACT.png","metadata":{},"created_at":"2021-03-05T17:32:41.000Z","updated_at":"2026-04-18T16:56:03.000Z","dependencies_parsed_at":"2022-08-12T12:10:24.374Z","dependency_job_id":null,"html_url":"https://github.com/NASA-IMPACT/marine_debris_ML","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NASA-IMPACT/marine_debris_ML","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NASA-IMPACT","download_url":"https://codeload.github.com/NASA-IMPACT/marine_debris_ML/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32938003,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-12T09:19:52.626Z","status":"ssl_error","status_checked_at":"2026-05-12T09:17:33.438Z","response_time":102,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"owner":{"login":"NASA-IMPACT","name":"Inter Agency Implementation and Advanced Concepts","uuid":"22798984","kind":"organization","description":"","email":"esds.dsig@gmail.com","website":null,"location":null,"twitter":null,"company":null,"icon_url":"https://avatars.githubusercontent.com/u/22798984?v=4","repositories_count":88,"last_synced_at":"2023-03-04T07:55:15.502Z","metadata":{"has_sponsors_listing":false},"html_url":"https://github.com/NASA-IMPACT","funding_links":[],"total_stars":null,"followers":null,"following":null,"created_at":"2022-11-14T15:35:50.518Z","updated_at":"2023-03-04T07:55:15.538Z","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NASA-IMPACT","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NASA-IMPACT/repositories"},"packages":[],"commits":{"id":1254240,"full_name":"NASA-IMPACT/marine_debris_ML","default_branch":"main","total_commits":68,"total_committers":3,"total_bot_commits":0,"total_bot_committers":0,"mean_commits":22.666666666666668,"dds":0.4852941176470589,"past_year_total_commits":0,"past_year_total_committers":0,"past_year_total_bot_commits":0,"past_year_total_bot_committers":0,"past_year_mean_commits":0.0,"past_year_dds":0.0,"last_synced_at":"2026-05-12T12:03:17.416Z","last_synced_commit":"9b7317c0ad881849ce5688aeb1eb368dfb85d39f","created_at":"2023-03-27T11:00:50.263Z","updated_at":"2026-05-12T12:03:11.879Z","committers":[{"name":"Lilly Thomas","email":"lillythomas@Lillys-MacBook-Pro.local","login":null,"count":35},{"name":"Lilly Thomas","email":"lilly@developmentseed.org","login":"lillythomas","count":32},{"name":"ankshah131","email":"61028925+ankshah131","login":"ankshah131","count":1}],"past_year_committers":[],"commits_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML/commits","host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-05-16T00:00:17.015Z","repositories_count":6234896,"commits_count":894640935,"contributors_count":34908805,"owners_count":1153693,"icon_url":"https://github.com/github.png","host_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://commits.ecosyste.ms/api/v1/hosts/GitHub/repositories"}},"issues_stats":{"full_name":"NASA-IMPACT/marine_debris_ML","html_url":"https://github.com/NASA-IMPACT/marine_debris_ML","last_synced_at":"2026-04-01T16:00:47.938Z","status":"error","issues_count":14,"pull_requests_count":10,"avg_time_to_close_issue":16082928.583333334,"avg_time_to_close_pull_request":223193.9,"issues_closed_count":12,"pull_requests_closed_count":10,"pull_request_authors_count":1,"issue_authors_count":5,"avg_comments_per_issue":0.0,"avg_comments_per_pull_request":0.0,"merged_pull_requests_count":10,"bot_issues_count":0,"bot_pull_requests_count":0,"past_year_issues_count":0,"past_year_pull_requests_count":0,"past_year_avg_time_to_close_issue":null,"past_year_avg_time_to_close_pull_request":null,"past_year_issues_closed_count":0,"past_year_pull_requests_closed_count":0,"past_year_pull_request_authors_count":0,"past_year_issue_authors_count":0,"past_year_avg_comments_per_issue":null,"past_year_avg_comments_per_pull_request":null,"past_year_bot_issues_count":0,"past_year_bot_pull_requests_count":0,"past_year_merged_pull_requests_count":0,"created_at":"2023-05-09T10:37:17.745Z","updated_at":"2026-04-01T16:00:47.938Z","repository_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML","issues_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories/NASA-IMPACT%2Fmarine_debris_ML/issues","issue_labels_count":{},"pull_request_labels_count":{},"issue_author_associations_count":{"COLLABORATOR":10,"NONE":2,"CONTRIBUTOR":2},"pull_request_author_associations_count":{"COLLABORATOR":10},"issue_authors":{"lillythomas":9,"ankshah131":2,"GuyYehezkel":1,"priftisg":1,"Ratheshan03":1},"pull_request_authors":{"lillythomas":10},"host":{"name":"GitHub","url":"https://github.com","kind":"github","last_synced_at":"2026-05-16T00:00:24.458Z","repositories_count":14616734,"issues_count":34236542,"pull_requests_count":112076338,"authors_count":11264022,"icon_url":"https://github.com/github.png","host_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/repositories","owners_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/owners","authors_url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors"},"past_year_issue_labels_count":{},"past_year_pull_request_labels_count":{},"past_year_issue_author_associations_count":{},"past_year_pull_request_author_associations_count":{},"past_year_issue_authors":{},"past_year_pull_request_authors":{},"maintainers":[{"login":"lillythomas","count":19,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/lillythomas"},{"login":"priftisg","count":1,"url":"https://issues.ecosyste.ms/api/v1/hosts/GitHub/authors/priftisg"}],"active_maintainers":[]},"events":{"total":{"ForkEvent":9,"WatchEvent":24},"last_year":{"ForkEvent":3,"WatchEvent":9}},"keywords":[],"dependencies":[{"ecosystem":"pypi","filepath":"requirements.txt","sha":null,"kind":"manifest","created_at":"2022-08-12T12:10:23.955Z","updated_at":"2022-08-12T12:10:23.955Z","repository_link":"https://github.com/NASA-IMPACT/marine_debris_ML/blob/main/requirements.txt","dependencies":[{"id":1323918796,"package_name":"numpy","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918797,"package_name":"matplotlib","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918798,"package_name":"geopandas","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918799,"package_name":"requests","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918800,"package_name":"tqdm","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918801,"package_name":"joblib","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918802,"package_name":"pillow","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918803,"package_name":"pyproj","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918804,"package_name":"shapely","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false},{"id":1323918805,"package_name":"label-maker","ecosystem":"pypi","requirements":"*","direct":true,"kind":"runtime","optional":false}]}],"score":5.817111159963204,"created_at":"2023-09-11T11:54:35.027Z","updated_at":"2026-05-23T17:30:32.165Z","avatar_url":"https://github.com/NASA-IMPACT.png","language":"Python","category":"Industrial Ecology","sub_category":"Circular Economy and Waste","monthly_downloads":0,"total_dependent_repos":0,"total_dependent_packages":0,"readme":"# Marine debris detection with commercial satellite imagery and deep learning.\n\nFloating marine debris is a global pollution problem which threatens marine and human life and leads to the loss of biodiversity. Large swaths of marine debris are also navigational hazards to vessels. Artificial intelligence, specifically deep learning, can be used to detect floating marine debris in satellite imagery. In this project, we seek to demonstrate the strong potential of using commercial small satellite imagery for detecting marine debris pollution and strengthening current and future efforts to clean the oceans. We present an application of a deep learning model designed for object detection in the TensorFlow framework for observing marine debris floating on the surface of the ocean. The model was trained on our custom-labeled dataset of 1370 polygons containing marine debris as observed in [Planetscope optical imagery](https://www.planet.com/products/planet-imagery/). An overall precision score of 0.78 and recall score of 0.70 were obtained on the test dataset. \n\n*Model performance on test images:*\n\u003cimg src=\"assets/predictions0.png\" width=\"800px\" height=\"auto\"\u003e\n\u003cimg src=\"assets/predictions1.png\" width=\"800px\" height=\"auto\"\u003e\n\n*Paper and dataset forthcoming.*\n\n## Overview\n\n### 1. Data\n\nPlanet small satellite imagery is utilized in this study. Specifically, the 3-meter imagery product called Planetscope. This imagery has four bands namely red, green, blue, and near-infrared. The combination of fairly high spatial resolution, high temporal resolution, availability of a near-infrared channel and global coverage of coastlines made this imagery quite advantageous for the purposes of this research. With these imagery specifications as well as plastic size and ghost fishing net size categories, we anticipated our model would be capable of detecting aggregated debris flotsam as well as some mega plastics including medium to large size ghost fishing nets.\n\nUsing the Planet Explorer, specific image scenes consisting of visible marine debris patches were selected for our training dataset. This step involved manually exploring Planetscope scenes and verifying the presence of marine debris. For this initial study, we decided to focus our efforts on detecting marine debris from optical (red, green, blue) channel imagery. Initial investigation into the utility of the Planetscope near-infrared channel was conducted, and future work will integrate the near-infrared channel.\n\nWe used [Image Labeler](https://impact.earthdata.nasa.gov/labeler/) to manually digitize bounding box annotations for observable debris on Planetscope optical imagery. A total of 1370 bounding boxes were labeled on the image scenes. This constituted the initial training, testing and validation dataset for object detection modeling.\n\nThe next task was to prepare the dataset in model-ready format, which entailed tiling the image scenes into smaller frames and encoding the bounding boxes into coordinate arrays with numerical class ids. The need for tiling the imagery stems from computational efficiency at model runtime. To accomplish these tasks, we used [Label Maker (LM)](https://github.com/developmentseed/label-maker). We used zoom level 16 as it most closely approximates the native spatial resolution of Planetscope imagery. An example configuration file for use with LM is located at *data_utils/config.json*. Finally, the dataset in compressed array format (.npz) was used to create binary TensorFlow Records datasets.\n\nTiled image with labels.npz entry. On the right are the bounding box annotation coordinates `[xmin, ymin, xmax, ymax]` and `class ID 1`, with the image array on the bottom:\n\u003cimg src=\"assets/tiled_example_npz.png\" width=\"800px\" height=\"auto\"\u003e\n\nTiled images with plotted annotations:\n\u003cimg src=\"assets/tiled_example.png\" width=\"800px\" height=\"auto\"\u003e\n\n### 2. Model\nOur architecture of choice for this project is [SSD Resnet 101 Feature Pyramid Network (FPN)](https://arxiv.org/abs/1708.02002), which we've implemented with the [Tensorflow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection). We employed a weighted sigmoid focal loss\nand transfer learning for our baseline model from a [pre-trained resnet 101 checkpoint](http://download.tensorflow.org/models/object_detection/ssd_resnet101_v1_fpn_shared_box_predictor_oid_512x512_sync_2019_01_20.tar.gz) hosted on Tensorflow model zoo. Our best model currently performs with a test F1 score of 0.74.\n\nAfter training is complete, we export the best model to [TensorFlow serving format](https://www.tensorflow.org/tfx/guide/serving), package the trained model weights and inference code into a [Docker](https://www.docker.com/) image and deploy at scale through our inference pipeline (shown below).\n\nFor inference, we use the [Planet tile endpoint](https://developers.planet.com/docs/basemaps/tile-services/) to request a list of [XYZ tiles](https://developers.planet.com/planetschool/xyz-tiles-and-slippy-maps/) for a given area of interest and time range. We send that list of tiles via [SQS](https://aws.amazon.com/sqs/) to our inference endpoint, and once deployed, we can inference at a rate of 3000 tiles of size 256x256 pixels per minute. The results written to the database include, for each XYZ tile, the original Planet image scene ID and XYZ tile name (containing the x coordinate, y coordinate and zoom level) and one or more bounding box coordinates, class values and confidence scores. We use the python utility, [Mercantile](https://github.com/mapbox/mercantile), to translate the XYZ coordinates to latitude and longitude coordinates and finally, export the final predictions with a minimum confidence threshold to GeoJSON format. The GeoJSON files are used for display in an online dashboard.\n\nScaled model inference pipeline:\n\u003cimg src=\"assets/model_inference.png\" width=\"800px\" height=\"auto\"\u003e\n \n## Implementation\n\n### 1. Model training and inference\n\nWe recommend creating a python 3.6+ virtual environment for this project. You can use [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv) to do so.\n\nInstall these Tensorflow versions in the activated environment.\n\n```\ntensorboard==1.14.0\ntensorboard-plugin-wit==1.6.0.post3\ntensorflow-estimator==1.14.0\ntensorflow-gpu==1.14.0\n```\n\n### 2. Setup TensorFlow Object Detection API\n\n#### 2a. Install TensorFlow object detection:\n- Download the necessary scripts with `git clone https://github.com/tensorflow/models.git`\n- Install TensorFlow Object Detection API by strictly following [these instructions](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1.md). Once you've successfully run `python object_detection/builders/model_builder_test.py` you are ready for the next step.\n- To access the necessary utility scripts, you'll need to run all the following commands from the `models/research/object_detection` directory from the cloned repo. **From here on we will refer the TensorFlow Object Detection directory `models/research/object_detection/` as the TOD directory.**\n\nYou could also work from this [codebase](https://github.com/NASA-IMPACT/marine_litter_ML/tree/main/object_detection_api) as a stable implementation with the above listed TF library versions. Just ensure that repo folder is set as `models/research/object_detection/`.\n\n### 3. Create TFRecords for model training\nTensorflow API supports a variety of file formats. The TFRecord file format is a simple record-oriented binary format that many TensorFlow applications use. We have example code in this repo which converts the `labels.npz` file to a TFRecords file:\n\n- Copy [`utils_convert_tfrecords.py` from this repo](https://github.com/NASA-IMPACT/marine_litter_ML/blob/main/data_utils/utils_convert_tfrecords.py) to the TOD directory, .\n- Your $folder will be the `data` path containing your `labels.npz` file and `tiles`.\n- From the TOD directory run:\n\n```shell\npython3 utils_convert_tfrecords.py    \\\n        --label_input=$folder/labels.npz   \\\n        --data_dir=tf_records   \\\n        --tiles_dir=$folder/tiles    \\\n        --pbtxt=classes.pbtxt\n```\nThis will create `train.record`, `val.record` and `test.record` files in a folder called `tf_records` in the TOD directory. Each record file contains different and non-overlapping partitions of the data (86,7,7 percents, respectively).\n\n### 4. Object detection model setup\nNow we're ready to set up the model architecture. For this walkthrough, we'll download a pre-trained model from the [TensorFlow model zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md). We'll demonstrate using [`ssd_resnet_101_fpn_oidv4`](http://download.tensorflow.org/models/object_detection/ssd_resnet101_v1_fpn_shared_box_predictor_oid_512x512_sync_2019_01_20.tar.gz) (download link):\n  - Download the model, unzip, and move the folder to the TOD directory\n  - Create a new folder `training` in the TOD directory.\n  - Copy a [model configuration file](https://github.com/NASA-IMPACT/marine_litter_ML/blob/main/configs/ssd_resnet101_v1_fpn_marine_debris.config) to the `training directory`. \n  - Copy a [class definitions file](https://github.com/NASA-IMPACT/marine_litter_ML/blob/main/configs/marine_debris.pbtxt) to the `data` directory.\n\nNow your current directory should be `models/research/object_detection/` and in addition to the files included in that repo originally, your folder structure should look like this:\n\n```\nmodels/research/object_detection/\n├── ssd_resnet101_v1_fpn_multilabel/\n├── training/\n│   └── ssd_resnet101_v1_fpn_marine_debris.config\n├── data/\n│   ├── train.record\n│   ├── val.record\n│   ├── test.record\n│   ├── marine_debris.pbtxt\n└───\n```\n\n### 5. Train the TensorFlow object detection model\nYou are now ready to train the model. From the `models/research/` directory, run:\n\n```shell\n#!/usr/bin/env bash\npyenv activate tf114_od\nexport PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim\ncd object_detection\nexport CUDA_VISIBLE_DEVICES=0\npython model_main.py --alsologtostderr --model_dir=training/ --pipeline_config_path=training/ssd_resnet101_v1_fpn_multilabel.config \n```\n\nThe model checkpoints and outputs for this task will save in the `training` folder. \n\n### 6. Visualize the Model\nUsing this [script](https://github.com/NASA-IMPACT/marine_litter_ML/tree/main/object_detection_api/export_inference_graph.py), create the marine debris detection model inference graph with:\n\n```shell\npython export_inference_graph.py --input_type image_tensor \\\n              --pipeline_config_path training/ssd_resnet101_v1_fpn_multilabel.config \\\n              --trained_checkpoint_prefix training/model.ckpt-500000 \\\n              --output_directory model_50k\n```\nWe can visualize this graph using [`tensorboard`](https://github.com/tensorflow/tensorboard):\n\n```shell\ntensorboard --logdir='training'\n```\n\nGo to `http://127.0.0.1:6006/` in your web browser and you will see:\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/tensorboard.png\" width=\"1000\" /\u003e\n\u003c/p\u003e\n\n### 7. Prediction\nNow let's run the model over our test tiles to predict where marine debris patches are. Copy [this script](https://github.com/NASA-IMPACT/marine_litter_ML/blob/v0_2/inference_utils/tf_od_predict_image_aug_to_geo_corrected.py) to the TOD directory then run:\n\n```shell\npython tf_od_predict_image_aug_to_geo_corrected.py --model_name=model_50k \\\n                        --path_to_label=data/marine_debris.pbtxt \\\n                        --test_image_path=path/to/test/image/tiles\n```\nThis code will read through all your test images in `path/to/test/image/tiles` folder and output the final prediction into the same folder. You will find new images in `test_image_path` with the `_test` suffixed to the end of the file basenames. The are images with the predicted bounding boxes and confidence scores plotted on top. As well, you will find a multipolygon geojson of predicted bounding boxes in the `test_image_path`. \n\nOption for flags:\n\n```\nexport base_dir=models/research/object_detection\nexport EXPORT_DIR=models/research/object_detection/model_50k\npython3 ${base_dir}/tf_od_predict_image_aug_to_geo_corrected.py --model_name=${EXPORT_DIR} --path_to_label=${base_dir}/marine_debris.pbtxt --test_image_path=${base_dir}/test/\n```\n\nDetections geo-registered and vectorized to GeoJSON format:\n\u003cimg src=\"assets/detections_geo.png\" width=\"800px\" height=\"auto\"\u003e\n\n### 8. Evaluation\nYou can use the [code](https://github.com/NASA-IMPACT/marine_litter_ML/tree/main/evaluation_utils) in this folder to compute standard evaluation metrics with your model. Runtime and background instructions live [here](https://github.com/NASA-IMPACT/marine_litter_ML/tree/main/evaluation_utils/evaluation.md).","funding_links":[],"readme_doi_urls":[],"works":{},"citation_counts":{},"total_citations":0,"keywords_from_contributors":[],"project_url":"https://ost.ecosyste.ms/api/v1/projects/535","html_url":"https://ost.ecosyste.ms/projects/535"}