DeepIceDrain
Mapping and monitoring deep subglacial water activity in Antarctica using remote sensing and machine learning.
https://github.com/weiji14/deepicedrain
Category: Cryosphere
Sub Category: Glacier and Ice Sheets
Keywords
analysis-ready-data antarctica big-data binder datashader hdf5 ice-sheet icesat-2 intake jupyter-lab open-science pygmt python3 zarr
Keywords from Contributors
measuring transformations compose archives observation reporter numeric optimize annotation animations
Last synced: about 20 hours ago
JSON representation
Repository metadata
Mapping and monitoring deep subglacial water activity in Antarctica using remote sensing and machine learning, with ICESat-2!
- Host: GitHub
- URL: https://github.com/weiji14/deepicedrain
- Owner: weiji14
- License: lgpl-3.0
- Created: 2019-10-09T02:02:05.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2022-10-18T21:10:22.000Z (over 2 years ago)
- Last Synced: 2025-04-20T06:16:13.340Z (7 days ago)
- Topics: analysis-ready-data, antarctica, big-data, binder, datashader, hdf5, ice-sheet, icesat-2, intake, jupyter-lab, open-science, pygmt, python3, zarr
- Language: Jupyter Notebook
- Homepage:
- Size: 29.6 MB
- Stars: 30
- Watchers: 2
- Forks: 8
- Open Issues: 12
- Releases: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
README.md
[poster]
DeepIceDrainMapping and monitoring deep subglacial water activity
in Antarctica using remote sensing and machine learning.
Ice Surface Elevation trends over Antactica | Active Subglacial Lake fill-drain event |
---|---|
![]() |
![]() |
Along track view of an ATL11 Ground Track | Elevation time-series at Crossover Points |
---|---|
![]() |
![]() |
Getting started
Quickstart
Launch in Binder (Interactive jupyter lab environment in the cloud).
Alternative Pangeo BinderHub link.
Requires a GitHub account and you'll have to install your own computing environment,
but it runs on AWS uswest2 which allows for
cloud access to ICESat-2!
Usage
Once you've properly installed the deepicedrain
package
(see installation instructions further below), you'll have access to a
wide range of tools
for downloading and performing quick calculations on ICESat-2 datasets.
The example below shows how to calculate ice surface elevation change
on a sample ATL11 dataset between ICESat's Cycle 3 and Cycle 4.
import deepicedrain
import xarray as xr
# Loads a sample ATL11 file from the intake catalog into xarray
atl11_dataset: xr.Dataset = deepicedrain.catalog.test_data.atl11_test_case.read()
# Calculate elevation change in metres from ICESat-2 Cycle 3 to Cycle 4
delta_height: xr.DataArray = deepicedrain.calculate_delta(
dataset=atl11_dataset, oldcyclenum=3, newcyclenum=4, variable="h_corr"
)
# Quick plot of delta_height along the ICESat-2 track
delta_height.plot()
Installation
Basic
To just try out the scripts, download the environment.yml
file from the repository and run the commands below:
cd deepicedrain
mamba env create --name deepicedrain --file environment.yml
pip install git+https://github.com/weiji14/deepicedrain.git
Intermediate
To help out with development, start by cloning this repo-url
git clone <repo-url>
Then I recommend using mamba
to install the non-python binaries.
A virtual environment will also be created with Python and
poetry installed.
cd deepicedrain
mamba env create --file environment.yml
Activate the virtual environment first.
mamba activate deepicedrain
Then install the python libraries listed in the pyproject.toml
/poetry.lock
file.
poetry install
Finally, double-check that the libraries have been installed.
poetry show
Advanced
This is for those who want full reproducibility of the virtual environment,
and more computing power by using Graphical Processing Units (GPU).
Making an explicit conda-lock file
(only needed if creating a new virtual environment/refreshing an existing one).
mamba env create -f environment.yml
mamba list --explicit > environment-linux-64.lock
Creating/Installing a virtual environment from a conda lock file.
See also https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#building-identical-conda-environments.
mamba create --name deepicedrain --file environment-linux-64.lock
mamba install --name deepicedrain --file environment-linux-64.lock
If you have a CUDA-capable GPU,
you can also install the optional "cuda" packages to accelerate some calculations.
poetry install --extras cuda
Running jupyter lab
mamba activate deepicedrain
python -m ipykernel install --user --name deepicedrain # to install virtual env properly
jupyter kernelspec list --json # see if kernel is installed
jupyter lab &
Related Projects
This work would not be possible without inspiration
from the following cool open source projects!
Go check them out if you have time.
Citing
The work in this repository has not been peer-reviewed, but if you do want to
cite it for some reason, use the following BibLaTeX code from this conference
proceedings (poster presentation):
@inproceedings{LeongSpatiotemporalvariabilityactive2021,
title = {{Spatiotemporal Variability of Active Subglacial Lakes in Antarctica from 2018-2020 Using ICESat-2 Laser Altimetry}},
author = {Leong, W. J. and Horgan, H. J.},
date = {2021-02-10},
publisher = {{Unpublished}},
location = {{Christchurch, New Zealand}},
doi = {10.13140/RG.2.2.27952.07680},
eventtitle = {{New Zealand Antarctic Science Conference}}},
langid = {english}
}
Python code for the DeepIceDrain package here on Github is also mirrored on Zenodo at https://doi.org/10.5281/zenodo.4071235.
Owner metadata
- Name: Wei Ji
- Login: weiji14
- Email:
- Kind: user
- Description: Geospatial Data Scientist/ML Practitioner @developmentseed. Towards GPU-native and cloud-native geospatial machine learning!
- Website: https://weiji14.xyz
- Location: Wellington
- Twitter:
- Company: @developmentseed
- Icon url: https://avatars.githubusercontent.com/u/23487320?u=227f2b53ce4b1eb9bcb1a3e5ba0eb7de61fce370&v=4
- Repositories: 67
- Last ynced at: 2024-05-01T10:09:42.150Z
- Profile URL: https://github.com/weiji14
GitHub Events
Total
Last Year
Committers metadata
Last synced: 5 days ago
Total Commits: 471
Total Committers: 3
Avg Commits per committer: 157.0
Development Distribution Score (DDS): 0.41
Commits in past year: 0
Committers in past year: 0
Avg Commits per committer in past year: 0.0
Development Distribution Score (DDS) in past year: 0.0
Name | Commits | |
---|---|---|
Wei Ji | w****g@v****z | 278 |
dependabot-preview[bot] | 2****] | 188 |
dependabot[bot] | 4****] | 5 |
Committer domains:
- vuw.ac.nz: 1
Issue and Pull Request metadata
Last synced: 2 days ago
Total issues: 13
Total pull requests: 331
Average time to close issues: 4 days
Average time to close pull requests: 6 days
Total issue authors: 3
Total pull request authors: 4
Average comments per issue: 0.54
Average comments per pull request: 0.74
Merged pull request: 247
Bot issues: 7
Bot pull requests: 276
Past year issues: 0
Past year pull requests: 0
Past year average time to close issues: N/A
Past year average time to close pull requests: N/A
Past year issue authors: 0
Past year pull request authors: 0
Past year average comments per issue: 0
Past year average comments per pull request: 0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- dependabot-preview[bot] (7)
- weiji14 (5)
- Bbbigcountry (1)
Top Pull Request Authors
- dependabot-preview[bot] (252)
- weiji14 (55)
- sourcery-ai[bot] (13)
- dependabot[bot] (11)
Top Issue Labels
- help wanted (2)
- feature :rocket: (2)
- enhancement :sparkles: (2)
- question :question: (1)
Top Pull Request Labels
- dependencies (283)
- python (228)
- skip-changelog (32)
- enhancement :sparkles: (23)
- docker :whale: (14)
- submodules (14)
- maintenance :toolbox: (12)
- data :card_file_box: (11)
- security :lock: (9)
- feature :rocket: (7)
- bug :beetle: (2)
- help wanted (1)
Dependencies
- cuml 21.10.00.*
- cuspatial 21.10.00.*
- geos 3.9.1.*
- gmt 6.2.0.*
- graphviz 2.49.1.*
- gxx_linux-64 11.2.0.*
- parallel 20210822.*
- pip 21.2.4.*
- poetry 1.1.11.*
- proj 8.0.1.*
- python 3.8.12.*
- 229 dependencies
- actions/checkout v2.2.0 composite
- actions/checkout v2.2.0 composite
- conda-incubator/setup-miniconda v2.1.1 composite
- base latest build
- buildpack-deps jammy-scm@sha256 build
Score: 4.836281906951478