Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Recent Releases of Digital Earth Australia Coastlines

Digital Earth Australia Coastlines - 3.0.1

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/3.0.0...3.0.1

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt about 1 month ago

Digital Earth Australia Coastlines - 3.0.0

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/2.2.0...3.0.0

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt about 2 months ago

Digital Earth Australia Coastlines - 2.2.0

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/2.1.1...2.2.0

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt 9 months ago

Digital Earth Australia Coastlines - 2.2.0

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/2.1.1...2.2.0

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt 9 months ago

Digital Earth Australia Coastlines - 2.1.1

What's Changed

  • Fixed bug with remote offshore tiles failing to process. Updated code to ensure empty arrays use a correct datetime datatype to allow them to be combined with arrays containing valid satellite data.

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 1 year ago

Digital Earth Australia Coastlines - 2.1.0

What's Changed

  • Added annual shoreline data for 2022. The 2022 shoreline is interim data that is subject to change, and will be updated to a final version in the following 2023 DEA Coastlines update (in July 2024).
  • Added dataset maturity filter. This ensures that the DEA Coastlines analysis is run on only the highest quality "final" maturity satellite data.

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 1 year ago

Digital Earth Australia Coastlines - 2.0.2

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/2.0.0...2.0.2

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt about 2 years ago

Digital Earth Australia Coastlines - 2.0.1

What's Changed

Updates to Python package dependencies to run on latest DEA Sandbox version 2.0.0 image.

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/2.0.0...2.0.1

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt about 2 years ago

Digital Earth Australia Coastlines - 2.0.0

What's Changed

  • Added annual shoreline data for 2021. The 2021 shoreline is interim data that is subject to change, and will be updated to a final version in the following 2022 DEA Coastlines update (in July 2023).
  • Improved data coverage over offshore islands and reefs. This includes new shorelines and rates of coastal change over the Indian Ocean, Great Barrier Reef and Coral Sea.
  • DEA Coastlines now uses data from the FES2014 global tidal model. This model produces accurate tide modelling across Australia and globally, and is more flexible and easier to install than the older TPXO8 tidal model.
  • Inclusion of Landsat 9 data to provide additional satellite data from late 2021 onwards.
  • Improved shoreline mapping certainty attributes to provide information about the quality of each individual annual shoreline (compared to the previous approach of identifying all annual shorelines in an area as low or good quality). These annual shoreline certainty attributes include:
  • good: Good quality annual shorelines with no low quality flags
  • insufficient data: The accuracy of this shoreline may be affected by limited good quality satellite observations at this location. This can lead to noisier and less reliable shorelines.
  • unstable data: The accuracy of this shoreline is affected by unstable data at this location. This may be caused by errors in the tidal model used to reduce the influence of tide, the presence of gently sloping intertidal mudflats or sandbars that can lead to inaccurate shoreline mapping, or noisy satellite imagery caused by high levels of cloud.
  • aerosol issues: The accuracy of this shoreline may be affected by aerosol issues caused by the 1991 eruption of Mount Pinatubo.
  • New rates of change point certainty attributes to provide information about the quality of each rate of coastal change. These new certainty attributes include:
  • good: Good quality rates of change points with no low quality flags
  • likely rocky shoreline: This coastline has been identified as a probable rocky or cliff shoreline. Rates of coastal change at this location may be less accurate due to noisy shoreline mapping caused by dark terrain shadows.
  • insufficient observations: There are insufficient years of good quality annual shoreline data (< 25 years) to calculate reliable rates of coastal change at this location.
  • extreme value (> 50 m): This location has been identified as having an extreme rate of coastal change (> 50 metres per year) and should be interpreted with caution.
  • high angular variability: This rate of coastal change is unlikely to be accurate due to high levels of angular variability from this point to each annual shoreline. This can occur in complex coastal environments like river mouths, sandbars and mudflats that do not follow linear patterns of coastal change over time.
  • baseline outlier: The baseline (i.e. most recent) annual shoreline is itself flagged as an outlier, potentially resulting in inaccurate rates of change at this location.
  • Updated coastal change hotspot layers: three coastal change hotspot layers are now included in the product, summarising coastal change within 1 km, 5 km and 10 km windows around each hotspot point. These layers now include attributes providing the median position of all annual shorelines around each hotspot point, which can be used to plot and identify regional trends of coastal change.
  • Rates of change points contain a new "uid" ID field containing a unique geohash ID that can be used to more easily track rates of change points as they are updated in future DEA Coastlines updates.
  • Rates of change points include a new "id_primary" field containing the ID of their Primary sediment compartment from the Australian Coastal Sediment Compartments framework. This enables points to be easily queried and summarised by coastal sediment compartment.

Backwards incompatible changes:

  • The annual shorelines WFS layer dea:coastlines has been renamed to dea:annual_shorelines.
  • The rates of change points WFS layer dea:coastlines_statistics has been renamed to dea:rates_of_change.

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.8...2.0.0

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt about 2 years ago

Digital Earth Australia Coastlines - 1.2.8

Allow tiles to be generated without an error even if no rates of change points are available (e.g. just shorelines). This may produce additional data over low data areas (e.g. remote oceans) where we may potentially be missing entire years of data.

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.7...1.2.8

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.7

Remove test of broken path after testing it works

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.6...1.2.7

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.6

Test of automatic re-try for tide model access issues

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.5...1.2.6

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.5

  • Make shapefile export in continental script optional to reduce run times while testing
  • Tweak all time raster thresholds for river and coastal study area extent generation

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.4...1.2.5

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.4

Implement mask modification file
Use combination of 15% and 50% layer for determining coastal study area and extracting rivers respectively

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.3...1.2.4

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.3

Add tide centre and hotspot generation flag to CLI so that they can be customised at run time.
Fix issues with requirements that caused vector code to fail with errors

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.2...1.2.3

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.2

Fixes a bug where NaN values weren't being properly dealt with in annual coastal masking code
Removed islands prior to river detection to avoid narrow channels being masked out as rivers:

Before:
image

After:
image

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.1...1.2.2

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.1

Fix bug with overwrite functionality failing when no vector version is supplied

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.2.0...1.2.1

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.2.0

  • Added run status outputs to ensure --no-overwrite functionality works correctly
  • Prepare code for second Argo continental run

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.16...1.2.0

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.16

A major overhaul of the contours_preprocess function to simplify the creation and application of masks, and add a new experimental automated river mask.

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.15...1.1.16

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.15

Make Geodata 100K load more efficient by removing dask load so data is not loaded again in each for loop iteration

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.14...1.1.15

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.14

Move Geodata 100K coastal dataset load outside for loop to reduce S3 requests

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.13...1.1.14

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.13

Remove reduntant tide modelling points for ocean water identification in favour of approach based on Geodata 100K dataset

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.12...1.1.13

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.12

Updated raster and vector code to support tiles with missing data for individual years in the time series (e.g. remote coral sea tiles with patchy Landsat history)

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.11...1.1.12

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.11

Restore code to use 48 km tiles, and rerun all notebooks

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.10...1.1.11

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.10

Experiment with 96 km tiles.

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.9...1.1.10

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.9

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.8...1.1.9

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.8

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.7...1.1.8

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.7

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.6...1.1.7

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.6

Use latest version of dea-tools Package that automatically points to the new location of tide modelling files.

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.5...1.1.6

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.5

Fix typo in tide modelling path

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.4...1.1.5

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.4

Updated tide modelling paths to account for new EFS mount location on Sandbox and Argo

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.3...1.1.4

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.3

What's Changed

Full Changelog: https://github.com/GeoscienceAustralia/dea-coastlines/compare/1.1.2...1.1.3

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.2

New release after fixing tests

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.1

Test release for update to tide modelling file directory name

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 2 years ago

Digital Earth Australia Coastlines - 1.1.0

1.1.0 (2020 annual shoreline)

In December 2021, the DEA Coastlines product was updated to version 1.1.0. This includes the following key changes to the data and web services:

Improvements and additions:

  • Added annual shoreline data for 2020. The 2020 shoreline is an interim dataset that is subject to change, and will be updated to a final version in the following 2021 DEA Coastlines update.
  • Implemented a new temporal filter that improves mapping performance across small islands and dynamic sandbars while reducing noise in areas with poor satellite coverage (#35).
  • Time series graphs are now available for non-significant rates of change points on DEA Maps (previously this was only available for retreating and growing points) (#40).
  • Australian Coastal Geomorphology Smartline and Australian Coastal Sediment Compartments datasets are now available in the "Supplementary data" folder on DEA Maps to assist in interpreting observed patterns of coastal change.
  • Added new valid_obs and valid_span fields to rates of change points that summarise the total number and temporal coverage of valid non-outlier shoreline observations. Results from valid_obs are now used to filter out unreliable rates of change points on DEA Maps by removing points with fewer than 25 valid observations (#38).
  • Climate indices (e.g. SOI) are now detrended prior to regression to remove false positives associated with trends in index values over time, and to allow climate index relationships to be modelled for coastlines with temporal erosional or progradation trends through time (#42).
  • Numerous minor improvements to inland water and estuary masking to fix areas of missing shoreline data and remove non-coastal features (#32, #33, #36).

Backwards incompatible changes:

  • Removed experimental breaks field from rates of change points (#41).
  • Removed retreat, growth fields from rates of change points (this data is already captured in rate_time, with negative values indicating retreat and positive values indicating growth).
  • Updated WMS layer group names from "dea:DEACoastLines" to "dea:DEACoastlines", and "dea:AnnualCoastlines" to "dea:AnnualShorelines"

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 3 years ago

Digital Earth Australia Coastlines - 1.0.0

Official 1.0.0 release (2019 annual shoreline; 15 March, 2020)

Official version 1.0.0 of DEA Coastlines code base, as submitted to Remote Sensing of Environment.

Updates

  • Implemented morphological operators to reduce false positive cloud detection along narrow, bright sandy beaches (see #37)

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt almost 4 years ago

Digital Earth Australia Coastlines - 1.0.0-beta

1.0.0-beta (1 Oct, 2020)

Updates

  • Added the following new columns to "Statistics" points:
    • rate_soi, sig_soi, se_soi, outl_soi: Columns giving the results of regression analysis of annual distances from the 2019 baseline coastline and the Southern Oscillation Index. This can be used to plot the spatial footprint of ENSO on coastal change (e.g. indicating if and by how much coastlines retreat during La Nina years). (#24)
    • breaks: A column listing any years that were identified as breakpoints. This can be useful for verifying that a significant trend is indeed linear, or identifying areas of rapid non-linear change e.g. associated with coastal development or management (experimental, interpret with care) (#21)
  • Continental summary includes SOI as well as temporal rates of change (#27)
  • Code cleaned to use virtual products and remove dependency on dea-notebooks code (#26)
  • Add support for multiple polygons and file export for DEA CoastLines utility notebook (#16)
  • Minor updates to waterbody mask to exclude additional poorly mapped tropical rivers
  • Fully documented all script functions (#23)
  • Re-written readme doc to follow official product description and provide examples of how to load data from WFS in both R and Python

Bug fixes

  • Greatly reduced data gaps caused by an issue with the way nodata pixels were identified for gapfill years (#19)

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt over 4 years ago

Digital Earth Australia Coastlines - 0.3.0

0.3.0 (July 15, 2020)

Run on almost complete GA Landsat Collection 3 archive (v.s. c3-samples database that was missing many scenes).

Major updates

  • Added the following new columns to "Statistics" points:
    • se_time: gives standard error of regression, will be used to add error bars to rates of change (#7)
    • growth/retreat: booleans indicating whether a point eroded or prograded, will be used in Terria config for plain english descriptions of change (#6 )
    • sce: Shoreline Change Envelope; gives the entire distance across which shorelines changed over the time series (#13)
    • nsm: Net Shoreline Movement; gives the difference between the first and the last valid shoreline position (#15)
    • max_year/min_year: gives the year where shorelines were at their most land-ward and most sea-ward (#15)
  • Set all 1991/1992 annual coastlines to "uncertain" northward of -23 degrees latitude on account of Mt Pinatubo aerosol issue (#12)
  • Standardise all continental outputs to use Australian Albers EPSG:3577 projection (#9)
  • Renamed year columns with dist_ prefix to clarify their purpose (i.e. distances from baseline) (#5)
  • Add notebook utility tool based on DEA CoastLines WFS for interactively drawing profiles and returning transects (#3, #8)

Bug fixes

  • Restored missing contours on western end of Rottnest Island, WA
  • Removed Papua New Guinea territory from data
  • Noise caused by low data availability is greatly reduced, particularly in southern Tasmania and Cambridge Gulf

image (1)

Outstanding issues

  • Two tiles fail due to KeyError in skimage.measure.find_contours (#11)

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt almost 5 years ago

Digital Earth Australia Coastlines - 0.2.0

0.2.0 (May 5, 2020)

Features

  • Add 'uncertainty' column to contour data for pixels with either less than 5 observations or > 0.25 MNDWI standard deviation in more than 50% of years
  • Add new DEACoastLines_utilities.ipynb notebook containing additional functions for using DEA CoastLines data
  • Add rates_hist utility function for plotting DEACoastLine statistics data as a histogram based on an interactive selection
  • Add ability to modify waterbody mask by providing a vector files of features to add or remove from the mask
  • Add new deacoastlines_summary.py script for generating continental summary of point statistics based on average values within a buffer of X km of a subset of points. This allows hotspots to be visualised at full zoom out

Bug fixes

  • In deacoastlines_generation.py: Fixed missing grid cell areas by buffering both the input grid cell extent and tide modelling extent by 0.05 degrees (i.e. a total of 0.10 degrees). This ensures that enough tidal modelling points are available for interpolation, and improves tidal modelling consistency between neighbouring grid cells
  • In deacoastlines_generation.py: Break script early if grid cell correspondes to 1 or less tidal modelling points, as this makes tidal interpolation impossible
  • In deacoastlines_generation.py: Increase dask_chunks to {'time': 1, 'x': 2000, 'y': 2000} for improved data load performance
  • In deacoastlines_statistics.py: Fixed missing shorelines caused by the waterbody mask by limiting mask to specific waterbody features ('Aquaculture Area', 'Estuary', 'Watercourse Area', 'Salt Evaporator', 'Settling Pond' and perennial 'Lakes')
  • In deacoastlines_statistics.py: Fix CRS of exported GeoJSON contours and statistics files by converting to EPSG:4326
  • In deacoastlines_statistics.py: Remove previous zeroing of yearly distance values to 1988. Yearly distances are now relative to the 2018 baseline (e.g. 0 for the 2018 contour), which should be simpler easier to interpret
  • In deacoastlines_statistics.py: Increase min_vertices for contour extraction to 30 to reduce noise
  • In deacoastlines_statistics.py: Move vector directory creation until after data load, so no directory is created if rasters do not exist

v0.1.0 (April 1, 2020)

Features

  • First continental run of DEACoastLines
  • Outlier detection now uses a more robust Median Absolute Deviation method

Bug fixes

  • In deacoastlines_statistics.py: Remove mask applied to contour generation due to excessive missing data

Hydrosphere - Coastal and Reefs - Jupyter Notebook
Published by robbibt almost 5 years ago