Fractional Cover

A remote sensing algorithm which measures the photosynthetic, non-photosynthetic and bare earth components of a Landsat image.
https://github.com/geoscienceaustralia/fc

Category: Natural Resources
Sub Category: Soil and Land

Keywords from Contributors

gdal opendatacube earthobservation earth-observation remotesensing ogc-services geospatial-data sentinel-2 geoscienceaustralia digitalearthaustralia

Last synced: about 18 hours ago
JSON representation

Repository metadata

Fractional Cover

• Host: GitHub
• URL: https://github.com/geoscienceaustralia/fc
• Owner: GeoscienceAustralia
• License: apache-2.0
• Created: 2016-06-10T05:15:32.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2024-10-09T03:15:48.000Z (7 months ago)
• Last Synced: 2025-02-13T03:04:04.408Z (2 months ago)
• Language: Python
• Size: 626 KB
• Stars: 12
• Watchers: 42
• Forks: 7
• Open Issues: 2
• Releases: 11
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

README.rst

          =======================
 Fractional Cover (FC)
=======================

|Build Status| |Coverage Status|

`Fractional Cover`_ is a remote sensing algorithm which measures the
photosynthetic, non-photosynthetic and bare earth components of a Landsat image.

This repository implements Fractional Cover as used by `Digital Earth Australia`_
for the production of the `Fractional Cover Data Product `_.
The implementation is Python, with the core algorithm in Fortran.

Data production is either by a command line application that runs on the NCI_,
or a `Virtual Product`_ class run by `Datacube Alchemist`_ on AWS.


.. contents::

.. _Fractional Cover: https://cmi.ga.gov.au/data-products/dea/119/dea-fractional-cover-landsat
.. _NCI: https://www.nci.org.au/
.. _Datacube Alchemist: https://github.com/opendatacube/datacube-alchemist/
.. _Virtual Product: https://datacube-core.readthedocs.io/en/latest/dev/api/virtual-products.html
.. _Digital Earth Australia: https://www.ga.gov.au/dea


.. |Build Status| image:: https://github.com/GeoscienceAustralia/fc/workflows/Tests/badge.svg
    :target: https://github.com/GeoscienceAustralia/fc/actions?query=workflow%3ATests

.. |Coverage Status| image:: https://codecov.io/gh/GeoscienceAustralia/fc/branch/master/graph/badge.svg?token=wpeulGrrUT
    :target: https://codecov.io/gh/GeoscienceAustralia/fc

Installation
============

The easiest way to install Fractional Cover is::

    pip install --extra-index-url=https://packages.dea.ga.gov.au/ fc

This package includes the compiled Fortran extensions to run about 15 times faster than the Python+scipy version
you get by default without a Fortran compiler.

Setup on VDI
============

The first time you try to use gadi PBS commands from VDI, you will need
to run::

    $ remote-hpc-cmd init

See http://vdi.nci.org.au/help#heading=h.u1kl1j7vdt16 for more details.

You will also need to setup datacube to work from VDI and gadi.

::

    $ ssh gadi "cat .pgpass" >> ~/.pgpass
    $ chmod 0600 ~/.pgpass

See http://geoscienceaustralia.github.io/digitalearthau/connect/nci_basics.html for
full details.

Running
-------

The Fractional Cover application works in 2 parts:

    #. Creating the task list
    #. Check for unexpected existing files - these were most likely created during an run that did not successfully
       finish.
    #. Submit the job to gadi.

To run fractional cover::

    $ module use /g/data/v10/public/modules/modulefiles/
    $ module load dea

This will list the available app configs::

    $ datacube-fc list
    ls5_fc_albers.yaml
    ls7_fc_albers.yaml
    ls8_fc_albers.yaml

To submit the job to ``gadi``, the datacube-fc app has a the ``datacube-fc submit`` command:
This command kick off two stage PBS job

    Stage 1 (Generate task file):
        The task-app machinery loads a config file, from a path specified on the
        command line, into a dict.

        If dry is enabled, a dummy DatasetType is created for tasks generation without indexing
        the product in the database.
        If dry run is disabled, generate tasks into file and queue PBS job to process them.

    Stage 2 (Run):
        During normal run, following are performed:
           1) Tasks (loadables (nbart,ps,dsm) + output targets) shall be yielded for dispatch to workers.
           2) Load data
           3) Run FC algorithm
           4) Attach metadata
           5) Write output files and
           6) Finally index the newly created FC output files

        If dry run is enabled, application only prepares a list of output files to be created and does not
        record anything in the database.

Tracking progress
-----------------

::

    $ qstat -u $USER

    $ qcat 7517348.gadi-pbs | head

    $ qcat 7517348.gadi-pbs | tail

    $ qps 7517348.gadi-pbs

File locations
--------------

The config file (eg. ls5_fc_albers.yaml) specifies the app settings, and is found in the module.

You will need to check the folder of the latest ``dea`` module::

    ls /g/data/v10/public/modules/dea//lib/python3.6/site-packages/fc/config

The config file lists the output `location` and file_path_template``, as shown in this snippet::

    source_type: ls5_nbar_albers
    output_type: ls5_fc_albers

    description: Landsat 5 Fractional Cover 25 metre, 100km tile, Australian Albers Equal Area projection (EPSG:3577)
    product_type: fractional_cover
    
    location: '/g/data/fk4/datacube/002/'
    file_path_template: 'LS5_TM_FC/{tile_index[0]}_{tile_index[1]}/LS5_TM_FC_3577_{tile_index[0]}_{tile_index[1]}_{start_time}_v{version}.nc'

So here the output files are saved to ``/g/data/fk4/datacube/002/FC/LS5_TM_FC//*.nc``

File naming
===========

Specify a template string used to name the output files. Uses the python ``format()`` string syntax, with the following placeholders available:


==============  ==============
  Placeholder    Description
==============  ==============
tile_index[0]    X Tile Index
tile_index[1]    Y Tile Index
region code      The region code of the input dataset
epoch_start      Start date of the epoch, format using `strftime` syntax
epoch_end        End date of the epoch, format using `strftime` syntax
version          Task timestamp
start_time       Start time in `%Y%m%d%H%M%S%f` format
end_time         End time in `%Y%m%d%H%M%S%f` format
==============  ==============

For example:

.. code-block:: yaml

       file_path_template: '{y}_{x}/LS_PQ_COUNT_3577_{y}_{x}_{epoch_start:%Y-%m-%d}_{epoch_end:%Y-%m-%d}.nc'

Will output filenames similar to:

.. code-block:: bash

    10_15/LS_PQ_COUNT_3577_10_15_2010-01-01_2011-01-01.nc


License
-------
This repository is licensed under the Apache License 2.0. See the `LICENSE file `_ in this repository for details.


Contacts
--------
Geoscience Australia developers:

**Joshua Sixsmith**
[email protected]

**Jeremy Hooke**
[email protected]

**Damien Ayers**
[email protected]

**Duncan Gray**
[email protected]

Algorithm developer:

**Peter Scarth**
[email protected]

        

---

Owner metadata

• Name: Geoscience Australia
• Login: GeoscienceAustralia
• Email:
• Kind: organization
• Description:
• Website: http://www.ga.gov.au/
• Location: Canberra, Australia
• Twitter:
• Company:
• Icon url: https://avatars.githubusercontent.com/u/4704285?v=4
• Repositories: 333
• Last ynced at: 2024-04-14T15:25:37.066Z
• Profile URL: https://github.com/GeoscienceAustralia

---

GitHub Events

Total

• Issues event: 1

Last Year

• Issues event: 1

---

Committers metadata

Last synced: 8 days ago

Name	Email	Commits
Damien Ayers	d****n@o****t	114
andrewdhicks	a****s@g****u	86
Duncan Gray	d****y@g****u	58
Jeremy Hooke	j****z@s****m	26
Santosh Mohan	s****n@g****u	24
Kirill Kouzoubov	k****8@g****m	7
Simon Oliver	s****r@g****u	5
Emma Ai	e****i	4
Alex Leith	a****h@g****m	3
Ariana Barzinpour	a****r@g****u	3
santoshamohan	3****n	2
Duncan Gray	d****7@r****)	2
David Gavin	G****s	1
FeiZhang2	f****g@g****m	1
dependabot[bot]	4****]	1
eloise-b	e****l@g****m	1
Emma Ai	e****i@g****u	1
Duncan Gray	d****7@r****)	1
Duncan Gray	d****7@r****)	1
Alex Vincent	a****t@g****u	1
harshurampur	h****r@g****u	1

Committer domains:

• ga.gov.au: 8
• raijin1.(none): 1
• raijin6.(none): 1
• raijin2.(none): 1
• stulk.com: 1
• omad.net: 1

---

Issue and Pull Request metadata

Last synced: 2 days ago

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/geoscienceaustralia/fc

---

---

---

Dependencies

Score: 5.683579767338681