esa_cci_sm

Readers and converters for ESA CCI Soil Moisture image and time series data.
https://github.com/TUW-GEO/esa_cci_sm

Category: Natural Resources
Sub Category: Soil and Land

Keywords

cci climers esa preprocess python read sm soil-moisture

Keywords from Contributors

transforms measur archiving observation conversion projection generic earth-science animals optimize

Last synced: 1 day ago
JSON representation

Repository metadata

Conversion of ESA CCI SM images into time series

• Host: GitHub
• URL: https://github.com/TUW-GEO/esa_cci_sm
• Owner: TUW-GEO
• License: mit
• Created: 2018-10-09T14:53:05.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2024-11-07T15:53:46.000Z (6 months ago)
• Last Synced: 2025-04-17T20:39:57.875Z (10 days ago)
• Topics: cci, climers, esa, preprocess, python, read, sm, soil-moisture
• Language: Python
• Homepage: https://www.esa-soilmoisture-cci.org/
• Size: 4.66 MB
• Stars: 14
• Watchers: 2
• Forks: 11
• Open Issues: 1
• Releases: 6
• Metadata Files:
  • Readme: README.rst
  • Changelog: CHANGELOG.rst
  • License: LICENSE.txt
  • Authors: AUTHORS.rst

README.rst

          ============
esa_cci_sm
============

|ci| |cov| |pip| |doc|

.. |ci| image:: https://github.com/TUW-GEO/esa_cci_sm/actions/workflows/ci.yml/badge.svg?branch=master
   :target: https://github.com/TUW-GEO/esa_cci_sm/actions

.. |cov| image:: https://coveralls.io/repos/TUW-GEO/esa_cci_sm/badge.png?branch=master
  :target: https://coveralls.io/r/TUW-GEO/esa_cci_sm?branch=master

.. |pip| image:: https://badge.fury.io/py/esa_cci_sm.svg
    :target: http://badge.fury.io/py/esa_cci_sm

.. |doc| image:: https://readthedocs.org/projects/esa_cci_sm/badge/?version=latest
   :target: http://esa-cci-sm.readthedocs.org/


Tools to convert `ESA CCI SM `_ image files into a time series format.

Installation
------------

This python package and all required dependencies can be installed from
`pypi `_
via `pip `_

.. code-block:: shell

    pip install esa_cci_sm

On macOS if you get ``ImportError: Pykdtree failed to import its C extension``,
then it might be necessary to install the `pykdtree` package from conda-forge

.. code-block:: shell

    conda install -c conda-forge pykdtree

Data download
-------------

Download ESA CCI SM data files either from the public
`CEDA data store via https `_
or the CEDA FTP server using, e.g. `FileZilla `_
or `wget `_

**Host (FTP)**: ``anon-ftp.ceda.ac.uk`` no password or user required

**Directory (FTP)**: ``/neodc/esacci/soil_moisture``)

E.g the following command will download v9.1 COMBINED data for the year 2023 from the CEDA FTP via wget

.. code-block:: shell

    wget -np -nH --cut-dirs 7 -r ftp://anon-ftp.ceda.ac.uk/neodc/esacci/soil_moisture/data/daily_files/COMBINED/v09.1/2023/

Time series creation
--------------------

After installing this package via pip, you have access to the command line tool to convert ESA CCI SM image files
to CF conform time series. We use an Orthogonal multidimensional array representation
as implemented in the `pynetCF `_ python library.

Note that we assume that the downloaded images are stored in yearly subfolders
like

.. code-block::

    /tmp/img/
    ├── 1978/
    │   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-19781101000000-fv09.1.nc
    │   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-19781102000000-fv09.1.nc
    │   ├── ...
    ...
    ├── 2023/
    │   ├── ...
    │   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-20231231000000-fv09.1.nc

The following command would then take the daily images from 1991
to 2023 in the path ``/tmp/img`` and convert data for grid cells over land into times series.
Time series are then stored in ``/tmp/ts``.

.. code-block:: shell

    ccism_reshuffle /tmp/img /tmp/ts 1991-01-01 2023-12-31 --land_points True

Afterwards, in python, the data can be read as pandas DataFrames.

.. code-block:: python

    >> from esa_cci_sm.interface import CCITs
    >> ds = CCITs("/tmp/ts", ioclass_kws={'read_bulk': True})
    >> ds.read(15, 45)  # lon, lat
                      sm  sm_uncertainty  flag  ...  mode  sensor            t0
    1991-01-01  0.424880        0.094507     0  ...     1       2   7670.175000
    1991-01-02       NaN             NaN    24  ...     2       2           NaN
    1991-01-03       NaN             NaN     8  ...     0       2           NaN
    ...              ...             ...   ...  ...   ...     ...           ...
    2023-12-29  0.495448        0.039983     0  ...     3   21536  19720.051575
    2023-12-30  0.426107        0.055060     0  ...     3   16416  19721.147066
    2023-12-31  0.390103        0.030294     0  ...     3   21600  19722.117129


Supported Products
==================

At the moment this package supports ESA CCI soil moisture data versions
3 to 9 in netCDF format (reading and time series creation)
with a spatial sampling of 0.25 degrees.

Contribute
==========

We are happy if you want to contribute. Please raise an issue explaining what
is missing or if you find a bug. We will also gladly accept pull requests
against our master branch for new features or bug fixes.

Setup
-----
Setup of a complete development environment with `conda
`_ can be performed using the following
commands:

.. code-block:: shell

  git clone [email protected]:TUW-GEO/esa_cci_sm.git --recursive esa_cci_sm
  cd ./esa_cci_sm
  conda create -n esa_cci_sm python=3.12
  conda activate esa_cci_sm
  pip install -e .[testing]

To checkout our testdata files, you need to have
`Git LFS `_ installed on your machine.

Guidelines
----------

If you want to contribute please follow these steps:

- Fork the esa_cci_sm repository to your account
- Clone the repository, make sure you use ``git clone ... --recursive`` to also get
  the test data repository.
- make a new feature branch from the esa_cci_sm master branch
- Add your feature
- Please include tests for your contributions in one of the test directories.
  We use pytest so a simple function called test_my_feature is enough
- submit a pull request to our master branch

        

---

Owner metadata

• Name: TU Wien - Department of Geodesy and Geoinformation
• Login: TUW-GEO
• Email:
• Kind: organization
• Description:
• Website: http://geo.tuwien.ac.at
• Location: Vienna, Austria
• Twitter:
• Company:
• Icon url: https://avatars.githubusercontent.com/u/5213740?v=4
• Repositories: 53
• Last ynced at: 2023-03-11T04:57:44.456Z
• Profile URL: https://github.com/TUW-GEO

---

GitHub Events

Total

• Release event: 1
• Issue comment event: 1
• Push event: 3
• Pull request event: 4
• Fork event: 1
• Create event: 1

Last Year

• Release event: 1
• Issue comment event: 1
• Push event: 3
• Pull request event: 4
• Fork event: 1
• Create event: 1

---

Committers metadata

Last synced: 6 days ago

Name	Email	Commits
Wolfgang Preimesberger	w****r@g****t	76
Manuel Schmitzer	m****r@t****t	45
wpreimes	M****3	5
c-hydro	3****o	3
FabioDelogu	f****u@c****g	1
dependabot[bot]	4****]	1
pbutting	3****g	1
tscanlon	t****n@g****t	1

Committer domains:

• geo.tuwien.ac.at: 2
• cimafoundation.org: 1
• tuwien.ac.at: 1

---

Issue and Pull Request metadata

Last synced: 2 days ago

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/TUW-GEO/esa_cci_sm

---

---

---

Package metadata

• Total packages: 1
• Total downloads: unknown
• Total dependent packages: 0
• Total dependent repositories: 0
• Total versions: 5

---

Dependencies

Score: -Infinity