Releases | Open Sustainable Technology

EOReader - v0.22.2

0.22.2 (2025-04-18)

FIX: Fix GPT graphs (mutiple errors, such a lonely Subset, a LinearToFromdB step not written, etc.)
FIX: Fixed TSX and TDX Products Regex to properly identify Staring Spotlight products as TsxProduct and TdxProduct - by @guillemc23

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 10 days ago

EOReader - v0.22.1

0.22.1 (2025-04-04)

FIX: Correct the logic behind saving a stack in uint16 in product.stack
FIX: Fix a regression disallowing to pass a custom nodata to product.stack
FIX: Fix COSMO GTC reading

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 24 days ago

EOReader - v0.22.0

0.22.0 (2025-03-25)

BREAKING CHANGES: Remove deprecated features (#94, #95, #96)
- resolution keyword if favor of pixel_size
- GREEN1 band in favor of GREEN_1
- utils.stack_dict function in favor of utils.stack (remove band argument)
- deprecated spectral indices in favor of awesome spectral index ones).
  - AFRI_1_6: AFRI1600
  - AFRI_2_1: AFRI2100
  - BSI: BI
  - NDGRI: NGRDI
  - NDRE1: NDREI
  - RGI: RGRI
  - WV_BI: NHFD
  - WI: WI2015
  - RDI: DSI
  - DSWI: DSWI5
  - GRI: DSWI4
  - WV_SI: NDSIWV
  - PANI: BITM
BREAKING CHANGES: Pass mask-related functions in private as their exposure is done via load now
BREAKING CHANGES: Update the resolution in band paths: 1000-00m instead of 1000m, or 0-50m instead of 0-5m. Set EOREADER_LEGACY_BAND_NAME_RESOLUTION to 1 to keep the older behavior
ENH: Adding the support of masks (#101)
ENH: Adding the support of Sentinel-2 L2A specific bands (AOT, SCL and WV) (#153)
ENH: Adding the support of WorldView Legion constellation (#183)
ENH: Adding the support of Umbra constellation (only GEC file) (#114)
ENH: Add the ability to open a product with a context manager (#133)
ENH: Use the window argument to subset SAR data during SNAP pre-process (#186)
ENH: Add a function computing the centroid of the footprint of the product
ENH: Write intermediary files as COG by default and add a environment variable EOREADER_DEFAULT_DRIVER to overload the default driver (#181)
ENH: Add a path attribute to the bands, corresponding to their path on disk
ENH: Manage COSMO-SkyMed GTC (orthorectified) products
ENH: Raise a proper exception in case of non-existing product path
FIX: Fix toa radiance to toa reflectance computation for VHR data
FIX: Save Cosmo quicklooks as PNG instead of GeoTiffs as they are not georeferenced
FIX: Fix to_str function with as_list=False
FIX: Correctly set the stack attributes when already existing on disk
FIX: Don't convert stack to uint16 in memory with save_as_int argument (only convert the version written on disk)
FIX: Fix reading bands for Sentinel-2 L1C data from Element-84 which have different AWS endpoints between STAC files and their assets
FIX: Don't warp multiple times VHR stacks
INTERNAL: Rationalize band paths and filename functions (still early work) (#31)
DOC: Add documentation on how EOReader can improve your data handling (#108)
DEPS: Add ephem for computing earth-sun distance (according to Maxar's method from docs)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 1 month ago

EOReader - v0.21.10

0.21.10 (2024-02-18)

ENH: Allow the user to change the default resampling per product and free the resolution keyword in load and stack (#103)
ENH: For Sentinel-2 L2A products, use the MSK_CLDPRB_20m for computing clouds, as they are more precise than legacy L1C MSK_CLOUDS_B00 masks. (#174)
FIX: Fix Sentinel-2 Theia footprints when the nodata area is wider than the data area (#201)
FIX: Pop driver keyword in stack function to only use it for writing, allowing people to save stack as COGs (#181, #202)
FIX: Simplify L7 footprint (#198)
FIX: Make RPC reprojection work with DEM stored on cloud (#200)
FIX: Fix DIMAP V2 mask reprojection when all mask values are equal to 1 (resulting in a failure of computing the bands in case of nodata)
FIX: Fix import of SaocomSensorMode and TsxProduct from eoreader.product
FIX: Upgrade to v2.0.0 of projection STAC extension
DEPS: Add tifffile as a dependency to allow to write COGs with dask (#181, #202)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 2 months ago

EOReader - v0.21.9

0.21.9 (2024-01-22)

FIX: Better management of temporary directories (name, cleaning, etc)
FIX: Align Sentinel-3 SLSTR clean band path creation on optical's method
DOC: Update documentation (intersphinx mapping, ...)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 3 months ago

EOReader - v0.21.8

0.21.8 (2024-01-13)

ENH: Add a new type (BandsType) for list of BandType
ENH: Add a new environment variable EOREADER_NOF_BANDS_IN_CHUNKS to control the number of the bands in chunks when using dask. (#178)
ENH: Allow 'auto' in EOREADER_TILE_SIZE, to set chunks="auto" when reading data. (#178)
FIX: Fix stack save_as_int to use updated int values - by @TabeaW
FIX: Fixed PAZ Product Regex to properly indentify PAZ ST products as PAZProduct - by @guillemc23
FIX: Fixed PNEO Product Regex to properly indentify PNEO products as PneoProduct - by @guillemc23
FIX: Fixed preprocessing graph paths in order to support relative paths in more complex environments or contexts - by @guillemc23
FIX: Remove useless _norm_diff function indices.py
FIX: Add a fallback in case map-overlay.kml is not readable for Sentinel-1 data (#180,#182)
FIX: Remove warning about Dask's lock and client
FIX: Don't throw an error in case of missing cloud coverage, only a warning and set the cloud coverage to 0 #159
FIX: Use the sun elevation angle rather than the sun zenith angle for STAC #158
FIX: Create comparison operators for BandNames, removing the xarray RuntimeWarning about sort order is undefined for incomparable objects.
FIX: Add some missing @cache around time-consuming functions
FIX: Set correctly the SAR product type, with adding two types (ORTHO and GEOCODED)
FIX: Fix the computation of parametric spectral indices #193
FIX: Fix retrieval of quicklook path for SAOCOM when already computed
FIX: Write data using windowed=True for very big rasters (> 50 Go) to avoid core dumps
FIX: Fix management of numpy temporary files saved on disk
OPTIM: Cache the access to any archived file list, as this operation is expensive when done with large archives stored on the cloud (and thus better done only once).
CI: Remove useless verbosity in CI
CI: GDAL performance tuning by tweaking rasterio's env
INTERNAL: Switch from setup.py to pyproject.toml #109
INTERNAL: Use ruff instead of black + flake8 + isort
DOC: Update conf.py (remove useless hunks and set Sphinx 7 as base)
DOC: Added the PAZ product guide to the PAZ Product documentation instead of the TerraSAR-X one - by @guillemc23
DEPS: Pin sertit>=1.44.1

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 3 months ago

EOReader - v0.21.7

0.21.7 (2024-11-08)

FIX: Handle ICEYE products with missing quicklook
FIX: Fix Sentinel-1 name with weird PDFs names (i.e. ending with .SAFE-report...)
FIX: Remove multi-swath workaround for Cosmo products if SNAP > 11.0
FIX: By default, try to assign a constellation (in a pure dummy way) to any Product created
FIX: Add ways of knowing if a constellation is a real one or not (i.e. CUSTOM or template such as Maxar)
FIX: Create TDX and PAZ (completely inherited) classes to disambiguate their constellations

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 6 months ago

EOReader - v0.21.6

0.21.6 (2024-10-17)

FIX: Fix (really) window's name coming from a vector with an underscore after it
FIX: Fix clean band path for Sentinel-3 SLSTR products
FIX: Remove an ignored exception when deleting a Product (ValueError: Unknown '__class__' name in 'covariance.compute' hyperparameters)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 6 months ago

EOReader -

0.21.5 (2024-10-17)

FIX: Fix window's name coming from a vector with an underscore after it
FIX: Allow to load numpy pickles stored in S3 buckets
FIX: Add MS and PAN resolution in Landsat Products

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 6 months ago

EOReader - v0.21.4

0.21.4 (2024-10-08)

DEPS: Don't force using geopandas 1.0.0, 0.14.4 should be enough.

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 7 months ago

EOReader - v0.21.3

0.21.3 (2024-10-08)

ENH: Allow the process of Sentinel-1 COGs (provided by the Copernicus DataSpace) for SNAP >= 10 (#172)
ENH: Add a BandType alias for any types that could be a band: a string, a BandNames or any of its children: Spectral, SAR, DEM or Cloud band names
FIX: Anticipate Sentinel C and D platforms in Reader's regexes
FIX: Resolve the inversion of resolution and pixel size between stripmap and sliding_spotlight types for Capella products
FIX: Get better window name (if available) when writing bands on disk (in tmp folder)
FIX: Reject buggy Maxar products (with version 28.4) as the workaround would be too heavy to implement. (#106)
FIX: Fix Despeckle graph with SNAP10 (#177)
OPTIM: Save rasterized masks of DIMAP V2 products on disk to avoid recomputing them (features.rasterize could be a heavy computation that shouldn't be done twice)
COMPAT: EOReader works correctly with SNAP 10 (#165)
PUBLISH: Use PyPI's Trusted Publisher Management mechanism

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 7 months ago

EOReader - v0.21.2

0.21.2 (2024-07-30)

ENH: to_str and to_band: add a as_list argument defaulting to True. When set as False, return a str from to_str and a band from to_band (#138). Thanks @jsetty!
FIX: Sentinel-2 product with StopIteration error (#142)
FIX: Fix error in looking for bands in Sentinel-2 L1C archived products (#168)
FIX: Fix issue with geocoding with unzipped Sentinel-3 OLCI product (#137)
FIX: In SPOT products, METADATA.DIM and IMAGERY.TIF must be at the root of the product (#145)
FIX: Fix Maxar product with QB02 satellite ID (#140)
FIX: Fix ICEYE product when extent file (*.kml) not found (#135)
FIX: Handle RCM and RS2 products that doesn't bundle their extent in a KML file (#155)
FIX: Handle wrongly recognized Planet products because of the recursive nested mtd in the Reader (#169)
FIX: Fix an unknown Planet bug that just appeared ('...Path' has no len())
FIX: Force the loading of DimapV1 bands in float32
FIX: Handle the case where fiona isn't installed anymore (with geopandas 1.0)
FIX: Don't make pystac a mandatory requirement
OPTIM: Search correctly nested metadata in the Reader (without accidentally using a recursive glob)
CI: Fix S3 endpoint management with sertit>=1.37
CI: Remove for now end-to-end tests with Python 3.11 and 3.10.
INSTALL: Remove pystac[validation] (as it is an optional dependency) from setup.py, and create a stac extra feature.

Sustainable Development - Environmental Satellites - Python
Published by remi-braun 9 months ago

EOReader - v0.21.1

0.21.1 (2024-04-03)

ENH: Add a is_stacked parameters for EOReader's Product to document either its bands are delivered stacked or file by file.
FIX: Correct SWIR_CIRRUS spectral band's enum value (to SWIR_CIRRUS instead of CIRRUS), avoiding shadowing cloud band CIRRUS (#131)
FIX: Raise proper exception (UnhandledArchiveError) for archived data that needs to be extracted before use. A warning wasn't enough.
FIX: Remove unused pixel_spacing for SAR Products
FIX: Fix workaround for corrupted Sentinel-2 mask.

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 1 year ago

EOReader - v0.21.0.post0

0.21.0.post0 (2024-01-08)

FIX: Don't force install planetary-computer or stac-asset to use EOReader

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 1 year ago

EOReader - v0.21.0

0.21.0 (2023-01-08)

BREAKING CHANGES: Rename utils.stack_dict to utils.stack since we are stacking datasets and not dict anymore.
BREAKING CHANGES: Band ID for Sentinel-3 OLCI are now int instead of band names (i.e. 7 instead of Oa07. The names don't change).
ENH: Allow to use bands IDs, names and common name added to mapped names when trying to load a spectral band. (#111)
ENH: Manage Sentinel-2 as formatted on the cloud (Element84 or Sinergise's way). (#104)
ENH: Handle Python 3.12. (#113)
ENH: Guard against S1 COG format, not yet handled by SNAP.
ENH: Calibration step for Capella products now exists in ESA SNAP. Add it in pre-processing.
ENH: Handling of Sentinel-1 ASF and MPC RTC products. (#112, #118)
ENH: Handling of Sentinel-1 SM products.
ENH: Better handling of calibration step in SNAP for SAR data.
FIX: Fix jpg, png... quicklooks management when plotting
FIX: Fix an xarray issue when trying to compute percentiles when stacking bands
DEPS: Remove as many mention as possible to cloudpathlib
DEPS: Update minimum versions of some libraries
DOC: Add example about the output management (in base notebook, #117)
DOC: Update copyright to 2024
GITHUB: Update bug template
CI: Enabling pre-commit.ci and dependabot bots
CI: Update pre-commit hooks
CI: Revamping test_satellites
CI: Some refactoring and speed-ups

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 1 year ago

EOReader - v0.20.4

0.20.4 (2023-09-26)

FIX: Don't collocate a raster on itself
FIX: Better management of default pixel size for multi-resolution products (such as PAN band in Landsat)
FIX: Fixing the PAN GSD for Landsat-OLI products
FIX: Update some code to match sertit==1.29.1

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 1 year ago

EOReader - v0.20.3

0.20.3 (2023-07-31)

Bug Fixes

FIX: Minor fix in RasterioError handling when reading bands
FIX: Fix Landsat L2 surface reflectance and temperature computation (#99)
FIX: Fixing TIR temperature conversion for Landsat-7
FIX: Test thermal bands in CI
FIX: Upgraded to EO STAC extension v1.1.0 (#83)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 1 year ago

EOReader - v0.20.2

0.20.2 (2023-06-22)

Bug Fixes

FIX: Use already computed bands stored in tmp for Planet products

Sustainable Development - Environmental Satellites - Python
Published by remi-braun almost 2 years ago

EOReader - v0.20.1

0.20.1 (2023-06-20)

Bug Fixes

FIX: Allow band aliases (such as "CA" instead of "COASTAL_AEROSOL") in string in band mapping when creating Custom Stacks

Other

COMPAT: Add the alias GREEN_1 for GREEN I band of PlanetScope data, in order to stay in the same pattern than VRE_x, SWIR_x... GREEN1 will be deprecated in another release.

Sustainable Development - Environmental Satellites - Python
Published by remi-braun almost 2 years ago

EOReader - v0.20.0

0.20.0 (2023-05-31)

Breaking Changes

BREAKING CHANGES: Switching from resolution to pixel_size to avoid confusion about the definitions (especially for SAR data) (#82)
BREAKING CHANGES: load function now returns a xarray.Dataset (#88)

Bug Fixes

FIX: Collocate bands before trying to create spectral indices: resolve the case where their size mismatches (i.e. in case of window or change of native pixel size)
FIX: Landsat band masking when specifying a custom resolution and a custom window
FIX: Round the default pixel_size of custom stacks
FIX: Convert some fields of STAC items from non JSON serializable dtypes to correct ones
FIX: Fix erroneous property set to _get_raw_crs for Maxar products

Other

ENH: Don't load data into memory when computing indices, ensuring tasks are delayed a bit longer (#58)
DOCS: Add documentation about default CRS (#87)
DEPS: Dropping support of Python 3.8 (#81)
DEPS: Pin sertit to 1.27.0
INTERNAL: Better management of logs for deprecation warnings
INTERNAL: Refactoring simplify_footprint in sertit library
CI: Test that STAC items are serializable when added to a catalog

Sustainable Development - Environmental Satellites - Python
Published by remi-braun almost 2 years ago

EOReader - v0.19.4

0.19.4 (2023-04-12)

Bug Fixes

FIX: Removing calibration step from SNAP pre-processing graph for multi-swath Cosmo-SkyMed 1st GEN products (to avoid ending up with empty images after pre-process)
FIX: Fixing the paths to Sentinel-2 quicklooks: using PVI instead of TCI file if no .jpg preview file is found (#84, #85, thanks a lot @floriandeboissieu)

Other

STAC: Updates in STAC management
INTERNAL: Use geopandas.estimate_utm_crs() when possible

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 2 years ago

EOReader - v0.19.3

0.19.3 (2023-03-24)

Bug Fixes

FIX: Fixing Custom Stacks when specifying datetime=None on creation
FIX: Fix regression for multi-swath DGM CSK data (huge region) (#78)
FIX: Fix calibration issues with CSK HR data (using fallback GPT graph by default)

Other

OPTIM: Always use chunks when reading rasters (#58)
OPTIM: Speed up VRT virtual warping
OPTIM: Better management of dask's usage
CI: Fix projection STAC extension's new version number (1.1.0)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 2 years ago

EOReader - v0.19.2

0.19.2 (2023-02-23)

Bug Fixes

FIX: Fixing stack when saved as integer for some special cases
FIX: Clipping negative reflectances to 0 (#79)
FIX: Fixing nodata mangement for Theia product
FIX: Fixing handling of SCS multi-swath Cosmo-SkyMed products (#78)
FIX: Writing spectral indices on disk to align with other bands (#80)
FIX: By default, calibration is not applied to slant range CSG data, avoiding producing an empty raster (#48)

Other

OPTIM: Using warped VRT instead of reprojecting DEM/VHR stacks to UTM (#58)
TYP: Fixing typos in typing
INTERNAL: Moving EOREADER_NAME and DATETIME_FMT into __init__.py
INTERNAL: Moving stacking function into utils
INTERNAL: Removing unused cache_property decorator
INTERNAL: Factorizing _load function
CI: Using assert_raster_almost_equal_magnitude in CI to better check according to bands' content (sertit 1.24.0)
DEPS: Officially handling Python 3.11 (adding weekly tests on Python 3.11) (#71)
DOC: Updating Custom notebook
DOC: Updating jupyter cache to match new way of handling outputs in readthedocs

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 2 years ago

EOReader - v0.19.1

0.19.1 (2023-01-12)

Bug Fixes

FIX: Fixing a bug for DIMAP V2 products whith GML masks opening without CRS: assigning first the raw CRS before converting to the product's CRS
FIX: Fixing index creation when exotic bands not handled by ASI have been loaded in the same time (i.e. stacking NDWI with Oa21 band)

Other

CLEAN: Removing useless GCP functions regarding Sentinel-3 data
DOC: Adding a Remove Clouds notebook
LIB: Pinning sertit to 1.22.0
CI: Don't run tests when only __init__ or __meta__ is updated
CI: Some factorizing in gitlab-ci

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.19.0

0.19.0 (2023-01-03)

Enhancements

ENH: Adding the support of Capella constellation (#74)
ENH: Allow the user to load bands with a window (pixels and geo) (#25, notebook)

Bug Fixes

FIX: Fix extent computation for CSG products with Shapely 2.0
FIX: Shapely 2.0 deprecation warnings

Other

DEPR: Add deprecation warning for EOReader spectral indices (used for legacy in 0.18.0) that are aliases of ASI names (#72):
- AFRI_1_6: AFRI1600,
- AFRI_2_1: AFRI2100,
- BSI: BI,
- NDGRI: NGRDI,
- NDRE1: NDREI,
- RGI: RGRI,
- WV_BI: NHFD,
- WI: WI2015,
- RDI: DSI,
- DSWI: DSWI5,
- GRI: DSWI4,
- WV_SI: NDSIWV,
- PANI: BITM
DOC: Changing copyright from 2022 to 2023

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.18.1

0.18.1 (2022-12-08)

Bug Fixes

FIX: Fix regression for missing EOReader aliases for spyndex spectral indices

Other

DOC: Add latest DOI link

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.18.0

0.18.0 (2022-12-06)

Breaking Changes

BREAKING CHANGES: Refactoring spectral indices management (#47)
- Using spyndex library, allowing to use all spectral indices
  listed here
- SAR products may now compute indices if possible (see this list)
- Old EOReader indices are still available for legacy purposes, with some changes:
  - For Sentinel-2 data, the band NIR and NARROW_NIR may be interchanged for some index (
    see this discussion)
  - OSAVI formula has changed to stick with the original paper definition (see issue)
  - NDRE2/3 formula are fixed, now using VRE_2/3 and NDRE1 corresponds to NDREI and uses VRE_1
  - CI1 is renamed CI32 and CI2 is renamed CI21 for readability purposes
  - NDWI21 can be written NDWI2100 for homogeneity purposes
  - RDI (or DSI) uses now SWIR_1 instead of SWIR_2 (see this issue)
  - PANI equivalent is now BITM and is normalised ! (divided by 3)
  - SBI is normalized (divided by 2) to fit with BIXS definition
  - ⚠ You may need to install the last spyndex directly from GitHub latest version to have all available indices
BREAKING CHANGES: Using pyresample to geocode Sentinel-3 data (#55)
- Cleaner: better conversion from swath to grid
- Faster: Up to 4 times faster
- Allows code refactoring between OLCI and SLSTR
BREAKING CHANGES: For SAR product types that are not available in the Data Access Portfolio, default resolution is now the pixel spacing instead of the rg x az resolution
- Changes mainly Sentinel-1 default resolutions (except from IW mode)

Enhancements

ENH: Adding the support of Harmonized Landsat-Sentinel constellation (#49)
ENH: Adding the support of GEOSAT-2 constellation (#59)

Bug Fixes

FIX: Fixing CustomProduct initialization when fields are set to None (instead of not declaring them)
FIX: SNAP cannot handle float predictors other than 1! Set it to 1 when saving ortho SAR images to disk, in order for SNAP to be able to despeckle
them (https://forum.step.esa.int/t/exception-found-when-reading-compressed-tif/654/7). (#62)
FIX: Fixing mix in Sentinel-2 mapping for B8 (NIR, 10m resolution, large spectral bandwidth) and B8A (NARROW_NIR, 20m resolution, narrow spectral bandwidth)

Other

DOC: Add FAQ entry concerning SAR constellations extent KML files failing to be read (TLDR: needs ogr2ogr in your
PATH)
DOC: Add Technical Note published in Remote Sensing MDPI in Readme
DOC: Update optical band mapping graphs (fix regression to 0.15.0 supported constellation)
DOC: Add information about DEM management in SAR notebook (#61)
DOC: Updating indices paragraphs
CI: Using actions/checkout@v3
CI: Updating versions of pre-commit hooks
LIBS: Updating requirements.txt and setup.py to add pyresample and zarr

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.17.0

0.17.0 (2022-10-12)

Enhancements

ENH: Adding the support of RapidEye constellation
ENH: Handling Planet data with multiple subdatasets (#45)
ENH: Adding the support of Landsat Level-2 products (#49)
ENH: Adding the support of Pleides Neo SEN and PRJ products (needs GDAL 3.5+ or rasterio 1.3.0+)
ENH: Adding the function bands.is_thermal_band
ENH: Adding the ability for optical custom stacks to load indices
ENH: Adding BAIM (MODIS Burned Area Index) spectral index
ENH: Better management of raw units of the bands of optical products
ENH: Copying files from tmp_process when changing product's output

Bug Fixes

FIX: Stacks saved as integers on disk keep their original dtype (float32) in Python
FIX: Stacks with bands loaded "as is" are correctly saved as integers on disk (#52)
FIX: Using stack CRS (if projected) for DIMAP products instead of recomputing from lat/lon, solving potential discrepencies between stack and product CRS
FIX: Workaround for JP2 bug when updating an existing raster (maybe related to this bug)
FIX: Better management of SkySat datetime conversion from JSON to XML (deterministic way)
FIX: Fixing computation of invalid pixels for Sentinel-2 and DIMAP products (do not remove straylight mask)
FIX: Fixing reprojection resolution of VHR data
FIX: Computing Brightness Temperature of Landsat TIR bands instead of leaving them as is
FIX: Better management of Landsat Instrument values
FIX: Better radiometry attribute (adding brightness temperature and reflectance and brightness temperature values)
FIX: Changing Brilliance Temperature to the correct Brightness Temperature
FIX: Fixing pandas FutureWarning The frame.append method is deprecated and will be removed from pandas in a future version.
FIX: Fixing DeprecationWarning invalid escape sequence \.
FIX: Manage correctly Planet dubious pixels (especially for 8 bands products)

Optimizations

OPTIM: Reduce memory usage when updating all the bands attributes
OPTIM: Reduce memory usage when stacking as integers

Other

DOC: Add the need of using SNAP 8.0 up-to-date or SNAP 9.0 (#42)
DOC: Add the STAC session in API documentation
DOC: Add warnings for shifts when orthorectifying DIMAP SEN products (using RPCs) (#53)
DOC: Add limitations to custom stacks
DEPS: Dropping support of Python 3.7 (#18)
DEPS: Update minimum version of libs (geopadans 0.11.0+, rasterio 1.3.0+...)

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.16.1

0.16.1 (2022-08-03)

Bug Fixes

FIX: Add the missing conversion to reflectance for Sentinel-3 OLCI
FIX: Better condition for the conversion to reflectance for Sentinel-2 THEIA
FIX: Add logs for SkySat data that cannot been converted to reflectance and fix the radiometry field of its band xarrays
FIX: Add the correct nodata (when overridden by the user) to stacks saved as uint16

Optimizations

OPTIM: Reduce memory usage during stacking

Other

CI: Test reflectance values

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.16.0

0.16.0 (2022-08-01)

Enhancements

ENH: Adding the support of SuperView-1 constellation (#21)
ENH: Adding the support of SPOT-4/5 constellations (#39)
ENH: Allow the possibility to pass a constellation (or a constellation list) to Reader().open() to speed up the opening of a product
ENH: Add a quicklook search for Sentinel-3 products

Bug Fixes

FIX: Fix quicklook media type with JP2 files
FIX: Fix Sentinel-3 SLSTR F1 bands based on F grid
FIX: Correct the UTM projection for Sentinel-3 data
FIX: Fix handling of zipped Sentinel-2 L2Ap
FIX: Fix zipped Sentinel-2 with other XML files in GRANULE subdirectories

Other

Renaming master branch to main

Sustainable Development - Environmental Satellites - Python
Published by remi-braun over 2 years ago

EOReader - v0.15.1

0.15.1 (2022-06-02)

Optimizations

OPTIM: Try to create Vision-1 footprint from the preview file instead of from the stack.
OPTIM: Create footprints for stacked products (i.e. Maxar, SkySat, Custom...) without mask by opening only the first band of the stack
OPTIM: Create footprints for Maxar Products with a resolution 10 times lower.
OPTIM: Footprints have now maximum 50 vertices in order to avoid pixelized footprints

Bug Fixes

FIX: Fixing condensed name to avoid duplicates:
- adding the job_id for VHR products
- adding the polarization channels for SAR products
FIX: Remove import of pystac in stac_utils
FIX: Fix bug for Vision-1 data looking for non-existing RPC files in case of ORTP product type
FIX: Fix quicklook regex for Vision-1 data
FIX: Fix regex for raw bands for extracted Sentinel-3 OLCI products
FIX: Fix PlanetScope identifying regex to handle products with a satellite_id containing a letter
FIX: Force metadata regex for Maxar products to look for a file with pattern {name}.XML to avoid other misplaced XML to be found in place of the true XML.
FIX: Fix regression for Landsat-7 footprint
FIX: Manage the case with cloud_cover = -999.0 for Maxar products (returns None)

Other

CI: Add new optical products to be tested for end-to-end tests

Sustainable Development - Environmental Satellites - Python
Published by remi-braun almost 3 years ago

EOReader - v0.15.0

0.15.0 (2022-05-30)

Breaking Changes

BREAKING CHANGES: Optical becomes Spectral when more appropriate
BREAKING CHANGES: Platform and Sensor become Constellation when more appropriate, to fit STAC vocabulary (#29):
- Platform enum becomes Constellation
- prod.platform becomes prod.constellation
- prod.sat_id becomes prod.constellation_id
BREAKING CHANGES: File alias is removed, replaced by *_bands files and proper imports in bands.__init__
BREAKING CHANGES: Product attribute band_names becomes bands in order to be STAC compliant (#29)
BREAKING CHANGES: Better use of NIR and NARROW_NIR in the indices file (according to the gsd of Sentinel-2 bands composing the indices)
BREAKING CHANGES: Correcting Landsat product types to better manage processing levels and instrument. Landsat-8/9 condensed name may change!

Enhancements

ENH: Adding the support of SkySat (Collect) products (#20)
ENH: Bands in mapping are now objects, instead of just IDs (#29). This allows us to:
- Add band metadata (such as center wavelength, bandwidth...)
- Map spectral bands between STAC spec and EOReader format (#29)
- Add a better __repr__ functions
ENH: Handling 8 bands PlanetScope data (#20)
ENH: Adding the GREEN1 mapped band, corresponding to PlanetScope GREEN I and Sentinel-3 OLCI Oa05 band
ENH: Handle some slightly broken Sentinel-2 products:
- when the metadata files are corrupted or when the detfoo vectors are empty (#34)
- with missing MSK prefix for QI_DATA files (i.e DETFOO instead of MSK_DETFOO)
ENH: Handle exception for corrupted bands (in Sentinel-2 and utils.read) (#34)
ENH: Add a STAC object that can be used to retrieve STAC Items from every Product (prod.stac.create_item()) (#29)
ENH: Add a get_mean_viewing_angles for Optical Products to fill STAC View Extension (#29)
ENH: Extending get_raw_band_paths to every product (#31)
ENH: Adding a is_ortho attribute corresponding to when the product is already orthorectified/geocoded, in order to avoid computing heavy processes without wanting it (i.e. footprint...)
ENH: Adding the instrument name of every constellation, under prod.instrument
ENH: Handling COSMO product with only the h5 file in it (if missing XML metadata file) (#36)

Optimizations

OPTIM: Retrieve name from filename if possible
OPTIM: Retrieve extent from metadata when possible (for VHR data)
OPTIM: Refactoring Landsat-XX products into LandsatProduct, this should bee invisible for user.

Bug Fixes

FIX: Fixing the band mapping of WorldView-2/3 Multi (8 bands)
FIX: Retrieval (if possible) of Sentinel-1 unique ID (was missing from the product name, as it is not in the product preview)
FIX: Fixing PAZ/TDX MTD regex
FIX: Optical products: Only set cloud cover and radiometry attributes if spectral bands are asked

Other

INTERNAL: File spot_6 and spot_7 are removed, replaced by a unique spot file. This shouldn't affect the user.
INTERNAL: Some refactoring in VHR files
WARNINGS: Filter warnings from __init__
CI: Do not process two times the zipped Sentinel-1 in end-to-end tests and manage when the runner kills SNAP
DOC: Adding a STAC notebook
DOC: Various updates

Sustainable Development - Environmental Satellites - Python
Published by remi-braun almost 3 years ago

EOReader - v0.14.0

Breaking Changes

BREAKING CHANGES: footprint, extent, wgs84_extent and crs properties are converted back to methods in order to prevent side effects of expensive computation when displaying the object when debugging (rollback before version 0.8.0)
BREAKING CHANGES: get_all_index becomes get_all_indices
BREAKING CHANGES: acquisition_datetime becomes datetime and default_resolutionbecomes resolution for CustomProduct in accepted keywords, and the metadata fields have been renamed according to the CustomFields enum

Enhancements

ENH: Adding spectral indices:
- Shadow Index (SI)
- Global Vegetation Moisture Index (GVMI)
- Soil Brightness Index (SBI), Soil Cuirass Index (SCI)
- Panchromatic mocking Index (PANI)
- Green-to-Red ratio Index (GRI)
- Soil Adjusted Vegetation Index (SAVI)
- Optimized Soil Adjusted Vegetation Index (OSAVI)
- Visible Atmospherically Resistant Index (Green) (VARI)
- Enhanced Vegetation Index (EVI)
- Chlorophyll Index RedEdge VRE_3/VRE_2 (CI1)
- Chlorophyll Index RedEdge VRE_2/VRE_1 (CI2)
- Normalized Difference Moisture Index (with SWIR_21) (NDMI21)
ENH: Making SAR attribute snap_filename public
ENH: Handling ICEYE pure SLC products
ENH: Allowing the user to choose if they want the GRD or SLC image for ICEYE products
ENH: Add the possibility to directly load the cloud cover for optical data (and add it in the band attributes) (#28)
ENH: Add the possibility to retrieve the quicklook path (if existing) and add the plot function allowing the user to plot the quicklook (if existing) (#28)
ENH: Add the possibility to retrieve the orbit direction (and add it in the band attributes) (#28)

Bug Fixes

FIX: Fixing the inversion between 8 and 8A bands for Sentinel-2 and Sentinel-2 Theia products
FIX: Loading every optical band in reflectance (fixed for Sentinel-2 THEIA, Maxar, Planet and Vision-1 data) (#30)
FIX: Fixing ReferenceError: weakly-referenced object no longer exists when deleting an object
FIX: Do not set sea values to nodata when orthorectifying SAR data with SNAP
FIX: Handle Sentinel-2 data with PB < 02.07 as L2Ap products
FIX: Fixing nodata and offset for Sentinel-2 data with PB > 04.00
FIX: Handle new ICEYE metadata name's nomenclature
FIX: Fixing harmless regex error when searching for B1 path for Landsat products
FIX: Fixing platform for Sentinel-2 Theia

Other

DOC: Creating a real base notebook and renaming the old one to optical
DOC: Better type hints (replacing XDS_TYPE by xr.DataArray)
CI: Using sertit.ci.reduce_verbosity instead of recreating the function

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 3 years ago

EOReader - v0.13.1

0.13.1 (2022-03-08)

FIX: Handling Sentinel-2 L2Ap data
FIX: Do not use --no-binary fiona,rasterio directly in requirements.txt (breaks on Windows)
FIX: Fixing stacking with string bands
FIX: Better __repr__ function
CI: Adding a tag for choosing the runners
DOC: Fixing cartopy/GEOS conflicts making the documentation build to fail

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 3 years ago

EOReader - v0.13.0

0.13.0 (2022-03-02)

ENH: Adding the support of Landsat-9 sensor
ENH: Support Sentinel-2 with missing datatake metadata file(sometimes happens with data downloaded from AWS buckets and converted to .SAFE)
FIX: Using default SAR resolution from official Copernicus Data Access Portfolio (2014-2022) (Sentinel-2 default resolution goes to 10.0 m !)
FIX: Use --no-binary fiona,rasterio directly in requirements.txt
FIX: Removing useless outputComplex line in GPT graphs that is breaking SNAP on Linux
FIX: Removing the workarounds caused by some bugs of cloudpathlib and enabling retrieval of nested SAR products (TSX, TDX, PAZ, RCM) from S3 compatible storage.
FIX: Do not process nodata for a band already existing
FIX: Fixing an error when reading TIR bands with Landsat-7
FIX: Fixing an error when additive/multiplicative coefficients are set to NULL for Landsat data
CI: Do not try to process SAR end to end if GPT cannot be found
CI: Publishing wheel from Github instead of Gitlab
REPO: Setting GitHub as the main repository and using new Gitlab runners

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 3 years ago

EOReader - v0.12.0

0.12.0 (2022-02-09)

ENH: Adding the support of Pleiades-Neo, Vision-1 and SAOCOM sensors
ENH: Adding a keyword to allow passing a specific DEM path in load/stack (for VHR orthorectification and DEM bands)
ENH: Adding the name of the DEM in DEM band (i.e. allow to compute the HILLSHADE with a DEM and the SLOPE with a DTM)
FIX: Sentinel-2 Processing Baseline 04.00: NARROW_NIR bands are now loaded correctly
FIX: Maxar products (with Multi band ID) are now correctly handled
FIX: Using COPDEM-30 (GLO-30) by default for SNAP as it appears that the retrieval has been fixed.
FIX: Fixing the default name for cleaned bands for Sentinel-3 SLSTR data (was set on CLEAN instead of NODATA)
FIX: Fixing default band for Custom stacks
FIX: Fixing get_existing_band_paths behavior for Custom stacks
FIX: Remove other never covered lines of code (archived RCM products, complex ICEYE products, others...)
FIX: Re-enabling loading str bands (regression)
FIX: Proper check for empty fields when parsing metadata
FIX: VHR _get_dem_path raises ValueError instead of TypeError
FIX: Pre-process SAR bands before despeckling if not existing (was OK in most of the cases, but broke in some cases, especially with CI folder activated and S3 compatible storage)
FIX: Remove warning invalid escape sequence \., \w, \D and \s
FIX: Do not set long_name for RAW_CLOUDS arrays
FIX: Providing a URL DEM on Windows throws a OSError instead of a bare Exception
OPTIM: Do not pre-process existing Sentinel-3 geocoded bands
OPTIM: Do not look for valid metadata further than a given nested level in product's directory (for extracted products)
CI: Using another (faster) runner
CI: Add on disk and end-to-end tests
CI: Do not write tmp files when running on disk tests
CI: Coverage:
- Get coverage as HTML
- Remove useless lines from coverage
- Combine coverage of S3 and on disk tests
DOC: Adding a DEM notebook

Sustainable Development - Environmental Satellites - Python
Published by remi-braun about 3 years ago