Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Recent Releases of wetterdienst

wetterdienst - Fix values coercing

  • Change integer dtypes in untidy format to float to prevent loosing information when converting units

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Change some hourly parameter mappings

  • Improve integrity of dataset, parameter and unit enumerations with further tests
  • Change source of hourly sunshine duration to dataset sun
  • Change source of hourly total cloud cover (+indicator) to dataset cloudiness

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Small fixes

  • Fix naming of sun dataset
  • Fix DWD Observation monthly test

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Fix monthly/annual data of DWD observations

  • Fix monthly/annual data of DWD observations

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Add Python 3.10

  • Simplify parameters using only one enumeration for flattened and detailed parameters
  • Rename dataset SUNSHINE_DURATION to SUN to avoid complications with similar named parameter and dataset
  • Rename parameter VISIBILITY to VISIBILITY_RANGE
  • Add datasets EXTREME_WIND (subdaily) and MORE_WEATHER_PHENOMENA (daily)
  • Add support for Python 3.10 and drop Python 3.7

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Extend explorer

  • Extend explorer to use all implemented APIs
  • Fix cli/restapi: return json and use NULL instead of NaN

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Several fixes

  • Fix missing station ids within values result
  • Add details about time interval for NOAA GHCN stations
  • Fix falsely calculated station distances

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Add Wetterdienst Settings

  • Add Wetterdienst.Settings to manage general settings like tidy, humanize,...
  • Rename DWD forecast to mosmix
  • Instead of "kind" use "network" attribute to differ between different data products of a provider
  • Change data source of NOAA GHCN after problems with timeouts when reaching the server
  • Fix problem with timezone conversion when having dates that are already timezone aware

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 3 years ago

wetterdienst - Fix cli

  • Fix cli error with upgraded click ^8.0 where default False would be converted to "False"

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 3 years ago

wetterdienst - Caching solely with fsspec

  • Fix access to ECCC stations listing using Google Drive storage
  • Remove/replace caching entirely by fsspec (+monkeypatch)
  • Fix bug with DWD intervals

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 3 years ago

wetterdienst - New NOAA GHCN API

  • Add NOAA GHCN API
  • Fix radar index by filtering out bz2 files

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 3 years ago

wetterdienst - Fix for timestamps of 1 minute / 10 minute data provided by DWD

  • [FIX] Add missing positional dataset argument for _create_empty_station_parameter_df
  • [FIX] Timestamps of 1 minute / 10 minutes DWD data now have a gap hour at the end of year 1999
    due to timezone shifts

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 3 years ago

wetterdienst - Start migrating from dogpile.cache to filesystem_spec

  • Start migrating from dogpile.cache to filesystem_spec

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 3 years ago

wetterdienst - Query specific parameters from dataset

Added

  • Enable selecting a parameter precisely from a dataset by passing a tuple like [("precipitation_height", "kl")] or
    [("precipitation_height", "precipitation_more")], or for cli/restapi use "precipitation_height/kl"
  • Test of restapi with recent period

Fixed

  • Bug when querying an entire DWD dataset for 10_minutes/1_minute resolution without providing start_date/end_date,
    which results in the interval of the request being None
  • Get rid of pandas performance warning from DWD Mosmix data

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 3 years ago

wetterdienst - Fix radar

  • Bugfix acquisition of DWD radar data
  • Adjust DWD radar composite parameters to new index

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 4 years ago

wetterdienst - Bugfix tidy method for DWD observation data

  • Bugfix tidy method for DWD observation data

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 4 years ago

wetterdienst - Bugfix show method

  • Update readme on sandbox developer installation
  • Fix show method

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 4 years ago

wetterdienst - New cli base, fixed InfluxDB export

  • Change cli base to click
  • Add support for wetterdienst core API in cli and restapi
  • Export: Use InfluxDBClient instead of DataFrameClient and improve connection handling with InfluxDB 1.x
  • Export: Add support for InfluxDB 2.x
  • Fix InfluxDB export by skipping empty fields
  • Add show() method with basic information on the wetterdienst instance

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 4 years ago

wetterdienst - Fix DWD Mosmix generator

  • Make tidy method a abstract core method of Values class
  • Fix DWD Mosmix generator to return all contained dataframes

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 4 years ago

wetterdienst - Add units

  • Add origin and si unit mappings to services
  • Use argument "si_units" in request classes to convert origin units to si, set to default
  • Improve caching behaviour by introducing optional WD_CACHE_DIR and
    WD_CACHE_DISABLE environment variables. Thanks, @meteoDaniel!
  • Add baseline test for ECCC observations
  • Add DWD Observation hourly moisture to catalogue

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 4 years ago

wetterdienst - Merge with wetterdienst-ui

  • Add capability to export data to Zarr format
  • Add Wetterdienst Explorer UI. Thanks, @meteoDaniel!
  • Add MAC ARM64 support with dependency restrictions
  • Radar: Verify HDF5 responses instead of returning invalid data
  • Add support for stations filtering via bbox and name
  • Add support for units in distance filtering
  • Rename station_name to name
  • Rename filter methods to .filter_by_station_id and .filter_by_name, use same convention for bbox, filter_by_rank
    (previously nearby_number), filter_by_distance (nearby_distance)
  • Mosmix: Use cached stations to improve performance

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 4 years ago

wetterdienst - Make .discover return lowercaser parameters and datasets

  • Make .discover return lowercase parameters and datasets

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 4 years ago

wetterdienst - Add API for Environment and Climate Change Canada

  • Use direct mapping to get a parameter set for a parameter
  • Rename DwdObservationParameterSet to DwdObservationDataset as well as corresponding
    columns
  • Merge metadata access into Request
  • Repair CLI and I/O subsystem
  • Add capability to export to Feather- and Parquet-files to I/O subsystem
  • Deprecate support for Python 3.6
  • Add --reload parameter to wetterdienst service for supporting development
  • Improve spreadsheet export
  • Increase I/O subsystem test coverage
  • Make all DWD observation field names lowercase
  • Make all DWD forecast (mosmix) field names lowercase
  • Add Environment and Climate Change Canada API
  • Rename humanize_parameters to humanize and tidy_data to tidy
  • Radar: Use OPERA as data source for improved list of radar sites

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 4 years ago

wetterdienst - Add new workflow to go from selected stations to values

  • Add StationsResult and ValuesResult to allow for new workflow and connect stations and
    values request
  • Add accessor .values to Stations class to get straight to values for a request
  • Rename Stations to Request and use upper camel case e.g. DwdObservationRequest
  • Add top-level API
  • Fix issue with Mosmix station location

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 4 years ago

wetterdienst - Fix date filtering

  • Fix date filtering of DWD observations, where accidentally an empty dataframe was
    returned

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 4 years ago

wetterdienst - Fix Mosmix station locations

  • DWD: Add missing radar site "Emden" (EMD, wmo=10204)
  • Mosmix stations: fix longitudes/latitudes to be decimal degrees (before they were
    degrees and minutes)
  • Change key STATION_HEIGHT to HEIGHT, LAT to LATITUDE, LON to LONGITUDE
  • Rename "Data" classes to "Values"
  • Make arguments singular

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj about 4 years ago

wetterdienst - Make use of common resolution and period

  • Create general Resolution and Period enumerations that can be used anywhere
  • Create a full dataframe even if no values exist at requested time
  • Add further attributes to the class structure
  • Make dates timezone aware
  • Restrict dates to isoformat

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Fix 10minutes fileindex intervals

  • Fix 10minutes file index interval range by adding timezone information

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Move functionality to core components, use timezones for dates

  • Move more functionality into core classes
  • Add more attributes to the core e.g. source and timezone
  • Make dates of internal data timezone aware, set start date and end date to UTC
  • Add issue date to Mosmix class that actually refers to the Mosmix run instead of start
    date and end date
  • Use Result object for every data related return
  • In accordance with typical naming conventions, DWDObservationSites is renamed to
    DWDObservationStations, the same is applied to DWDMosmixSites
  • The name ELEMENT is removed and replaced by parameter while the acutal parameter set
    e.g. CLIMATE_SUMMARY is now found under PARAMETER_SET
  • Remove StorageAdapter and its dependencies
  • Methods self.collect_data() and self.collect_safe() are replaced by self.query() and
    self.all() and will deprecate at some point

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst -

  • Bump h5py to version 3.1.0 in order to satisfy installation on Python 3.9

Atmosphere - Meteorological Observation and Forecast - Python
Published by amotl over 4 years ago

wetterdienst - Fix InfluxDB exports and increase efficiency and performance

  • InfluxDB export: Fix export in non-tidy format (#230). Thanks, @wetterfrosch!
  • InfluxDB export: Use "quality" column as tag (#234). Thanks, @wetterfrosch!
  • InfluxDB export: Use a batch size of 50000 to handle larger amounts of data (#235). Thanks, @wetterfrosch!
  • Update radar examples to use wradlib>=1.9.0. Thanks, @kmuehlbauer!
  • Change wherever possible column type to category
  • Increase efficiency by downloading only historical files with overlapping dates if start_date and end_date are given
  • Use periods dynamically depending on start and end date
  • Fix inconsistency within 1 minute precipitation data where historical files have more columns
  • Improve DWD PDF parser to extract quality information and select language.
    Also, add an example at example/dwd_describe_fields.py as well as
    respective documentation.

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst -

Atmosphere - Meteorological Observation and Forecast - Python
Published by amotl over 4 years ago

wetterdienst - Improve MOSMIX subsystem and various refactorings

  • CLI: Obtain "--tidy" argument from command line
  • Extend MOSMIX support to equal the API of observations
  • DWDObservationSites now filters for those stations which have a file on the server
  • DWDObservationData now also takes an individual parameter
    independent of the pre-configured DWD datasets by using DWDObservationParameter or
    similar names e.g. "precipitation_height"
  • Newly introduced coexistence of DWDObservationParameter and DWDObservationParameterSet
    to address parameter sets as well as individual parameters
  • Imports are changed to submodule thus now one has to import everything from
    wetterdienst.dwd
  • Renaming of time_resolution to resolution, period_type to period, several other
    relabels

Atmosphere - Meteorological Observation and Forecast - Python
Published by amotl over 4 years ago

wetterdienst - Large refactoring and full coverage of DWD radar data

  • Large refactoring
  • Make period type in DWDObservationData and cli optional
  • Activate SQL querying again by using DuckDB 0.2.2.dev254. Thanks, @Mytherin!
  • Fix coercion of integers with nans
  • Fix problem with storing IntegerArrays in HDF
  • Rename DWDStationRequest to DWDObservationData
  • Add DWDObservationSites API wrapper to acquire station information
  • Move discover_climate_observations to DWDObservationMetadata.discover_parameters
  • Add PDF-based DWDObservationMetadata.describe_fields()
  • Upgrade Docker images to Python 3.8.6
  • Move intermediate storage of HDF out of data collection
  • Fix bug with date filtering for empty/no station data for a given parameter
  • Radar data: Add non-RADOLAN data acquisition

Atmosphere - Meteorological Observation and Forecast - Python
Published by amotl over 4 years ago

wetterdienst - Add multiple storage pipelines, MOSMIX forecast and API

  • Add TTL-based persistent caching using dogpile.cache
  • Add example/radolan.py and adjust documentation
  • Export dataframe to different data sinks like SQLite, DuckDB, InfluxDB and CrateDB
  • Query results with SQL, based on in-memory DuckDB
  • Split get_nearby_stations into two functions, get_nearby_stations_by_number and
    get_nearby_stations_by_distance
  • Add MOSMIX client and parser. Thanks, @jlewis91!
  • Add basic HTTP API

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Enhance documentation, add test for example notebook, fix bug with TimeResolution enumeration

  • Add test for Jupyter notebook
  • Add function to discover available climate observations (time resolution, parameter, period type)
  • Make the CLI work again and add software tests to prevent future havocs
  • Use Sphinx Material theme for documentation
  • Fix typo in enumeration for TimeResolution.MINUTES_10

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Enhance get_nearby_stations

  • Enhance the usage of get_nearby_stations
  • Output of get_nearby_stations is a slive of meta_data DataFrame

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Add RADOLAN support

  • add RADOLAN support
  • change module and function naming in accordance with RADOLAN

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Extend DWDStationRequest to work with multiple parameters

  • extend DWDStationRequest to take multiple parameters as request
  • add documentation at readthedocs.io
  • [cli] Adjust methods to work with multiple parameters

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj over 4 years ago

wetterdienst - Establish black codestyle, fix type casting, add tidy data option

  • establish code style black
  • setup nox session that can be used to run black via nox -s black for one of the supported
    Python versions
  • add option for data collection to tidy the DataFrame (properly reshape) with the
    "tidy_data" keyword and set it to be used as default
  • fix integer type casting for cases with nans in the column/series
  • fix humanizing of column names for tidy data

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 5 years ago

wetterdienst - Major updates to parsing speed, client extension, more specific dtypes, further small fixes

  • [cli] Add geospatial filtering by distance.
  • [cli] Filter stations by station identifiers.
  • [cli] Add GeoJSON output format for station data.
  • improvements to parsing high resolution data by setting specific datetime formats and changing to concurrent.futures
  • fix na value detection for cases where cells have leading and trailing whitespace
  • change column name mapping to more explicit one with columns being individually addressable
  • add full column names for every individual parameter
  • more specific type casting for integer fields and string fields

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 5 years ago

wetterdienst - Bug fixes, new parameter definition, new option for near stations

The release includes some bug fixes and fixes for tests.

Besides that it is now possible to use the enumeration names themselves for parameters e.g. "climate_summary" instead of "kl" to define those parameters. Also the function get_nearby_stations (former get_nearest_station) has now an argument "max_distance_in_km" to select stations based on distance instead of simple ranking.

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 5 years ago

wetterdienst - Initial release

The first release of wetterdienst. The release brings a new name, updates to the readme, a changelog, etc...

Atmosphere - Meteorological Observation and Forecast - Python
Published by gutzbenj almost 5 years ago