Data

Tools

Indexes

Applications

Experiments

Open Sustainable Technology

Recent Releases of C-Star

C-Star - v0.4.0

v0.4.0

This is a bug-fix release focused on stabilizing the orchestration of blueprints.

Breaking Changes

  • Rename env var group name constants to omit leading underscore
  • Modified the interpretation of Status.is_running to exclude Unsubmitted

New features

  • Enable status retrieval for multiple SLURM jobs in a single sacct call
  • The orchestrator writes a sentinel file to disk to dynamically alter dependencies
  • Add trace log level to reduce log frequency during Workplan execution
  • Persist WorkplanRun record for every invocation of the orchestrator for run history and lookup
  • Add cstar workplan monitor --run-id <run-id> for reattaching to a running Workplan

Security Fixes

  • N/A

Bug Fixes

  • Fix failure to override output directories for orchestrated blueprints
  • Fix defect where steps defined before dependencies were not mapped correctly after time-splitting
  • Fix unhandled exceptions in cstar [blueprint|workplan] check with invalid paths
  • Fix posix-path conversion bug when passing blueprint URLs to cstar blueprint run
  • Fix case-sensitivity bug when configuring split frequency via CSTAR_ORCH_TRX_FREQ env variable
  • Fix a JSON serialization bug with pydantic models using field aliases
  • Fix incorrect working directories resulting from LiveStep.from_step when a parent is specified
  • Fix failure to terminate health check thread
  • Fix issue when parsing SLURM status of the form CANCELLED by <username>
  • Fix incorrect env var discovery if env_ appears repeatedly in variable name
  • Fix unhandled exception caused by attempting to execute typer app
  • Fix cstar workplan status ... triggering a re-build of the entire workplan

Improvements

  • Improve error handling when CachedRemoteRepositoryRetriever.refresh fails
  • Validate inputs prior to execution with cstar [blueprint|workplan] run
  • Improve module load times
  • Extract nested/duplicate YAML representer functions for re-use
  • Replace use of incorrect mp.Queue where queue.Queue is appropriate
  • Replace blocking time.sleep usage in health check thread
  • Removed slow status retrieval loop for previously submitted jobs
  • Ensure env var display consistency in cstar env show output
  • Enable "path-less" CLI commands, such as cstar workplan status --run-id <id>.
  • Display detailed status information for tasks in a Workplan

Miscellaneous

  • N/A

Full commit history

For more details, please refer to the commit history.

What's Changed

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.3.0...v0.4.0

Emissions - Carbon Capture - Python
Published by ankona 3 months ago

C-Star - v0.3.0

v0.3.0

This is a bug-fix release focused on stabilizing the orchestration of blueprints.

Breaking Changes

  • Environment variables CSTAR_HOME and CSTAR_OUTDIR are deprecated in favor of CSTAR_CONFIG_HOME and CSTAR_STATE_HOME
  • Relocated command conversion functions from cstar.orchestration.launch.slurm to cstar.orchestration.converter
  • Promote blueprint overrides to always-on, deprecate and remove feature flag CSTAR_FF_ORCH_TRX_OVERRIDE

New features

  • Add remote repository caching for re-use across simulations
  • XDG base directory standard is implemented

Security Fixes

  • N/A

Bug Fixes

  • Fix path concatenation bug resulting in invalid source data paths
  • Fix indentation creating an unreachable code block in git utils
  • Add conda installation of rsync to environment.yml (fixes bug related to a flag used in newer ucla-roms makefiles that is not available in MacOS's default rsync installation)
  • Reimplement CSTAR_CLOBBER_WORKING_DIR for single-blueprint cases
  • Fix bug handling user-relative paths in JobFileSystemManager

Improvements

  • Avoid automatic failures by retrying failed git clone operations
  • Add informational logging when tasks fail and additional tasks are cancelled
  • Update default log format to include file name and line number
  • Add constants for the set of available feature flags
  • Implement CSTAR_LOG_LEVEL env-var
  • Automatically retry partitioning without coarse dims if it fails when they are included
  • Enable orchestrated simulation execution using the local runner

Miscellaneous

  • Additional details added in docstrings
  • Silence OMP warning originating from numba
  • Modify a blueprint in the tests directory for a refactored layout in the remote data repository
  • Move env-var handling methods to separate module to avoid circular imports
  • Remove __init__-based import shortcuts
  • Adjust error handling and presentation in the worker/service layer

What's Changed

New Contributors

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.2.0...v0.3.0

Emissions - Carbon Capture - Python
Published by ankona 4 months ago

C-Star - v0.2.0

Summary

This release focuses on introducing capabilities enabling the orchestration of
one-to-many heterogeneous simulations.

Breaking Changes

  • The blueprint schema introduced in :ref:v0.0.1-alpha is deprecated
  • Deprecate the use of the user environment file

New features

  • Add SimulationRunner entrypoint for blueprint execution
  • Add public orchestration models in cstar.orchestration.models
  • Add Workplan and Blueprint orchestration in cstar.orchestration.orchestration
  • Add cstar CLI for command-line execution of Workplan's and Blueprint's
  • Add optional dependencies when launching jobs with SLURM

Security Fixes

N/A

Bug Fixes

  • Fix logging failure when file directories do not exist
  • Fix incorrect number of tracers in vertical mixing configuration
  • Fix failure occurring in worker post-run handler
  • Fix/modify how the "active allocation" is identified
  • Fix env var expansion failures due to lmod and environment loading ordering
  • Fix missing partitioning information in ROMSInputDataset
  • Fix simulation failures due to file collisions

Improvements

  • Overhaul documentation with a focus on Blueprint orchestration
  • Improved error logging and message de-duplication
  • Improved developer experience; additional docstrings, increased typehint coverage, expanded linting rules
  • Centralized calculation of various shared strings and paths
  • Parallelized ncjoin operations

Miscellaneous

  • Mitigate various linter warnings
  • Pin roms-tools to 3.4.0

What's Changed (Autogenerated)

New Contributors

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.1.0...v0.2.0

Emissions - Carbon Capture - Python
Published by ankona 4 months ago

C-Star - v0.1.0

Summary

New features:

  • Add support for river forcing
  • Add support for forcing corrections (used in legacy ROMS configurations)

Breaking Changes:

  • Merge Case and Component classes into a single class, Simulation, simplifying internal/blueprint structure. Remove old Case and Component modules.
  • Rename caseroot to directory
  • Rename BaseModel to ExternalCodeBase
  • Remove start_time and end_time parameters from InputDataset.get(), these are now obtained from the corresponding attributes
  • Rename ROMSComponent.namelists to ROMSSimulation.runtime_code and ROMSComponent.additional_source_code to ROMSSimulation.compile_time_code

Internal Changes:

  • Update calls to roms-tools to reflect latest changes in API
  • Update internal/test blueprints to reflect new structure
  • New backend data retrieval system cstar.io
  • Add ROMSRuntimeSettings class with ability to parse and create roms .in files
  • Save partitioned ROMSInputDatasets in the same directory as their un-partitioned versions, rather than a subdirectory "PARTITIONED"

What's Changed (Autogenerated)

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.7...v0.1.0

Emissions - Carbon Capture - Python
Published by dafyddstephenson 6 months ago

C-Star - v0.08-alpha

Improve Case execution handling and state persistence between sessions

Summary

  • Add Case.persist() and Case.restore() methods to allow continuation of work in a new session
  • Add ExecutionHandler class to track tasks run locally (LocalProcess subclass) or submitted to a job scheduler (SchedulerJob subclass)
  • Improved tracking of local InputDataset and AdditionalCode files to prevent repeat fetching
  • Add ability to read blueprint from URL
  • Change remote yaml handling so checksums no longer required for verification

What's Changed (Autogenerated)

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.7...v0.08-alpha

Emissions - Carbon Capture - Python
Published by dafyddstephenson over 1 year ago

C-Star - v0.0.7

Attempt to correct version number on PyPI

What's Changed

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.6...v0.0.7

Emissions - Carbon Capture - Python
Published by dafyddstephenson over 1 year ago

C-Star - v0.0.6

What's Changed

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.5-alpha...v0.0.6

Emissions - Carbon Capture - Python
Published by NoraLoose over 1 year ago

C-Star - v0.0.5-alpha

Attempt to correct version number displayed on PyPI.

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.4-alpha...v0.0.5-alpha

Emissions - Carbon Capture - Python
Published by TomNicholas over 1 year ago

C-Star - v0.0.4-alpha

This is the same release as v0.0.3-alpha but re-released to trigger the automatic publishing workflow to PyPI.

What's Changed

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.3-alpha...v0.0.4-alpha

Emissions - Carbon Capture - Python
Published by NoraLoose over 1 year ago

C-Star - v0.0.3-alpha

What's Changed

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.2-alpha...v0.0.3-alpha

Emissions - Carbon Capture - Python
Published by dafyddstephenson over 1 year ago

C-Star - v0.0.2-alpha

What's Changed

This is the same release as v0.0.1-alpha but re-released to trigger the automatic publishing workflow to PyPI.

Full Changelog: https://github.com/CWorthy-ocean/C-Star/compare/v0.0.1-alpha...v0.0.2-alpha

Emissions - Carbon Capture - Python
Published by NoraLoose over 1 year ago

C-Star - v0.0.1-alpha

The first release of C-Star!

This release provides basic functionality, including the ability to create, import, and export a blueprint for reproducible ocean model simulations (a C-Star Case) using supported ocean models (ROMS, optionally with MARBL biogeochemistry) and run those Cases locally or on supported HPC systems (via Slurm and PBS).
There is support for using existing model input data in netCDF format, or creating new input data via integration with the roms-tools library.

Note that the python API is not yet stable, and some aspects of the schema for the blueprint will likely evolve.
Therefore whilst you are welcome to try out using the package, we cannot yet guarantee backwards compatibility.
We expect to reach a more stable version in Q1 2025.

Full Changelog: https://github.com/CWorthy-ocean/C-Star/commits/v0.0.1-alpha

Emissions - Carbon Capture - Python
Published by TomNicholas over 1 year ago