MARIO - v1.1.1

MARIO v1.1.1

This bugfix release rebuilds the GTAP Power MRIO parser around a record-based
assembly engine shared by the CSV and GDX backends, fixing crashes and silent
data losses on recent GTAP exports, and adds an opt-in structured row layout
for the GTAP factor and satellite blocks.

GTAP parser rewrite

The parser now maps every record onto integer axis positions and accumulates
values directly into dense blocks, so the build cost scales with the number
of records instead of the size of the cartesian label space. A full GTAP
2023 Power MRIO CSV bundle (~7 GB, ~140M records) parses end-to-end in
about 80–90 seconds, where the previous implementation ran for 20+ minutes
before failing.
CSV reading prefers the multithreaded pyarrow engine (with a silent
fallback to the default engine) and logs per-file and per-block progress.

Fixes

Duplicated record keys no longer crash the CSV path. Recent GTAP CSV
exports flatten several GDX symbols into one file (e.g. combustion and
non-combustion emissions), so the same key can legitimately appear more than
once: duplicated keys are now summed instead of raising
ValueError: cannot handle a non-unique multi-index.
Output- and value-added-based emission accounts are captured. The CSV
export encodes them with the SRC="TOT" placeholder; they were silently
dropped — about one third of the emission record mass in the GTAP 2023
bundle — and now land in the domestic satellite rows of the destination
region (e.g. EMI_CH4_dms_QO), mirroring how the GDX bundles store them.
No more spurious EMI_0_* satellite rows produced by the fill step of
the previous CSV implementation (which also raised a TypeError on the
pandas 3 string dtype).
E and EY expose one dense, aligned satellite row set on both
backends, instead of the data-dependent asymmetric row sets of the previous
CSV path.
Any satellite record mass the parser cannot attribute is now reported with a
parser warning instead of being silently dropped.

New: optional structured row layouts for GTAP

parse_gtap gained an opt-in matrix_layouts argument:
```
db = mario.parse_gtap(
    path="path/to/bundle",
    input_format="auto",
    matrix_layouts={"V": ("Region", "Sector"), "E": ("Region", "Sector")},
)
```
Factor and satellite rows are exposed as a semantic
(Region, Sector, item) MultiIndex instead of the historical flat string
names (MTAX_AUS_GRO becomes ("AUS", "GRO", "MTAX")), and the matching
block specifications are registered on the database. Values are identical to
the flat parse — only the index representation changes.
Row families without a region or sector of their own (PTAX, VAAD/VTAX
endowments, ETAX, DTAX/ITAX, domestic satellite accounts) carry the
"TOTAL" sentinel on the levels that do not apply. The sentinel is not
part of the Region/Sector sets and passes through aggregation untouched, so
no extra entry is needed in aggregation mapping files.
The default (matrix_layouts=None) keeps the historical flat rows, so
existing workflows are unaffected.

Notes

The GDX backend requires the GAMS Python API; the pip package
gamsapi[transfer] is sufficient to read GDX bundles.
GTAP databases parsed with earlier MARIO versions differ from v1.1.1 output
by design: the emission block now includes the previously dropped output-
and value-added-based accounts, and E/EY share one aligned row set.

Compatibility

Requires Python 3.11–3.13.
No API breaking changes since v1.1.0; the matrix_layouts argument is
additive and off by default.

Full changelog: https://github.com/it-is-me-mario/MARIO/compare/v1.1.0...v1.1.1

Industrial Ecology - Input Output Model - Python
Published by LorenzoRinaldi 24 days ago

MARIO - v1.1.0

MARIO v1.1.0

This release centers on scenario-based composition updates — rewriting the
electricity supply and trade mixes of a table while preserving its column
totals — together with a reworked sector disaggregation workflow, RAS
balancing, and richer shock authoring.

Electricity supply & trade mixes

Database.update_supply_mix — rewrite one region's generation mix in
place, preserving every buyer's total: across z/Y on IOT and u/Yc
(commodity) or the supply block s (activity market shares) on SUT. The
"electricity" mode derives the mix from EMBER generation for a chosen
year (aggregating to the compatible fuel groups and redistributing back).
Database.get_mix reads a production mix back. update_supply_mix_iot /
update_mix_iot remain as IOT-only aliases.
Database.pool_trade — pool a commodity's trade behind a
"- supply"/"- need" pass-through layer, so its technology mix and trade
mix live in two independent market-share columns.
Database.update_trade_mix — the dual of the supply mix: redistribute one
item across its origin regions per destination market. The "electricity"
mode reads the ENTSO-E import mix.
Live data fetch — the "electricity" modes can fetch EMBER / ENTSO-E
directly with an API key (mario.set_api_keys(...), a <PROVIDER>_API_KEY
env var, or mario/settings/api_keys.yaml); packaged snapshots keep both
methods working offline with no key.
New "Updating the electricity supply and trade mix" user-guide workflow
and API reference.

Sector disaggregation & balancing

Database.add_sectors — new VA_fix (augment missing factor-of-production
rows from parent sectors) and accept_non_unitary_sum options;
read_add_sectors_excel reads grouped inventories from multiple sheets and
accepts split=; corrected total-output/flow computations after inserting a
sector.
Sector split (split=True) — reworked routine: exclusions and
factor-of-production clusters, a residue threshold, a faster and better
disaggregation formulation, and no accidental output-folder overwrite. A
complete end-to-end example lives in
MRIO-disaggregation-model.
Database.ras — balance the Z block of an IOT scenario to target row
and column margins with the biproportional RAS algorithm.

Shocks

shock_calc(base_scenario=...) — build a shock on top of any existing
scenario, so shocks can be chained.
New Supply mix N and Trade mix N shock types in the shock
workbook (IOT z/Y; SUT u/s/Yc), with template picklists — the mix
updates authored without code.

GHG & plotting

calc_ghg names the aggregated satellite account after its GWP basis when
label is omitted (e.g. "GHG AR6 GWP-100").
Plot helpers always write the figure HTML to disk and no longer fail when the
inline Jupyter backend is unavailable.

Settings & dependencies

Default compute.compute_method changed from auto to inverse.
Adds entsoe-py (live ENTSO-E fetch, raw XML client only — pandas-3 safe) and
nbformat as dependencies.

Full changelog: https://github.com/it-is-me-mario/MARIO/compare/v1.0.3...v1.1.0

Industrial Ecology - Input Output Model - Python
Published by LorenzoRinaldi 28 days ago

MARIO - v1.0.3

MARIO v1.0.3

New features

EMERGING-E parser

parse_emerging now supports the variant="E" (EMERGING-E) bundle, parsing the power-disaggregated table (146 sectors, with electricity split into 14 generation sub-sectors) from EMERGING_E_<year>.mat.
A sibling label workbook is auto-detected, with fallback to a packaged EMERGING-E sector classification. Use labels_path= to override detection.
CO₂ projection: when only the standard EMERGING_CO2_<year> companion is available, the 7 fuel rows of the aggregated electricity sector are projected onto the EMERGING-E generation sub-sectors through a fixed fuel-to-technology mapping. Use co2_path= to point to a specific companion file.

Matrix-specific export — `Database.export`

Added Database.export(matrices, scenarios=..., format="csv"|"txt"|"parquet", split="scenario"|"matrix", include_meta=False, **meta_overrides), which writes selected matrices as flat long-format files with Scenario, Matrix, Unit and Value columns.
Flow matrices carry the numerator unit; coefficient and exploded matrices (f_ex, m_ex, p_ex and SUT variants) carry the numerator/denominator ratio.
Output can be split one-per-scenario or one-per-matrix, with an optional metadata header (one row per scenario, or key/value metadata in the parquet schema). release_date and other metadata fields can be supplied at export time.
A filters argument restricts exported rows to selected axis labels, e.g. filters={"Satellite account": ["CO2"]}. Keys may be suffixed with _from/_to to target a single side.
In one-per-matrix mode, a flow and its coefficient (Z/z) no longer collide on case-insensitive filesystems: the coefficient token is doubled (zz).

Documentation and packaging

Added the API reference page for Database.export and updated the export methods overview.
Aligned the documentation version string to 1.0.3.
Updated installation notes: Python 3.11, 3.12 and 3.13 are supported; Python 3.14 is not supported yet (the numpy==2.1.1 pin has no wheel for 3.14 and the source build fails).

Compatibility

Requires Python 3.11–3.13.
No breaking changes since v1.0.2.

Full changelog: https://github.com/it-is-me-mario/MARIO/compare/v1.0.2...v1.0.3

Industrial Ecology - Input Output Model - Python
Published by LorenzoRinaldi about 2 months ago

MARIO - v1.0.2

v1.0.2

Packaging and installation


* Installed Parquet support by default by shipping the package with the
  required dependency out of the box, refreshed the installation guidance,
  refined optional-dependency messaging, and added package-metadata regression
  coverage. Resolved issue `#142 <https://github.com/it-is-me-mario/MARIO/issues/142>`_.

SUT scenarios and native matrix workflows

Extended SUT scenario updates so unified Z, Y and E inputs are
accepted directly and normalized through the shared ordering logic, with
dedicated regression tests for the new update paths. Resolved issue
#144 <https://github.com/it-is-me-mario/MARIO/issues/144>_.
Switched SUT export and parser roundtrips to native SUT blocks instead of
legacy unified IOT-style matrices, while preserving backward-compatible
parsing of historical TXT, Parquet and Excel layouts. Resolved issue
#146 <https://github.com/it-is-me-mario/MARIO/issues/146>_.

Parser and export roundtrip fixes


* Fixed ``coefficients``-mode parsing of native SUT and special-layout IOT
  bundles from TXT and Parquet, which raised ``KeyError: 'Z'`` because the
  index builders only looked up the upper-case flow matrix name.
* Fixed the explicit SUT Excel export so productive columns are written in the
  same commodity-first order as the rows, allowing workbooks with overlapping
  ``Activity``/``Commodity`` item labels to round-trip correctly.
* Fixed flat TXT/CSV parsing of standard IOT bundles, which raised
  ``ValueError: not enough values to unpack`` due to an inconsistent parser
  return signature.
* Added a full export/parse roundtrip test matrix covering the ``standard`` and
  ``special`` layouts across Excel, TXT and Parquet in both ``flows`` and
  ``coefficients`` modes.

Industrial Ecology - Input Output Model - Python
Published by LorenzoRinaldi 2 months ago

MARIO - v1.0.1

MARIO v1.0.1 expands core calculation workflows with new exploded price-contribution accessors and improved regional subset/export handling, including sectorized external-trade layouts for IOT databases.
This release also strengthens Parquet support via the new mariopy[parquet] extra, hardens PyPI packaging metadata, and refreshes installation, API, shock-workflow, and transformation documentation.

Industrial Ecology - Input Output Model - Python
Published by LorenzoRinaldi 2 months ago

MARIO - v1.0.0

v1.0.0 is a major step forward from the 0.3.x series. MARIO has been substantially reorganized around a clearer architecture and a more explicit public API centered on Database, with legacy logic moved out of core/tools into dedicated api, compute, ops, parsers, views, storage, and internal modules. The release also expands compute capabilities with stronger SUT support, product-based technology assumptions, new trade-analysis APIs, built-in GHG calculation profiles, exploded multiplier/footprint accessors, and a new unified Database.plot(...) workflow.

Compared with 0.3.x, v1.0.0 also significantly broadens data handling. Parsing and export are now unified across Excel, TXT, and Parquet; richer matrix layouts are supported; direct parser coverage has been expanded across major datasets; and downloaders have been cleaned up into a clearer interface. Structural operations were upgraded as well, including a refactored add_sectors workflow, better aggregation behavior, support for VY, and improved handling of SUT assumptions and transformations. Finally, the release comes with a much larger automated test suite, stronger regression coverage, pandas 3 compatibility fixes, and a broad documentation refresh.

Industrial Ecology - Input Output Model - Python
Published by LorenzoRinaldi 3 months ago

MARIO - v0.3.5

exiobase 3.9.4 parser added
oecd parser added
documents improvements

Industrial Ecology - Input Output Model - Python
Published by mohammadamint over 1 year ago

MARIO - v0.3.4

Parsing functions error fixed
Recent pandas versions have changed the way they interpret "None" in DataFrames indices and values, which are currently interpreted it as NaN. This mario update fixes the issue by replacing NaN with the string "None" when parsing excel files.

Deprecated functions
Parser for old-fashioned Eurostat SUTs is deprecated. This function relied on peculiarly structured SUTs formats. In case you need to parse such SUTs, please rearrange them into the standard MARIO format. You can check the MARIO format from 'SUT.xlsx' file in the mario/test directory in this repository.

Industrial Ecology - Input Output Model - Python
Published by mohammadamint almost 2 years ago

MARIO - v0.3.3

What's Changed

To excel fix bug fixed in flow model
new figaro donwloader to opt for the new table release

Industrial Ecology - Input Output Model - Python
Published by mohammadamint almost 2 years ago

MARIO - v0.3.2

Bug fixes in installation and setup.py

Industrial Ecology - Input Output Model - Python
Published by mohammadamint about 2 years ago

MARIO - v0.3.1

Bug fixes on FIGARO_SUT_parser

Industrial Ecology - Input Output Model - Python
Published by mohammadamint about 2 years ago

MARIO - V0.3.0

Release History

v 0.3.0

Settings


New functionalities are provided to allow the user to change some naming convensions in mario indexing and input-output nomenclature convensions in mario.

Isard to Chenery-Moses Transformation

The transformation implies moving from trades accounted in the USE matrix to trades accounted in the SUPPLY matrix.

Data Templates


New functionalities are added to create an enpty IO/SU tables  from tabular data.

Figaro Parser
~~~~~~~~~~~~~

New download and parsing functionalities are added to parser figaro database.


Table Downloader

Donwload functions are added to the software. Some of the download functions are using pymrio database download functionalities, and some other databases are mario exclusive.

Deprecated functions


is_productive and backup methods are deprecated.

Improvements
~~~~~~~~~~~~

* The add_sector function imprvements are implemented to make the code faster.
* Updating dependencies versioning (specifically pandas, numpy and xlsxwriter) 


Documentation
~~~~~~~~~~~~~

* The tutorials are updated to improve the readiblity and quality of the juputer notebook functionalities.
* New templates for the readthedocs.

Industrial Ecology - Input Output Model - Python
Published by mohammadamint about 2 years ago

MARIO - v0.2.2

Industrial Ecology - Input Output Model - Python
Published by mohammadamint almost 3 years ago

MARIO -

Minor bugs in V0.2.0 Fixed

Industrial Ecology - Input Output Model - Python
Published by mohammadamint about 3 years ago

MARIO -

Industrial Ecology - Input Output Model - Python
Published by mohammadamint over 3 years ago

MARIO - V0.2.0

New Release of mario with new features, and improved performance!

What's Changed

Pymrio integration by @mohammadamint in https://github.com/it-is-me-mario/MARIO/pull/14
Unit test features
Unifying Exiobase parser functions
new get_data function with improved functionality
Documentation update
Parse eurostat improvements
Parse EORA SUT
Bugs fixed

Full Changelog: https://github.com/it-is-me-mario/MARIO/compare/v0.1.0...v2.0.0

Industrial Ecology - Input Output Model - Python
Published by mohammadamint over 3 years ago

MARIO - MARIO v0.1.0

Initial release

Industrial Ecology - Input Output Model - Python
Published by nigolred over 4 years ago

Recent Releases of MARIO

MARIO - v1.1.1

MARIO v1.1.1

GTAP parser rewrite

Fixes

New: optional structured row layouts for GTAP

Notes

Compatibility

MARIO - v1.1.0

MARIO v1.1.0

Electricity supply & trade mixes

Sector disaggregation & balancing

Shocks

GHG & plotting

Settings & dependencies

MARIO - v1.0.3

MARIO v1.0.3

New features

EMERGING-E parser

Matrix-specific export — Database.export

Documentation and packaging

Compatibility

MARIO - v1.0.2

v1.0.2

MARIO - v1.0.1

MARIO - v1.0.0

MARIO - v0.3.5

MARIO - v0.3.4

MARIO - v0.3.3

What's Changed

MARIO - v0.3.2

MARIO - v0.3.1

MARIO - V0.3.0

v 0.3.0

MARIO - v0.2.2

MARIO -

MARIO -

MARIO - V0.2.0

What's Changed

MARIO - MARIO v0.1.0

Matrix-specific export — `Database.export`