Skip to contents

PESTO Ensemble Manifest (S7 class)

Source: R/manifest.R
pesto_ensemble_manifest.Rd

A versioned, hashed, provenance-tracked container for the result of a PESTO ensemble run. It is a portable data contract: a documented, versioned format that any downstream tool can read without depending on PESTO internals, carrying enough provenance to make a run independently reproducible and independently verifiable (via verify_manifest()).

Arguments

schema_version

Character. Semantic version of the manifest schema itself (default "1.1.0"; 1.1.0 adds the optional obs_schema grounding descriptor over 1.0.0).

run_id

Character. Short identifier (auto-generated by as_manifest() if not supplied).

obs_schema

Optional grounded semantic descriptor (or NULL). A list(outputs = <data.frame>, params = <data.frame>) giving the meaning of each output / parameter column so downstream consumers can check correspondence (unit + quantity) by name rather than trusting positional convention. outputs columns: name, quantity, unit (+ optional provenance verified_on, oracle_kind, evidence_path); params columns: name, apsim_node, unit (+ same optional provenance). Build one with pesto_obs_schema(). The descriptor is provenance metadata — it is not folded into data_hash (correspondence is grounded by a second manifest that agrees on units, not by self-hash). Default NULL (absent).

params

data.frame / data.table. Parameter ensemble; first column real_name, subsequent columns one per parameter.

outputs

data.frame / data.table. Simulated observation ensemble (same shape conventions as params).

weights

Named numeric. IES observation weights (1/obs_sd).

obs_target

Named numeric. Target observations.

seed

Integer scalar. RNG seed used (NA_integer_ if unrecorded).

data_hash

Character. sha256:... checksum over (params, outputs, weights, obs_target, seed).

fidelity

Structured provenance list or NULL. For a multi-fidelity run pesto_ies_callback() records list(type, schedule, final_level, n_levels, costs); NULL for a single-fidelity run. A legacy named-numeric tag is still accepted.

apsim_version

Character. APSIM version string (NA_character_ when forward model is non-APSIM).

pesto_version

Character. PESTO package version that produced the run.

timestamp

POSIXct. Construction time.

method

Character. One of "ies_callback", "ies_pst", "mda", "surrogate".

noptmax

Integer. Number of IES iterations.

lambda_schedule

Numeric vector. Marquardt lambdas per iteration.

failure_rate

Numeric in [0, 1]. Fraction of forward evaluations that returned NA.

Details

Construct directly via the class constructor, or — preferred — via as_manifest() applied to a pesto_ies_callback_result.

Persistence

See write_manifest() / read_manifest() for YAML serialisation and verify_manifest() for integrity checking. The YAML file is the authoritative human-readable entry; params, outputs, and the assimilation context (weights + obs_target) are emitted as sidecar files whose relative paths are recorded inside the YAML. Three sidecar modes:

  • format = "rds" (default, verifiable) — RDS only. IEEE 754 doubles round-trip bit-exactly; SHA-256 integrity check holds.

  • format = "both" (verifiable) — RDS sidecars plus parallel *_inspection.csv files. Hash bound to RDS; CSVs are decorative.

  • format = "csv_unverified" (not verifiable, renamed from "csv" in PESTO 0.3.2) — CSV-only sidecars. Hash is recorded but cannot be recomputed from disk (CSV formatter precision loss ~1 ULP). verify_manifest() returns ok = NA + message. Use only for one-way export to non-R analysts; round-trip integrity is by construction broken.

The YAML carries an explicit integrity: field ("verifiable" / "not_verifiable") derived from format so downstream non-R tools can branch on the integrity contract without needing to know the PESTO-specific format vocabulary.

Examples

set.seed(1)
npar <- 2L; nobs <- 3L; nreal <- 30L
G <- matrix(stats::rnorm(nobs * npar), nobs, npar)
y <- as.numeric(G %*% c(0.5, -1.0)) + stats::rnorm(nobs, sd = 0.05)
names(y) <- paste0("o", seq_len(nobs))
prior <- matrix(stats::rnorm(nreal * npar), nreal, npar,
                dimnames = list(NULL, c("a", "b")))
fit <- pesto_ies_callback(function(t) t %*% t(G), prior, y, 0.05,
                          noptmax = 3, verbose = FALSE)
m <- as_manifest(fit, seed = 1L)
verify_manifest(m)$ok
#> [1] TRUE