Skip to contents

PESTO

Parameter ESTimation Optimised – the PEST approach (Doherty 2015), brought to R and optimised, with APSIM as its flagship simulator partner.

PESTO is a high-performance R package for model-independent parameter estimation and uncertainty quantification. It brings the algorithms of PEST (Parameter ESTimation; Doherty 2015) and its C++ successor PEST++ (White et al. 2020) – notably the iterative ensemble smoother (IES) – natively into R, and is the first R-native implementation of that algorithm family. It adds a typed forward-model contract, an in-process simulator callback, multi-fidelity acceleration, covariance inflation / localisation, and surrogate methods.

PESTO is model-independent: any R function mapping parameters to outputs can be the forward model. Its flagship partner is APSIM, the agricultural-systems simulator, coupled in-process through the apsimx package – the use case PESTO’s callback was built for. The same callback couples any other process-based simulator just as directly: the groundwater / hydrology models of PEST’s own heritage, other crop models, and compartmental ODE systems.

Key Features

Feature Description
Fast ensemble solvers C++ (RcppEigen) implementation of IES and GLM update equations
In-process callback Couple any R forward model directly – no file-exchange overhead
APSIM coupling First-class apsim_callback() adapter drives apsimx ensembles in-process – PESTO’s flagship simulator partner
Adaptive SVD Automatic backend selection: randomised SVD, LAPACK, or Eigen BDCSVD
Surrogate-accelerated IES Gaussian-process surrogates skip the model on confident realisations – regime-dependent savings (large on smooth responses; near-zero with graceful fallback otherwise)
Inflation & localisation Counter finite-ensemble under-dispersion and spurious correlations
Adaptive ensemble sizing ESS-based diagnostics prevent over/under-sampling
PEST++ integration Read/write .pst control files, run PEST++ executables from R
Publication-ready plots Convergence, ensemble distributions, identifiability, surrogate diagnostics

Performance

The optimised in the name is concrete – three measured properties, with the full definitions and maths in the Benchmarking PESTO against PEST and PEST++ vignette:

  • Faster, at matched accuracy. Against PEST 18.25 and pestpp-ies 5.2.16 on identical problems, PESTO is roughly 40–860x faster in wall-clock (median per inversion) while matching their posterior accuracy – the cause is in-process evaluation and a C++ update kernel avoiding per-solve file exchange, not fewer model solves.
  • Recovers the right answer. It recovers known-true parameters in a twin experiment and brackets the independent apsimx optimiser’s optimum on real observed data (the APSIM case study vignette).
  • Fewer expensive solves when it can. The GP surrogate skips the model on confident realisations; the saving is measured and regime-dependent, with a graceful fall-back to full evaluation (the Surrogate-accelerated IES vignette).

Where PESTO does not win – raw-interval calibration, total solve count – is stated just as plainly in the vignettes.

Installation

The canonical home for PESTO is max578/PESTO, published through the author’s personal CRAN-track channel and the max578.r-universe.dev registry. The AAGI-AUS/PESTO remote is retained as a read-only mirror.

# Development version from GitHub
# install.packages("pak")
pak::pak("max578/PESTO")

# Pre-built binaries from r-universe
install.packages("PESTO", repos = c(
  "https://max578.r-universe.dev",
  "https://cloud.r-project.org"
))

CRAN submission is in preparation.

Vignettes. GitHub source installs (pak, install_github) do not build vignettes by default, so browseVignettes("PESTO") will be empty. Install from r-universe (above) for pre-built vignettes, or add build_vignettes = TRUE to an install_github() call. All articles are also on the package website.

Bayesian inference with the Iterative Ensemble Smoother (IES)

PESTO performs approximate Bayesian inference for parameter estimation. You specify a prior as a parameter ensemble – a matrix of draws encoding your prior belief about each parameter (rows are realisations, columns are parameters) – together with the observations and their error standard deviation. PESTO conditions the ensemble on the observations through the IES update and returns an approximate posterior ensemble with uncertainty diagnostics. Any R function mapping a parameter matrix to an observation matrix can be the forward model.

library(PESTO)
set.seed(1)

# Forward model: y = G %*% theta (any R function theta-matrix -> obs-matrix works)
G <- matrix(rnorm(6 * 3), 6, 3)
forward <- function(theta) theta %*% t(G)
truth <- c(a = 1, b = -0.5, c = 2)
obs <- stats::setNames(as.numeric(G %*% truth) + rnorm(6, sd = 0.05),
                       paste0("o", 1:6))

# Prior: an ensemble of parameter draws (here a broad Gaussian over 3 parameters)
prior <- matrix(rnorm(80 * 3), 80, 3, dimnames = list(NULL, names(truth)))

fit <- pesto_ies_callback(forward, prior, obs, obs_sd = 0.05, noptmax = 6,
                          verbose = FALSE)
colMeans(as.matrix(fit$par_ensemble[, -1]))   # posterior mean -> (1, -0.5, 2)
#>          a          b          c 
#>  1.0136949 -0.4936717  1.9847292

Optional convergence-based early stopping (phi_tol), covariance inflation (pesto_inflation()), and localisation (pesto_localisation()) are documented in ?pesto_ies_callback and the Getting started vignette.

Coupling to APSIM and other simulators

The forward model above is a plain R function. To calibrate a process-based simulator, supply an adapter that maps a parameter vector to that simulator’s outputs. PESTO’s flagship partner is APSIM: apsim_callback() turns an .apsimx model file plus a parameter map into exactly the forward-model callable the IES driver expects, running the ensemble in-process via apsimx.

# Requires the apsimx package and a working APSIM installation.
fm <- apsim_callback(
  template         = "wheat.apsimx",
  param_map        = list(rue = "[Wheat].Leaf.Photosynthesis.RUE.FixedValue"),
  output_extractor = function(sim) sim$Wheat.AboveGround.Wt
)
fit <- pesto_ies_callback(fm, prior, obs, obs_sd = 0.05, noptmax = 6)

The full worked example is in the Calibrating APSIM with PESTO vignette (it needs APSIM installed, so its run blocks are disabled on CRAN). The same pesto_ies_callback() pattern couples any other simulator: a CRAN-safe, fully-runnable agricultural demonstration using the built-in crop_growth_forward_model() is in that vignette, and the apsim-callback article shows Python-bridge and multi-fidelity variants.

Low-level kernels

The C++ update kernels are exported for advanced use (no PEST++ binary needed):

set.seed(42)
npar <- 100; nobs <- 200; nreal <- 50
upgrade <- ensemble_solution(
  par_diff   = matrix(rnorm(npar * nreal), npar, nreal),
  obs_diff   = matrix(rnorm(nobs * nreal), nobs, nreal),
  obs_resid  = matrix(rnorm(nobs * nreal), nobs, nreal),
  par_resid  = matrix(rnorm(npar * nreal), npar, nreal),
  weights    = abs(rnorm(nobs)) + 0.1,
  parcov_inv = abs(rnorm(npar)) + 0.1,
  Am         = matrix(rnorm(npar * (nreal - 1)), npar, nreal - 1),
  cur_lam    = 1.0
)
dim(upgrade)
#> [1]  50 100

The classical .pst-file path is also supported:

pars <- data.table::data.table(
  parnme = paste0("k", 1:10), partrans = "log", parchglim = "factor",
  parval1 = runif(10, 0.1, 10), parlbnd = 0.001, parubnd = 1000,
  pargp = "hydraulic"
)
obs <- data.table::data.table(
  obsnme = paste0("h", 1:20), obsval = rnorm(20, 5, 1),
  weight = 1.0, obgnme = "heads"
)
pst <- create_pest_scenario(pars, obs, "python model.py")
write_pst(pst, file.path(tempdir(), "model.pst"))

Dependencies

  • R >= 4.1.0
  • C++17 compiler (clang on macOS, g++ on Linux, Rtools on Windows)
  • LAPACK/BLAS (bundled with R)

R-package dependencies are declared in DESCRIPTION (Imports: Rcpp, data.table, ggplot2, S7, yaml, digest).

Documentation

Contributing

Contributions are welcome. See CONTRIBUTING.md for the development workflow and pull-request convention; bug reports and feature requests go through GitHub Issues. All participants abide by the Code of Conduct.

Citation 

citation("PESTO")

Moldovan, M. (2026). PESTO: Parameter ESTimation Optimised. R package version 0.8.0. https://github.com/max578/PESTO

Acknowledgements

PESTO builds on the algorithmic legacy of the PEST++ project (US Geological Survey) and the underlying PEST framework by John Doherty. Developed at Adelaide University; the surrogate-acceleration, adaptive-ensemble-sizing, multi-fidelity, and convergence-aware components are original contributions of the author.

License

GPL (>= 3). See LICENSE.md for the full text.