KL-Optimal Gaussian Mixture Proxies for Arbitrary Target Densities • proxymix

proxymix fits multivariate Gaussian-mixture proxies that are Kullback–Leibler optimal to user-supplied target densities on $\mathbb{R}^p$ . Three regimes from Hoek and Elliott (2024) are unified under one verb:

Regime	When it applies	Method
(i) moment	$N = 1$ component	Closed-form moment matching
(ii) sample	i.i.d. samples from the target are available	Classical EM
(iii) kld	target density `f(x)` can be evaluated but not (cheaply) sampled	KLD-EM with importance sampling

Regime (iii) is the reason the package exists. The sample-based mixture packages (mclust, mixtools, flexmix) all assume i.i.d. draws from the target; proxymix fits directly against an evaluable (possibly unnormalised) log-density. The nearest CRAN neighbour is AdMit, which adaptively fits a mixture of Student-t distributions to an evaluable kernel as an importance/proposal density; proxymix differs in fitting a Gaussian mixture that is Kullback–Leibler optimal, precisely so that the fitted object then supports the closed-form operator calculus (marginals, conditionals, Bayes updates, products, convolutions, filtering) and carries a fit-quality certificate through every operation.

Why not MCMC? If you can evaluate the unnormalised density you can always run a sampler and then fit a mixture to the draws. The regime-(iii) fit is the shortcut when what you want is the compact closed-form object: no chain tuning or convergence diagnostics, a deterministic pipeline given the seed, and a mixture whose marginals, conditionals, moments and samples are then available in closed form through the operator calculus. The trade-off is dimension: the importance sampling that drives regime (iii) loses effective sample size sharply beyond roughly $p = 5$ – $10$ , and every fit reports its effective sample size so that limit is visible rather than silent. For high-dimensional posteriors, sample with your favourite MCMC and use regime (ii) on the draws.

Installation

From GitHub:

# install.packages("remotes")
remotes::install_github("max578/proxymix")

or locally from the source tree:

R CMD build proxymix
R CMD INSTALL proxymix_*.tar.gz

Documentation site: https://max578.github.io/proxymix/.

Quick start

library(proxymix)

## A target you can evaluate but not sample from — a 2D "banana".
banana <- banana_target()

## Fit a 3-component Gaussian mixture proxy via KLD-EM with importance sampling.
fit <- fit_proxymix(banana, N = 3L, regime = "kld",
                    proposal = is_mvt(n_dim = 2L, df = 5),
                    is_size = 2000L, max_iter = 60L, seed = 1L)

print(fit)
#> <gmm_fit>: regime = "kld", K = 3, p = 2
#>   target     : banana
#>   iterations : 37
#>   converged  : TRUE
#>   [1] w = 0.6456, |mu| = 0.3299, tr(Sigma) = 1.3601
#>   [2] w = 0.2473, |mu| = 1.2996, tr(Sigma) = 2.2312
#>   [3] w = 0.1071, |mu| = 1.8882, tr(Sigma) = 4.9044

## Closed-form operations on the fitted mixture.
gmm_marginalise(fit, keep = 1L)
#> <marginalise(kld_em[N=3] on banana)>: K = 3 components in p = 1 dimensions
#>   [1] w = 0.6456, |mu| = 0.1818, tr(Sigma) = 0.4620
#>   [2] w = 0.2473, |mu| = 1.1518, tr(Sigma) = 0.4846
#>   [3] w = 0.1071, |mu| = 1.5954, tr(Sigma) = 0.6321
gmm_conditionalise(fit, given = c(NA, 0.5))
#> <conditionalise(kld_em[N=3] on banana)>: K = 3 components in p = 1 dimensions
#>   [1] w = 0.6730, |mu| = 0.2620, tr(Sigma) = 0.4524
#>   [2] w = 0.2576, |mu| = 1.1133, tr(Sigma) = 0.2352
#>   [3] w = 0.0694, |mu| = 1.4188, tr(Sigma) = 0.1199

The unified fitting verb

fit_proxymix(
  target,
  N        = 1L,
  regime   = c("auto", "moment", "sample", "kld"),
  ...
)

target is an S7 gmm_target produced by gmm_target() (from a log-density), gmm_target_from_samples() (from samples), or one of the built-in factories (banana_target(), donut_target(), mixture_target()).
regime = "auto" picks the cheapest applicable regime from the target’s contents.

Closed-form mixture operators

dgmm(x, fit)              # density evaluation
rgmm(n, fit)              # exact sampling
gmm_marginalise(fit, keep)
gmm_conditionalise(fit, given)
gmm_kld(fit_p, fit_q)     # Monte Carlo + variational bounds

Mapping the optima of an objective

fit   <- from_objective(f, lower, upper, N = 10L)  # a mixture map of the optima
gmm_modes(fit)$modes                               # the distinct optima

from_objective() treats an objective f as the Gibbs measure exp(-f / T) — a regime-(iii) target you can evaluate but not sample — and returns a closed-form mixture over its low regions, so a multimodal f is recovered as a whole rather than one optimum at a time. gmm_modes() resolves the fitted map into the recovered optima.

Which function do I need?

You have	You want	Reach for
an evaluable (unnormalised) log-density	a compact closed-form proxy	`gmm_target()` then `fit_proxymix(regime = "kld")`
i.i.d. samples	a mixture fit	`gmm_target_from_samples()` then `fit_proxymix()`
an objective function	a map of its optima	`from_objective()`, `gmm_modes()`
a kernel density estimate	a small closed-form surrogate	`from_kde()`
a fitted mixture	marginals, conditionals, updates	`gmm_marginalise()`, `gmm_conditionalise()`, `gmm_observe()`, `gmm_affine()`
data with holes	multiple imputation	`gmm_impute()` with `mar()` / `mnar()` / `censored()`
a time series + state-space model	filtering / stability testing	`gmm_filter()`, `gmm_eos_test()`
a fitted mixture	information-theoretic diagnostics	`gmm_entropy()`, `gmm_divergence()`, `gmm_mutual_information()`
a fitted mixture + treatment data	causal / decision quantities	`gmm_intervene()`, `fit_uplift()`, `proxy_cate()`
an unnormalised posterior	the marginal likelihood	`gmm_evidence()`
a fitted proxy	error bars on any functional	`gmm_fit_ensemble()`, `proxy_functional_ci()`
an evaluable-only target	the component count	`select_N()`
a fit collapsing in high dimension	an adaptive proposal	`fit_kld_em(adapt = "pmc")`

Vignettes

The package ships with twelve vignettes:

quickstart — one-page tour.
posterior_proxy — the flagship workflow: a real unnormalised Bayesian posterior compressed to a proxy, with the evidence, closed-form reads, and bootstrap error bars.
three_regimes — a walk-through of regimes (i)–(iii) on toy 2-D targets, including the agreement of (i) and (iii) at $N = 1$ .
density_shapes — the regime-(iii) demonstration: banana, donut, three-mixture targets fit by importance-sampled KLD-EM.
operator_calculus — closed-form pushforward, Bayesian update, aggregation and conditioning on a fitted mixture.
from_kde — compressing a kernel density estimate into a Gaussian-mixture proxy.
many_methods — one fitted mixture in place of regression, clustering, PCA and ridge regression.
entropy — closed-form entropy, divergence and mutual-information diagnostics.
calibration — mapping the optima of an objective via its Gibbs measure.
missing_data — multiple imputation by conditioning the fitted mixture (missing at random).
missing_data_mnar — imputation under value-dependent missingness and censoring, with sensitivity analysis.
end_of_sample — testing whether the last few observations of a series are consistent with a fitted state-space model.

Interactive tutorial

A standalone, single-file HTML primer for proxymix is hosted at

https://max578.github.io/proxymix-tutorial/

The page is targeted at Adelaide University Mathematics and Statistics undergraduates (Guided depth, default) and postgraduate / PhD readers (Technical depth, via the in-page dial). It runs in the browser with no R install required, and works the same way offline by double-click. Source at max578/proxymix-tutorial.

Reference

Hoek, J. van der and Elliott, R. J. (2024). Mixtures of multivariate Gaussians. Stochastic Analysis and Applications. doi:10.1080/07362994.2024.2372605.

proxymix