Returns the fit-quality certificate: a small list recording the fitting
regime, convergence, degeneracy, the effective-sample-size profile of
the importance weights, and (when a validation split was drawn) the
held-out validation gap. The certificate is stamped into the object's
metadata at fit time and carried through every closed-form operator: a
unary operator (marginal, conditional, affine map, Bayes update) preserves
it unchanged, and an operator with several operands (product, convolution,
mixing) combines their certificates conservatively – worst-case of each
field, with the individual per-operand certificates retained under
quality_sources and regime set to "composite". It can therefore be
read off a marginal, a conditional, a product, a filtered belief, or any
other derived mixture – alongside the provenance vector recording the
chain of operations that produced it.
Value
A list with elements regime, converged, degenerate,
ess, ess_relative, min_component_ess, max_weight,
support_fraction, kld_final, kld_approx, validation_gap, and
heldout_kld (fields not applicable to the regime are NA), or NULL
for a mixture that was never fitted (e.g. built directly with gmm()).
A certificate produced by an operator over several operands has
regime = "composite", the conservative worst-case value of each numeric field across the
operands, and an additional quality_sources element holding the
operands' own certificates. kld_final is estimated on the importance
draws the fit was tuned to, so it reads low; validation_gap is the
held-out validation_kld (see ess_summary()) minus kld_final, and is
NA when the fit was made with validation_size = 0. heldout_kld is
the KL divergence from the target to the proxy estimated on the
validation draws. For a target with an unknown normalising constant it
is validation_kld minus a log normalising constant estimated from the
same draws. kld_approx is half the importance-weighted variance of
log f - log g over the fitting draws. It approximates the KL
divergence only when the proxy is close to the target, and it does not
depend on the target's normalising constant. Both are NA outside
regime "kld".
Details
Downstream verbs read the same certificate and raise a one-shot
advisory (class proxymix_low_quality) when the source fit is flagged. A
fit is flagged when it did not converge, when it is degenerate (fewer
effective importance draws than min_ess), or when its held-out KL
divergence heldout_kld exceeds 0.3 nats. A fit made with
validation_size = 0 has no held-out estimate, and is flagged when
kld_approx exceeds 0.3. That fallback misses a proxy wider than the
target, because such a proxy can keep kld_approx below 0.3 while its KL
divergence is large. Relative ESS is not used: with a fixed proposal it
describes the proposal, not the fit, and stays the same however good the
fit is.
Examples
fit <- fit_proxymix(banana_target(), N = 2L, regime = "kld",
is_size = 1500L, max_iter = 15L, seed = 1L)
gmm_fit_quality(fit)
#> $regime
#> [1] "kld"
#>
#> $converged
#> [1] FALSE
#>
#> $degenerate
#> [1] FALSE
#>
#> $ess
#> [1] 1048.118
#>
#> $ess_relative
#> [1] 0.6987454
#>
#> $min_component_ess
#> [1] 165.3181
#>
#> $max_weight
#> [1] 0.005472202
#>
#> $support_fraction
#> [1] 1
#>
#> $kld_final
#> [1] 0.144013
#>
#> $kld_approx
#> [1] 0.1029181
#>
#> $validation_gap
#> [1] -0.008892184
#>
#> $heldout_kld
#> [1] 0.1351208
#>
## The certificate survives the operator calculus.
gmm_fit_quality(gmm_marginalise(fit, keep = 1L))
#> $regime
#> [1] "kld"
#>
#> $converged
#> [1] FALSE
#>
#> $degenerate
#> [1] FALSE
#>
#> $ess
#> [1] 1048.118
#>
#> $ess_relative
#> [1] 0.6987454
#>
#> $min_component_ess
#> [1] 165.3181
#>
#> $max_weight
#> [1] 0.005472202
#>
#> $support_fraction
#> [1] 1
#>
#> $kld_final
#> [1] 0.144013
#>
#> $kld_approx
#> [1] 0.1029181
#>
#> $validation_gap
#> [1] -0.008892184
#>
#> $heldout_kld
#> [1] 0.1351208
#>