Renyi-2 or Shannon entropy of a conditional Gaussian mixture

Returns the order-2 Renyi entropy (the default) or the Shannon differential entropy of the conditional mixture \(g_{Y \mid X = x}\) obtained from gmm_conditionalise() – the predictive uncertainty of the target coordinates given the conditioned ones. The Renyi-2 entropy is closed-form; order = "shannon" is a Monte Carlo estimate. Multiple conditioning configurations are evaluated row-by-row.

Usage

gmm_conditional_entropy(
  g,
  given,
  order = c("renyi2", "shannon"),
  n_mc = 5000L,
  seed = NULL
)

Arguments

g

A gmm (or gmm_fit) joint mixture.

given

Either a numeric vector with one entry per coordinate, or a matrix whose rows are such vectors. NA marks a target (kept) coordinate; a numeric value conditions on that coordinate (the gmm_conditionalise() convention).

order

"renyi2" (closed-form, the default) or "shannon".

n_mc, seed

Passed to gmm_entropy() for order = "shannon".

Value

The entropy of the chosen order in nats: a numeric scalar for a single configuration, or a numeric vector with one entropy per row of given.

See also

gmm_entropy(), gmm_conditionalise()

Other diagnostics: bic_aic(), ess_summary(), ess_trace(), gmm_anneal_path(), gmm_entropy(), gmm_evidence(), gmm_fit_ensemble(), gmm_fit_quality(), gmm_independence_graph(), gmm_mutual_information(), hellinger_mc(), kld_trace(), proxy_functional_ci()

Examples

## Joint over (Y, X); predictive entropy of Y at several X values.
s <- matrix(c(2, 0.8, 0.8, 1), 2, 2)
g <- gmm(weights = 1, means = list(c(0, 0)), covariances = list(s))
gmm_conditional_entropy(g, given = rbind(c(NA, 0), c(NA, 1)))
#> [1] 1.419254 1.419254