Divergence between two Gaussian mixtures

Computes a divergence between two Gaussian mixtures of the same ambient dimension. The Cauchy-Schwarz divergence $$D_{\mathrm{CS}}(p, q) = \tfrac{1}{2}\log V(p, p) + \tfrac{1}{2}\log V(q, q) - \log V(p, q),$$ with \(V(p, q) = \int p(x) q(x)\, dx\), is closed-form, symmetric, non-negative, and zero exactly when \(p \propto q\). The "kl" option delegates to gmm_kld(), a Monte-Carlo estimate of the asymmetric Kullback-Leibler divergence \(\mathrm{KL}(p \Vert q)\).

Usage

gmm_divergence(p, q, type = c("cs", "kl"), n_mc = 5000L)

Arguments

p, q

Two gmm (or gmm_fit) objects of the same ambient dimension.

type

"cs" (closed-form symmetric Cauchy-Schwarz divergence, the default) or "kl" (delegates to gmm_kld()).

n_mc

Number of Monte Carlo samples used when type = "kl".

Value

For type = "cs", a non-negative numeric scalar. For type = "kl", the list returned by gmm_kld().

See also

gmm_entropy(), gmm_kld()

Other ops: dgmm(), gmm_canonicalise(), gmm_conditionalise(), gmm_kld(), gmm_marginalise(), gmm_mean(), gmm_modes(), pgmm(), rgmm()

Examples

p <- gmm(weights = c(0.5, 0.5),
         means = list(c(-1, 0), c(1, 0)),
         covariances = list(diag(2), diag(2)))
q <- gmm(weights = 1, means = list(c(0, 0)),
         covariances = list(diag(2) * 2))
gmm_divergence(p, q)
#> [1] 0.03561544
gmm_divergence(p, p)
#> [1] 0