Cauchy-Schwarz mutual information between two coordinate blocks

Measures the dependence between two disjoint coordinate blocks of a fitted joint Gaussian mixture as the Cauchy-Schwarz divergence between the joint over the two blocks and the product of their marginals, $$I_{\mathrm{CS}}(A; B) = D_{\mathrm{CS}}(p_{AB},\ p_A\, p_B).$$ The product of the marginals is itself a Gaussian mixture, so the quantity is closed-form. It is non-negative and zero exactly when the two blocks are independent. (The naive combination \(H_2(A) + H_2(B) - H_2(A, B)\) is not a valid mutual information: order-2 Renyi entropies are not additive over independent blocks and that difference can be negative.)

Usage

gmm_mutual_information(g, block_a, block_b)

Arguments

g

A gmm (or gmm_fit) joint mixture.

block_a, block_b

Disjoint integer vectors of coordinate indices (in 1..p) naming the two blocks.

Value

A non-negative numeric scalar.

See also

gmm_entropy(), gmm_divergence()

Other diagnostics: bic_aic(), ess_summary(), ess_trace(), gmm_anneal_path(), gmm_conditional_entropy(), gmm_entropy(), gmm_evidence(), gmm_fit_ensemble(), gmm_fit_quality(), gmm_independence_graph(), hellinger_mc(), kld_trace(), proxy_functional_ci()

Examples

## A correlated bivariate Gaussian: mutual information grows with |rho|.
s <- matrix(c(1, 0.7, 0.7, 1), 2, 2)
g <- gmm(weights = 1, means = list(c(0, 0)), covariances = list(s))
gmm_mutual_information(g, 1L, 2L)
#> [1] 0.102997