DeWeese-like Measures¶
Mike DeWeese has introduced a family of multivariate information measures based on a multivariate extension of the data processing inequality. The general idea is the following: local modification of a single variable can not increase the amount of correlation or dependence it has with the other variables. Consider, however, the triadic distribution:
In [1]: In [1]: from dit.example_dists import dyadic, triadic
This particular distribution has zero coinformation:
In [2]: In [3]: from dit.multivariate import coinformation
Yet the distribution is a product of a giant bit (coinformation \(1.0\)) and the xor (coinformation \(-1.0\)), and so there exists within it the capability of having a coinformation of \(1.0\) if the xor component were dropped. This is exactly what the DeWeese construction captures:
In [3]: In [5]: from dit.multivariate import deweese_coinformation
DeWeese version of the total_correlation, dual_total_correlation, and caekl_mutual_information are also available, and operate on an arbitrary number of variables with optional conditional variables.
API¶
-
deweese_coinformation
(dist, rvs=None, crvs=None, niter=None, deterministic=False, rv_mode=None)¶ Compute the DeWeese coinformation.
- Parameters
dist (Distribution) – The distribution of interest.
rvs (iter of iters, None) – The random variables of interest. If None, use all.
crvs (iter, None) – The variables to condition on. If None, none.
niter (int, None) – If specified, the number of optimization steps to perform.
deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
- Returns
val – The value of the DeWeese coinformation.
- Return type
-
deweese_total_correlation
(dist, rvs=None, crvs=None, niter=None, deterministic=False, rv_mode=None)¶ Compute the DeWeese total correlation.
- Parameters
dist (Distribution) – The distribution of interest.
rvs (iter of iters, None) – The random variables of interest. If None, use all.
crvs (iter, None) – The variables to condition on. If None, none.
niter (int, None) – If specified, the number of optimization steps to perform.
deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
- Returns
val – The value of the DeWeese total correlation.
- Return type
-
deweese_dual_total_correlation
(dist, rvs=None, crvs=None, niter=None, deterministic=False, rv_mode=None)¶ Compute the DeWeese dual total correlation.
- Parameters
dist (Distribution) – The distribution of interest.
rvs (iter of iters, None) – The random variables of interest. If None, use all.
crvs (iter, None) – The variables to condition on. If None, none.
niter (int, None) – If specified, the number of optimization steps to perform.
deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
- Returns
val – The value of the DeWeese dual total correlation.
- Return type
-
deweese_caekl_mutual_information
(dist, rvs=None, crvs=None, niter=None, deterministic=False, rv_mode=None)¶ Compute the DeWeese caekl mutual information.
- Parameters
dist (Distribution) – The distribution of interest.
rvs (iter of iters, None) – The random variables of interest. If None, use all.
crvs (iter, None) – The variables to condition on. If None, none.
niter (int, None) – If specified, the number of optimization steps to perform.
deterministic (bool) – Whether the functions to optimize over should be deterministic or not. Defaults to False.
rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
- Returns
val – The value of the DeWeese caekl mutual information.
- Return type