Estimating the decomposition of predictive information in multivariate systems
TL;DR: A framework for the model-free estimation of information storage and information transfer computed as the terms composing the predictive information about the target of a multivariate dynamical process is presented, resulting in physiologically well-interpretable information decompositions of cardiovascular and cardiorespiratory interactions during head-up tilt and of joint brain-heart dynamics during sleep.
read more
Abstract: In the study of complex systems from observed multivariate time series, insight into the evolution of one system may be under investigation, which can be explained by the information storage of the system and the information transfer from other interacting systems. We present a framework for the model-free estimation of information storage and information transfer computed as the terms composing the predictive information about the target of a multivariate dynamical process. The approach tackles the curse of dimensionality employing a nonuniform embedding scheme that selects progressively, among the past components of the multivariate process, only those that contribute most, in terms of conditional mutual information, to the present target process. Moreover, it computes all information-theoretic quantities using a nearest-neighbor technique designed to compensate the bias due to the different dimensionality of individual entropy terms. The resulting estimators of prediction entropy, storage entropy, transfer entropy, and partial transfer entropy are tested on simulations of coupled linear stochastic and nonlinear deterministic dynamic processes, demonstrating the superiority of the proposed approach over the traditional estimators based on uniform embedding. The framework is then applied to multivariate physiologic time series, resulting in physiologically well-interpretable information decompositions of cardiovascular and cardiorespiratory interactions during head-up tilt and of joint brain-heart dynamics during sleep.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Figures
FIG. 6. Estimation of measures of information dynamics for the first application (X, systolic arterial pressure; Y , heart period; and Z, respiratory activity). Plots depict the estimated values (median and 10th–90th percentiles over 15 subjects) of the PE of Y , SE of Y , TE from Z to Y , and PTE from X to Y conditioned to Z, computed in the supine position and upright position by means of (a) uniform embedding and (b) nonuniform embedding. The Prediction Entropy is computed through the direct estimation (squares) as well as the sum of the estimated SE, TE and PTE (triangles). The number of subjects (out of 15) for which any measure was detected as statistically significant is reported close to its distribution. Asterisks indicate statistically significant differences of supine vs upright (paired t-test, p < 0.05). 
FIG. 3. Estimation of measures of information dynamics for simulation A (linear VAR process) with delay δ = 10. Plots and symbols are the same as in Fig. 2. 
FIG. 2. Estimation of measures of information dynamics for simulation A (linear VAR process) with delay δ = 1. Plots depict the theoretical values (dots) and the estimated values (black and white symbols) (median and 10th–90th percentiles over 100 process realizations) of the PE of Y , SE of Y , TE from Z to Y , and PTE from X to Y conditioned to Z, computed as a function of the coupling parameter C by means of (a) the uniform embedding approach and (b) the nonuniform embedding approach. The Prediction Entropy is computed through the direct estimation (white squares) as well as the sum of the estimated SE, TE and PTE (black triangles). 
FIG. 1. Exact computation of the measures of information dynamics for the benchmark simulation example of Eq. (6). Plots depict the trends of PE (solid lines), SE (dotted lines), TE (dashed lines), and PTE (gray dash-dotted lines) computed with varying one or two of the simulation parameters (as indicated in the x-axis label of each plot), keeping the remaining parameters at the constant values specified below the plot. 
FIG. 4. Estimation of measures of information dynamics for simulation B (nonlinear Hénon maps) with M = 5 processes, assuming the target process Y = X3 and source process (a) and (b) X = X1 or (c) and (d) X = X2. Plots depict the estimated values (median and 10th–90th percentiles over 100 process realizations) of the PE of Y , SE of Y , TE from Z to Y , and PTE from X to Y conditioned to Z, computed as a function of the coupling parameter C by means of (a) and (c) the uniform embedding approach and (b) and (d) the nonuniform embedding approach. The number of realizations (out of 100) for which a measure was detected as statistically significant is reported close to its distribution. The Prediction Entropy is computed through the direct estimation (white squares) as well as the sum of the estimated SE, TE and PTE (black triangles). 
FIG. 7. Estimation of measures of information dynamics for the second application (time series of EEG and cardiac HF variability band powers). Plots depict the estimated values (median and 10th– 90th percentiles over 10 subjects) of the PE of Y , SE of Y , TE from Z to Y , and PTE from X to Y conditioned to Z, computed by means of (a) uniform embedding and (b) nonuniform embedding. The analysis is performed by taking the cardiac HF activity as the target process and the EEG δ power as the source process and vice versa. The Prediction Entropy is computed through the direct estimation (squares) as well as the sum of the estimated SE, TE and PTE (triangles). The number of subjects (out of 10) for which any measure was detected as statistically significant is reported close to its distribution. Asterisks indicate statistically significant differences between the two directions of interaction (paired t-test, p < 0.05).
Citations
Networks beyond pairwise interactions: Structure and dynamics
Federico Battiston,Giulia Cencetti,Iacopo Iacopini,Iacopo Iacopini,Vito Latora,Maxime Lucas,Alice Patania,Jean-Gabriel Young,Giovanni Petri +8 more
TL;DR: A complete overview of the emerging field of networks beyond pairwise interactions, and focuses on novel emergent phenomena characterizing landmark dynamical processes, such as diffusion, spreading, synchronization and games, when extended beyond Pairwise interactions.
1.4K
Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations.
TL;DR: It is found that entropy measures can only differentiate changes of specific types in cardiac dynamics and that appropriate preprocessing is vital for correct estimation and interpretation.
Wiener–Granger Causality in Network Physiology With Applications to Cardiovascular Control and Neuroscience
Alberto Porta,Luca Faes +1 more
- 01 Feb 2016
TL;DR: This tutorial reviews predictability improvement, information-based and frequency domain methods for inferring WGC among physiological processes from multivariate realizations and quantifying the strength of the cause-effect relations in network physiology.
166
Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics
TL;DR: A thorough evaluation of SE, TE, CE and cSE as quantities related to the causal statistical structure of coupled dynamic processes allows to infer the causal effects associated with the observed effects.
Linear and non-linear brain-heart and brain-brain interactions during sleep.
TL;DR: A significant contribution of nonlinear dynamics was found in the TE, particularly involving the information transferred out of the δ node in the two networks, suggesting the importance of nonparametric TE estimation for evidencing the fine structure of the physiological networks underlying the autonomic regulation of cardiac and brain functions during sleep.
96
References
Multivariate Granger causality and generalized variance
TL;DR: This study establishes a principled framework for Granger causality in the context of causal interactions among two or more multivariate sets of variables based on the generalized variances of residual errors and motivates reformulations of "causal density" and "Granger autonomy."
Local active information storage as a tool to understand distributed neural information processing
TL;DR: It is suggested that LAIS will be a useful quantity to test theories of cortical function, such as predictive coding, when measured on a local scale in time and space in voltage sensitive dye imaging data from area 18 of the cat.
Directed Information Measures in Neuroscience
Michael Wibral,Raul Vicente,Joseph T. Lizier +2 more
- 04 Apr 2014
TL;DR: Highlights include the theoretical quantification and practical estimation of information transfer, description of transfer locally in space and time, multivariate directed measures, information decomposition among a set of stimulus/responses variables and the relation between interventional and observational causality.
Nonuniform state-space reconstruction and coupling detection
TL;DR: This work investigates in records of scalp epileptic EEG the information flow across brain areas and proposes a method for building embedding vectors progressively using information measure criteria regarding past, current, and future states.
205
•Book
The Local Information Dynamics of Distributed Computation in Complex Systems
Joseph T. Lizier
- 06 Nov 2012
TL;DR: A complete information-theoretic framework to quantify these operations on information, and in particular their dynamics in space and time, is applied to cellular automata, and delivers important insights into the fundamental nature of distributed computation and the dynamics of complex systems.
199