Data-Driven Encoding: A New Numerical Method for Computation of the Koopman Operator

Q: What is the convergence of the approximate inner product integral Rij?

The convergence of the approximate inner product integral Rij is demonstrated by increasing the number of partition points N and letting the maximum subintervals |x p | approach zero. As P tends to infinity, the approximate inner product integral Rij converges to its true integral value. This convergence is based on the Riemann sum formulation, which is a weighted sum of functions that are bounded and continuous over the subdomain of interest. According to Numerical Integration theory, sequences of this form converge to the Riemann integral value over the subdomain. This is supported by references [22, Section 1.5].

Q: How can integrals be streamlined using node-based computations?

Integrals can be streamlined by converting summations over partitions to computations over nodes. By calculating the volume associated to node k, rather than partition p, the repetition of integrand Gij(x) can be eliminated. This is achieved by using a membership function I(k, p) that takes value 1 when node k is an apex of triangle p. The new weight is then used to rewrite equations (30) and (32) as EQUATION. This conversion simplifies the computation into three steps: Graph Creation, Weighting Calculation, and Matrix Calculation. These steps involve connecting data to create partitions, calculating weights for each data point, and finding the R and Q matrices to determine matrix A, respectively.

Q: What is the purpose of the two-part numerical experiment in the IV. EXPERIMENTS section?

The two-part numerical experiment in the IV. EXPERIMENTS section aims to evaluate the DDE algorithm's validity and compare its modeling accuracy to EDMD. The first part focuses on variations in dataset size and distribution, while the second part varies the usage of observable functions. The experiment involves generating trajectories from 100 initial conditions, simulating them forward for a set number of time steps. The datasets include uniform distributions and excluded samples outside the dynamic range. Both DDE and EDMD models use two-dimensional radial basis functions (RBFs) as observable functions, with a total order of 27th order and 25 RBFs for two state variables. The accuracy of the models is tested through calculating sum of squared errors (SSE) for one-step ahead predictions over the dynamic range of the datasets. A visualization of the SSE is provided.

Q: How does DDE outperform EDMD in total SSE?

The numerical experiments show that DDE outperforms EDMD in total SSE. For uniform datasets, both models are nearly equivalent, but DDE has slightly lower SSE in all cases. This is expected as all data points are weighted equally in a uniform distribution. DDE not only has lower total SSE but also smaller variance. Fig. 5-(c) demonstrates that DDE has a uniformly low error distribution across the dynamic range, while EDMD in Fig. 5-(d) has two regions with significantly larger error. These regions coincide with sparsity in the dataset, indicating EDMD's bias towards regions of high data density. In trajectory datasets, the total SSE of DDE converges with increasing data size and density, as shown in Fig. 6. The second experiment confirms that the effect of DDE remains even with significant increases in observable functions, outperforming EDMD over the dynamic range for non-uniform datasets.

Q: How does Deep Koopman improve approximation accuracy?

Deep Koopman improves approximation accuracy by incorporating the DDE method into the neural network architecture. The input layer receives training data of independent state variables, and successive hidden layers produce observable functions. These functions feed into the output layer consisting of linear activation functions, which corresponds to the A matrix that maps the observables of the current time to those of the next time step. In the Deep Koopman approach, the output layer, or the A matrix, is trained together with observable functions in the hidden layers. By using observable functions learned from deep learning, the output layer weights are replaced with the A matrix obtained from DDE. This method has been applied to a multi-cable manipulation system, resulting in a 46th order model. However, it may be limiting and inconsistent for systems that are eighth order or higher due to computational cost and numerical instability. Alternative numerical integration approaches or partitioning methods may be necessary for higher order nonlinear systems.

Question

1. What are the two avenues of research for improving accuracy in Koopman-based DMD models?

2. What is the Koopman Operator in discrete-time dynamical systems?

3. How can the A matrix be obtained directly?

4. What is the limitation of LSE in estimating the Koopman Operator?

Accepted Answer

The two avenues of research for improving accuracy in Koopman-based DMD models are the selection of observable functions and the formulation of the linear transition matrix. Observable functions are crucial in creating an accurate linear model, and various methods have been developed for selecting them, including deep neural networks and optimization techniques. However, an efficient selection of observables does not solve all the issues that arise when attempting to construct an accurate linear model. For example, unstable modes are involved in Koopman-based DMD models, even though the underlying nonlinear systems are known to be stable. The second avenue of research involves the formulation of the linear transition matrix. Extensive studies have been done to create stable linear models to remedy situations where an outright use of DMD would lead to the creation of an unstable linear model. Recently, an extension of DMD called Robust Dynamic Mode Decomposition (RDMD) utilizes statistical measures to suppress the effect of outliers on modeling the linear Koopman matrix. Additionally, the current work aims to fill the gap between the Koopman Direct Encoding (DE) method and data-driven approaches by converting the DE formula of the Koopman Operator to a data-driven formula and providing a computational algorithm and proof of convergence to the true inner products that constitute the DE formula. Numerical experiments demonstrate that the proposed method does not exhibit biases to data distribution and can produce consistently higher accuracy compared to EDMD. Finally, the DDE algorithm is utilized in modeling a high-order nonlinear system in combination with deep learning.

Accepted Answer

The Koopman Operator K is an infinite-dimensional linear operator acting on observable functions g. It is defined as Kg = g * f, where g * f is the composition of function g with function f. In discrete-time dynamical systems, the Koopman Operator is used to analyze the system's behavior by considering observable functions of the state variables. The operator is constructed using Extended Dynamic Mode Decomposition (EDMD), which involves augmenting the state vector with real-valued observable functions of the independent state vector x t. The linear state transition matrix A, which relates z t+1 to z t, is determined by solving a least squares regression that minimizes the Sum of Squared Error (SSE). Singular Value Decomposition (SVD) is used for the least squares optimization in this process.

Accepted Answer

The A matrix can be obtained directly by encoding the selfmap and nonlinear state transition function f(x) using an independent and complete set of observable functions through inner product computations. This method involves concatenating the basis functions g1, g2, g3, ... and expressing the state transition as a linear state with matrix A. The matrix A f can be computed directly from the selfmap, state transition function, and observables. Post-multiplying the transpose of zt and integrating over X yields the formula A f = R^-1 * (zt^T * R * zt). This formula is derived from the Direct Encoding method, which guarantees the existence of inner products in Hilbert space H.

Accepted Answer

The limitation of LSE in estimating the Koopman Operator is that it assumes the model structure is correct. When this assumption is violated, LSE is unable to create an unbiased estimator. This dependency on distribution occurs because LSE applies equal weighting to all data points, leading to a model heavily tuned to the behavior of densely populated regions. In practical applications, non-uniform data distributions inevitably occur, and LSE struggles to provide a consistent estimate of the Koopman Operator.

Accepted Answer

The dynamic range X D is defined as the minimum domain in the space X that includes all the data points in D N, and is convex. For any two states in X D, x and x X D, the condition 0 <= a <= 1 holds. The convex hull that encapsulates all data points is in grey, as shown in Fig. 1. This dynamic range ensures that the state space of the nonlinear system remains closed within X D. The objective is to compute inner products over X D, which can be achieved by partitioning the domain into many segments X 1, * * * X P, as shown in Fig. 1. Delaunay Triangulation is used to generate a triangular mesh structure with desirable properties, where each triangular element is convex and has no internal node. The volume of the dynamic range V(X D) is the sum of the volumes of all the elements. This approach allows for the approximation of the integrals R ij and Q components using the meshing technique applied to the data set D N.

Accepted Answer

The convergence of the approximate inner product integral Rij is demonstrated by increasing the number of partition points N and letting the maximum subintervals |x p | approach zero. As P tends to infinity, the approximate inner product integral Rij converges to its true integral value. This convergence is based on the Riemann sum formulation, which is a weighted sum of functions that are bounded and continuous over the subdomain of interest. According to Numerical Integration theory, sequences of this form converge to the Riemann integral value over the subdomain. This is supported by references [22, Section 1.5].

Accepted Answer

Integrals can be streamlined by converting summations over partitions to computations over nodes. By calculating the volume associated to node k, rather than partition p, the repetition of integrand Gij(x) can be eliminated. This is achieved by using a membership function I(k, p) that takes value 1 when node k is an apex of triangle p. The new weight is then used to rewrite equations (30) and (32) as EQUATION. This conversion simplifies the computation into three steps: Graph Creation, Weighting Calculation, and Matrix Calculation. These steps involve connecting data to create partitions, calculating weights for each data point, and finding the R and Q matrices to determine matrix A, respectively.

Accepted Answer

The two-part numerical experiment in the IV. EXPERIMENTS section aims to evaluate the DDE algorithm's validity and compare its modeling accuracy to EDMD. The first part focuses on variations in dataset size and distribution, while the second part varies the usage of observable functions. The experiment involves generating trajectories from 100 initial conditions, simulating them forward for a set number of time steps. The datasets include uniform distributions and excluded samples outside the dynamic range. Both DDE and EDMD models use two-dimensional radial basis functions (RBFs) as observable functions, with a total order of 27th order and 25 RBFs for two state variables. The accuracy of the models is tested through calculating sum of squared errors (SSE) for one-step ahead predictions over the dynamic range of the datasets. A visualization of the SSE is provided.

Accepted Answer

The numerical experiments show that DDE outperforms EDMD in total SSE. For uniform datasets, both models are nearly equivalent, but DDE has slightly lower SSE in all cases. This is expected as all data points are weighted equally in a uniform distribution. DDE not only has lower total SSE but also smaller variance. Fig. 5-(c) demonstrates that DDE has a uniformly low error distribution across the dynamic range, while EDMD in Fig. 5-(d) has two regions with significantly larger error. These regions coincide with sparsity in the dataset, indicating EDMD's bias towards regions of high data density. In trajectory datasets, the total SSE of DDE converges with increasing data size and density, as shown in Fig. 6. The second experiment confirms that the effect of DDE remains even with significant increases in observable functions, outperforming EDMD over the dynamic range for non-uniform datasets.

Accepted Answer

Deep Koopman improves approximation accuracy by incorporating the DDE method into the neural network architecture. The input layer receives training data of independent state variables, and successive hidden layers produce observable functions. These functions feed into the output layer consisting of linear activation functions, which corresponds to the A matrix that maps the observables of the current time to those of the next time step. In the Deep Koopman approach, the output layer, or the A matrix, is trained together with observable functions in the hidden layers. By using observable functions learned from deep learning, the output layer weights are replaced with the A matrix obtained from DDE. This method has been applied to a multi-cable manipulation system, resulting in a 46th order model. However, it may be limiting and inconsistent for systems that are eighth order or higher due to computational cost and numerical instability. Alternative numerical integration approaches or partitioning methods may be necessary for higher order nonlinear systems.

Accepted Answer

The major contributions of the new data-driven approach include converting the analytical formula of Direct Encoding to a numerical formula for computing inner product integrals from given data. An efficient algorithm is developed, and its convergence conditions to the true results are analyzed. Numerical experiments demonstrate greater accuracy compared to EDMD, lower sensitivity to data distribution, and rapid convergence of inner product computation. Additionally, the DDE method is incorporated into Deep Koopman, improving prediction accuracy through neural network-based methods for constructing the Koopman Operator.

Data-Driven Encoding: A New Numerical Method for Computation of the Koopman Operator

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the two avenues of research for improving accuracy in Koopman-based DMD models?

2. What is the Koopman Operator in discrete-time dynamical systems?

3. How can the A matrix be obtained directly?

4. What is the limitation of LSE in estimating the Koopman Operator?

5. How is the dynamic range X D defined?

6. What is the convergence of the approximate inner product integral Rij?

7. How can integrals be streamlined using node-based computations?

8. What is the purpose of the two-part numerical experiment in the IV. EXPERIMENTS section?

9. How does DDE outperform EDMD in total SSE?

10. How does Deep Koopman improve approximation accuracy?

11. What are the major contributions of the new data-driven approach?

Citations

Table of Contents

A Monte Carlo Approach to Koopman Direct Encoding and Its Application to the Learning of Neural-Network Observables

References

Dynamic mode decomposition of numerical and experimental data

Methods of Numerical Integration

Hamiltonian Systems and Transformation in Hilbert Space

Deep learning for universal linear embeddings of nonlinear dynamics.

A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition

Related Papers (5)

Wide-Area Monitoring of Power System Using Dynamic Mode Decomposition on Nonlinear Observables

A robust approach to the missile defence location problem

Robust Approximation of the Stochastic Koopman Operator

Robust Approximation of the Stochastic Koopman Operator

Applied Koopman Theory for Partial Differential Equations and Data-Driven Modeling of Spatio-Temporal Systems