Complexification

Abstract: Over the last decade, kernel methods for nonlinear processing have successfully been used in the machine learning community. The primary mathematical tool employed in these methods is the notion of the reproducing kernel Hilbert space (RKHS). However, so far, the emphasis has been on batch techniques. It is only recently, that online techniques have been considered in the context of adaptive signal processing tasks. Moreover, these efforts have only been focussed on real valued data sequences. To the best of our knowledge, no adaptive kernel-based strategy has been developed, so far, for complex valued signals. Furthermore, although the real reproducing kernels are used in an increasing number of machine learning problems, complex kernels have not, yet, been used, in spite of their potential interest in applications that deal with complex signals, with Communications being a typical example. In this paper, we present a general framework to attack the problem of adaptive filtering of complex signals, using either real reproducing kernels, taking advantage of a technique called complexification of real RKHSs, or complex reproducing kernels, highlighting the use of the complex Gaussian kernel. In order to derive gradients of operators that need to be defined on the associated complex RKHSs, we employ the powerful tool of Wirtinger's Calculus, which has recently attracted attention in the signal processing community. Wirtinger's calculus simplifies computations and offers an elegant tool for treating complex signals. To this end, in this paper, the notion of Wirtinger's calculus is extended, for the first time, to include complex RKHSs and use it to derive several realizations of the complex kernel least-mean-square (CKLMS) algorithm. Experiments verify that the CKLMS offers significant performance improvements over several linear and nonlinear algorithms, when dealing with nonlinearities.

Abstract: Two major goals in machine learning are the discovery and improvement of solutions to complex problems. In this paper, we argue that complexification, i.e. the incremental elaboration of solutions through adding new structure, achieves both these goals. We demonstrate the power of complexification through the NeuroEvolution of Augmenting Topologies (NEAT) method, which evolves increasingly complex neural network architectures. NEAT is applied to an open-ended coevolutionary robot duel domain where robot controllers compete head to head. Because the robot duel domain supports a wide range of strategies, and because coevolution benefits from an escalating arms race, it serves as a suitable testbed for studying complexification. When compared to the evolution of networks with fixed structure, complexifying evolution discovers significantly more sophisticated strategies. The results suggest that in order to discover and improve complex solutions, evolution, and search in general, should be allowed to complexify as well as optimize.

Abstract: We show that the semi-classical analysis of generic Euclidean path integrals necessarily requires complexification of the action and measure, and consideration of complex saddle solutions.We demonstrate that complex saddle points have a natural interpretation in terms of the Picard–Lefschetz theory. Motivated in part by the semi-classical expansion of QCD with adjoint matter on $\mathbb{R}^3 \times S^1$, we study quantum-mechanical systems with bosonic and fermionic (Grassmann) degrees of freedom with harmonic degenerate minima, as well as (related) purely bosonic systems with harmonic nondegenerate minima. We find exact finite action non-BPS bounce and bion solutions to the holomorphic Newton equations. We find not only real solutions, but also complex solution with non-trivial monodromy, and finally complex multi-valued and singular solutions. Complex bions are necessary for obtaining the correct nonperturbative structure of these models. In the supersymmetric limit the complex solutions govern the ground state properties, and their contribution to the semiclassical expansion is necessary to obtain consistency with the supersymmetry algebra. The multi-valuedness of the action is either related to the hidden topological angle or to the resurgent cancellation of ambiguities. We also show that in the approximate multi-instanton description the integration over the complex quasi-zero mode thimble produces the most salient features of the exact solutions. While exact complex saddles are more difficult to construct in quantum field theory, the relation to the approximate thimble construction suggests that such solutions may be underlying some remarkable features of approximate bion saddles in quantum field theories.

Abstract: This paper presents a wide framework for non-linear online supervised learning tasks in the context of complex valued signal processing. The (complex) input data are mapped into a complex reproducing kernel Hilbert space (RKHS), where the learning phase is taking place. Both pure complex kernels and real kernels (via the complexification trick) can be employed. Moreover, any convex, continuous and not necessarily differentiable function can be used to measure the loss between the output of the specific system and the desired response. The only requirement is the subgradient of the adopted loss function to be available in an analytic form. In order to derive analytically the subgradients, the principles of the (recently developed) Wirtinger's calculus in complex RKHS are exploited. Furthermore, both linear and widely linear (in RKHS) estimation filters are considered. To cope with the problem of increasing memory requirements, which is present in almost all online schemes in RKHS, the sparsification scheme, based on projection onto closed balls, has been adopted. We demonstrate the effectiveness of the proposed framework in a non-linear channel identification task, a non-linear channel equalization problem and a quadrature phase shift keying equalization scheme, using both circular and non circular synthetic signal sources.