Abstract: We present a generic extension of variational mode decomposition (VMD) algorithm to multivariate or multichannel data. The proposed method utilizes a model for multivariate modulated oscillations that is based on the presence of a joint or common frequency component among all channels of input data. We then formulate a variational optimization problem that aims to extract an ensemble of band-limited modes containing inherent multivariate modulated oscillations present in the data. The cost function to be minimized is the sum of bandwidths of all signal modes across all input data channels, which is a generic extension of the cost function used in standard VMD to multivariate data. Minimization of the resulting variational model is achieved through the alternating direction method of multipliers (ADMM) that yields an optimal set of multivariate modes in terms of narrow bandwidth and corresponding center frequencies. The proposed extension is elegant as it does not require any extra user-defined parameters for its operation i.e., it uses the same parameters as standard VMD. We demonstrate the effectiveness of the proposed method through results obtained from extensive simulations involving test (synthetic) and real world multivariate data sets. Specifically, we highlight the utility of the proposed method in two real world applications which include the separation of alpha rhythms in multivariate electroencephalogram (EEG) data and the decomposition of bivariate cardiotocographic signals that consist of fetal heart rate and maternal uterine contraction (FHR-UC) as its two channels.

Abstract: This paper presents a new and very simple strategy for torque and flux control of ac machines. The method is based on model predictive control and uses one cost function for the torque and a separate cost function for the flux. This strategy introduces a drastic simplification, achieving a very fast dynamic behavior in the controlled machines. Experimental results obtained with an induction machine confirm the drive's very good performance.

Abstract: The Quantum Approximate Optimization Algorithm (QAOA) is a general-purpose algorithm for combinatorial optimization problems whose performance can only improve with the number of layers $p$. While QAOA holds promise as an algorithm that can be run on near-term quantum computers, its computational power has not been fully explored. In this work, we study the QAOA applied to the Sherrington-Kirkpatrick (SK) model, which can be understood as energy minimization of $n$ spins with all-to-all random signed couplings. There is a recent classical algorithm by Montanari that, assuming a widely believed conjecture, can be tailored to efficiently find an approximate solution for a typical instance of the SK model to within $(1-\epsilon)$ times the ground state energy. We hope to match its performance with the QAOA. Our main result is a novel technique that allows us to evaluate the typical-instance energy of the QAOA applied to the SK model. We produce a formula for the expected value of the energy, as a function of the $2p$ QAOA parameters, in the infinite size limit that can be evaluated on a computer with $O(16^p)$ complexity. We evaluate the formula up to $p=12$, and find that the QAOA at $p=11$ outperforms the standard semidefinite programming algorithm. Moreover, we show concentration: With probability tending to one as $n\to\infty$, measurements of the QAOA will produce strings whose energies concentrate at our calculated value. As an algorithm running on a quantum computer, there is no need to search for optimal parameters on an instance-by-instance basis since we can determine them in advance. What we have here is a new framework for analyzing the QAOA, and our techniques can be of broad interest for evaluating its performance on more general problems where classical algorithms may fail.

Abstract: Attribute reduction is one of the most important applications of fuzzy rough sets in machine learning and pattern recognition. Most existing methods employ the intersection operation of fuzzy relations to construct the dependency function of attribute reduction. However, the intersection operation may lead to low discrimination of fuzzy decision in high-dimensional data space. In this study, we introduce distance measures into fuzzy rough sets and propose a novel method for attribute reduction. We first construct a fuzzy rough set model based on distance measure with a fixed parameter. Then, the fixed distance parameter is replaced by a variable one to better characterize attribute reduction with fuzzy rough sets. Some iterative formulas for computing fuzzy rough dependency and attribute significance are presented, and an iterative computation model based on a variable distance parameter is proposed. Based on this, a greedy convergent algorithm for attribute reduction is designed. The experimental comparison demonstrates that the proposed reduction algorithm is effective and performs better than some of the other existing algorithms.

Abstract: Grey wolf optimization algorithm (GWO) is a new meta-heuristic optimization technology. Its principle is to imitate the behavior of grey wolves in nature to hunt in a cooperative way. GWO is different from others in terms of model structure. It is a large-scale search method centered on three optimal samples, and which is also the research object of many scholars. In the course of its research, this paper find that GWO is flawed. It has good performance for the optimization problem whose optimal solution is 0, however, for other problems, its advantage is not as obvious as before or even worse. Then it is further found that when GWO solves the same optimization function, the farther the function’s optimal solution is from 0, the worse its performance, and this flaw also appears in other optimization algorithms. Through the study of this defect, the analysis is carried out, and the reason is determined. Finally, although there is no way to make GWO normal, this paper provides a verification method to avoid the same problem, and hopes to help the development of the optimization algorithm.

Abstract: We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE). We obtain finite-time bounds on the mean-square error in the case of constant step-size algorithms by considering the drift of an appropriately chosen Lyapunov function. The Lyapunov function can be interpreted either in terms of Stein's method to obtain bounds on steady-state performance or in terms of Lyapunov stability theory for linear ODEs. We also provide a comprehensive treatment of the moments of the square of the 2-norm of the approximation error. Our analysis yields the following results: (i) for a given step-size, we show that the lower-order moments can be made small as a function of the step-size and can be upper-bounded by the moments of a Gaussian random variable; (ii) we show that the higher-order moments beyond a threshold may be infinite in steady-state; and (iii) we characterize the number of samples needed for the finite-time bounds to be of the same order as the steady-state bounds. As a by-product of our analysis, we also solve the open problem of obtaining finite-time bounds for the performance of temporal difference learning algorithms with linear function approximation and a constant step-size, without requiring a projection step or an i.i.d. noise assumption.

Abstract: This paper proposes an optimized tracking control approach using neural network (NN) based reinforcement learning (RL) for a class of nonlinear dynamic systems, which requires both tracking and optimizing to be performed simultaneously. Generally, for obtaining optimal control solution, Hamilton–Jacobi–Bellman equation is expected to be solvable, but, owing to strong nonlinearity, the equation is solved difficultly or even impossibly by analytical methods. Therefore, adaptive NN approximation based RL is usually considered. In the optimized control design, for driving output state following to the desired trajectory, an error term is split from optimal performance index function, and then both actor and critic NNs are built to perform RL algorithm. Actor NN aims to execute control behaviors, and critic NN aims to appraise control performance and make feedback to actor. The proof of stability concludes that the desired control performances are obtained. A numerical simulation is designed and implemented, and the desired results are shown.

Abstract: Current state-of-the-art instance segmentation methods are not suited for real-time applications like autonomous driving, which require fast execution times at high accuracy. Although the currently dominant proposal-based methods have high accuracy, they are slow and generate masks at a fixed and low resolution. Proposal-free methods, by contrast, can generate masks at high resolution and are often faster, but fail to reach the same accuracy as the proposal-based methods. In this work we propose a new clustering loss function for proposal-free instance segmentation. The loss function pulls the spatial embeddings of pixels belonging to the same instance together and jointly learns an instance-specific clustering bandwidth, maximizing the intersection-over-union of the resulting instance mask. When combined with a fast architecture, the network can perform instance segmentation in real-time while maintaining a high accuracy. We evaluate our method on the challenging Cityscapes benchmark and achieve top results (5\% improvement over Mask R-CNN) at more than 10 fps on 2MP images. Code will be available at this https URL .

Abstract: We consider the classical statistical learning/regression problem, when the value of a real random variable Y is to be predicted based on the observation of another random variable X. Given a class of functions F and a sample of independent copies of (X, Y ), one needs to choose a function f from F such that f(X) approximates Y as well as possible, in the mean-squared sense. We introduce a new procedure, the so-called median-of-means tournament, that achieves the optimal tradeoff between accuracy and confidence under minimal assumptions, and in particular outperforms classical methods based on empirical risk minimization.

Abstract: A temporal point process is a mathematical model for a time series of discrete events, which covers various applications. Recently, recurrent neural network (RNN) based models have been developed for point processes and have been found effective. RNN based models usually assume a specific functional form for the time course of the intensity function of a point process (e.g., exponentially decreasing or increasing with the time since the most recent event). However, such an assumption can restrict the expressive power of the model. We herein propose a novel RNN based model in which the time course of the intensity function is represented in a general manner. In our approach, we first model the integral of the intensity function using a feedforward neural network and then obtain the intensity function as its derivative. This approach enables us to both obtain a flexible model of the intensity function and exactly evaluate the log-likelihood function, which contains the integral of the intensity function, without any numerical approximations. Our model achieves competitive or superior performances compared to the previous state-of-the-art methods for both synthetic and real datasets.

Abstract: Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorithms and theory that provide strong problem-dependent bounds could help illuminate the key features of what makes a RL problem hard and reduce the barrier to using RL algorithms in practice. As a step towards this we derive an algorithm for finite horizon discrete MDPs and associated analysis that both yields state-of-the art worst-case regret bounds in the dominant terms and yields substantially tighter bounds if the RL environment has small environmental norm, which is a function of the variance of the next-state value functions. An important benefit of our algorithmic is that it does not require apriori knowledge of a bound on the environmental norm. As a result of our analysis, we also help address an open learning theory question~\cite{jiang2018open} about episodic MDPs with a constant upper-bound on the sum of rewards, providing a regret bound with no $H$-dependence in the leading term that scales a polynomial function of the number of episodes.

Abstract: We propose a new, generic and flexible methodology for non-parametric function estimation, in which we first estimate the number and locations of any features that may be present in the function and then estimate the function parametrically between each pair of neighbouring detected features. Examples of features handled by our methodology include change points in the piecewise constant signal model, kinks in the piecewise linear signal model and other similar irregularities, which we also refer to as generalized change points. Our methodology works with only minor modifications across a range of generalized change point scenarios, and we achieve such a high degree of generality by proposing and using a new multiple generalized change point detection device, termed narrowest-over-threshold (NOT) detection. The key ingredient of the NOT method is its focus on the smallest local sections of the data on which the existence of a feature is suspected. For selected scenarios, we show the consistency and near optimality of the NOT algorithm in detecting the number and locations of generalized change points. The NOT estimators are easy to implement and rapid to compute. Importantly, the NOT approach is easy to extend by the user to tailor to their own needs. Our methodology is implemented in the R package not.

Abstract: A supervised learning algorithm searches over a set of functions $A\rightarrow B$ parametrised by a space $P$ to find the best approximation to some ideal function $f:A\rightarrow B$ . It does this by taking examples $(a, f(a))\in A\times B$ , and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent-with respect to a fixed step size and an error function satisfying a certain property-defines a monoidal functor from a category of parametrised functions to this category of update rules. A key contribution is the notion of request function. This provides a structural perspective on backpropagation, giving a broad generalisation of neural networks and linking it with structures from bidirectional programming and open games.

Abstract: Real parameter optimization problems are often very complex and computationally expensive. We can find such problems in engineering and scientific applications. In this paper, a new algorithm is proposed to tackle the 100-Digit Challenge. There are 10 functions representing 10 optimization problems, and the goal is to compute each function’s minimum value to 10 digits of accuracy. There is no limit on either time or the maximum number of function evaluations. The proposed algorithm is based on the self-adaptive differential evolution algorithm jDE. Our algorithm uses two populations and some other mechanisms when tackling the challenge. We provide the score for each function as required by the organizers of this challenge competition.

Abstract: Deep neural networks have revolutionized many real world applications, due to their flexibility in data fitting and accurate predictions for unseen data. A line of research reveals that neural networks can approximate certain classes of functions with an arbitrary accuracy, while the size of the network scales exponentially with respect to the data dimension. Empirical results, however, suggest that networks of moderate size already yield appealing performance. To explain such a gap, a common belief is that many data sets exhibit low dimensional structures, and can be modeled as samples near a low dimensional manifold. In this paper, we prove that neural networks can efficiently approximate functions supported on low dimensional manifolds. The network size scales exponentially in the approximation error, with an exponent depending on the intrinsic dimension of the data and the smoothness of the function. Our result shows that exploiting low dimensional data structures can greatly enhance the efficiency in function approximation by neural networks. We also implement a sub-network that assigns input data to their corresponding local neighborhoods, which may be of independent interest.

Abstract: In this article, strength of fractional neural networks (FrNNs) is exploited to find the approximate solutions of nonlinear systems based on Riccati equations of arbitrary order. The feed-forward artificial FrNN are used to develop the energy function of the system by defining an error function in mean square sense. Design parameters for optimization of the energy function are adapted using viable local search with interior point methods (IPMs). The performance of design methodology in terms of accuracy and convergence is analyzed for two different variants of the nonlinear system. Comparison of the results with the exact solutions, as well as approximate numerical results, illustrates the correctness of the methodology. The worth of the scheme is established through statistical inferences based on a large number of simulation runs.

Abstract: This paper outlines further properties concerning the fractional derivative of the Riemann ζ function. The functional equation, computed by the introduction of the Grunwald–Letnikov fractional derivative, is rewritten in a simplified form that reduces the computational cost. Additionally, a quasisymmetric form of the aforementioned functional equation is derived (symmetric up to one complex multiplicative constant). The second part of the paper examines the link with the distribution of prime numbers. The Dirichlet η function suggests the introduction of a complex strip as a fractional counterpart of the critical strip. Analytic properties are shown, particularly that a Dirichlet series can be linked with this strip and expressed as a sum of the fractional derivatives of ζ. Finally, Theorem 4.3 links the fractional derivative of ζ with the distribution of prime numbers in the left half-plane.

Abstract: We tackle the change-point problem with data belonging to a general set. We build a penalty for choosing the number of change-points in the kernel-based method of Harchaoui and Cappe (2007). This penalty generalizes the one proposed by Lebarbier (2005) for one-dimensional signals. We prove a non-asymptotic oracle inequality for the proposed method, thanks to a new concentration result for some function of Hilbert-space valued random variables. Experiments on synthetic data illustrate the accuracy of our method, showing that it can detect changes in the whole distribution of data, even when the mean and variance are constant.

Abstract: In this paper, we analyze the behaviours of two different fractional derivative operators defined in the last decade. One of them is defined with the normalized sinc function (NSF) and the other one is defined with the Mittag-Leffler function (MLF). Both of them have a non-singular kernel. The fractional derivative operator defined with the MLF is developed by Atangana and Baleanu (ABO) in 2016 and the other operator defined with the normalized sinc function (NSFDO) is created by Yang et al. in 2017. These mentioned operators have some advantages to model the real life problems and to solve them. On the other hand, since the Laplace transform (LT) of the ABO can be calculated more easily, it can be preferred to solve linear/nonlinear problems. In this study, we use the perturbation method with coupled the LTs of these operators to analyze their performance in solving some fractional differential equations. Furthermore, by constructing the error analysis, we test the practicability and usefulness of the method.

Abstract: A common technique to solve bilevel optimization problems is by reducing the problem to a single level and then solving it as a standard optimization problem. A number of single level reduction formulations exist, but one of the most common ways is to replace the lower level optimization problem with its Karush-Kuhn-Tucker (KKT) conditions. Such a reduction strategy has been widely used in the classical optimization as well as the evolutionary computation literature. However, KKT conditions contain a set of non-linear equality constraints that are often found hard to satisfy. In this paper, we discuss a single level reduction of a bilevel problem using recently proposed relaxed KKT conditions. The conditions are relaxed; therefore, approximate, but the error in terms of distance from the true lower level KKT point is bounded. There is a proximity measure associated to the new KKT conditions, which gives an idea of the KKT error and distance from the optimum. We utilize this reduction method within an evolutionary algorithm to solve bilevel optimization problems. The proposed algorithm is compared against a number of recently proposed approaches. The idea is found to lead to significant computational savings, especially, in the lower level function evaluations. The idea is promising and might be useful for further developments on bilevel optimization both in the domain of classical as well as evolutionary optimization research.