Abstract: There has been a growing effort in studying the distributed optimization problem over a network. The objective is to optimize a global function formed by a sum of local functions, using only local computation and communication. The literature has developed consensus-based distributed (sub)gradient descent (DGD) methods and has shown that they have the same convergence rate $O(\frac{\log t}{\sqrt{t}})$ as the centralized (sub)gradient methods (CGD), when the function is convex but possibly nonsmooth. However, when the function is convex and smooth, under the framework of DGD, it is unclear how to harness the smoothness to obtain a faster convergence rate comparable to CGD's convergence rate. In this paper, we propose a distributed algorithm that, despite using the same amount of communication per iteration as DGD, can effectively harnesses the function smoothness and converge to the optimum with a rate of $O(\frac{1}{t})$ . If the objective function is further strongly convex, our algorithm has a linear convergence rate. Both rates match the convergence rate of CGD. The key step in our algorithm is a novel gradient estimation scheme that uses history information to achieve fast and accurate estimation of the average gradient. To motivate the necessity of history information, we also show that it is impossible for a class of distributed algorithms like DGD to achieve a linear convergence rate without using history information even if the objective function is strongly convex and smooth.

Abstract: This paper presents a new algorithm, termed truncated amplitude flow (TAF), to recover an unknown vector $ {x}$ from a system of quadratic equations of the form $y_{i}=|\langle {a}_{i}, {x}\rangle |^{2}$ , where $ {a}_{i}$ ’s are given random measurement vectors. This problem is known to be NP-hard in general. We prove that as soon as the number of equations is on the order of the number of unknowns, TAF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with both the number of unknowns and the number of equations. Our TAF approach adapts the amplitude-based empirical loss function and proceeds in two stages. In the first stage, we introduce an orthogonality-promoting initialization that can be obtained with a few power iterations. Stage two refines the initial estimate by successive updates of scalable truncated generalized gradient iterations , which are able to handle the rather challenging nonconvex and nonsmooth amplitude-based objective function. In particular, when vectors $ {x}$ and ${a}_{i}$ ’s are real valued, our gradient truncation rule provably eliminates erroneously estimated signs with high probability to markedly improve upon its untruncated version. Numerical tests using synthetic data and real images demonstrate that our initialization returns more accurate and robust estimates relative to spectral initializations. Furthermore, even under the same initialization, the proposed amplitude-based refinement outperforms existing Wirtinger flow variants, corroborating the superior performance of TAF over state-of-the-art algorithms.

Abstract: We introduce instancewise feature selection as a methodology for model interpretation. Our method is based on learning a function to extract a subset of features that are most informative for each given example. This feature selector is trained to maximize the mutual information between selected features and the response variable, where the conditional distribution of the response variable given the input is the model to be explained. We develop an efficient variational approximation to the mutual information, and show the effectiveness of our method on a variety of synthetic and real data sets using both quantitative metrics and human evaluation.

Abstract: This brief addresses the trajectory tracking control problem of a fully actuated surface vessel subjected to asymmetrically constrained input and output. The controller design process is based on the backstepping technique. An asymmetric time-varying barrier Lyapunov function is proposed to address the output constraint. To overcome the difficulty of nondifferentiable input saturation, a smooth hyperbolic tangent function is employed to approximate the asymmetric saturation function. A Nussbaum function is introduced to compensate for the saturation approximation and ensure the system stability. The command filters and auxiliary systems are integrated with the control law to avoid the complicated calculation of the derivative of the virtual control in backstepping. In addition, the bounds of uncertainties and disturbances are estimated and compensated with an adaptive algorithm. With the proposed control, the constraints will never be violated during operation, and all system states are bounded. Simulation results and comparisons with standard method illustrate the effectiveness and advantages of the proposed controller.

Abstract: In this work, we present a unified performance analysis of a free-space optical (FSO) link that accounts for pointing errors and both types of detection techniques (i.e. intensity modulation/direct detection (IM/DD) as well as heterodyne detection). More specifically, we present unified exact closed-form expressions for the cumulative distribution function, the probability density function, the moment generating function, and the moments of the end-to-end signal-to-noise ratio (SNR) of a single link FSO transmission system, all in terms of the Meijer's G function except for the moments that is in terms of simple elementary functions. We then capitalize on these unified results to offer unified exact closed-form expressions for various performance metrics of FSO link transmission systems, such as, the outage probability, the scintillation index (SI), the average error rate for binary and $M$-ary modulation schemes, and the ergodic capacity (except for IM/DD technique, where we present closed-form lower bound results), all in terms of Meijer's G functions except for the SI that is in terms of simple elementary functions. Additionally, we derive the asymptotic results for all the expressions derived earlier in terms of Meijer's G function in the high SNR regime in terms of simple elementary functions via an asymptotic expansion of the Meijer's G function. We also derive new asymptotic expressions for the ergodic capacity in the low as well as high SNR regimes in terms of simple elementary functions via utilizing moments. All the presented results are verified via computer-based Monte-Carlo simulations.

Abstract: In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models. In this work, we investigate this tension between complexity and generalization through an extensive empirical exploration of two natural metrics of complexity related to sensitivity to input perturbations. Our experiments survey thousands of models with various fully-connected architectures, optimizers, and other hyper-parameters, as well as four different image classification datasets. We find that trained neural networks are more robust to input perturbations in the vicinity of the training data manifold, as measured by the norm of the input-output Jacobian of the network, and that it correlates well with generalization. We further establish that factors associated with poor generalization $-$ such as full-batch training or using random labels $-$ correspond to lower robustness, while factors associated with good generalization $-$ such as data augmentation and ReLU non-linearities $-$ give rise to more robust functions. Finally, we demonstrate how the input-output Jacobian norm can be predictive of generalization at the level of individual test points.

Abstract: In this paper, we generalize the soft set to the hypersoft set by transforming the function F into a multi-attributefunction. Then we introduce the hybrids of Crisp, Fuzzy, Intuitionistic Fuzzy, Neutrosophic, and Plithogenic Hypersoft Set.

Abstract: We have developed a framework to convert an arbitrary integer factorization problem to an executable Ising model by first writing it as an optimization function then transforming the k-bit coupling (k ≥ 3) terms to quadratic terms using ancillary variables. Our resource-efficient method uses $${\mathscr{O}}({\mathrm{log}}^{2}(N))$$ binary variables (qubits) for finding the factors of an integer N. We present how to factorize 15, 143, 59989, and 376289 using 4, 12, 59, and 94 logical qubits, respectively. This method was tested using the D-Wave 2000Q for finding an embedding and determining the prime factors for a given composite number. The method is general and could be used to factor larger integers as the number of available qubits increases, or combined with other ad hoc methods to achieve better performances for specific numbers.

Abstract: The increasing prevalence of electric vehicles (EVs) calls for the effective planning of the charging infrastructure. In this study, a multi-objective, multistage collaborative planning model is proposed for the coupled EV charging station infrastructure and power distribution network. The planning model aims to minimize the investment and operation costs of the distribution system while maximize the annually captured traffic flow. The uncertainties of EV charging loads are modeled for three different types of charging stations. The FISK's stochastic traffic assignment model is utilized to model realistic traffic flows. And a new class of volume-delay functions, conical congestion functions, is employed to overcome the shortcomings of the conventional Bureau of Public Roads function. The multi-objective evolutionary algorithm based on decomposition (MOEA/D) algorithm is applied to find the nondominated solutions of the proposed collaborative planning model. Finally, simulations based on a 54-node distribution system are conducted to validate the effectiveness of the proposed method.

Abstract: In this paper, using the concept of horizontal membership functions, a new definition of fuzzy derivative called granular derivative is proposed based on granular difference. Moreover, a new definition of fuzzy integral called granular integral is defined, and its relation with the granular derivative is given. A new definition of a metric—granular metric—on the space of type-1 fuzzy numbers, and a concept of continuous fuzzy functions are also presented. Restrictions associated to previous approaches—Hukuhara differentiability, strongly generalized Hukuhara differentiability, generalized Hukuhara differentiability, generalized differentiability, Zadeh's extension principle, and fuzzy differential inclusions—dealing with fuzzy differential equations (FDEs) are expressed. It is shown that the proposed approach does not have the drawbacks of the previous approaches. It is also demonstrated how this approach enables researchers to solve FDEs more conveniently than ever before. Moreover, we showed that this approach does not necessitate that the diameter of the fuzzy function be monotonic. It is also proved that the result of each of the four basic operations on fuzzy numbers introduced based on the proposed approach leads to a fuzzy number. Moreover, the condition for the existence of the granular derivative of a fuzzy function is provided by a theorem. Additionally, by two examples, it is shown that the existence of the granular derivative of a fuzzy function does not imply the existence of the generalized Hukuhara differentiability of the fuzzy function, and vice versa. The terms doubling property and unnatural behavior in modeling phenomenon are also introduced. Furthermore, using some examples, the paper proceeds to elaborate on the efficiency and effectiveness of the proposed approach. Moreover, as an application of the proposed approach, the response of Boeing 747 to impulsive elevator input is obtained in the presence of uncertain initial conditions and parameters.

Abstract: In the article, we present several quadratic transformation inequalities for Gaussian hypergeometric function and find the analogs of duplication inequalities for the generalized Grotzsch ring function

Abstract: Training neural networks involves finding minima of a high-dimensional non-convex loss function. Knowledge of the structure of this energy landscape is sparse. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFAR100. Surprisingly, the paths are essentially flat in both the training and test landscapes. This implies that neural networks have enough capacity for structural changes, or that these changes are small between minima. Also, each minimum has at least one vanishing Hessian eigenvalue in addition to those resulting from trivial invariance.

Abstract: Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning Disentangling methods aim to make explicit compositional or factorial structure We combine these two active but independent lines of research and propose a new paradigm suitable for both goals We propose and evaluate a novel loss function based on the $F$ statistic, which describes the separation of two or more distributions By ensuring that distinct classes are well separated on a subset of embedding dimensions, we obtain embeddings that are useful for few-shot learning By not requiring separation on all dimensions, we encourage the discovery of disentangled representations Our embedding method matches or beats state-of-the-art, as evaluated by performance on recall@$k$ and few-shot learning tasks Our method also obtains performance superior to a variety of alternatives on disentangling, as evaluated by two key properties of a disentangled representation: modularity and explicitness The goal of our work is to obtain more interpretable, manipulable, and generalizable deep representations of concepts and categories

Abstract: In this paper, convergence properties are established for the newly developed discrete-time local value iteration adaptive dynamic programming (ADP) algorithm. The present local iterative ADP algorithm permits an arbitrary positive semidefinite function to initialize the algorithm. Employing a state-dependent learning rate function, for the first time, the iterative value function and iterative control law can be updated in a subset of the state space instead of the whole state space, which effectively relaxes the computational burden. A new analysis method for the convergence property is developed to prove that the iterative value functions will converge to the optimum under some mild constraints. Monotonicity of the local value iteration ADP algorithm is presented, which shows that under some special conditions of the initial value function and the learning rate function, the iterative value function can monotonically converge to the optimum. Finally, three simulation examples and comparisons are given to illustrate the performance of the developed algorithm.

Abstract: In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiased estimates of the gradients of their local cost functions, we consider a distributed stochastic gradient tracking method (DSGT) and a gossip-like stochastic gradient tracking method (GSGT). We show that, in expectation, the iterates generated by each agent are attracted to a neighborhood of the optimal solution, where they accumulate exponentially fast (under a constant stepsize choice). Under DSGT, the limiting (expected) error bounds on the distance of the iterates from the optimal solution decrease with the network size $n$, which is a comparable performance to a centralized stochastic gradient algorithm. Moreover, we show that when the network is well-connected, GSGT incurs lower communication cost than DSGT while maintaining a similar computational cost. Numerical example further demonstrates the effectiveness of the proposed methods.

Abstract: Entropy-SGD is a first-order optimization method which has been used successfully to train deep neural networks. This algorithm, which was motivated by statistical physics, is now interpreted as gradient descent on a modified loss function. The modified, or relaxed, loss function is the solution of a viscous Hamilton–Jacobi partial differential equation (PDE). Experimental results on modern, high-dimensional neural networks demonstrate that the algorithm converges faster than the benchmark stochastic gradient descent (SGD). Well-established PDE regularity results allow us to analyze the geometry of the relaxed energy landscape, confirming empirical evidence. Stochastic homogenization theory allows us to better understand the convergence of the algorithm. A stochastic control interpretation is used to prove that a modified algorithm converges faster than SGD in expectation.

Abstract: The Quantum Approximate Optimization Algorithm, QAOA, uses a shallow depth quantum circuit to produce a parameter dependent state. For a given combinatorial optimization problem instance, the quantum expectation of the associated cost function is the parameter dependent objective function of the QAOA. We demonstrate that if the parameters are fixed and the instance comes from a reasonable distribution then the objective function value is concentrated in the sense that typical instances have (nearly) the same value of the objective function. This applies not just for optimal parameters as the whole landscape is instance independent. We can prove this is true for low depth quantum circuits for instances of MaxCut on large 3-regular graphs. Our results generalize beyond this example. We support the arguments with numerical examples that show remarkable concentration. For higher depth circuits the numerics also show concentration and we argue for this using the Law of Large Numbers. We also observe by simulation that if we find parameters which result in good performance at say 10 bits these same parameters result in good performance at say 24 bits. These findings suggest ways to run the QAOA that reduce or eliminate the use of the outer loop optimization and may allow us to find good solutions with fewer calls to the quantum computer.

Abstract: Fuzzy neural networks (FNNs) are quite useful for nonlinear system identification when only the input/output information is available. A new FNN framework is first proposed by combining an AutoRegressive with exogenous input (ARX) with the nonlinear Tanh function in the Takagi–Sugeno (T-S) type fuzzy consequent part. An improved genetic algorithm is then designed to optimize the structure and parameters of the FNN simultaneously under unknown plant dynamics. The hybrid encoding/decoding, neighborhood search operator, and maintain operator are presented to optimize the input structure of the ARX plus the nonlinear function submodel, the number of the fuzzy rules, and the parameters of the membership function. Three benchmarks and a liquid level modeling problem in an industrial coke furnace are utilized to compare the performance of several typical methods. Simulation results show that the proposed method is superior in structure simplification, modeling precision, and generalization capability.

Abstract: We consider the problem of constructing confidence intervals (CIs) for a linear functional of a regression function, such as its value at a point, the regression discontinuity parameter, or a regression coefficient in a linear or partly linear regression. Our main assumption is that the regression function is known to lie in a convex function class, which covers most smoothness and/or shape assumptions used in econometrics. We derive finite-sample optimal CIs and sharp efficiency bounds under normal errors with known variance. We show that these results translate to uniform (over the function class) asymptotic results when the error distribution is not known. When the function class is centrosymmetric, these efficiency bounds imply thatminimax CIs are close to efficient at smooth regression functions. This implies, in particular, that it is impossible to form CIs that are tighter using data-dependent tuning parameters, and maintain coverage over the whole function class. We specialize our results to inference in a linear regression, and inference on the regression discontinuity parameter, and illustrate them in an empirical application.

Abstract: Alongside instrumental-variables and fixed-effects approaches, the control function approach is the most widely used in production function estimation. Olley and Pakes (1996, Econometrica 64: 1263–...

Abstract: Full-subsets information theoretic approaches are becoming an increasingly popular tool for exploring predictive power and variable importance where a wide range of candidate predictors are being considered. Here, we describe a simple function in the statistical programming language R that can be used to construct, fit, and compare a complete model set of possible ecological or environmental predictors, given a response variable of interest and a starting generalized additive (mixed) model fit. Main advantages include not requiring a complete model to be fit as the starting point for candidate model set construction (meaning that a greater number of predictors can potentially be explored than might be available through functions such as dredge); model sets that include interactions between factors and continuous nonlinear predictors; and automatic removal of models with correlated predictors (based on a user defined criterion for exclusion). The function takes continuous predictors, which are fitted using smoothers via either gam, gamm (mgcv) or gamm4, as well as factor variables which are included on their own or as two-level interaction terms within the gam smooth (via use of the "by" argument), or with themselves. The function allows any model to be constructed and used as a null model, and takes a range of arguments that allow control over the model set being constructed, including specifying cyclic and linear continuous predictors, specification of the smoothing algorithm used, and the maximum complexity allowed for smooth terms. The use of the function is demonstrated via case studies that highlight how appropriate model sets can be easily constructed and the broader utility of the approach for exploratory ecology.

Abstract: This paper studies the projected saddle-point dynamics associated to a convex–concave function, which we term saddle function. The dynamics consists of gradient descent of the saddle function in variables corresponding to convexity and (projected) gradient ascent in variables corresponding to concavity. We examine the role that the local and/or global nature of the convexity–concavity properties of the saddle function plays in guaranteeing convergence and robustness of the dynamics. Under the assumption that the saddle function is twice continuously differentiable, we provide a novel characterization of the omega-limit set of the trajectories of this dynamics in terms of the diagonal blocks of the Hessian. Using this characterization, we establish global asymptotic convergence of the dynamics under local strong convexity–concavity of the saddle function. When strong convexity–concavity holds globally, we establish three results. First, we identify a Lyapunov function (that decreases strictly along the trajectory) for the projected saddle-point dynamics when the saddle function corresponds to the Lagrangian of a general constrained convex optimization problem. Second, for the particular case when the saddle function is the Lagrangian of an equality-constrained optimization problem, we show input-to-state stability (ISS) of the saddle-point dynamics by providing an ISS Lyapunov function. Third, we use the latter result to design an opportunistic state-triggered implementation of the dynamics. Various examples illustrate our results.

Abstract: We present new constructions of multi-input functional encryption (MIFE) schemes for the inner-product functionality that improve the state of the art solution of Abdalla et al. (Eurocrypt 2017) in two main directions.

Abstract: Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new paradigm suitable for both goals. We propose and evaluate a novel loss function based on the $F$ statistic, which describes the separation of two or more distributions. By ensuring that distinct classes are well separated on a subset of embedding dimensions, we obtain embeddings that are useful for few-shot learning. By not requiring separation on all dimensions, we encourage the discovery of disentangled representations. Our embedding method matches or beats state-of-the-art, as evaluated by performance on recall@$k$ and few-shot learning tasks. Our method also obtains performance superior to a variety of alternatives on disentangling, as evaluated by two key properties of a disentangled representation: modularity and explicitness. The goal of our work is to obtain more interpretable, manipulable, and generalizable deep representations of concepts and categories.

Abstract: This paper addresses the problem of state estimation for delayed genetic regulatory networks (DGRNs) with reaction-diffusion terms using Dirichlet boundary conditions. The nonlinear regulation function of DGRNs is assumed to exhibit the Hill form. The aim of this paper is to design a state observer to estimate the concentrations of mRNAs and proteins via available measurement techniques. By introducing novel integral terms into the Lyapunov-Krasovskii functional and by employing the Wirtinger-type integral inequality, the convex approach, Green’s identity, the reciprocally convex approach, and Wirtinger’s inequality, an asymptotic stability criterion of the error system was established in terms of linear matrix inequalities (LMIs). The stability criterion depends upon the bounds of delays and their derivatives. It should be noted that if the set of LMIs is feasible, then the desired observation of DGRNs is possible, and the state estimation can be determined. Finally, two numerical examples are presented to illustrate the availability and applicability of the proposed scheme design.