Abstract: In this paper, we initiate a systematic investigation of differentially private algorithms for convex empirical risk minimization. Various instantiations of this problem have been studied before. We provide new algorithms and matching lower bounds for private ERM assuming only that each data point's contribution to the loss function is Lipschitz bounded and that the domain of optimization is bounded. We provide a separate set of algorithms and matching lower bounds for the setting in which the loss functions are known to also be strongly convex. Our algorithms run in polynomial time, and in some cases even match the optimal non-private running time (as measured by oracle complexity). We give separate algorithms (and lower bounds) for $(\epsilon,0)$- and $(\epsilon,\delta)$-differential privacy; perhaps surprisingly, the techniques used for designing optimal algorithms in the two cases are completely different. Our lower bounds apply even to very simple, smooth function families, such as linear and quadratic functions. This implies that algorithms from previous work can be used to obtain optimal error rates, under the additional assumption that the contributions of each data point to the loss function is smooth. We show that simple approaches to smoothing arbitrary loss functions (in order to apply previous techniques) do not yield optimal error rates. In particular, optimal algorithms were not previously known for problems such as training support vector machines and the high-dimensional median.

Abstract: We consider the problem of maximizing a nonnegative submodular set function $f:2^N \rightarrow {\mathbb R}_+$ over a ground set $N$ subject to a variety of packing-type constraints including (multiple) matroid constraints, knapsack constraints, and their intersections. In this paper we develop a general framework that allows us to derive a number of new results, in particular, when $f$ may be a nonmonotone function. Our algorithms are based on (approximately) maximizing the multilinear extension $F$ of $f$ over a polytope $P$ that represents the constraints, and then effectively rounding the fractional solution. Although this approach has been used quite successfully, it has been limited in some important ways. We overcome these limitations as follows. First, we give constant factor approximation algorithms to maximize $F$ over a downward-closed polytope $P$ described by an efficient separation oracle. Previously this was known only for monotone functions. For nonmonotone functions, a constant factor was ...

Abstract: In this paper, we address the optimal control problem for Boolean control networks (BCNs). We first consider the problem of finding the input sequences that minimize a given cost function over a finite time horizon. The problem solution is obtained by means of a recursive algorithm that represents the analogue for BCNs of the difference Riccati equation for linear systems. We prove that a significant number of optimal control problems for BCNs can be easily reframed into the present setup. In particular, the cost function can be adjusted so as to include penalties on the switchings, provided that we augment the size of the BCN state variable. In the second part of the paper, we address the infinite horizon optimal control problem and we provide necessary and sufficient conditions for the problem solvability. The solution is obtained as the limit of the solution over the finite horizon [0,T], and it is always achieved in a finite number of steps. Finally, the average cost problem over the infinite horizon, investigated in “Optimal control of logical control networks” (Y. Zhao , IEEE Trans. Autom. Control, vol 56, no. 8, pp. 1766-1776, Aug. 2011), is addressed by making use of the results obtained in the previous sections.

Abstract: What do auto-encoders learn about the underlying data-generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of data. This paper clarifies some of these previous observations by showing that minimizing a particular form of regularized reconstruction error yields a reconstruction function that locally characterizes the shape of the data-generating density. We show that the auto-encoder captures the score (derivative of the log-density with respect to the input). It contradicts previous interpretations of reconstruction error as an energy function. Unlike previous results, the theorems provided here are completely generic and do not depend on the parameterization of the auto-encoder: they show what the auto-encoder would tend to if given enough capacity and examples. These results are for a contractive training criterion we show to be similar to the denoising auto-encoder training criterion with small corruption noise, but with contraction applied on the whole reconstruction function rather than just encoder. Similarly to score matching, one can consider the proposed training criterion as a convenient alternative to maximum likelihood because it does not involve a partition function. Finally, we show how an approximate Metropolis-Hastings MCMC can be setup to recover samples from the estimated distribution, and this is confirmed in sampling experiments.

Abstract: We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjective linear map $\cal M$, with the proximal mappings of $\tau P$, $\tau > 0$, simple to compute. This problem is nonconvex in general and encompasses many important applications in engineering and machine learning. In this paper, we examined two types of splitting methods for solving this nonconvex optimization problem: alternating direction method of multipliers and proximal gradient algorithm. For the direct adaptation of the alternating direction method of multipliers, we show that, if the penalty parameter is chosen sufficiently large and the sequence generated has a cluster point, then it gives a stationary point of the nonconvex problem. We also establish convergence of the whole sequence under an additional assumption that the functions $h$ and $P$ are semi-algebraic. Furthermore, we give simple sufficient conditions to guarantee boundedness of the sequence generated. These conditions can be satisfied for a wide range of applications including the least squares problem with the $\ell_{1/2}$ regularization. Finally, when $\cal M$ is the identity so that the proximal gradient algorithm can be efficiently applied, we show that any cluster point is stationary under a slightly more flexible constant step-size rule than what is known in the literature for a nonconvex $h$.

Abstract: In the statistical and actuarial literature several generalizations of quantiles have been considered, by means of the minimization of a suitable asymmetric loss function. All these generalized quantiles share the important property of elicitability , which has received a lot of attention recently since it corresponds to the existence of a natural backtesting methodology. In this paper we investigate the case of M -quantiles as the minimizers of an asymmetric convex loss function, in contrast to Orlicz quantiles that have been considered in Bellini and Rosazza Gianin (2012). We discuss their properties as risk measures and point out the connection with the zero utility premium principle and with shortfall risk measures introduced by Follmer and Schied (2002). In particular, we show that the only M -quantiles that are coherent risk measures are the expectiles , introduced by Newey and Powell (1987) as the minimizers of an asymmetric quadratic loss function. We provide their dual and Kusuoka representations and discuss their relationship with CVaR. We analyze their asymptotic properties for α → 1 and show that for very heavy tailed distributions expectiles are more conservative than the usual quantiles. Finally, we show their robustness in the sense of lipschitzianity with respect to the Wasserstein metric.

Abstract: The Sylvester equation is often encountered in mathematics and control theory. For the general time-invariant Sylvester equation problem, which is defined in the domain of complex numbers, the Bartels-Stewart algorithm and its extensions are effective and widely used with an O(n³) time complexity. When applied to solving the time-varying Sylvester equation, the computation burden increases intensively with the decrease of sampling period and cannot satisfy continuous realtime calculation requirements. For the special case of the general Sylvester equation problem defined in the domain of real numbers, gradient-based recurrent neural networks are able to solve the time-varying Sylvester equation in real time, but there always exists an estimation error while a recently proposed recurrent neural network by Zhang et al [this type of neural network is called Zhang neural network (ZNN)] converges to the solution ideally. The advancements in complex-valued neural networks cast light to extend the existing real-valued ZNN for solving the time-varying real-valued Sylvester equation to its counterpart in the domain of complex numbers. In this paper, a complex-valued ZNN for solving the complex-valued Sylvester equation problem is investigated and the global convergence of the neural network is proven with the proposed nonlinear complex-valued activation functions. Moreover, a special type of activation function with a core function, called sign-bi-power function, is proven to enable the ZNN to converge in finite time, which further enhances its advantage in online processing. In this case, the upper bound of the convergence time is also derived analytically. Simulations are performed to evaluate and compare the performance of the neural network with different parameters and activation functions. Both theoretical analysis and numerical simulations validate the effectiveness of the proposed method.

Abstract: In distributed optimization and iterative consensus literature, a standard problem is for $N$ agents to minimize a function $f$ over a subset of Euclidean space, where the cost function is expressed as a sum $\sum f_i$. In this paper, we study the private distributed optimization (PDOP) problem with the additional requirement that the cost function of the individual agents should remain differentially private. The adversary attempts to infer information about the private cost functions from the messages that the agents exchange. Achieving differential privacy requires that any change of an individual's cost function only results in unsubstantial changes in the statistics of the messages. We propose a class of iterative algorithms for solving PDOP, which achieves differential privacy and convergence to the optimal value. Our analysis reveals the dependence of the achieved accuracy and the privacy levels on the the parameters of the algorithm. We observe that to achieve $\epsilon$-differential privacy the accuracy of the algorithm has the order of $O(\frac{1}{\epsilon^2})$.

Abstract: We consider the logarithmic negativity of a finite interval embedded in an infinite one dimensional system at finite temperature. We focus on conformal invariant systems and we show that the naive approach based on the calculation of a two-point function of twist fields in a cylindrical geometry yields a wrong result. The correct result is obtained through a four-point function of twist fields in which two auxiliary fields are inserted far away from the interval, and they are sent to infinity only after having taken the replica limit. In this way, we find a universal scaling form for the finite temperature negativity which depends on the full operator content of the theory and not only on the central charge. In the limit of low and high temperatures, the expansion of this universal form can be obtained by means of the operator product expansion. We check our results against exact numerical computations for the critical harmonic chain.

Abstract: We present the four-loop remainder function for six-gluon scattering with maximal helicity violation in planar = 4 super-Yang-Mills theory, as an analytic function of three dual-conformal cross ratios. The function is constructed entirely from its analytic properties, without ever inspecting any multi-loop integrand. We employ the same approach used at three loops, writing an ansatz in terms of hexagon functions, and fixing coefficients in the ansatz using the multi-Regge limit and the operator product expansion in the near-collinear limit. We express the result in terms of multiple polylogarithms, and in terms of the coproduct for the associated Hopf algebra. From the remainder function, we extract the BFKL eigenvalue at next-to-next-to-leading logarithmic accuracy (NNLLA), and the impact factor at N(3)LLA. We plot the remainder function along various lines and on one surface, studying ratios of successive loop orders. As seen previously through three loops, these ratios are surprisingly constant over large regions in the space of cross ratios, and they are not far from the value expected at asymptotically large orders of perturbation theory.

Abstract: It is known that a reliable geometric quantifier of discord-like correlations can be built by employing the so-called trace distance, which is used to measure how far the state under investigation is from the closest ‘classical-quantum’ state. To date, the explicit calculation of this indicator for two qubits has only been accomplished for states where the reduced density matrix of the measured party is maximally mixed, a class that includes Bell-diagonal states. Here, we first reduce the required optimization for a general two-qubit state to the minimization of an explicit two-variable function. Using this framework, we show that the minimum can be analytically worked out in a number of relevant cases, including quantum-classical and X states. This provides an explicit and compact expression for the trace distance discord of an arbitrary state belonging to either of these important classes of density matrices.

Abstract: In fuzzy logic, an imprecise (“fuzzy”) property is described by its membership function (x), i.e., by a function which describes, for each real number x, to what degree this real number satisfies the desired property. In principle, membership functions can be of different shape, but in practice, trapezoidal and triangular membership functions are most frequently used. In this paper, we provide an interval-based theoretical explanation for this empirical fact. c

Abstract: A function f is extractable if it is possible to algorithmically "extract," from any adversarial program that outputs a value y in the image of f; a preimage of y. When combined with hardness properties such as one-wayness or collision-resistance, extractability has proven to be a powerful tool. However, so far, extractability has not been explicitly shown. Instead, it has only been considered as a non-standard knowledge assumption on certain functions. We make two headways in the study of the existence of extractable one-way functions (EOWFs). On the negative side, we show that if there exist indistinguishability obfuscators for a certain class of circuits then there do not exist EOWFs where extraction works for any adversarial program with auxiliary-input of unbounded polynomial length. On the positive side, for adversarial programs with bounded auxiliary input (and unbounded polynomial running time), we give the first construction of EOWFs with an explicit extraction procedure, based on relatively standard assumptions (e.g., sub-exponential hardness of Learning with Errors). We then use these functions to construct the first 2-message zero-knowledge arguments and 3-message zero-knowledge arguments of knowledge, against the same class of adversarial verifiers, from essentially the same assumptions.

Abstract: We present the four-loop remainder function for six-gluon scattering with maximal helicity violation in planar N=4 super-Yang-Mills theory, as an analytic function of three dual-conformal cross ratios. The function is constructed entirely from its analytic properties, without ever inspecting any multi-loop integrand. We employ the same approach used at three loops, writing an ansatz in terms of hexagon functions, and fixing coefficients in the ansatz using the multi-Regge limit and the operator product expansion in the near-collinear limit. We express the result in terms of multiple polylogarithms, and in terms of the coproduct for the associated Hopf algebra. From the remainder function, we extract the BFKL eigenvalue at next-to-next-to-leading logarithmic accuracy (NNLLA), and the impact factor at NNNLLA. We plot the remainder function along various lines and on one surface, studying ratios of successive loop orders. As seen previously through three loops, these ratios are surprisingly constant over large regions in the space of cross ratios, and they are not far from the value expected at asymptotically large orders of perturbation theory.

Abstract: This paper is concerned with a new iterative theta-adaptive dynamic programming (ADP) technique to solve optimal control problems of infinite horizon discrete-time nonlinear systems. The idea is to use an iterative ADP algorithm to obtain the iterative control law which optimizes the iterative performance index function. In the present iterative theta-ADP algorithm, the condition of initial admissible control in policy iteration algorithm is avoided. It is proved that all the iterative controls obtained in the iterative theta-ADP algorithm can stabilize the nonlinear system which means that the iterative theta-ADP algorithm is feasible for implementations both online and offline. Convergence analysis of the performance index function is presented to guarantee that the iterative performance index function will converge to the optimum monotonically. Neural networks are used to approximate the performance index function and compute the optimal control policy, respectively, for facilitating the implementation of the iterative theta-ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the established method.

Abstract: Using the quantum spectral curve approach we compute, exactly, an observable (called slope function) in the planar Aharony-Bergman-Jafferis-Maldacena theory in terms of an unknown interpolating function h(λ) which plays the role of the coupling in any integrability based calculation in this theory. We verified our results with known weak coupling expansion in the gauge theory and with the results of semiclassical string calculations. Quite surprisingly at strong coupling the result is given by an explicit rational function of h(λ) to all orders. By comparing the structure of our result with that of an exact localization based calculation for a similar observable in Marino and Putrov [J. High Energy Phys. 06 (2010) 011], we conjecture an exact expression for h(λ).

Abstract: We often encounter pairs of variables in the world whose mutual relationship can be described by a function. After training, human responses closely correspond to these functional relationships. Here we study how humans predict unobserved segments of a function that they have been trained on and we compare how human predictions differ to those made by various function-learning models in the literature. Participants' performance was best predicted by the polynomial functions that generated the observations. Further, participants were able to explicitly report the correct generating function in most cases upon a post-experiment survey. This suggests that humans can abstract functions. To understand how they do so, we modeled human learning using an hierarchical Bayesian framework organized at two levels of abstraction: function learning and parameter learning, and used it to understand the time course of participants' learning as we surreptitiously changed the generating function over time. This Bayesian model selection framework allowed us to analyze the time course of function learning and parameter learning in relative isolation. We found that participants acquired new functions as they changed and even when parameter learning was not completely accurate, the probability that the correct function was learned remained high. Most importantly, we found that humans selected the simplest-fitting function with the highest probability and that they acquired simpler functions faster than more complex ones. Both aspects of this behavior, extent and rate of selection, present evidence that human function learning obeys the Occam's razor principle.

Abstract: We present an optimal, combinatorial $1-1/e$ approximation algorithm for monotone submodular optimization over a matroid constraint. Compared to the continuous greedy algorithm [G. Calinescu et al., IPCO, Springer, Berlin, 2007, pp. 182--196] our algorithm is extremely simple and requires no rounding. It consists of the greedy algorithm followed by a local search. Both phases are run not on the actual objective function, but on a related auxiliary potential function, which is also monotone and submodular. In our previous work on maximum coverage [Y. Filmus and J. Ward, FOCS, IEEE, Piscataway, NJ, 2012, pp. 659--668], the potential function gives more weight to elements covered multiple times. We generalize this approach from coverage functions to arbitrary monotone submodular functions. When the objective function is a coverage function, both definitions of the potential function coincide. Our approach generalizes to the case where the monotone submodular function has restricted curvature. For any curvatu...

Abstract: A method of inputting a multi-point input is provided. The method includes detecting a multi-point input generated by one or more types of input means, extracting a coordinate of the multi-point input, excluding at least one coordinate among the extracted coordinates; determining whether a hovering state of a first input means among one or more types of input means is detected, configuring an additional recognition area based on a hovering point of the first input means when the hovering state has been detected, and performing a function according to an operation corresponding to the coordinate in the additional recognition area.

Abstract: The problem of reconstruction of a function from spherical means is at the heart of several modern imaging modalities and other applications. In this paper we derive universal back-projection-type reconstruction formulas for recovering a function in arbitrary dimension from averages over spheres centered on the boundary of an arbitrarily shaped bounded convex domain with smooth boundary. Provided that the unknown function is supported inside that domain, the derived formulas recover the unknown function up to an explicitly computed integral operator. For elliptical domains the integral operator is shown to vanish and hence we establish exact inversion formulas for recovering a function from spherical means centered on the boundary of elliptical domains in arbitrary dimension.

Abstract: Using symmetry considerations, we derive Ward identities which relate the three point function of scalar perturbations produced during inflation to the scalar four point function, in a particular limit. The derivation assumes approximate conformal invariance, and the conditions for the slow roll approximation, but is otherwise model independent. The Ward identities allow us to deduce that the three point function must be suppressed in general, being of the same order of magnitude as in the slow roll model. They also fix the three point function in terms of the four point function, upto one constant which we argue is generically suppressed. Our approach is based on analyzing the wave function of the universe, and the Ward identities arise by imposing the requirements of spatial and time reparametrization invariance on it.

Abstract: Quantum algorithms are known for providing more efficient solutions to certain computational tasks than any corresponding classical algorithm. Here we show that a single qudit is sufficient to implement an oracle based quantum algorithm, which can solve a black-box problem faster than any classical algorithm. For $2d$ permutation functions defined on a set of $d$ elements, deciding whether a given permutation is even or odd, requires evaluation of the function for at least two elements. We demonstrate that a quantum circuit with a single qudit can determine the parity of the permutation with only one evaluation of the function. Our algorithm provides an example for quantum computation without entanglement since it makes use of the pure state of a qudit. We also present an experimental realization of the proposed quantum algorithm with a quadrupolar nuclear magnetic resonance using a single four-level quantum system, i.e., a ququart.