Abstract: In this paper, we introduce a new stochastic approximation type algorithm, namely, the randomized stochastic gradient (RSG) method, for solving an important class of nonlinear (possibly nonconvex) stochastic programming problems. We establish the complexity of this method for computing an approximate stationary point of a nonlinear programming problem. We also show that this method possesses a nearly optimal rate of convergence if the problem is convex. We discuss a variant of the algorithm which consists of applying a postoptimization phase to evaluate a short list of solutions generated by several independent runs of the RSG method, and we show that such modification allows us to improve significantly the large-deviation properties of the algorithm. These methods are then specialized for solving a class of simulation-based optimization problems in which only stochastic zeroth-order information is available.

Abstract: In this paper, we introduce a new stochastic approximation (SA) type algorithm, namely the randomized stochastic gradient (RSG) method, for solving an important class of nonlinear (possibly nonconvex) stochastic programming (SP) problems. We establish the complexity of this method for computing an approximate stationary point of a nonlinear programming problem. We also show that this method possesses a nearly optimal rate of convergence if the problem is convex. We discuss a variant of the algorithm which consists of applying a post-optimization phase to evaluate a short list of solutions generated by several independent runs of the RSG method, and show that such modification allows to improve significantly the large-deviation properties of the algorithm. These methods are then specialized for solving a class of simulation-based optimization problems in which only stochastic zeroth-order information is available.

Abstract: Many optimal decision problems in scientific, engineering, and public sector applications involve both discrete decisions and nonlinear system dynamics that affect the quality of the final design or plan. These decision problems lead to mixed-integer nonlinear programming (MINLP) problems that combine the combinatorial difficulty of optimizing over discrete variable sets with the challenges of handling nonlinear functions. We review models and applications of MINLP, and survey the state of the art in methods for solving this challenging class of problems.Most solution methods for MINLP apply some form of tree search. We distinguish two broad classes of methods: single-tree and multitree methods. We discuss these two classes of methods first in the case where the underlying problem functions are convex. Classical single-tree methods include nonlinear branch-and-bound and branch-and-cut methods, while classical multitree methods include outer approximation and Benders decomposition. The most efficient class of methods for convex MINLP are hybrid methods that combine the strengths of both classes of classical techniques.Non-convex MINLPs pose additional challenges, because they contain non-convex functions in the objective function or the constraints; hence even when the integer variables are relaxed to be continuous, the feasible region is generally non-convex, resulting in many local minima. We discuss a range of approaches for tackling this challenging class of problems, including piecewise linear approximations, generic strategies for obtaining convex relaxations for non-convex functions, spatial branch-and-bound methods, and a small sample of techniques that exploit particular types of non-convex structures to obtain improved convex relaxations.We finish our survey with a brief discussion of three important aspects of MINLP. First, we review heuristic techniques that can obtain good feasible solution in situations where the search-tree has grown too large or we require real-time solutions. Second, we describe an emerging area of mixed-integer optimal control that adds systems of ordinary differential equations to MINLP. Third, we survey the state of the art in software for MINLP.

Abstract: Preface. List of Notations. List of Acronyms. Part I: Theory. 1.Introduction. 2. Auxiliary Results. 3. Algorithms of Nonsmooth Optimization. 4. Generalized Equations. 5. Stability of Solutions to Perturbed Generalized Equations. 6. Derivatives of Solutions to Perturbed Generalized Equations. 7. Optimality Conditions and a Solution Method. Part II: Applications. 8. Introduction. 9. Membrane with Obstacle. 10. Elasticity Problems with Internal Obstacles. 11. Contact Problem with Coulomb Friction. 12. Economic Applications. Appendices: A. Cookbook. B. Basic Facts on Elliptic Boundary Value problems. C. Complementarity Problems. References. Index.

Abstract: Optimization on manifolds is a rapidly developing branch of nonlinear optimization. Its focus is on problems where the smooth geometry of the search space can be leveraged to design efficient numerical algorithms. In particular, optimization on manifolds is well-suited to deal with rank and orthogonality constraints. Such structured constraints appear pervasively in machine learning applications, including low-rank matrix completion, sensor network localization, camera network registration, independent component analysis, metric learning, dimensionality reduction and so on. The Manopt toolbox, available at this http URL, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms. We aim particularly at reaching practitioners outside our field.

Abstract: This article focuses on optimizing a passenger train timetable in a heavily congested urban rail corridor. When peak-hour demand temporally exceeds the maximum loading capacity of a train, passengers may not be able to board the next arrival train, and they may be forced to wait in queues for the following trains. A binary integer programming model incorporated with passenger loading and departure events is constructed to provide a theoretic description for the problem under consideration. Based on time-dependent, origin-to-destination trip records from an automatic fare collection system, a nonlinear optimization model is developed to solve the problem on practically sized corridors, subject to the available train-unit fleet. The latest arrival time of boarded passengers is introduced to analytically calculate effective passenger loading time periods and the resulting time-dependent waiting times under dynamic demand conditions. A by-product of the model is the passenger assignment with strict capacity constraints under oversaturated conditions. Using cumulative input–output diagrams, we present a local improvement algorithm to find optimal timetables for individual station cases. A genetic algorithm is developed to solve the multi-station problem through a special binary coding method that indicates a train departure or cancellation at every possible time point. The effectiveness of the proposed model and algorithm are evaluated using a real-world data set.

Abstract: This paper treats the problem of minimizing a general continuously differentiable function subject to sparsity constraints. We present and analyze several different optimality criteria which are based on the notions of stationarity and coordinatewise optimality. These conditions are then used to derive three numerical algorithms aimed at finding points satisfying the resulting optimality criteria: the iterative hard thresholding method and the greedy and partial sparse-simplex methods. The first algorithm is essentially a gradient projection method, while the remaining two algorithms are of a coordinate descent type. The theoretical convergence of these techniques and their relations to the derived optimality conditions are studied. The algorithms and results are illustrated by several numerical examples.

Abstract: The difficulties faced by existing multiobjective evolutionary algorithms (MOEAs) in handling many-objective problems relate to the inefficiency of selection operators, high computational cost, and difficulty in visualization of objective space. While many approaches aim to counter these difficulties by increasing the fidelity of the standard selection operators, the objective reduction approach attempts to eliminate objectives that are not essential to describe the Pareto-optimal front (POF). If the number of essential objectives is found to be two or three, the problem could be solved by the existing MOEAs. It implies that objective reduction could make an otherwise unsolvable (many-objective) problem solvable. Even when the essential objectives are four or more, the reduced representation of the problem will have favorable impact on the search efficiency, computational cost, and decision-making. Hence, development of generic and robust objective reduction approaches becomes important. This paper presents a principal component analysis and maximum variance unfolding based framework for linear and nonlinear objective reduction algorithms, respectively. The major contribution of this paper includes: 1) the enhancements in the core components of the framework for higher robustness in terms of applicability to a range of problems with disparate degree of redundancy; mechanisms to handle input data that poorly approximates the true POF; and dependence on fewer parameters to minimize the variability in performance; 2) proposition of an error measure to assess the quality of results; 3) sensitivity analysis of the proposed algorithms for the critical parameter involved, and the characteristics of the input data; and 4) study of the performance of the proposed algorithms vis-a-vis dominance relation preservation based algorithms, on a wide range of test problems (scaled up to 50 objectives) and two real-world problems.

Abstract: This paper presents a one-layer projection neural network for solving nonsmooth optimization problems with generalized convex objective functions and subject to linear equalities and bound constraints. The proposed neural network is designed based on two projection operators: linear equality constraints, and bound constraints. The objective function in the optimization problem can be any nonsmooth function which is not restricted to be convex but is required to be convex (pseudoconvex) on a set defined by the constraints. Compared with existing recurrent neural networks for nonsmooth optimization, the proposed model does not have any design parameter, which is more convenient for design and implementation. It is proved that the output variables of the proposed neural network are globally convergent to the optimal solutions provided that the objective function is at least pseudoconvex. Simulation results of numerical examples are discussed to demonstrate the effectiveness and characteristics of the proposed neural network.

Abstract: Cooperative localization (also known as sensor network localization) using received signal strength (RSS) measurements when the source transmit powers are different and unknown is investigated. Previous studies were based on the assumption that the transmit powers of source nodes are the same and perfectly known which is not practical. In this paper, the source transmit powers are considered as nuisance parameters and estimated along with the source locations. The corresponding Cramer-Rao lower bound (CRLB) of the problem is derived. To find the maximum likelihood (ML) estimator, it is necessary to solve a nonlinear and nonconvex optimization problem, which is computationally complex. To avoid the difficulty in solving the ML estimator, we derive a novel semidefinite programming (SDP) relaxation technique by converting the ML minimization problem into a convex problem which can be solved efficiently. The algorithm requires only an estimate of the path loss exponent (PLE). We initially assume that perfect knowledge of the PLE is available, but we then examine the effect of imperfect knowledge of the PLE on the proposed SDP algorithm. The complexity analyses of the proposed algorithms are also studied in detail. Computer simulations showing the remarkable performance of the proposed SDP algorithm are presented.

Abstract: The optimal trajectory planning problem for train operations under constraints and fixed arrival time is considered. The varying line resistance, variable speed restrictions, and varying maximum traction force are included in the problem definition. The objective function is a trade-off between the energy consumption and the riding comfort. Two approaches are proposed to solve this optimal control problem. First, we propose to use the pseudospectral method, a state-of-the-art method for optimal control problems, which has not used for train optimal control before. In the pseudospectral method, the optimal trajectory planning problem is recast into a multiple-phase optimal control problem, which is then transformed into a nonlinear programming problem. However, the calculation time for the pseudospectral method is too long for the real-time application in an automatic train operation system. To shorten the computation time, the optimal trajectory planning problem is reformulated as a mixed-integer linear programming (MILP) problem by approximating the nonlinear terms in the problem by piecewise affine functions. The MILP problem can be solved efficiently by existing solvers that guarantee to return the global optimum for the proposed MILP problem. Simulation results comparing the pseudospectral method, the new MILP approach, and a discrete dynamic programming approach show that the pseudospectral method has the best control performance, but that if the required computation time is also take into consideration, the MILP approach yields the best overall performance. More specifically, for the given case study the control performance of the pseudospectral approach is about 10% better than that of the MILP approach, and the computation time of the MILP approach is two to three orders of magnitude smaller than that of the pseudospectral method and the discrete dynamic programming approach.

Abstract: In power system operation, transmission congestion can drastically limit more economical generation units from being dispatched. In this paper, optimal transmission switching as a congestion management tool is utilized to change network topology which, in turn, would lead to higher electricity market efficiency. Transmission switching (TS) is formulated as an optimization problem to determine the most influential lines as candidates for disconnection. In order to relieve congestion without violating voltage security, TS is embedded in an optimal power flow (OPF) problem with AC constraints and binary variables, i.e., a mixed-integer nonlinear programming (MINLP) problem, solved using Benders decomposition. Also, a methodology is presented which provides a guideline to the system operator showing the order of switching manoeuvres that have to be followed in order to relieve congestion. It is also shown that TS based on DC optimal power flow (DCOPF) formulation as used in the literature may jeopardize system security and in some cases result in voltage collapse due to the shortcomings in its simplified models. In order to evaluate the applicability and effectiveness of the proposed method, the IEEE 57-bus and IEEE 300-bus test systems are used.

Abstract: Nonconvex constraints are valuable regularizers in many optimization problems. In particular, sparsity constraints have had a significant impact on sampling theory, where they are used in compressed sensing and allow structured signals to be sampled far below the rate traditionally prescribed. Nearly, all of the theory developed for compressed sensing signal recovery assumes that samples are taken using linear measurements. In this paper, we instead address the compressed sensing recovery problem in a setting where the observations are nonlinear. We show that, under conditions similar to those required in the linear setting, the iterative hard thresholding algorithm can be used to accurately recover sparse or structured signals from few nonlinear observations. Similar ideas can also be developed in a more general nonlinear optimization framework. In the second part of this paper, we therefore present related result that shows how this can be done under sparsity and union of subspaces constraints, whenever a generalization of the restricted isometry property traditionally imposed on the compressed sensing system holds.

Abstract: The main purpose of this paper is to develop methodologies for simultaneous generation and transmission expansion planning problem of power networks while investigating interactions between these two important sections. In the proposed methodologies, a transmission operator (TO) operating under different incentives decides about investment in transmission network while anticipating the outcome of a purely competitive electricity market where several rival generation firms complete with each other to determine operating level of their generators, amount of their sales and amount of their investment in generation capacity. The proposed methodologies rely on bi-level programming models whose upper level model represent the problem of investment in transmission by TO and the lower level problems represent market outcomes obtained from clearing the market. These bi-level models are reduced to mixed-integer linear and nonlinear programming using the duality theory and Karush-Kuhn-Tucker (KKT) optimality conditions. The results of the proposed models are analyzed and compared using two illustrative examples.

Abstract: This paper addresses the discrete network design problem (DNDP) with multiple capacity levels, or multi-capacity DNDP for short, which determines the optimal number of lanes to add to each candidate link in a road network. We formulate the problem as a bi-level programming model, where the upper level aims to minimize the total travel time via adding new lanes to candidate links and the lower level is a traditional Wardrop user equilibrium (UE) problem. We propose two global optimization methods by taking advantage of the relationship between UE and system optimal (SO) traffic assignment principles. The first method, termed as SO-relaxation, exploits the property that an optimal network design solution under SO principle can be a good approximate solution under UE principle, and successively sorts the solutions in the order of increasing total travel time under SO principle. Optimality is guaranteed when the lower bound of the total travel time of the unexplored solutions under UE principle is not less than the total travel time of a known solution under UE principle. The second method, termed as UE-reduction, adds the objective function of the Beckmann-McGuire-Winsten transformation of UE traffic assignment to the constraints of the SO-relaxation formulation of the multi-capacity DNDP. This constraint is convex and strengthens the SO-relaxation formulation. We also develop a dynamic outer-approximation scheme to make use of the state-of-the-art mixed-integer linear programming solvers to solve the SO-relaxation formulation. Numerical experiments based on a two-link network and the Sioux-Falls network are conducted.

Abstract: Discrete-continuous optimization problems are commonly modeled in algebraic form as mixed-integer linear or nonlinear programming models. Since these models can be formulated in different ways, leading either to solvable or nonsolvable problems, there is a need for a systematic modeling framework that provides a fundamental understanding on the nature of these models. This work presents a modeling framework, generalized disjunctive programming (GDP), which represents problems in terms of Boolean and continuous variables, allowing the representation of constraints as algebraic equations, disjunctions and logic propositions. An overview is provided of major research results that have emerged in this area. Basic concepts are emphasized as well as the major classes of formulations that can be derived. These are illustrated with a number of examples in the area of process systems engineering. As will be shown, GDP provides a structured way for systematically deriving mixed-integer optimization models that exhibit strong continuous relaxations, which often translates into shorter computational times. © 2013 American Institute of Chemical Engineers AIChE J, 59: 3276–3295, 2013

Abstract: Short term voltage stability poses a significant threat to system stability and reliability. This paper applies dynamic VAr injection to ensure short term voltage stability following a large disturbance in a power system with high concentration of induction motor loads. Decelerating and stalling of induction motor loads is considered to be the major cause of fault induced delayed voltage recovery (FIDVR) and short term voltage stability. If system dynamics are not taken into account properly, the proposed control solution may be an expensive over design or an under design that is not capable of eliminating FIDVR problems completely. In this work, the optimal amount and locations for installing dynamic reactive resources are found by control vector parameterization (CVP), a dynamic optimization approach. The efficiency and effectiveness of this approach is improved by utilizing results from trajectory sensitivity analysis, singular value decomposition and linear programming optimization. Dynamic optimization based on CVP approach is tested in an IEEE 162-bus system and a realistic large scale utility power system.

Abstract: The elastic behavior of the demand consumption jointly used with other available resources such as distributed generation (DG) can play a crucial role for the success of smart grids. The intensive use of Distributed Energy Resources (DER) and the technical and contractual constraints result in large-scale non linear optimization problems that require computational intelligence methods to be solved. This paper proposes a Particle Swarm Optimization (PSO) based methodology to support the minimization of the operation costs of a virtual power player that manages the resources in a distribution network and the network itself. Resources include the DER available in the considered time period and the energy that can be bought from external energy suppliers. Network constraints are considered. The proposed approach uses Gaussian mutation of the strategic parameters and contextual self-parameterization of the maximum and minimum particle velocities. The case study considers a real 937 bus distribution network, with 20310 consumers and 548 distributed generators. The obtained solutions are compared with a deterministic approach and with PSO without mutation and Evolutionary PSO, both using self-parameterization.

Abstract: Various optimization problems in engineering and management are formulated as nonlinear programming problems. Because of the nonconvexity nature of this kind of problems, no efficient approach is available to derive the global optimum of the problems. How to locate a global optimal solution of a nonlinear programming problem is an important issue in optimization theory. In the last few decades, piecewise linearization methods have been widely applied to convert a nonlinear programming problem into a linear programming problem or a mixed-integer convex programming problem for obtaining an approximated global optimal solution. In the transformation process, extra binary variables, continuous variables, and constraints are introduced to reformulate the original problem. These extra variables and constraints mainly determine the solution efficiency of the converted problem. This study therefore provides a review of piecewise linearization methods and analyzes the computational efficiency of various piecewise linearization methods.

Abstract: This paper describes how PID controllers can be designed by optimizing performance subject to robustness constraints. The optimization problem is solved using convex-concave programming. The method admits general process descriptions in terms of frequency response data and it can cope with many different constraints. Examples are presented and some pitfalls in optimization are discussed.

Abstract: Engineering design problems often involve non-linear criterion functions, including inequality and equality constraints, and a mixture of discrete and continuous design variables. Optimization approaches entail substantial challenges when solving such an all-inclusive design problem. In this paper, a modification of the Particle Swarm Optimization (PSO) algorithm is presented, which can adequately address system constraints while dealing with mixed-discrete variables. Continuous search (particle motion), as in conventional PSO, is implemented as the primary search strategy; subsequently, the discrete variables are updated using a deterministic nearest-feasible-vertex criterion. This approach is expected to alleviate the undesirable difference in the rates of evolution of discrete and continuous variables. The premature stagnation of candidate solutions (particles) due to loss of diversity is known to be one of the primary drawbacks of the basic PSO dynamics. To address this issue in high dimensional design problems, a new adaptive diversity-preservation technique is developed. This technique characterizes the population diversity at each iteration. The estimated diversity measure is then used to apply (i) a dynamic repulsion away from the best global solution in the case of continuous variables, and (ii) a stochastic update of the discrete variables. For performance validation, the Mixed-Discrete PSO algorithm is applied to a wide variety of standard test problems: (i) a set of 9 unconstrained problems, and (ii) a comprehensive set of 98 Mixed-Integer Nonlinear Programming (MINLP) problems. We also explore the applicability of this algorithm to a large scale engineering design problem---wind farm layout optimization.

Abstract: In this paper, we investigate the energy-efficient resource allocation in orthogonal frequency division multiplexing (OFDM)-based cognitive radio (CR) networks, where we try to maximize the system energy-efficiency under the consideration of many practical limitations, such as transmission power budget of the CR system, interference threshold of primary users and traffic demands of secondary users. Our general objective formulation leads to a challenging mixed integer programming problem that is hard to solve. To make it computationally tractable, we employ a time-sharing method to transform it into a non-linear fractional programming problem, which can be further converted into an equivalent convex optimization problem by using its hypogragh form. Based on these transformations, it is possible to obtain (near) optimal solution by standard optimization technique. However, the complexity of the standard technique is too high for this real-time optimization task. By exploiting the structure of the problem extensively, we develop an efficient barrier method to work out the (near) optimal solution with a reasonable complexity, significantly better than the standard technique. Numerical results show that our proposal can maximize the energy efficiency of the CR system, whilst the proposed algorithm performs quickly and stably.

Abstract: Distributed generations (DGs) are recently in growing attention as a solution to environmental and economical challenges caused by conventional power plants. In this study, a multi-objective framework as a nonlinear programming (NLP) is proposed for optimal placement and sizing of DG units. Objective functions include minimising the number of DGs and power losses as well as maximising voltage stability margin formulated as a function of decision variables. The objective functions are combined into one objective function. To avoid problems with choosing appropriate weighting factors, fuzzification is applied to objective functions to bring them into the same scale. DG units are placed at more efficient buses rather than end buses of radial links as usually determined by previous methods for improving voltage stability. Also, power system constraints including branch and voltage limits are observed in the problem. The proposed method not only is able to model all types of DG technologies but also it employs adaptive reactive limits for DGs rather than fixed limits. In addition, a three-stage procedure is proposed to gradually solve the multi-objective problem in order to prevent infeasible solutions. Also, a new technique is proposed to formulate the number of DGs without converting the NLP problem into mixed-integer NLP. Results of testing the proposed method show its efficiency.

Abstract: A new robust proportional-integrative-derivative (PID) tuning method based on nonlinear optimization is developed. The closed-loop bandwidth is maximized for specified gain and phase margins (GPM) with constraint on overshoot ratio, so that criteria of closed-loop performance and robustness are both satisfied. The equations of open-loop amplitude ratio and phase change are derived based on frequency analysis for the first-order plus time delay system with a PID controller in parallel form. The equation of closed-loop amplitude ratio is also explicitly given. The formulated optimization problem from the tuning method based on GPM with constraint on closed-loop amplitude ratio is further given. The method is demonstrated in simulation examples and compared with previous work on this topic.

Abstract: Optimization is key to solve many problems in computational biology. Global optimization methods provide a robust methodology, and metaheuristics in particular have proven to be the most efficient methods for many applications. Despite their utility, there is limited availability of metaheuristic tools. We present MEIGO, an R and Matlab optimization toolbox (also available in Python via a wrapper of the R version), that implements metaheuristics capable of solving diverse problems arising in systems biology and bioinformatics: enhanced scatter search method (eSS) for continuous nonlinear programming (cNLP) and mixed-integer programming (MINLP) problems, and variable neighborhood search (VNS) for Integer Programming (IP) problems. Both methods can be run on a single-thread or in parallel using a cooperative strategy. The code is supplied under GPLv3 and is available at \url{this http URL}. Documentation and examples are included. The R package has been submitted to Bioconductor. We evaluate MEIGO against optimization benchmarks, and illustrate its applicability to a series of case studies in bioinformatics and systems biology, outperforming other state-of-the-art methods. MEIGO provides a free, open-source platform for optimization, that can be applied to multiple domains of systems biology and bioinformatics. It includes efficient state of the art metaheuristics, and its open and modular structure allows the addition of further methods.