Open AccessPosted Content
Strong mixed-integer programming formulations for trained neural networks
TL;DR: In this article, strong mixed-integer programming (MIP) formulations for high-dimensional piecewise linear functions that correspond to trained neural networks are presented, which can be used for a number of important tasks such as verifying that an image classification network is robust to adversarial inputs, or solving decision problems where the objective function is a machine learning model.
read more
Abstract: We present strong mixed-integer programming (MIP) formulations for high-dimensional piecewise linear functions that correspond to trained neural networks. These formulations can be used for a number of important tasks, such as verifying that an image classification network is robust to adversarial inputs, or solving decision problems where the objective function is a machine learning model. We present a generic framework, which may be of independent interest, that provides a way to construct sharp or ideal formulations for the maximum of d affine functions over arbitrary polyhedral input domains. We apply this result to derive MIP formulations for a number of the most popular nonlinear operations (e.g. ReLU and max pooling) that are strictly stronger than other approaches from the literature. We corroborate this computationally, showing that our formulations are able to offer substantial improvements in solve time on verification tasks for image classification networks.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Pattern Recognition and Machine Learning
Christopher M. Bishop
- 01 Jan 2006
TL;DR: Probability distributions of linear models for regression and classification are given in this article, along with a discussion of combining models and combining models in the context of machine learning and classification.
10.1K
On-Time Last-Mile Delivery: Order Assignment with Travel-Time Predictors
TL;DR: This work studies how delivery data can be applied to improve the on-time performance of last-mile delivery services and chooses a food delivery service provider to test this approach.
173
Efficient Verification of ReLU-Based Neural Networks via Dependency Analysis
Elena Botoeva,Panagiotis Kouvaros,Jan Kronqvist,Alessio Lomuscio,Ruth Misener +4 more
- 03 Apr 2020
TL;DR: An automated procedure is derived that exploits dependency relations between the ReLU nodes, thereby pruning the search tree that needs to be considered by MILP-based formulations of the verification problem, and is augmented with methods for input domain splitting and symbolic interval propagation.
ReLU networks as surrogate models in mixed-integer linear programs
Bjarne Grimstad,Henrik Andersson +1 more
TL;DR: In this article, the authors consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems and devise and study several bound tightening procedures that consider both input and output bounds.
138
•Posted Content
SoK: Certified Robustness for Deep Neural Networks.
TL;DR: This paper provides a taxonomy for the robustness verification and training approaches, and provides an open-sourced unified platform to evaluate 20+ representative verification and corresponding robust training approaches on a wide range of DNNs.
References
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
Deep learning
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
67K
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
- 01 Jan 1998
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
53.5K
Pattern Recognition and Machine Learning
TL;DR: This book covers a broad range of topics for regular factorial designs and presents all of the material in very mathematical fashion and will surely become an invaluable resource for researchers and graduate students doing research in the design of factorial experiments.
30.8K
•Book
Pattern Recognition and Machine Learning
Christopher M. Bishop
- 17 Aug 2006
TL;DR: Probability Distributions, linear models for Regression, Linear Models for Classification, Neural Networks, Graphical Models, Mixture Models and EM, Sampling Methods, Continuous Latent Variables, Sequential Data are studied.