Weighted Random Search for Hyperparameter Optimization
TL;DR: The authors proposed an improved version of Random Search (RS) for hyperparameter optimization of machine learning algorithms, which generates new values for each hyper-parameter with a probability of change, based on the intuition that a value that already triggered a good result is a good candidate for the next step, and should be tested in new combinations of hyperparameters values.
read more
Abstract: We introduce an improved version of Random Search (RS), used here for hyperparameter optimization of machine learning algorithms. Unlike the standard RS, which generates for each trial new values for all hyperparameters, we generate new values for each hyperparameter with a probability of change. The intuition behind our approach is that a value that already triggered a good result is a good candidate for the next step, and should be tested in new combinations of hyperparameter values. Within the same computational budget, our method yields better results than the standard RS. Our theoretical results prove this statement. We test our method on a variation of one of the most commonly used objective function for this class of problems (the Grievank function) and for the hyperparameter optimization of a deep learning CNN architecture. Our results can be generalized to any optimization problem defined on a discrete domain.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Model-predictive control and reinforcement learning in multi-energy system case studies
Glenn Ceusters,Román Cantú Rodríguez,Alberte Bouso García,Rüdiger Franke,Geert Deconinck,Lieve Helsen,Ann Nowé,Maarten Messagie,Luis Ramirez Camargo +8 more
TL;DR: It is concluded that reinforcement learning is a viable optimal control technique for multi-energy systems given adequate constraint handling and pre-training, to avoid unsafe interactions and long training periods, as is proposed in fundamental future work.
109
Hyperparameter optimization in learning systems
Razvan Andonie
- 01 Dec 2019
TL;DR: This work presents an integrated view of methods used in hyperparameter optimization of learning systems, with an emphasis on computational complexity aspects, and creates the framework for a future separation between parameters and hyperparameters in adaptive P systems.
98
•Posted Content
Machine Learning Optimization of Quantum Circuit Layouts
TL;DR: This work introduces QXX, a novel search-based layout method, which includes a configurable Gaussian function used to estimate the depth of the generated circuits and determine the circuit region that influences most the depth, and presents empiric evidence for the feasibility of learning the layout method using approximation.
37
An adaptive hyper parameter tuning model for ship fuel consumption prediction under complex maritime environments
TL;DR: It is confirmed that the use of Bayesian optimisation for hyperparameter tuning can effectively improve the fuel consumption prediction accuracy, and the proposed model can therefore serve as a significant reference for calculating fuel consumption.
37
Stock Price Predictions with LSTM Neural Networks and Twitter Sentiment
Marah-Lisanne Thormann,Jan Farchmin,Christoph Weisser,René-Marcel Kruse,Benjamin Säfken,Alexander Silbersdorff +5 more
TL;DR: This paper presents a guide for collecting past tweets, processing for sentiment analysis and combining them with technical financial indicators to forecast the stock prices of Apple 30m and 60m ahead using a Long Short-Term Memory (LSTM) as a baseline model.
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
•Journal Article
Scikit-learn: Machine Learning in Python
Fabian Pedregosa,Gaël Varoquaux,Alexandre Gramfort,Vincent Michel,Bertrand Thirion,Olivier Grisel,Mathieu Blondel,Peter Prettenhofer,Ron Weiss,Vincent Dubourg,Jake Vanderplas,Alexandre Passos,David Cournapeau,Matthieu Brucher,Matthieu Perrot,Edouard Duchesnay +15 more
TL;DR: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems, focusing on bringing machine learning to non-specialists using a general-purpose high-level language.
Going deeper with convolutions
Christian Szegedy,Wei Liu,Yangqing Jia,Pierre Sermanet,Scott Reed,Dragomir Anguelov,Dumitru Erhan,Vincent Vanhoucke,Andrew Rabinovich +8 more
- 07 Jun 2015
TL;DR: Inception as mentioned in this paper is a deep convolutional neural network architecture that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).
LIBSVM: A library for support vector machines
Chih-Chung Chang,Chih-Jen Lin +1 more
TL;DR: Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.