Topic
Adaptive optimization
About: Adaptive optimization is a research topic. Over the lifetime, 727 publications have been published within this topic receiving 9959 citations.
Papers published on a yearly basis
Papers
Posted Content•
TL;DR: In this article, the authors provide an explicit convex optimization example where training the BNNs with the traditionally adaptive optimization methods still faces the risk of non-convergence, and identify that constraining the range of gradients is critical for optimizing the deep binary model to avoid highly suboptimal solutions.
Abstract: Recent methods have significantly reduced the performance degradation of Binary Neural Networks (BNNs), but guaranteeing the effective and efficient training of BNNs is an unsolved problem. The main reason is that the estimated gradients produced by the Straight-Through-Estimator (STE) mismatches with the gradients of the real derivatives. In this paper, we provide an explicit convex optimization example where training the BNNs with the traditionally adaptive optimization methods still faces the risk of non-convergence, and identify that constraining the range of gradients is critical for optimizing the deep binary model to avoid highly suboptimal solutions. For solving above issues, we propose a BAMSProd algorithm with a key observation that the convergence property of optimizing deep binary model is strongly related to the quantization errors. In brief, it employs an adaptive range constraint via an errors measurement for smoothing the gradients transition while follows the exponential moving strategy from AMSGrad to avoid errors accumulation during the optimization. The experiments verify the corollary of theoretical convergence analysis, and further demonstrate that our optimization method can speed up the convergence about 1:2x and boost the performance of BNNs to a significant level than the specific binary optimizer about 3:7%, even in a highly non-convex optimization problem.
TL;DR: This paper proposes ViP-DUN, a deep unfolding network that learns adaptive prior terms and optimizes convex or non-convex methods for image Compressive Sensing reconstruction, achieving improved quality at multiple compression rates through a data-driven and multi-scale approach.
Abstract: Recently, deep unfolding networks (DUNs) have emerged as a promising technique for image Compressive Sensing (CS) reconstruction by unfolding optimization algorithms, where each stage of the DUNs corresponds to an iteration of the optimization algorithm. DUNs can be divided into convex optimization based methods and non-convex optimization based methods. On the one hand, DUNs based on convex optimization algorithms cannot handle non-convex optimization problems, thereby limiting their use when the prior term is a non-convex function. On the other hand, although DUNs based on non-convex optimization algorithms can handle more complex prior terms to make global optimal solutions closer to the ground truth, there is a high probability that they converge only to a local optimum. Therefore, in practical applications, it is necessary to consider the various characteristics of the problem comprehensively, then design appropriate prior terms and choose convex or non-convex optimization in DUN. This paper proposes ViP-DUN method to learn suitable prior terms and adaptively use convex or non-convex optimization. ViP-DUN learns deep prior terms and variable metrics in a data-driven manner to achieve adaptive use of convex or non-convex optimization. Moreover, we designed a lightweight multi-scale information fusion module in ViP-DUN at the network structure level to further enhance the network's processing capability. Experiments demonstrate that our proposed method can improve image reconstruction quality at multiple compression rates through the adaptive capabilities of the network. Our complete code will be made publicly available upon acceptance.
TL;DR: This paper presents the development of an auto-tuning adaptive optimization system (a strategy to find good compiler optimizations), in order to obtain the best possible performance; that is, the reduction of runtime.
Abstract: An important component of virtual machines is the adaptive optimization system, which decides what methods to optimize and what compiler optimization set to enable. In this context, this paper presents the development of an auto-tuning adaptive optimization system (a strategy to find good compiler optimizations), in order to obtain the best possible performance; that is, the reduction of runtime. Such system was implemented over the Jikes Research Virtual Machine, and the results indicate that the proposal is capable of achieving 11% better average performance, at a cost of less than 10% of the total runtime.
1 Sep 2012
TL;DR: The problem of adaptive strategy in intelligent argumentation-based negotiation is discussed, a generating process of adaptive strategies is presented, and the process is optimizes and improves the process by using a method of machine learning to help negotiator to determine valid candidate concessional attributes and concessional values.
Abstract: In argumentation-based negotiation based on multi-agent, if a negotiator agent is endowed with ability of self-learning, then it can acquire much more information about opponent's costs and benefits to achieve the purpose of improving negotiated efficiency This paper discusses the problem of adaptive strategy in intelligent argumentation-based negotiation, presents a generating process of adaptive strategy, optimizes and improves the process by using a method of machine learning to help negotiator to determine valid candidate concessional attributes and concessional values Finally, this paper also describes an implementing process of the strategy model and explains it in details The research results of this paper provide new ideas and measures for solving the problem that how to generate reasonable adaptive strategies in argumentation-based negotiation
13 Jun 2002
TL;DR: This work believes that mobile computing requires new curricular directions for compilers and the Java Programming Language that focuses on adaptive techniques, has a performance orientation, and is empirical.
Abstract: Dynamic, adaptive optimization is quickly becoming vital to the future of high-performance, mobile computing using Java. These compilation environments have the potential to enable ubiquitous computing on resources that together represent greater computing power than that which can be extracted from existing supercomputers. As a result, we believe that mobile computing requires new curricular directions for compilers and the Java Programming Language that focuses on adaptive techniques, has a performance orientation, and is empirical. We describe such a course that we recently implemented at the University of California, Santa Barbara.
Related Topics (5)
Performance Metrics
| Year | Papers |
|---|---|
| 2026 | 1 |
| 2025 | 1 |
| 2024 | 6 |
| 2023 | 11 |
| 2022 | 18 |
| 2021 | 57 |