Abstract: With the aim of developing a fast yet accurate algorithm for compressive sensing (CS) reconstruction of natural images, we combine in this paper the merits of two existing categories of CS methods: the structure insights of traditional optimization-based methods and the speed of recent network-based ones. Specifically, we propose a novel structured deep network, dubbed ISTA-Net, which is inspired by the Iterative Shrinkage-Thresholding Algorithm (ISTA) for optimizing a general $$ norm CS reconstruction model. To cast ISTA into deep network form, we develop an effective strategy to solve the proximal mapping associated with the sparsity-inducing regularizer using nonlinear transforms. All the parameters in ISTA-Net (e.g. nonlinear transforms, shrinkage thresholds, step sizes, etc.) are learned end-to-end, rather than being hand-crafted. Moreover, considering that the residuals of natural images are more compressible, an enhanced version of ISTA-Net in the residual domain, dubbed ISTA-Net+, is derived to further improve CS reconstruction. Extensive CS experiments demonstrate that the proposed ISTA-Nets outperform existing state-of-the-art optimization-based and network-based CS methods by large margins, while maintaining fast computational speed. Our source codes are available: http://jianzhang.tech/projects/ISTA-Net.

Abstract: We present a new image reconstruction method that replaces the projector in a projected gradient descent (PGD) with a convolutional neural network (CNN). Recently, CNNs trained as image-to-image regressors have been successfully used to solve inverse problems in imaging. However, unlike existing iterative image reconstruction algorithms, these CNN-based approaches usually lack a feedback mechanism to enforce that the reconstructed image is consistent with the measurements. We propose a relaxed version of PGD wherein gradient descent enforces measurement consistency, while a CNN recursively projects the solution closer to the space of desired reconstruction images. We show that this algorithm is guaranteed to converge and, under certain conditions, converges to a local minimum of a non-convex inverse problem. Finally, we propose a simple scheme to train the CNN to act like a projector. Our experiments on sparse-view computed-tomography reconstruction show an improvement over total variation-based regularization, dictionary learning, and a state-of-the-art deep learning-based direct reconstruction technique.

Abstract: Compressive sensing (CS) has proved effective for tomographic reconstruction from sparsely collected data or under-sampled measurements, which are practically important for few-view computed tomography (CT), tomosynthesis, interior tomography, and so on. To perform sparse-data CT, the iterative reconstruction commonly uses regularizers in the CS framework. Currently, how to choose the parameters adaptively for regularization is a major open problem. In this paper, inspired by the idea of machine learning especially deep learning, we unfold the state-of-the-art “fields of experts”-based iterative reconstruction scheme up to a number of iterations for data-driven training, construct a learned experts’ assessment-based reconstruction network (LEARN) for sparse-data CT, and demonstrate the feasibility and merits of our LEARN network. The experimental results with our proposed LEARN network produces a superior performance with the well-known Mayo Clinic low-dose challenge data set relative to the several state-of-the-art methods, in terms of artifact reduction, feature preservation, and computational speed. This is consistent to our insight that because all the regularization terms and parameters used in the iterative reconstruction are now learned from the training data, our LEARN network utilizes application-oriented knowledge more effectively and recovers underlying images more favorably than competing algorithms. Also, the number of layers in the LEARN network is only 50, reducing the computational complexity of typical iterative algorithms by orders of magnitude.

Abstract: In this paper, we propose a model-driven deep learning network for multiple-input multiple-output (MIMO) detection. The structure of the network is specially designed by unfolding the iterative algorithm. Some trainable parameters are optimized through deep learning techniques to improve the detection performance. Since the number of trainable variables of the network is equal to that of the layers, the network can be easily trained within a very short time. Furthermore, the network can handle time-varying channel with only a single training. Numerical results show that the proposed approach can improve the performance of the iterative algorithm significantly under Rayleigh and correlated MIMO channels.

Abstract: The present investigation designs a systematic method for finding the latent roots and the principal axes of a matrix, without reducing the order of the matrix. It is characterized by a wide field of applicability and great accuracy, since the accumulation of rounding errors is avoided, through the process of "minimized iterations". Moreover, the method leads to a well convergent successive approximation procedure by which the solution of integral equations of the Fredholm type and the solution of the eigenvalue problem of linear differential and integral operators may be accomplished.

Abstract: The optimal power flow (OPF) problem determines a network operating point that minimizes a certain objective such as generation cost or power loss. Traditionally, OPF is solved in a centralized manner. With increasing penetration of renewable energy in distribution system, we need faster and distributed solutions for real-time feedback control. This is difficult due to the nonlinearity of the power flow equations. In this paper, we propose a solution for balanced radial networks. It exploits recent results that suggest solving for a globally optimal solution of OPF over a radial network through the second-order cone program relaxation. Our distributed algorithm is based on alternating direction method of multiplier (ADMM), but unlike standard ADMM-based distributed OPF algorithms that require solving optimization subproblems using iterative method, our decomposition allows us to derive closed form solutions for these subproblems, greatly speeding up each ADMM iteration. We illustrate the scalability of the proposed algorithm by simulating it on a real-world 2065-bus distribution network.

Abstract: This paper considers the joint design of a multiple-input multiple-output (MIMO) radar with co-located antennas and a MIMO communication system. The degrees of freedom under the designer's control are the waveforms transmitted by the radar transmit array, the filter at the radar array and the code-book employed by the communication system to form its space-time code matrix. Two formulations of the spectrum sharing problem are proposed. First, the design problem is stated as the constrained maximization of the signal-to-interference-plus-noise ratio at the radar receiver, where interference is due to both clutter and the coexistence structure, and the constraints concern both the similarity with a standard radar waveform and the rate achievable by the communication system, on top of those on the transmit energy. The resulting problem is nonconvex, but a reduced-complexity iterative algorithm, based on iterative alternating maximization of three suitably designed subproblems, is proposed and analyzed. In addition, the constrained maximization of the communication rate is also investigated. The convergence of all the devised algorithms is guaranteed. Finally, a thorough performance assessment is presented, aimed at showing the merits of the proposed approach.

Abstract: Higher levels of renewable electricity generation increase uncertainty in power system operation. To ensure secure system operation, new tools that account for this uncertainty are required. In this paper, we adopt a chance-constrained AC optimal power flow formulation, which guarantees that generation, power flows, and voltages remain within their bounds with a predefined probability. We then discuss different chance-constraint reformulations and solution approaches for the problem. We first describe an analytical reformulation based on partial linearization, which enables us to obtain a tractable representation of the optimization problem. We then provide an efficient algorithm based on an iterative solution scheme which alternates between solving a deterministic AC optimal power flow problem and assessing the impact of uncertainty. The flexibility of the iterative scheme enables not only scalable implementations, but also alternative chance-constraint reformulations. In particular, we suggest two sample-based reformulations that do not require any approximation or relaxation of the AC power flow equations. In a case study based on four different IEEE systems, we assess the performance of the method, and demonstrate scalability of the iterative scheme. We further show that the analytical reformulation accurately and efficiently enforces chance constraints in both in- and out-of-sample tests, and that the analytical reformulations outperforms the two alternative, sample-based chance constraint reformulations.

Abstract: Dimensionality reduction has attracted increasing attention, because high-dimensional data have arisen naturally in numerous domains in recent years. As one popular dimensionality reduction method, nonnegative matrix factorization (NMF), whose goal is to learn parts-based representations, has been widely studied and applied to various applications. In contrast to the previous approaches, this paper proposes a novel semisupervised NMF learning framework, called robust structured NMF, that learns a robust discriminative representation by leveraging the block-diagonal structure and the $\ell _{2,p}$ -norm (especially when $0 ) loss function. Specifically, the problems of noise and outliers are well addressed by the $\ell _{2,p}$ -norm ( $0 ) loss function, while the discriminative representations of both the labeled and unlabeled data are simultaneously learned by explicitly exploring the block-diagonal structure. The proposed problem is formulated as an optimization problem with a well-defined objective function solved by the proposed iterative algorithm. The convergence of the proposed optimization algorithm is analyzed both theoretically and empirically. In addition, we also discuss the relationships between the proposed method and some previous methods. Extensive experiments on both the synthetic and real-world data sets are conducted, and the experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods.

Abstract: This letter introduces a new iterative method for contact dynamics problems. The proposed method is based on an efficient bisection method which iterates over each contact. We compared our approach to two existing ones for the same model and found that it is about twice as fast as the existing ones. We also introduce four different robotic simulation experiments and compare the proposed method to the most common contact solver, the projected Gauss-Seidel (PGS) method. We show that, while both methods are very efficient in solving simple problems, the proposed method significantly outperforms the PGS method in more complicated contact scenarios. Simulating one time step of an 18-DOF quadruped robot with multiple contacts took less than 20 μs with a single core of a CPU. This is at least an order of magnitude faster than many other simulators which employ multiple relaxation methods to the major dynamic principles in order to boost their computational speed. The proposed simulation method is also stable at 50 Hz due to its strict adherence to the dynamical principles. Although the accuracy might be compromised at such a low update rate, this means that we can simulate an 18-DOF robot more than thousand times faster than the real time.

Abstract: We present a MATLAB software package with efficient, robust, and flexible implementations of algebraic iterative reconstruction (AIR) methods for computing regularized solutions to discretizations of inverse problems. These methods are of particular interest in computed tomography and similar problems where they easily adapt to the particular geometry of the problem. All our methods are equipped with stopping rules as well as heuristics for computing a good relaxation parameter, and we also provide several test problems from tomography. The package is intended for users who want to experiment with algebraic iterative methods and their convergence properties. The present software is a much expanded and improved version of the package AIR Tools from 2012, based on a new modular design. In addition to improved performance and memory use, we provide more flexible iterative methods, a column-action method, new test problems, new demo functions, and—perhaps most important—the ability to use function handles instead of (sparse) matrices, allowing larger problems to be handled.

Abstract: Low-precision floating-point arithmetic is a powerful tool for accelerating scientific computing applications, especially those in artificial intelligence. Here, we present an investigation showing that other high-performance computing (HPC) applications can also harness this power. Specifically, we use the general HPC problem, Ax = b, where A is a large dense matrix, and a double precision (FP64) solution is needed for accuracy. Our approach is based on mixed-precision (FP16→FP64) iterative refinement, and we generalize and extend prior advances into a framework, for which we develop architecture-specific algorithms and highly tuned implementations. These new methods show how using half-precision Tensor Cores (FP16-TC) for the arithmetic can provide up to 4X speedup. This is due to the performance boost that the FP16-TC provide as well as to the improved accuracy over the classical FP16 arithmetic that is obtained because the GEMM accumulation occurs in FP32 arithmetic.

Abstract: This letter considers the minimization of the offloading delay for nonorthogonal multiple access assisted mobile edge computing (NOMA-MEC). By transforming the delay minimization problem into a form of fractional programming, two iterative algorithms based on, respectively, Dinkelbach's method and Newton's method are proposed. The optimality of both methods is proved and their convergence is compared. Furthermore, criteria for choosing between three possible modes, namely orthogonal multiple access, pure NOMA, and hybrid NOMA, for MEC offloading are established.

Abstract: In tomographic medical imaging (PET, SPECT, CT), differences in data acquisition and organization are a major hurdle for the development of tomographic reconstruction software. The implementation of a given reconstruction algorithm is usually limited to a specific set of conditions, depending on the modality, the purpose of the study, the input data, or on the characteristics of the reconstruction algorithm itself. It causes restricted or limited use of algorithms, differences in implementation, code duplication, impractical code development, and difficulties for comparing different methods. This work attempts to address these issues by proposing a unified and generic code framework for formatting, processing and reconstructing acquired multi-modal and multi-dimensional data. The proposed iterative framework processes in the same way elements from list-mode (i.e. events) and histogrammed (i.e. sinogram or other bins) data sets. Each element is processed separately, which opens the way for highly parallel execution. A unique iterative algorithm engine makes use of generic core components corresponding to the main parts of the reconstruction process. Features that are specific to different modalities and algorithms are embedded into specific components inheriting from the generic abstract components. Temporal dimensions are taken into account in the core architecture. The framework is implemented in an open-source C++ parallel platform, called CASToR (customizable and advanced software for tomographic reconstruction). Performance assessments show that the time loss due to genericity remains acceptable, being one order of magnitude slower compared to a manufacturer's software optimized for computational efficiency for a given system geometry. Specific optimizations were made possible by the underlying data set organization and processing and allowed for an average speed-up factor ranging from 1.54 to 3.07 when compared to more conventional implementations. Using parallel programming, an almost linear speed-up increase (factor of 0.85 times number of cores) was obtained in a realistic clinical PET setting. In conclusion, the proposed framework offers a substantial flexibility for the integration of new reconstruction algorithms while maintaining computation efficiency.

Abstract: In this work, we aim to present a new fractional extension of regularized long-wave equation. The regularized long-wave equation is a very important mathematical model in physical sciences, which unfolds the nature of shallow water waves and ion acoustic plasma waves. The existence and uniqueness of the solution of the regularized long-wave equation associated with Atangana–Baleanu fractional derivative having Mittag-Leffler type kernel is verified by implementing the fixed-point theorem. The numerical results are derived with the help of an iterative algorithm. In order to show the effects of various parameters and variables on the displacement, the numerical results are presented in graphical and tabular form.

Abstract: Conventional approaches to image registration consist of time consuming iterative methods. Most current deep learning (DL) based registration methods extract deep features to use in an iterative setting. We propose an end-to-end DL method for registering multimodal images. Our approach uses generative adversarial networks (GANs) that eliminates the need for time consuming iterative methods, and directly generates the registered image with the deformation field. Appropriate constraints in the GAN cost function produce accurately registered images in less than a second. Experiments demonstrate their accuracy for multimodal retinal and cardiac MR image registration.

Abstract: This paper proposes a new distance-based distributionally robust unit commitment (DB-DRUC) model via Kullback–Leibler (KL) divergence, considering volatile wind power generation. The objective function of the DB-DRUC model is to minimize the expected cost under the worst case wind distributions restricted in an ambiguity set. The ambiguity set is a family of distributions within a fixed distance from a nominal distribution. The distance between two distributions is measured by KL divergence. The DB-DRUC model is a “min-max-min” programming model; thus, it is intractable to solve. Applying reformulation methods and stochastic programming technologies, we reformulate this “min-max-min” DB-DRUC model into a one-level model, referred to as the reformulated DB-DRUC (RDB-DRUC) model. Using the generalized Benders decomposition, we then propose a two-level decomposition method and an iterative algorithm to address the RDB-DRUC model. The iterative algorithm for the RDB-DRUC model guarantees global convergence within finite iterations. Case studies are carried out to demonstrate the effectiveness, global optimality, and finite convergence of a proposed solution strategy.

Abstract: This paper examines the properties of the Iterated Ensemble Smoother (IES) and the Multiple Data Assimilation Ensemble Smoother (ES–MDA) for solving the history matching problem. The iterative methods are compared with the standard Ensemble Smoother (ES) to improve the understanding of the similarities and differences between them. We derive the three smoothers from Bayes’ theorem for a scalar case which allows us to compare the equations solved by the three methods, and we can better understand which assumptions are applied and their consequences. When working with a scalar model, it is possible to use a vast ensemble size, and we can construct the sample distributions for both priors and posteriors, as well as intermediate iterates. For a linear model, all three methods give the same result. For a nonlinear model, the iterative methods improve on the ES result, but the two iterative methods converge to different solutions, and it is not clear which should be the preferred choice. It is clear that the ensemble of cost functions used to define the IES solution does not represent an exact sampling of the posterior-Bayes’ probability density function. Also, the use of an ensemble representation for the gradient in IES introduces an additional approximation compared to using an exact analytic gradient. For ES–MDA, the convergence, as a function of increasing number of uniform update steps, is studied for a huge ensemble size. We illustrate that ES–MDA converges to a solution that differs from the Bayesian posterior. The convergence is also examined using a realistic sample size to study the impact of the number of realizations relative to the number of update steps. We have run multiple ES–MDA experiments to examine the impact of using different schemes for choosing the lengths of the update steps, and we have tried to understand which properties of the inverse problem imply that a non-uniform update step length is beneficial. Finally, we have examined the smoother methods with a highly nonlinear model to examine their properties and limitations in more extreme situations.

Abstract: This paper considers unbalanced multiphase distribution systems with generic topology and different load models, and extends the $Z$ -bus iterative load-flow algorithm based on a fixed-point interpretation of the AC load-flow equations. Explicit conditions for existence and uniqueness of load-flow solutions are presented. These conditions also guarantee convergence of the load-flow algorithm to the unique solution. The proposed methodology is applicable to generic systems featuring i) wye connections; ii) ungrounded delta connections; iii) a combination of wye-connected and delta-connected sources/loads; and iv) a combination of line-to-line and line-to-grounded-neutral devices at the secondary of distribution transformers. Further, a sufficient condition for the nonsingularity of the load-flow Jacobian is proposed. Finally, linear load-flow models are derived, and their approximation accuracy is analyzed. Theoretical results are corroborated through experiments on IEEE test feeders.

Abstract: Accompanied with the rising popularity of compressed sensing, the Alternating Direction Method of Multipliers (ADMM) has become the most widely used solver for linearly constrained convex problems with separable objectives. In this work, we observe that many existing ADMMs update the primal variable by minimizing different majorant functions with their convergence proofs given case by case. Inspired by the principle of majorization minimization, we respectively present the unified frameworks of Gauss-Seidel ADMMs and Jacobian ADMMs, which use different historical information for the current updating. Our frameworks generalize previous ADMMs to solve the problems with non-separable objectives. We also show that ADMMs converge faster when the used majorant function is tighter. We then propose the Mixed Gauss-Seidel and Jacobian ADMM (M-ADMM) which alleviates the slow convergence issue of Jacobian ADMMs by absorbing merits of the Gauss-Seidel ADMMs. M-ADMM can be further improved by backtracking and wise variable partition. We also propose to solve the multi-blocks problems by Proximal Gauss-Seidel ADMM which is of the Gauss-Seidel type. It convegences for non-strongly convex objective. Experiments on both synthesized and real-world data demonstrate the superiority of our new ADMMs. Finally, we release a toolbox that implements efficient ADMMs for many problems in compressed sensing.

Abstract: This paper proposes a novel data-driven control approach to address the problem of adaptive optimal tracking for a class of nonlinear systems taking the strict-feedback form. Adaptive dynamic programming (ADP) and nonlinear output regulation theories are integrated for the first time to compute an adaptive near-optimal tracker without any a priori knowledge of the system dynamics. Fundamentally different from adaptive optimal stabilization problems, the solution to a Hamilton-Jacobi–Bellman (HJB) equation, not necessarily a positive definite function, cannot be approximated through the existing iterative methods. This paper proposes a novel policy iteration technique for solving positive semidefinite HJB equations with rigorous convergence analysis. A two-phase data-driven learning method is developed and implemented online by ADP. The efficacy of the proposed adaptive optimal tracking control methodology is demonstrated via a Van der Pol oscillator with time-varying exogenous signals.

Abstract: To recover the corrupted pixels, traditional inpainting methods based on low-rank priors generally need to solve a convex optimization problem by an iterative singular value shrinkage algorithm. In this paper, we propose a simple method for image inpainting using low rank approximation, which avoids the time-consuming iterative shrinkage. Specifically, if similar patches of a corrupted image are identified and reshaped as vectors, then a patch matrix can be constructed by collecting these similar patch-vectors. Due to its columns being highly linearly correlated, this patch matrix is low-rank. Instead of using an iterative singular value shrinkage scheme, the proposed method utilizes low rank approximation with truncated singular values to derive a closed-form estimate for each patch matrix. Depending upon an observation that there exists a distinct gap in the singular spectrum of patch matrix, the rank of each patch matrix is empirically determined by a heuristic procedure. Inspired by the inpainting algorithms with component decomposition, a two-stage low rank approximation (TSLRA) scheme is designed to recover image structures and refine texture details of corrupted images. Experimental results on various inpainting tasks demonstrate that the proposed method is comparable and even superior to some state-of-the-art inpainting algorithms.

Abstract: Plug-and-play priors (PnP) is a powerful framework for regularizing imaging inverse problems by using advanced denoisers within an iterative algorithm Recent experimental evidence suggests that PnP algorithms achieve state-of-the-art performance in a range of imaging applications In this paper, we introduce a new online PnP algorithm based on the iterative shrinkage/thresholding algorithm (ISTA) The proposed algorithm uses only a subset of measurements at every iteration, which makes it scalable to very large datasets We present a new theoretical convergence analysis, for both batch and online variants of PnP-ISTA, for denoisers that do not necessarily correspond to proximal operators We also present simulations illustrating the applicability of the algorithm to image reconstruction in diffraction tomography The results in this paper have the potential to expand the applicability of the PnP framework to very large and redundant datasets

Abstract: Recent works have proposed two L1-norm distance measure-based linear discriminant analysis (LDA) methods, L1-LD and LDA-L1, which aim to promote the robustness of the conventional LDA against outliers. In LDA-L1, a gradient ascending iterative algorithm is applied, which, however, suffers from the choice of stepwise. In L1-LDA, an alternating optimization strategy is proposed to overcome this problem. In this paper, however, we show that due to the use of this strategy, L1-LDA is accompanied with some serious problems that hinder the derivation of the optimal discrimination for data. Then, we propose an effective iterative framework to solve a general L1-norm minimization–maximization ( minmax ) problem. Based on the framework, we further develop a effective L1-norm distance-based LDA (called L1-ELDA) method. Theoretical insights into the convergence and effectiveness of our algorithm are provided and further verified by extensive experimental results on image databases.

Abstract: High-resolution scanning radar mapping of the surface is an effective tool for addressing concerns in local environmental and social investigation fields. Regrettably, the azimuth resolution of a scanning radar is constrained by the antenna beamwidth. Multiple super-resolution approaches have been applied to the scanning radar to enhance the azimuth resolution, but they suffer from limited resolution improvement. In this paper, a methodology to derive surface estimates from the scanning radar at an improved azimuth resolution is proposed. We first consider the truncated spectrum by discarding the unreliable frequencies to suppress the noise amplification. Then, based on the iterative adaptive approach (IAA), a novel inverse filtering method is formulated to obtain lower sidelobes and a higher resolution. Finally, by taking advantage of the Fourier property of the steering matrix and the Toeplitz structure of the covariance matrix, we exploit the Gohberg-Semencul representation and the data-dependent trigonometric polynomials to derive a fast IAA (FIAA)-based inverse filtering to mitigate the computational burden. Simulation results and real data processing demonstrate that the proposed FIAA-based inverse filtering outperforms the existing super-resolution approaches in resolution improvement and results in a higher computational efficiency.