Abstract: In our work, we bridge deep neural network design with numerical differential equations. We show that many effective networks, such as ResNet, PolyNet, FractalNet and RevNet, can be interpreted as different numerical discretizations of differential equations. This finding brings us a brand new perspective on the design of effective deep architectures. We can take advantage of the rich knowledge in numerical analysis to guide us in designing new and potentially more effective deep networks. As an example, we propose a linear multi-step architecture (LM-architecture) which is inspired by the linear multi-step method solving ordinary differential equations. The LM-architecture is an effective structure that can be used on any ResNet-like networks. In particular, we demonstrate that LM-ResNet and LM-ResNeXt (i.e. the networks obtained by applying the LM-architecture on ResNet and ResNeXt respectively) can achieve noticeably higher accuracy than ResNet and ResNeXt on both CIFAR and ImageNet with comparable numbers of trainable parameters. In particular, on both CIFAR and ImageNet, LM-ResNet/LM-ResNeXt can significantly compress ($>50$\%) the original networks while maintaining a similar performance. This can be explained mathematically using the concept of modified equation from numerical analysis. Last but not least, we also establish a connection between stochastic control and noise injection in the training process which helps to improve generalization of the networks. Furthermore, by relating stochastic training strategy with stochastic dynamic system, we can easily apply stochastic training to the networks with the LM-architecture. As an example, we introduced stochastic depth to LM-ResNet and achieve significant improvement over the original LM-ResNet on CIFAR10.

Abstract: Full waveform inversion (FWI) is a powerful method for reconstructing subsurface parameters from local measurements of the seismic wavefield. This method consists in minimizing the distance between predicted and recorded data. The predicted data are computed as the solution of a wave-propagation problem. Conventional numerical methods for the solution of FWI problems are gradient-based methods, such as the preconditioned steepest descent, the nonlinear conjugate gradient, or more recently the $l$-BFGS quasi-Newton algorithm. In this study, we investigate the desirability of applying a truncated Newton method to FWI. The inverse Hessian operator plays a crucial role in the parameter reconstruction, as it should help to mitigate finite-frequency effects and to better remove artifacts arising from multiscattered waves. For multiparameter reconstruction, the inverse Hessian operator also offers the possibility of better removing trade-offs due to coupling effects between parameter classes. The truncated Newto...

Abstract: By means of Non-Uniform Rational B-Splines (NURBS) curves, it is possible to describe arbitrary shapes with holes and discontinuities. These peculiar shapes can be taken into account to describe the reference domain of several nanoplates, where a nanoplate refers to a flat structure reinforced with Carbon Nanotubes (CNTs). In the present paper, a micromechanical model based on the agglomeration of these nanoparticles is considered. Indeed, when this kind of reinforcing phase is inserted into a polymeric matrix, CNTs tend to increase their density in some regions. Nevertheless, some nanoparticles can be still scattered within the matrix. The proposed model allows to control the agglomeration by means of two parameters. In this way, several parametric studies are presented to show the influence of this agglomeration on the free vibrations. The considered structures are characterized also by a gradual variation of CNTs along the plate thickness. Thus, the term Functionally Graded Carbon Nanotubes (FG-CNTs) is introduced to specify these plates. Some additional parametric studies are also performed to analyze the effect of a mesh distortion, by considering several geometric and mechanical configurations. The validity of the current methodology is proven through a comparative assessment of our results with those available from the literature or obtained with different numerical approaches, such as the Finite Element Method (FEM). The strong form of the equations governing a plate is solved by means of the Generalized Differential Quadrature (GDQ) method.

Abstract: Nonlinear eigenvalue problems arise in a variety of science and engineering applications and in the past ten years there have been numerous breakthroughs in the development of numerical methods. This article surveys nonlinear eigenvalue problems associated with matrix-valued functions which depend nonlinearly on a single scalar parameter, with a particular emphasis on their mathematical properties and available numerical solution techniques. Solvers based on Newton's method, contour integration, and sampling via rational interpolation are reviewed. Problems of selecting the appropriate parameters for each of the solver classes are discussed and illustrated with numerical examples. This survey also contains numerous MATLAB code snippets that can be used for interactive exploration of the discussed methods.

Abstract: In this research, a Bernoulli wavelet operational matrix of fractional integration is presented Bernoulli wavelets and their properties are employed for deriving a general procedure for forming this matrix The application of the proposed operational matrix for solving the fractional delay differential equations is explained Also, upper bound for the error of operational matrix of the fractional integration is given This operational matrix is utilized to transform the problem to a set of algebraic equations with unknown Bernoulli wavelet coefficients Several numerical examples are solved to demonstrate the validity and applicability of the presented technique

Abstract: In this paper, we present a novel second order in time mixed finite element scheme for the Cahn–Hilliard–Navier–Stokes equations with matched densities. The scheme combines a standard second order Crank–Nicolson method for the Navier–Stokes equations and a modification to the Crank–Nicolson method for the Cahn–Hilliard equation. In particular, a second order Adams-Bashforth extrapolation and a trapezoidal rule are included to help preserve the energy stability natural to the Cahn–Hilliard equation. We show that our scheme is unconditionally energy stable with respect to a modification of the continuous free energy of the PDE system. Specifically, the discrete phase variable is shown to be bounded in $$\ell ^\infty \left( 0,T;L^\infty \right) $$ and the discrete chemical potential bounded in $$\ell ^\infty \left( 0,T;L^2\right) $$ , for any time and space step sizes, in two and three dimensions, and for any finite final time T. We subsequently prove that these variables along with the fluid velocity converge with optimal rates in the appropriate energy norms in both two and three dimensions.

Abstract: We consider numerical methods for solving the fractional-in-space Allen–Cahn equation which contains small perturbation parameters and strong nonlinearity. A standard fully discretized scheme for this equation is considered, namely, using the conventional second-order Crank–Nicolson scheme in time and the second-order central difference approach in space. For the resulting nonlinear scheme, we propose a nonlinear iteration algorithm, whose unique solvability and convergence can be proved. The nonlinear iteration can avoid inverting a dense matrix with only $$\mathcal {O}(N\log N)$$ computation complexity. One major contribution of this work is to show that the numerical solutions satisfy discrete maximum principle under reasonable time step constraint. Based on the maximum stability, the nonlinear energy stability for the fully discrete scheme is established, and the corresponding error estimates are investigated. Numerical experiments are performed to verify the theoretical results.

Abstract: Differential Equations Classification of Differential Equations Linear Equations Non-Linear Equations Existence and Uniqueness of Solutions Numerical Methods Computer Programming First Ideas and Single-Step Methods Analytical and Numerical Solutions A First Example The Taylor Series Method Runge-Kutta Methods Second and Higher Order Equations Error Considerations Definitions Local Truncation Error for the Taylor Series Method Local Truncation Error for the Runge-Kutta Method Local Truncation and Global Errors Local Error and LTE Runge-Kutta Methods Error Criteria A Third Order Formula Fourth Order Formulae Fifth and Higher Order Formulae Rationale for Higher Order Formulae Computational Examples Step-Size Control Steplength Prediction Error Estimation Local Extrapolation Error Estimation with RK Methods More Runge-Kutta Pairs Application of RK Embedding Dense Output Construction of Continuous Extensions Choice of Free Parameters Higher-Order Formulae Computational Aspects of Dense Output Inverse Interpolation Stability and Stiffness Absolute Stability Non-Linear Stability Stiffness Improving the Stability of RK Methods Multistep Methods The Linear Multistep Process Selection of Parameters A Third Order Implicit Formula A Third Order Explicit Formula Predictor-Corrector Schemes Error Estimation A Predictor-Corrector Program Multistep Formulae from Quadrature Quadrature Applied to Differential Equations The Adams-Bashforth Formulae The Adams-Moulton Formulae Other Multistep Formulae Varying the Step Size Numerical Results Stability of Multistep Methods Some Numerical Experiments Zero-Stability Weak Stability Theory Stability Properties of Some Formulae Stability of Predictor-Corrector Pairs Methods for Stiff Systems Differentiation Formulae Implementation of BDF Schemes A BDF Program Implicit Runge-Kutta Methods A Semi-Implicit RK Program Variable Coefficient Multistep Methods Variable Coefficient Integrators Practical Implementation Step-Size Estimation A Modified Approach An Application of STEP90 Global Error Estimation Classical Extrapolation Solving for the Correction An Example of Classical Extrapolation The Correction Technique Global Embedding A Global Embedding Program Second Order Equations Transformation of the RK Process A Direct Approach to the RKNG Processes The Special Second Order Problem Dense Output for RKN Methods Multistep Methods Partial Differential Equations Finite Differences Semi-Discretization of the Heat Equation Highly Stable Explicit Schemes Equations with Two Space Dimensions Non-Linear Equations Hyperbolic Equations Appendix A: Programs for Single Step Methods A Variable Step Taylor Method An Embedded Runge-Kutta Program A Sample RK Data File An Alternative Runge-Kutta Scheme Runge-Kutta with Dense Output A Sample Continuous RK Data File Appendix B: Multistep Programs A Constant Steplength Program A Variable Step Adams PC Scheme A Variable Coefficient Multistep Package Appendix C: Programs for Stiff Systems A BDF Program A Diagonally Implicit RK Program Appendix D: Global Embedding Programs The Gem Global Embedding Code The GEM90 Package with Global Embedding A Driver Program for GEM90 Appendix E: A Runge-Kutta Nystroem Program Bibliography Index Each chapter also includes an introduction and a section of exercise problems.

Abstract: In this paper, a class of nonlinear Riesz space-fractional Schrodinger equations are considered. Based on the standard Galerkin finite element method in space and Crank-Nicolson difference method in time, the semi-discrete and fully discrete systems are constructed. By Brouwer fixed point theorem and fractional Gagliardo-Nirenberg inequality, we prove the fully discrete system is uniquely solvable. Moreover, we focus on a rigorous analysis and consideration of the conservation and convergence properties for the semi-discrete and fully discrete systems. Finally, a linearized iterative finite element algorithm is introduced and some numerical examples are given to confirm the theoretical results.

Abstract: In this paper we analyze the convergence properties of two-level and W-cycle multigrid solvers for the numerical solution of the linear system of equations arising from hp-version symmetric interior penalty discontinuous Galerkin discretizations of second-order elliptic partial differential equations on polygonal/polyhedral meshes. We prove that the two-level method converges uniformly with respect to the granularity of the grid and the polynomial approximation degree p, provided that the number of smoothing steps, which depends on p, is chosen sufficiently large. An analogous result is obtained for the W-cycle multigrid algorithm, which is proved to be uniformly convergent with respect to the mesh size, the polynomial approximation degree, and the number of levels, provided the latter remains bounded and the number of smoothing steps is chosen sufficiently large. Numerical experiments are presented which underpin the theoretical predictions; moreover, the proposed multilevel solvers are shown to be convergent in practice, even when some of the theoretical assumptions are not fully satisfied.

Abstract: In recent years, considerable attention has been devoted to distributed-order differential equations mainly because they appear to be more effective for modelling complex processes which obey a mixture of power laws or flexible variations in space In this paper, we propose a novel finite volume method (FVM) for a distributed-order space-fractional diffusion equation (FDE) Firstly, we use the mid-point quadrature rule to transform the space distributed-order diffusion equation into a multi-term fractional equation Secondly, the transformed multi-term fractional equation is solved by discretising in space using the finite volume method and then in time using the Crank–Nicolson scheme Thirdly, we prove that the Crank–Nicolson scheme with FVM is unconditionally stable and convergent with second order accuracy in both time and space Finally, two numerical examples are presented to show the effectiveness of the numerical method These methods and techniques can also be used to solve other types of fractional partial differential equations

Abstract: This paper is devoted to a study of the numerical solution of elliptic hemivariational inequalities with or without convex constraints by the finite element method. For a general family of elliptic...

Abstract: In this paper, the numerical solutions of conformable fractional-order linear and nonlinear equations are obtained by employing the constructed conformable Adomian decomposition method (CADM). We found that CADM is an effective method for numerical solution of conformable fractional-order differential equations. Taking the conformable fractional-order simplified Lorenz system as an example, the numerical solution and chaotic behaviors of the conformable fractional-order simplified Lorenz system are investigated. It is found that rich dynamics exist in the conformable fractional-order simplified Lorenz system, and the minimum order for chaos is even less than 2. The results are validated by means of bifurcation diagram, Lyapunov characteristic exponents and phase portraits.

Abstract: Unconstrained optimization is the search for the maximum or minimum of a function with no restriction on the values of the variables. At the same time, it forms the basis for methods of constrained optimization in the next chapter. Zero-order methods use only function values, progress made in the previous step pointing the way to the next step. The Hooke and Jeeves method is one such method, suitable for small problems with little programming effort. First-order methods employ the gradient of the function, usually obtained by finite difference, to derive a search direction. This is followed by a line search along this direction for the current maximum or minimum, performed either by progressive reduction of the region in which the maximum or minimum is to be found or by polynomial interpolation. In its simplest form, this is the steepest descent method. However, by the use of gradient data from the previous iteration, an improved search direction can be found, with faster convergence. This is the Fletcher–Reeves method. A more general formulation is based on a quadratic approximation to the objective function, referred to as a second-order method or quasi-Newton method. This involves progressively building up an approximation to the inverse of the Hessian matrix of second derivatives to deduce a search direction. A spreadsheet program for the Hooke and Jeeves method is also used in the next chapter for the penalty function method for constrained optimization.

Abstract: In recent years considerable progress has been made in the continuum modelling of granular flows, in particular the -rheology, which links the local viscosity in a flow to the strain rate and pressure through the non-dimensional inertial number . This formulation greatly benefits from its similarity to the incompressible Navier–Stokes equations as it allows many existing numerical methods to be used. Unfortunately, this system of equations is ill posed when the inertial number is too high or too low. The consequence of ill posedness is that the growth rate of small perturbations tends to infinity in the high wavenumber limit. Due to this, numerical solutions are grid dependent and cannot be taken as being physically realistic. In this paper changes to the functional form of the curve are considered, in order to maximise the range of well-posed inertial numbers, while preserving the overall structure of the equations. It is found that when the inertial number is low there exist curves for which the equations are guaranteed to be well posed. However when the inertial number is very large the equations are found to be ill posed regardless of the functional dependence of on . A new curve, which is inspired by the analysis of the governing equations and by experimental data, is proposed here. In order to test this regularised rheology, transient granular flows on inclined planes are studied. It is found that simulations of flows, which show signs of ill posedness with unregularised models, are numerically stable and match key experimental observations when the regularised model is used. This paper details two-dimensional transient computations of decelerating flows where the inertial number tends to zero, high-speed flows that have large inertial numbers, and flows which develop into granular rollwaves. This is the first time that granular rollwaves have been simulated in two dimensions, which represents a major step towards the simulation of other complex granular flows.

Abstract: In the current decade, the meshless methods have been developed for solving partial differential equations. The meshless methods may be classified in two basic parts: 1.The meshless methods based on the strong form2.The meshless methods based on the weak form The element-free Galerkin (EFG) method is a meshless method based on the global weak form. The test and trial functions in element-free Galerkin are shape functions of moving least squares (MLS) approximation. Also, the traditional MLS shape functions have not the ź-Kronecker property. Recently, a new class of MLS shape functions has been presented. These are well-known as the interpolating MLS (IMLS) shape functions. The IMLS shape functions have the ź-Kronecker property; thus the essential boundary conditions can be applied directly. The main aim of this paper is to combine the alternating direction implicit approach with the IEFG method. To this end, we apply the mentioned technique on the distributed order time-fractional diffusion-wave equation. For comparing the numerical results, we propose three schemes based on the trapezoidal, Simpson, and Gauss-Legendre quadrature techniques. Also, we investigate the uniqueness, existence and stability analysis of the new schemes and we obtain an error estimate for the full-discrete schemes. The time-fractional derivative has been described in Caputo's sense. Numerical examples demonstrate the theoretical results and the efficiency of the proposed schemes.

Abstract: We present and assess a Bayesian method to interpret gravitational wave signals from binary black holes Our method directly compares gravitational wave data to numerical relativity (NR) simulations In this study, we present a detailed investigation of the systematic and statistical parameter estimation errors of this method This procedure bypasses approximations used in semianalytical models for compact binary coalescence In this work, we use the full posterior parameter distribution for only generic nonprecessing binaries, drawing inferences away from the set of NR simulations used, via interpolation of a single scalar quantity (the marginalized log likelihood, lnL) evaluated by comparing data to nonprecessing binary black hole simulations We also compare the data to generic simulations, and discuss the effectiveness of this procedure for generic sources We specifically assess the impact of higher order modes, repeating our interpretation with both l ≤ 2 as well as l ≤ 3 harmonic modes Using the l ≤ 3 higher modes, we gain more information from the signal and can better constrain the parameters of the gravitational wave signal We assess and quantify several sources of systematic error that our procedure could introduce, including simulation resolution and duration; most are negligible We show through examples that our method can recover the parameters for equal mass, zero spin, GW150914-like, and unequal mass, precessing spin sources Our study of this new parameter estimation method demonstrates that we can quantify and understand the systematic and statistical error This method allows us to use higher order modes from numerical relativity simulations to better constrain the black hole binary parameters

Abstract: We present a 50-line MATLAB implementation of the lowest order virtual element method for the two-dimensional Poisson problem on general polygonal meshes. The matrix formulation of the method is discussed, along with the structure of the overall algorithm for computing with a virtual element method. The purpose of this software is primarily educational, to demonstrate how the key components of the method can be translated into code.

Abstract: We introduce an exponential-type time-integrator for the KdV equation and prove its first-order convergence in $$H^1$$ for initial data in $$H^3$$ . Furthermore, we outline the generalization of the presented technique to a second-order method.

Abstract: In this work, the smoothed finite element method using four-node quadrilateral elements (SFEM-Q4) is employed to resolve underwater acoustic radiation problems. The SFEM-Q4 can be regarded as a com...