Multimodal distribution

Abstract: Multilayer perceptrons (MLPs) or neural networks are popular models used for nonlinear regression and classification tasks As regressors, MLPs model the conditional distribution of the predictor variables Y given the input variables X However, this predictive distribution is assumed to be unimodal (eg Gaussian) For tasks involving structured prediction, the conditional distribution should be multi-modal, resulting in one-to-many mappings By using stochastic hidden variables rather than deterministic ones, Sigmoid Belief Nets (SBNs) can induce a rich multimodal distribution in the output space However, previously proposed learning algorithms for SBNs are not efficient and unsuitable for modeling real-valued data In this paper, we propose a stochastic feedforward network with hidden layers composed of both deterministic and stochastic variables A new Generalized EM training procedure using importance sampling allows us to efficiently learn complicated conditional distributions Our model achieves superior performance on synthetic and facial expressions datasets compared to conditional Restricted Boltzmann Machines and Mixture Density Networks In addition, the latent features of our model improves classification and can learn to generate colorful textures of objects

Abstract: Ensemble smoother (ES) has been widely used in inverse modeling of hydrologic systems. However, for problems where the distribution of model parameters is multimodal, using ES directly would be problematic. One popular solution is to use a clustering algorithm to identify each mode and update the clusters with ES separately. However, this strategy may not be very efficient when the dimension of parameter space is high or the number of modes is large. Alternatively, we propose in this paper a very simple and efficient algorithm, i.e., the iterative local updating ensemble smoother (ILUES), to explore multimodal distributions of model parameters in nonlinear hydrologic systems. The ILUES algorithm works by updating local ensembles of each sample with ES to explore possible multimodal distributions. To achieve satisfactory data matches in nonlinear problems, we adopt an iterative form of ES to assimilate the measurements multiple times. Numerical cases involving nonlinearity and multimodality are tested to illustrate the performance of the proposed method. It is shown that overall the ILUES algorithm can well quantify the parametric uncertainties of complex hydrologic models, no matter whether the multimodal distribution exists.