Abstract: We consider the task of estimating, from observed data, a probabilistic model that is parameterized by a finite number of parameters. In particular, we are considering the situation where the model probability density function is unnormalized. That is, the model is only specified up to the partition function. The partition function normalizes a model so that it integrates to one for any choice of the parameters. However, it is often impossible to obtain it in closed form. Gibbs distributions, Markov and multi-layer networks are examples of models where analytical normalization is often impossible. Maximum likelihood estimation can then not be used without resorting to numerical approximations which are often computationally expensive. We propose here a new objective function for the estimation of both normalized and unnormalized models. The basic idea is to perform nonlinear logistic regression to discriminate between the observed data and some artificially generated noise. With this approach, the normalizing partition function can be estimated like any other parameter. We prove that the new estimation method leads to a consistent (convergent) estimator of the parameters. For large noise sample sizes, the new estimator is furthermore shown to behave like the maximum likelihood estimator. In the estimation of unnormalized models, there is a trade-off between statistical and computational performance. We show that the new method strikes a competitive trade-off in comparison to other estimation methods for unnormalized models. As an application to real data, we estimate novel two-layer models of natural image statistics with spline nonlinearities.

Abstract: Background: Huge progress has been made in the development of array- or sequencing-based technologies for DNA methylation ana lysis. The Illumina Infinium ® Human Methylation 450K BeadChip (Illumina Inc., CA, USA) allows the simultaneous quantitative monitoring of more than 480,000 CpG positions, enabling large-scale epigenotyping studies. However, the assay combines two different assay chemistries, which may cause a bias in the ana lysis if all signals are merged as a unique source of methylation measurement. Materials & methods: We confirm in three 450K data sets that Infinium I signals are more stable and cover a wider dynamic range of methylation values than Infinium II signals. We evaluated the methylation profile of Infinium I and II probes obtained with different normalization protocols and compared these results with the methylation values of a subset of CpGs analyzed by pyrosequencing. Results: We developed a subset quantile normalization approach for the processing of 450K BeadChips. The Infinium I signals were used as ‘anchors’ to normalize Infinium II signals at the level of probe coverage categories. Our normalization approach outperformed alternative normalization or correction approaches in terms of bias correction and methylation signal estimation. We further implemented a complete preprocessing protocol that solves most of the issues currently raised by 450K array users. Conclusion: We developed a complete preprocessing pipeline for 450K BeadChip data using an original subset quantile normalization approach that performs both sample normalization and efficient Infinium I/II shift correction. The scripts, being freely available from the authors, will allow researchers to concentrate on the biological ana lysis of data, such as the identification of DNA methylation signatures.

Abstract: We discuss the identification of genes that are associated with an outcome in RNA sequencing and other sequence-based comparative genomic experiments. RNA-sequencing data take the form of counts, so models based on the Gaussian distribution are unsuitable. Moreover, normalization is challenging because different sequencing experiments may generate quite different total numbers of reads. To overcome these difficulties, we use a log-linear model with a new approach to normalization. We derive a novel procedure to estimate the false discovery rate (FDR). Our method can be applied to data with quantitative, two-class, or multiple-class outcomes, and the computation is fast even for large data sets. We study the accuracy of our approaches for significance calculation and FDR estimation, and we demonstrate that our method has potential advantages over existing methods that are based on a Poisson or negative binomial model. In summary, this work provides a pipeline for the significance analysis of sequencing data.

Abstract: Markov Chain Monte Carlo (MCMC) algorithms are routinely used to draw samples from distributions with intractable normalization constants. However, standard MCMC algorithms do not apply to doubly-intractable distributions in which there are additional parameter-dependent normalization terms; for example, the posterior over parameters of an undirected graphical model. An ingenious auxiliary-variable scheme (Moeller et al., 2004) offers a solution: exact sampling (Propp and Wilson, 1996) is used to sample from a Metropolis-Hastings proposal for which the acceptance probability is tractable. Unfortunately the acceptance probability of these expensive updates can be low. This paper provides a generalization of Moeller et al. (2004) and a new MCMC algorithm, which obtains better acceptance probabilities for the same amount of exact sampling, and removes the need to estimate model parameters before sampling begins.

Abstract: Shotgun proteomic data are affected by a variety of known and unknown systematic biases as well as high proportions of missing values. Typically, normalization is performed in an attempt to remove systematic biases from the data before statistical inference, sometimes followed by missing value imputation to obtain a complete matrix of intensities. Here we discuss several approaches to normalization and dealing with missing values, some initially developed for microarray data and some developed specifically for mass spectrometry-based data.

Abstract: The elasticity of substitution between capital and labor and, in turn, the direction of technical change are critical parameters in many fields of economics. Until recently, though, the application of production functions with specifically non-unitary substitution elasticities (i.e., non-Cobb–Douglas) was hampered by empirical and theoretical uncertainties. As recently revealed, ‘normalization’ of production-technology systems holds out the promise of resolving many of those uncertainties. We survey and assess the intrinsic links between production (as conceptualized in a production function), factor substitution (as made most explicit in Constant Elasticity of Substitution functions) and normalization (defined by the fixing of baseline values for relevant variables). First, we recall how the normalized Constant Elasticity of Substitution function came into existence and what normalization implies for its formal properties. Then we deal with the key role of normalization in recent advances in the theory of business cycles and of economic growth. Next, we discuss the benefits normalization brings for empirical estimation and empirical growth research. Finally, we identify promising areas of future research.

Abstract: Recent work has shown that visual attributes are a powerful approach for applications such as recognition, image description and retrieval However, fusing multiple attribute scores — as required during multi-attribute queries or similarity searches — presents a significant challenge Scores from different attribute classifiers cannot be combined in a simple way; the same score for different attributes can mean different things In this work, we show how to construct normalized “multi-attribute spaces” from raw classifier outputs, using techniques based on the statistical Extreme Value Theory Our method calibrates each raw score to a probability that the given attribute is present in the image We describe how these probabilities can be fused in a simple way to perform more accurate multiattribute searches, as well as enable attribute-based similarity searches A significant advantage of our approach is that the normalization is done after-the-fact, requiring neither modification to the attribute classification system nor ground truth attribute annotations We demonstrate results on a large data set of nearly 2 million face images and show significant improvements over prior work We also show that perceptual similarity of search results increases by using contextual attributes

Abstract: Over the last century, Component Analysis (CA) methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Canonical Correlation Analysis (CCA), Locality Preserving Projections (LPP), and Spectral Clustering (SC) have been extensively used as a feature extraction step for modeling, classification, visualization, and clustering. CA techniques are appealing because many can be formulated as eigen-problems, offering great potential for learning linear and nonlinear representations of data in closed-form. However, the eigen-formulation often conceals important analytic and computational drawbacks of CA techniques, such as solving generalized eigen-problems with rank deficient matrices (e.g., small sample size problem), lacking intuitive interpretation of normalization factors, and understanding commonalities and differences between CA methods. This paper proposes a unified least-squares framework to formulate many CA methods. We show how PCA, LDA, CCA, LPP, SC, and its kernel and regularized extensions correspond to a particular instance of least-squares weighted kernel reduced rank regression (LS--WKRRR). The LS-WKRRR formulation of CA methods has several benefits: 1) provides a clean connection between many CA techniques and an intuitive framework to understand normalization factors; 2) yields efficient numerical schemes to solve CA techniques; 3) overcomes the small sample size problem; 4) provides a framework to easily extend CA methods. We derive weighted generalizations of PCA, LDA, SC, and CCA, and several new CA techniques.

Abstract: The growing importance of sustainable development as a policy objective has initiated a debate about those suitable frameworks and tools useful for policy makers when making a sustainable decision. Composite indicators (CIs) aggregate multidimensional issues into one index, thus providing comprehensive information. However, it is frequently argued that CIs are too subjective, as their results undesirably depend on the normalization method, a specific weighting scheme, and the aggregation method of sub-indicators. This article applies different combinations of normalization, weighting, and aggregation methods for the assessment of an industrial case study, with the aim of determining the best scheme for constructing CIs. The applied methodology gradually aggregates sustainable development indicators into sustainability sub-indices and, finally, to a composite sustainability index. The normalization methods included in this analysis are: minimum–maximum, distance to a reference, and the percentages of annual differences over consecutive years. Equal weightings, the ‘benefit of the doubt’ approach, and budget allocation process were used for determining the weights of individual indicators and sustainability sub-indices. The linear, geometric, and non-compensatory multi-criteria approaches (NCMCs) were used as aggregation methods. The NCMC is modified to fit the two-level aggregation, then to sub-indices, and finally to a composite sustainable index. Also, a penalty criterion is introduced into the evaluation process with the aim of motivating the company to move towards sustainable development. The results are analyzed by variance-based sensitivity analysis. According to the results the recommended scheme for CIs’ construction is: distance to a reference–benefit of the doubt–linear aggregation.

Abstract: Interaction intent prediction and the Midas touch have been a longstanding challenge for eye-tracking researchers and users of gaze-based interaction. Inspired by machine learning approaches in biometric person authentication, we developed and tested an offline framework for task-independent prediction of interaction intents. We describe the principles of the method, the features extracted, normalization methods, and evaluation metrics. We systematically evaluated the proposed approach on an example dataset of gaze-augmented problem-solving sessions. We present results of three normalization methods, different feature sets and fusion of multiple feature types. Our results show that accuracy of up to 76% can be achieved with Area Under Curve around 80%. We discuss the possibility of applying the results for an online system capable of interaction intent prediction.

Abstract: Normalization of feature vectors of datasets is widely used in a number of fields of data mining, in particular in cluster analysis, where it is used to prevent features with large numerical values...

Abstract: Authors who report ground reaction force (GRF), free moment (FM), and resultant joint moments usually normalize these variables by division normalization. Normalization parameters include body weight (BW), body weight x height (BWH), and body weight x leg length (BWL). The purpose of this study was to explore the appropriateness of division normalization, power curve normalization, and offset normalization on peak GRF, FM, and resultant joint moments. Kinematic and kinetic data were collected on 98 subjects who walked at 1.2 and 1.8 m/s and ran at 3.4 and 4.0 m/s. Linear curves were best fit to the data, and regression analyses performed to test the significance of the correlations. It was found that the relationship between peak force and BW, as well as joint moments and BW, BWH, and BWL, were not always linear. After division normalization, significant correlations were still found. Power curve and offset normalization, however, were effective at normalizing all variables; therefore, when attempting to normalize GRF and joint moments, perhaps nonlinear or offset methods should be implemented.

Abstract: Educationalists are well able to find fault with rankings on numerous grounds and may reject them outright. However, given that they are here to stay, we could also try to improve them wherever possible. All currently published university rankings combine various measures to produce an overall score using an additive approach. The individual measures are first normalized to make the figures ‘comparable’ before they are combined. Various normalization procedures exist but, unfortunately, they lead to different results when applied to the same data: hence the compiler’s choice of normalization actually affects the order in which universities are ranked. Other difficulties associated with the additive approach include differing treatments of the student to staff ratio, and unexpected rank reversals associated with the removal or inclusion of institutions. We show that a multiplicative approach to aggregation overcomes all of these difficulties. It also provides a transparent interpretation for the weights. The proposed approach is very general and can be applied to many other types of ranking problem.

Abstract: I-vector extraction and Probabilistic Linear Discriminant Analysis (PLDA) has become the state-of-the-art configuration for speaker verification. Recently, Gaussian-PLDA has been improved by a preliminary length normalization of i-vectors. This normalization, known to increase the Gaussianity of the i-vector distribution, also improves performance of systems based on standard Linear Discriminant Analysis (LDA) and ”two-covariance model” scoring. But this technique follows a standardization of the i-vectors (centering and whitening ivectors based on the first and second order moments of the development data). We propose in this paper two techniques of normalization based on total, between- and within-speaker variance spectra 1 . These ”spectral” techniques both normalize the i-vectors length for Gaussianity, but the first adapts the ivectors representation to a speaker recognition system based on LDA and two-covariance scoring when the second adapts it to a Gaussian-PLDA model. Significant performance improvements are demonstrated on the male and female telephone portion of NIST SRE 2010. Index Terms: i-vectors, probabilistic linear discriminant analysis, speaker recognition.

Abstract: SUMMARY The estimation of the Green's function between two points on the Earth's surface by the cross-correlation of seismic noise time-series became a widely used method in seismology. In general, very long time-series (months to years) as well as massive normalization and/or data selection are necessary to obtain useful cross-correlation functions. One task of this study is to evaluate the influence of different established normalization methods on the obtained cross-correlation functions. Furthermore, we evaluate two waveform preserving time domain normalizations as well as a new fully automated data selection approach. The cross-correlation functions analysed in this study are obtained from 12 months of seismic noise recorded in 2004 at five seismic stations in the United States with station distances on a continental scale. For practical reasons, the cross-correlation functions of such long time-series are calculated by stacking the cross-correlation functions obtained from shorter time windows. We use this stacking process for the implementation of the waveform preserving time domain normalizations. The time window length is in general an important parameter of the cross-correlation processing, as it influences the normalization and data selection. Therefore, we evaluate the cross-correlation functions obtained with 47 different time window lengths between one hr and 24 hr. The time domain normalizations intend to suppress the influence of transient signals like earthquake waves as well as long-term (e.g. seasonal) amplitude variations. We compare the proposed waveform preserving time domain normalizations with the established running absolute mean normalization and the one-bit normalization. We demonstrate that a waveform preserving time domain normalization can replace a non-linear time domain normalization, if a time window length similar to the duration of the typically occurring transient signals is used. Next to the time domain normalizations also the spectral whitening in the frequency domain is evaluated. Spectral whitening is a powerful normalization to improve the emergence of broad-band signals in seismic noise cross-correlations. Nevertheless, we observe spectral whitening to depend strongly on the time window length. An unwanted amplification of a persistent microseism signal is observed on the continental scale with time windows shorter than 12 hr. Our approach of automated data selection is based on a statistical time-series classification and reliably excludes time windows with transient signals occurring contemporaneously at both sites (e.g. earthquake waves). This data selection approach is capable to replace a non-linear time domain normalization, but no improvement of the waveform symmetry or the signal-to-noise ratio of the cross-correlation functions is observed in general.

Abstract: The electromyogram is the summation of the motor unit action potentials occurring during contraction measured at a given electrode location. The voltage potential of the surface electromyographic signal detected by electrodes strongly depends on several factors, varying between individuals and also over time within an individual. Thus, the amplitude of the EMG signal itself is not useful in group comparisons, or to follow events over a long period of time. The fact that the recorded electromyographic amplitude is never absolute is mainly because impedance varies between the active muscle fibers and electrodes and its value is unknown. The EMG signal is highly variable and is dependent upon many factors. Thus, the amplitude of the temporally processed electromyography can only be used to assess short-term changes in the activity of a single muscle from the same individual when the electrode setup has not been altered. To allow comparison of activity between different muscles, across time, and between individuals, the EMG signal should be normalized, i.e. expressed in relation to a reference value obtained during standardized and reproducible conditions. Notwithstanding the importance of electromyographic amplitude normalization, studies on functional activities, such as gait, do not seem to show a uniform methodology. Taking this into account, the main purpose of this chapter is to review and discuss different normalization procedures to relate the most appropriate method for specific situations, based on how the normalization method might influence data interpretation. In addition, this review supports the development of proper normalization procedures for biomechanical studies of functional activities like human gait.

Abstract: Common extraction protocols and sampling procedures for GC-MS metabolite profiling were applied to the MCF-7 breast cancer cell culture as a model system of adherent growing cells and validated for repeatability and reproducibility. For normalization of a concentration series after methanolic extraction, results obtained with cell count normalization equalled normalization to the chromatogram total ion current as a chromatogram-intrinsic parameter, indicating that cell counting as an additional experimental step could be omitted. However, we show here that both normalization strategies should only be applied for comparison of extracts with similar concentrations complicating comparisons of different samples, e.g. with different biological origin. Therefore, the application of TIC thresholds for the comparison of differently concentrated extracts is recommended with respect to the accuracy of the data, the working effort and complexity of the biological experiment. For proof of concept, MCF-7 cell samples generated by different sampling procedures were assessed using these thresholds. Within this context, direct extraction of adherent growing cells without any further preparation steps, which allows for rapid quenching of cell metabolism and efficient sample extraction, is discussed as an alternative to conventional cell counting and extraction. In conclusion, we recommend the consideration of chromatogram intensity thresholds for the mean comparison of differently concentrated sample replicates in GC-MS metabolite profiling.

Abstract: Normalization of Notch expression restores enlarged blood vessels to microvessels through EphB4-mediated reprogramming of arterial endothelial cells. Reducing Inflation Arteriovenous malformations (AVMs) are a class of vascular abnormalities in which arteries connect directly with veins, thus bypassing the capillary beds and diverting blood flow away from tissues. In these vascular diseases, blood vessels, particularly the veins, become inflated in size and eventually rupture, resulting in hemorrhage and ischemia. AVMs, which can be found in any tissue, are particularly problematic in the brain, where surgical options are limited, and they often result in stroke or death. In a tour-de-force study, Murphy et al. now show that dialing down Notch4 receptor signaling in established AVMs in mouse brain reduces the size of enlarged blood vessels, resulting in restoration of blood flow to capillary beds and the reversal of hypoxia in mouse brain tissue. The Notch receptor is a master regulator of arteriovenous development and is up-regulated in AVMs in human brain. Overexpression of a constitutively active form of Notch4 in endothelial cells lining blood vessel walls is sufficient to induce AVMs in mice. In their new work, Murphy et al. first wanted to establish whether correction of Notch4 signaling could induce the regression of AVMs. Using their mouse brain AVM model, they obtained four-dimensional imaging data of the mouse brain vasculature viewed through a window cut into the cranium with two-photon fluorescence microscopy. When Notch4 signaling was normalized, they found regression of enlarged AVMs, which became similar in size to capillaries. This shrinkage in size enabled blood flow to return to oxygen-deprived tissues in the mouse brain. Surprisingly, the authors discovered that AVM regression was not induced by loss of endothelial cells, thrombotic occlusion, or vessel rupture. Rather, it required reprogramming of arterial endothelial cells in the enlarged AVM vessels to a venous endothelial cell specification. This reprogramming was activated by a decrease in Notch4 receptor signaling, which prompted arterial endothelial cells to start expressing the venous marker EphB4. These findings suggest that strategies to manipulate Notch receptor signaling in blood vessel endothelial cells may help to shrink AVMs and may be a new approach to treating AVMs and other vascular diseases. Abnormally enlarged blood vessels underlie many life-threatening disorders including arteriovenous (AV) malformations (AVMs). The core defect in AVMs is high-flow AV shunts, which connect arteries directly to veins, “stealing” blood from capillaries. Here, we studied mouse brain AV shunts caused by up-regulation of Notch signaling in endothelial cells (ECs) through transgenic expression of constitutively active Notch4 (Notch4*). Using four-dimensional two-photon imaging through a cranial window, we found that normalizing Notch signaling by repressing Notch4* expression converted large-caliber, high-flow AV shunts to capillary-like vessels. The structural regression of the high-flow AV shunts returned blood to capillaries, thus reversing tissue hypoxia. This regression was initiated by vessel narrowing without the loss of ECs and required restoration of EphB4 receptor expression by venous ECs. Normalization of Notch signaling resulting in regression of high-flow AV shunts, and a return to normal blood flow suggests that targeting the Notch pathway may be useful therapeutically for treating diseases such as AVMs.

Abstract: Fast and effective automated indexing is critical for search and personalized services. Key phrases that consist of one or more words and represent the main concepts of the document are often used for the purpose of indexing. In this paper, we investigate the use of additional semantic features and pre-processing steps to improve automatic key phrase extraction. These features include the use of signal words and freebase categories. Some of these features lead to significant improvements in the accuracy of the results. We also experimented with 2 forms of document pre-processing that we call light filtering and co-reference normalization. Light filtering removes sentences from the document, which are judged peripheral to its main content. Co-reference normalization unifies several written forms of the same named entity into a unique form. We also needed a “Gold Standard” ― a set of labeled documents for training and evaluation. While the subjective nature of key phrase selection precludes a true “Gold Standard”, we used Amazon's Mechanical Turk service to obtain a useful approximation. Our data indicates that the biggest improvements in performance were due to shallow semantic features, news categories, and rhetorical signals (nDCG 78.47% vs. 68.93%). The inclusion of deeper semantic features such as Freebase sub-categories was not beneficial by itself, but in combination with pre-processing, did cause slight improvements in the nDCG scores.

Abstract: In metabolomic research, it is important to reduce systematic error in experimental conditions. To ensure that metabolomic data from different studies are comparable, it is necessary to remove unwanted systematic factors by data normalization. Several normalization methods are used for metabolomic data, but the best method has not yet been identified. In this study, to reduce variation from non-biological systematic errors, we applied 1-norm, 2-norm, and quantile normalization methods to liquid chromatography-mass spectrometry (LC-MS)-based metabolomic data from human urine samples after oral administration of cyclosporine (high- and low-dose) in healthy volunteers and compared the effectiveness of the three methods. The principal component analysis (PCA) score plot showed more obvious groupings according to the cyclosporine dose after quantile normalization than after the other two methods and prior to normalization. Quantile normalization is a simple and effective method to reduce non-biological systematic variation from human LC-MS-based metabolomic data, revealing the biological variance.