Abstract: Self-organizing maps are popular algorithms for unsupervised learning and data visualization. Exploiting the link between vector quantization and mixture modeling, we derive expectation-maximization (EM) algorithms for self-organizing maps with and without missing values. We compare self-organizing maps with the elastic-net approach and explain why the former is better suited for the visualization of high-dimensional data. Several extensions and improvements are discussed. As an illustration we apply a self-organizing map based on a multinomial distribution to market basket analysis.

Abstract: In this paper we demonstrate that a recently introduced shape descriptor, the "shape context", can be used to quickly prune a search for similar shapes. Our representation for a shape is a discrete set of n points sampled from its internal and external contours. For each of these points, the shape context is a histogram of the relative positions of the n - 1 remaining points. We present two methods for rapid shape retrieval: one that does comparisons based on a small number of shape contexts and another that uses vector quantization in the space of shape contexts. We verify the discriminative power of these methods with tests on the Columbia (COIL-100) 3D object database and the Snodgrass and Vanderwart line drawings. The shape context-based methods are shown to quickly produce an accurate shortlist of candidates suitable for a more exact matching engine in spite of pose variation and occlusion.

Abstract: In lossy image compression schemes utilizing the discrete cosine transform (DCT), quantization of the DCT coefficients introduces error in the image representation and a loss of signal information. At high compression ratios, this introduced error produces visually undesirable compression artifacts that can dramatically lower the perceived quality of a particular image. This paper provides a spatial domain model of the quantization error based on a statistical noise model of the error introduced when quantizing the DCT coefficients. The resulting theoretically derived spatial domain quantization noise model shows that, in general, the compression noise in the spatial domain is both correlated and spatially varying. This provides some justification for many of the ad hoc artifact removal filters that have been proposed. An accurate description of quantization noise is essential if one hopes to remove, or at least alleviate, the visibility of compression artifacts.

Abstract: We propose a method to automatically determine the relevance of the input dimensions of a learning vector quantization (LVQ) architecture during training The method is based on Hebbian learning and introduces weighting factors of the input dimensions which are automatically adapted to the speci c problem The bene ts are twofold: On the one hand, the incorporation of relevance factors in the LVQ architecture increases the overall performance of the classi cation and adapts the metric to the speci c data used for training On the other hand, the method induces a pruning algorithm, ie an automatic detection of the input dimensions which do not contribute to the overall classi er Hence we obtain a possibly more eAEcient classi cation and we gain insight to the role of the data dimensions

Abstract: In this paper, we propose a systematic procedure to design a quantization table based on the human visual system model for the baseline JPEG coder. By incorporating the human visual system model with a uniform quantizer, a perceptual quantization table is derived. The quantization table can be easily adapted to the specified resolution for viewing and printing. Experimental results indicate that the derived HVS-based quantization table can achieve better performance in rate-distortion sense than the JPEG default quantization table.

Abstract: Anomaly detection involves characterizing the behaviors of individuals or systems and recognizing behavior that is outside the norm. This paper describes some preliminary results concerning the robustness and generalization capabilities of machine learning methods in creating user profiles based on the selection and subsequent classification of command line arguments. We base our method on the belief that legitimate users can be classified into categories based on the percentage of commands they use in a specified period. The hybrid approach we employ begins with the application of expert rules to reduce the dimensionality of the data, followed by an initial clustering of the data and subsequent refinement of the cluster locations using a competitive network called Learning Vector Quantization. Since Learning Vector Quantization is a nearest neighbor classifier, and new record presented to the network that lies outside a specified distance is classified as a masquerader. Thus, this system does not require anomalous records to be included in the training set.

Abstract: In financial market risk measurement, Value-at-Risk (VaR) techniques have proven to be a very useful and popular tool. Unfortunately, most VaR estimation models suffer from major drawbacks: the lognormal (Gaussian) modeling of the returns does not take into account the observed fat tail distribution and the non-stationarity of the financial instruments severely limits the efficiency of the VaR predictions. In this paper, we present a new approach to VaR estimation which is based on ideas from the field of information theory and lossless data compression. More specifically, the technique of context modeling is applied to estimate the VaR by conditioning the probability density function on the present context. Tree-structured vector quantization is applied to partition the multi-dimensional state space of both macroeconomic and microeconomic priors into an increasing but limited number of context classes. Each class can be interpreted as a state of aggregation with its own statistical and dynamic behavior, or as a random walk with its own drift and step size. Results on the US S&P500 index, obtained using several evaluation methods, show the strong potential of this approach and prove that it can be applied successfully for, amongst other useful applications, VaR and volatility prediction. The October 1997 crash is indicated in time.

Abstract: A network source code is an optimal source code for a network. To design network source codes, we require each node to have a single encoder, which jointly encodes all messages transmitted by that node, and a single decoder, which jointly decodes all messages arriving at that node. Given a distribution over the sources, the design of the network source code jointly optimizes all encoders and decoders to obtain the best performance with respect to a user-defined priority schedule over the rates and distortions of the system. In this paper we focus on fixed-rate codes and address the implementation of an existing design algorithm for optimal network vector quantizers. Implementing the design algorithm is not straightforward since each encoder must choose its reproduction based on the expected behavior of sources that are unknown to it. We describe a new implementation approach and demonstrate its performance on a three-node network. In addition, we extend the design algorithm to allow the decoder at each node to use side information (specifically, the messages that are to be encoded by the encoder at the same node).

Abstract: Quantization matrices facilitate digital audio encoding and decoding. An audio encoder generates and compresses quantization matrices; an audio decoder decompresses and applies the quantization matrices. The invention includes several techniques and tools, which can be used in combination or separately. For example, the audio encoder can generate quantization matrices from critical band patterns for blocks of audio data. The encoder can compute the quantization matrices directly from the critical band patterns, which can be computed from the same audio data that is being compressed. The audio encoder/decoder can use different modes for generating/applying quantization matrices depending on the coding channel mode of multi-channel audio data. The audio encoder/decoder can use different compression/decompression modes for the quantization matrices, including a parametric compression/decompression mode.

Abstract: A method for adaptive quantization of video frames based on bit rate prediction that includes increasing quantization in sectors of a video frame where coding artifacts would be less noticeable to the human visual system and decreasing quantization in sectors where coding artifacts would be more noticeable to the human visual system. In one embodiment the method reverts to uniform quantization for video frames in which adaptive quantization would require extra bits.

Abstract: Addresses the problem of the detection of speaker changes and clustering speakers when no information is available regarding speaker classes or even the total number of classes. We assume that no previous information on speakers is available (no speaker model, no training phase) and that people do not speak simultaneously. The aim is to apply speaker grouping information to speaker adaptation for speech recognition. We use vector quantization (VQ) distortion as the criterion. A speaker model is created from successive utterances as a codebook by a VQ algorithm, and the VQ distortion is calculated between the model and an utterance. A result was obtained by the experiment on speaker detection and speaker clustering. The speaker change detection experiment was compared with results by generalized likelihood ratio and Bayesian information criterion. We show the superiority of our proposed method.

Abstract: To improve the assessment capability of power transformers, this paper proposes a new intelligent decision support system based on fuzzy learning vector quantization (HVQ) networks. In constructing the system, a fuzzy-based classifier is designed to divide the historical data for dissolved gas analysis (DGA) into various categories with different levels of gas attributes. For each category of gas attributes, a learning vector quantization (LVQ) network is trained to be responsible for the classification of the potential faults due to insulation deterioration. The assessment approach has been tested on the DGA data from Taiwan Power Company (TPC) and compared with the previous fuzzy diagnosis system and the existing multi-layered backpropagation based artificial neural networks (BPANN) methods. Remarkable classification accuracy and far less training efforts of the proposed approach are achieved in this paper.

Abstract: In vector bit-plane encoding schemes, codebooks must be uniformly distributed on a hyper-sphere. Shells of regular lattices are often used, but they provide only a limited choice of number of vectors K and dimension N. The authors propose a method to generate codebooks in dimension N with arbitrary number K of vectors, almost uniformly distributed on a hyper-sphere. The uniform distribution of an arbitrary number of points on the surface of a hyper-sphere is still an open problem. Some mathematicians indeed consider it one of the mathematical challenges of the 21st century. The proposed method uses a combination of geometric and stochastic approaches to generate approximate solutions. The generated codebooks are tested in vector bit-plane encoding schemes. The results show that the proposed method is effective in generating codebooks almost uniformly distributed on a hyper-sphere.

Abstract: A quantizer finds a quantization threshold using a quantization loop with a heuristic approach. Following the heuristic approach reduces the number of iterations in the quantization loop required to find an acceptable quantization threshold, which instantly improves the performance of an encoder system by eliminating costly compression operations. A heuristic model relates actual bit-rate of output following compression to quantization threshold for a block of a particular type of data. The quantizer determines an initial approximation for the quantization threshold based upon the heuristic model. The quantizer evaluates actual bit-rate following compression of output quantized by the initial approximation. If the actual bit-rate satisfies a criterion such as proximity to a target bit-rate, the quantizer sets accepts the initial approximation as the quantization threshold. Otherwise, the quantizer adjusts the heuristic model and repeats the process with a new approximation of the quantization threshold. In an illustrative example, a quantizer finds a uniform, scalar quantization threshold using a quantization loop with a heuristic model adapted to spectral audio data. During decoding, a dequantizer applies the quantization threshold to decompressed output in an inverse quantization operation.

Abstract: A modified adaptive resonance theory (ART2) learning algorithm, which we employ in this paper, belongs to the family of NN algorithms whose main goal is the discovery of input data clusters, without considering their actual size. This feature makes the modified ART2 algorithm very convenient for image compression tasks, particularly when dealing with images with large background areas containing few details. Moreover, due to the ability to produce hierarchical quantization (clustering), the modified ART2 algorithm is proved to significantly reduce the computation time required for coding, and therefore enhance the overall compression process. Examples of the results obtained are presented, suggesting the benefits of using this algorithm for the purpose of VQ, i.e., image compression, over the other NN learning algorithms.

Abstract: An edge preserving image compression algorithm based on an unsupervised competitive neural network is proposed. The proposed neural network, the called weighted centroid neural network (WCNN), utilizes the characteristics of image blocks from edge areas. The mean/residual vector quantization (M/RVQ) scheme is utilized in this proposed approach as the framework of the proposed algorithm. The edge strength of image block data is utilized as a tool to allocate the proper code vectors in the proposed WCNN. The WCNN successfully allocates more code vectors to the image block data from edge area while it allocates less code vectors to the image black data from shade or non-edge area when compared to conventional neural networks based on VQ algorithm. As a result, a simple application of WCNN to an image compression problem gives improved edge characteristics in reconstructed images over conventional neural network based on VQ algorithms such as self-organizing map (SOM) and adaptive SOM.

Abstract: A theoretical analysis of recursive speech spectrum coding, where predictive and finite state schemes are special cases, is presented. We evaluate the spectral distortion (SD) theoretically and design coders that minimize the SD. The analysis rests on three cornerstones: high-rate theory, PDF modeling, and an approximation of SD. A derivation of the mean L/sub 2/-norm distortion of a recursive quantizer operating at high rate is provided. Also, the distortion distribution is supplied. The evaluation of the distortion expressions requires a model of the joint PDF of two consecutive spectrum vectors. The LPC spectrum source considered here has outcomes in a bounded region, and this is taken into account in the choice of model and modeling algorithm. It is further shown how to approximate the SD with an L/sub 2/-norm measure. Combining the results, we show theoretically that 16 bits are needed to achieve an average SD of 1 dB when quantizing ten-dimensional (10-D) spectrum vectors using a first-order recursive scheme. A gain of six bits per frame is noted compared to memoryless quantization. These results rely on high-rate assumptions which are validated in experiments. There, actual high-rate optimal coders are designed and evaluated.

Abstract: A novel approach to speaker recognition is presented. The method, called Support Vector Data Description (SVDD), was originally suggested by Vapnik, interpreted as a novelty detector by D. Tax and R. Duin (1999). In this paper, we use this data domain description as a classifier. It contains support vectors describing the sphere separating the samples. With a minimal radius R, this classifier achieves good performance in finding abnormal samples within the open set test. We use it in the speaker identification application. The results on YOHO database are presented.

Abstract: The increase of the need for image storage and transmission in computer systems has increased the importance of signal and image compression algorithms. The approach involving vector quantization (VQ) relies on the design of a finite set of codes which will substitute the original signal during transmission with a minimal of distortion, taking advantage of the spatial redundancy of image to compress them. Algorithms such as LBG and SOM work in an unsupervised way toward finding a good codebook for a given training data. However, the number of code vectors (N) needed for VQ increases with the vector dimension, and full-search algorithms such as LBG and SOM can lead to large training and coding times. An alternative for reducing the computational complexity is the use of a tree-structured vector quantization algorithm. This paper presents an application of a hierarchical SOM for image compression which reduces the search complexity from O(N) to O(log N), enabling a faster training and image coding. Results are given for conventional SOM, LBG and HSOM, showing the advantage of the proposed method.

Abstract: This paper addresses the problem of speech recognition in noisy conditions when low complexity is required like in embedded systems. In such systems, vector quantization is generally used to reduce the complexity of the recognition systems (e.g. HMMs). A novel approach for vector quantization based on the missing data theory is proposed. This approach allows to increase the robustness of the system against the noise perturbations with only a small increase of the computational requirements. The proposed algorithm is composed of two parts. The first part consists in dividing the spectral temporal features of the noisy signal into two subspaces: the unreliable (or missing) features and the reliable (or present) features. The second part of the proposed approach consists in defining a robust distance measure for vector quantization that compensates for the unreliable features. The proposed approach obtains similar results in noisy conditions than a more classical approach that consists in adapting the codebook of the vector quantization to the noisy conditions using model compensation. However the computation requirements are lower in the proposed approach and it is more suitable for a low complexity speech recognition system.

Abstract: We propose a new algorithm for vector quantization, the Activity Equalization Vector quantization (AEV). It is based on the winner takes all rule with an additional supervision of the average node activities over a training interval and a subsequent re-positioning of those nodes with low average activities. The re-positioning is aimed to both an exploration of the data space and a better approximation of already discovered data clusters by an equalization of the node activities. We introduce a learning scheme for AEV which requires as previous knowledge about the data only their bounding box. Using an example of Martinetz et al. l1r, AEV is compared with the Neural Gas, Frequency Sensitive Competitive Learning (FSCL) and other standard algorithms. It turns out to converge much faster and requires less computational effort.

Abstract: The paper describes the K-winner machine (KWM) model for classification. KWM training uses unsupervised vector quantization and subsequent calibration to label data-space partitions. A K-winner classifier seeks the largest set of best-matching prototypes agreeing on a test pattern, and provides a local-level measure of confidence. A theoretical analysis characterizes the growth function of a K-winner classifier, and the result leads to tight bounds to generalization performance. The method proves suitable for high-dimensional multiclass problems with large amounts of data. Experimental results on both a synthetic and a real domain (NIST handwritten numerals) confirm the approach effectiveness and the consistency of the theoretical framework.

Abstract: Mean-normalized vector quantization (M-NVQ) has been demonstrated to be the preferred technique for lossless compression of hyperspectral data. In this paper, a jointly optimized spatial M-NVQ/spectral DCT technique is shown to produce compression ratios significantly better than those obtained by the optimized spatial M-NVQ technique alone.

Abstract: Considerable research attention has been directed towards speech and audio coding algorithms capable of producing high quality coded speech and audio, however few of these use signal representations which account for temporal as well as spectral detail. This paper presents a new technique for 16 kHz wideband speech and audio coding, whereby analysis and synthesis are performed using a linear phase gammatone filter bank. The outputs of these critical band filters are processed to obtain a series of pulse trains that represent neural firing. Auditory masking is then applied to reduce the number of pulses, producing a more compact time-frequency parameterization. The critical band gains and pulse amplitudes and positions are then coded using a combination of non-uniform quantization, arithmetic coding and vector quantization. This coding paradigm produces high quality coded speech and audio, is based upon well-known models of the auditory system, is highly scalable, and has moderate complexity.

Abstract: A method and an apparatus to realize high quality image signal encoding at a variable bit rate, which is provided by the first quantization scale—maximum data volume calculation unit that calculates the maximum data volume and the first quantization scale based on encoding results of an encoded image frame, the quantization control buffer calculation unit that calculates a predicted data volume for a given encoding block that belongs to the encoding target image frame from the maximum data volume while receiving the originated data volume of the image frame that is quantized, encoding block (macroblock) by encoding unit (macroblock), by the quantization scale and calculates the second quantization scale based on the difference between the predicted data volume and the originated data volume, and the quantization control unit 10 that compares the first quantization scale with the second quantization scale to output the larger of the two scales.