Abstract: Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations.Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).

Abstract: Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.

Abstract: A computationally efficient, high quality, vector quantization scheme based on a parametric probability density function (PDF) is proposed. In this scheme, the observations are modeled as i.i.d realizations of a multivariate Gaussian mixture density. The mixture model parameters are efficiently estimated using the expectation maximization (EM) algorithm. A low complexity quantization scheme using transform coding and bit allocation techniques which allows for easy mapping from observation to quantized value is developed for both fixed rate and variable rate systems. An attractive feature of this method is that source encoding using the resultant codebook involves very few searches and its computational complexity is minimal and independent of the rate of the system. Furthermore, the proposed scheme is bit scalable and can switch seamlessly between a memoryless quantizer and a quantizer with memory. The usefulness of the approach is demonstrated for speech coding where Gaussian mixture models are used to model speech line spectral frequencies. The performance of the memoryless quantizer is 1-3 bits better than conventional quantization schemes.

Abstract: This paper presents a novel algorithm for license plate detection and license plate character segmentation problems by using the Gabor transform in detection and local vector quantization in segmentation. As of our knowledge this is the first application of Gabor filters to license plate segmentation problem. Even though much of the research efforts are devoted to the edge or global thresholding-based approaches, it is more practical and efficient to analyze the image in certain directions and scales utilizing the Gabor transform instead of error-prone edge detection or thresholding. Gabor filter response only gives a rough estimate of the plate boundary. Then binary split tree is used for vector quantization in order to extract the exact boundary and segment the plate region into disjoint characters which become ready for the optical character recognition.

Abstract: We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization (LVQ). The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show 1) that annealing in the dispersion parameter of the Gaussian kernels improves classification accuracy; 2) that classification results are better than those obtained with standard learning vector quantization (LVQ 2.1, LVQ 3) for equal numbers of prototypes; and 3) that annealing of the width parameter improved the classification capability. Additionally, the principled approach provides an explanation of a number of features of the (heuristic) LVQ methods.

Abstract: High dimensional source vectors, such as those that occur in hyperspectral imagery, are partitioned into a number of subvectors of different length and then each subvector is vector quantized (VQ) individually with an appropriate codebook. A locally adaptive partitioning algorithm is introduced that performs comparably in this application to a more expensive globally optimal one that employs dynamic programming. The VQ indices are entropy coded and used to condition the lossless or near-lossless coding of the residual error. Motivated by the need for maintaining uniform quality across all vector components, a percentage maximum absolute error distortion measure is employed. Experiments on the lossless and near-lossless compression of NASA AVIRIS images are presented. A key advantage of the approach is the use of independent small VQ codebooks that allow fast encoding and decoding.

Abstract: The present invention relates to a method and system for multi-rate lattice vector quantization of a source vector x representing a frame from a source signal to be used, for example, in digital transmission and storage systems. The multi-rate lattice quantization encoding method comprises the steps of associating to x a lattice point y in a unbounded lattice Λ; verifying if y is included in a base codebook C derived from the lattice Λ; if it is the case then indexing y in C so as to yield quantization indices if not then extending the base codebook using, for example a Voronoi based extension method, yielding an extended codebook; associating to y a codevector c from the extended codebook, and indexing y in the extended codebook C. The extension technique allows to obtain higher bit rate codebooks from the base codebooks compared to quantization method and system from the prior art.

Abstract: Bucklew's (1984) high-rate vector quantizer mismatch result is extended from fixed-rate coding to variable-rate coding using a Lagrangian formulation. It is shown that if an asymptotically (high-rate) optimal sequence of variable rate codes is designed for a k-dimensional probability density function (PDF) g and then applied to another PDF f for which f/g is bounded, then the resulting mismatch or loss of performance from the optimal possible is given by the relative entropy or Kullback-Leibler (1968) divergence I(f/spl par/g). It is also shown that under the same assumptions, an asymptotically optimal code sequence for g can be converted to an asymptotically optimal code sequence for a mismatched source f by modifying only the lossless component of the code. Applications to quantizer design using uniform and Gaussian densities are described, including a high-rate analog to the Shannon rate-distortion result of Sakrison (1975) and Lapidoth (1997) showing that the Gaussian is the "worst case" for lossy compression of a source with known covariance. By coupling the mismatch result with composite quantizers, the worst case properties of uniform and Gaussian densities are extended to conditionally uniform and Gaussian densities, which provides a Lloyd clustering algorithm for fitting mixtures to general densities.

Abstract: Lately, Gaussian mixture (GM) models have found new applications in speech processing, and particularly in speech coding. This paper provides a review of GM based quantization and prediction. The main contribution is a discussion on GM model optimization. Two previously presented algorithms of EM-type are analyzed in some detail, and models are estimated and evaluated experimentally using theoretical measures as well as GM based speech spectrum coding and prediction. It has been argued that since many sources have a bounded support, this should be utilized in both the choice of model, and the optimization algorithm. By low-dimensional modeling examples, illustrating the behavior of the two algorithms graphically, and by full-scale evaluation of GM based systems, the advantages of a bounded support approach are quantified. For all evaluation techniques in the study, model accuracy is improved when the bounded support approach is adopted. The gains are typically largest for models with diagonal covariance matrices.

Abstract: A system and method for translating a written document into a computer readable document by recognizing the character written on the document aim at recognizing typed or printed, especially hand-printed or handwritten characters, in the various fields of a form. Providing a pixel representation of the written document, the method allows translating a written document into a computer readable document by i) identifying at least one field into the pixel representation of the document; ii) segmenting each field so as to yield at least one segmented symbol; iii) applying a character recognition method on each segmented symbol; and iii) assigning a computer-readable code to each recognized character resulting from the character recognition method. The character recognition method includes doing a vector quantization on each segmented symbol, and doing a vector classification using a vector base. A learning base is also created based on the optimal elliptic separation method. System and method according to the present invention allow to achieve a substitution rate of near zero.

Abstract: We investigate high-rate quantization for various detection and reconstruction loss criteria. A new distortion measure is introduced which accounts for global loss in best attainable binary hypothesis testing performance. The distortion criterion is related to the area under the receiver-operating-characteristic (ROC) curve. Specifically, motivated by Sanov's theorem, we define a performance curve as the trajectory of the pair of optimal asymptotic Type I and Type II error rates of the most powerful Neyman-Pearson test of the hypotheses. The distortion measure is then defined as the difference between the area-under-the-curve (AUC) of the optimal pre-encoded hypothesis test and the AUC of the optimal post-encoded hypothesis test. As compared to many previously introduced distortion measures for decision making, this distortion measure has the advantage of being independent of any detection thresholds or priors on the hypotheses, which are generally difficult to specify in the code design process. A high-resolution Zador-Gersho type of analysis is applied to characterize the point density and the inertial profile associated with the optimal high-rate vector quantizer. The analysis applies to a restricted class of high-rate quantizers that have bounded cells with vanishing volumes. The optimal point density is used to specify a Lloyd-type algorithm which allocates its finest resolution to regions where the gradient of the pre-encoded likelihood ratio has greatest magnitude.

Abstract: A fast search method for searching for an optimum codeword for nearest neighbor vector quantization. An upper boundary value and a lower boundary value between which an optimum codeword will exist in a codebook are calculated using a distortion of a designated element in an input vector and an experimentally determined threshold. Further, a start point and an end point for codebook search are determined using a binary search method from a codebook rearranged in descending order, and a full search scheme is applied only within a search range calculated by the determined start point and end point, thereby determining an optimum codeword for nearest neighbor vector quantization.

Abstract: The present invention relates to a method and device for quantizing linear prediction parameters in variable bit-rate sound signal coding, in which an input linear prediction parameter vector is received, a sound signal frame corresponding to the input linear prediction parameter vector is classified, a prediction vector is computed, the computed prediction vector is removed from the input linear prediction parameter vector to produce a prediction error vector, and the prediction error vector is quantized. Computation of the prediction vector comprises selecting one of a plurality of prediction schemes in relation to the classification of the sound signal frame, and processing the prediction error vector through the selected prediction scheme. The present invention further relates to a method and device for dequantizing linear prediction parameters in variable bit-rate sound signal decoding, in which at least one quantization index and information about classification of a sound signal frame corresponding to the quantization index are received, a prediction error vector is recovered by applying the index to at least one quantization table, a prediction vector is reconstructed, and a linear prediction parameter vector is produced in response to the recovered prediction error vector and the reconstructed prediction vector. Reconstruction of the prediction vector comprises processing the recovered prediction error vector through one of a plurality of prediction schemes depending on the frame classification information.

Abstract: The set partitioning in hierarchical trees (SPIHT) approach for still-image compression proposed by Said and Pearlman (1996) is one of the most efficient embedded monochrome image compression schemes known to date. The algorithm relies on a very efficient scanning and bit-allocation scheme for quantizing the coefficients obtained by a wavelet decomposition of an image. In this paper, we adopt this approach to scan groups (vectors) of wavelet coefficients, and use successive refinement vector quantization (VQ) techniques with staggered bit-allocation to quantize the groups at once. The scheme is named vector SPIHT (VSPIHT). We present discussions on possible models for the distributions of the coefficient vectors, and show how trained classified tree-multistage VQ techniques can be used to efficiently quantize them. Extensive coding results comparing VSPIHT to scalar SPIHT in the mean-squared-error sense, are presented for monochrome images. VSPIHT is found to yield superior performance for most images, especially those with high detail content. The method is also applied to color video coding, where a partially scalable bitstream is generated. We present the coding results on QCIF sequences as compared against H.263.

Abstract: The optimization criterion and a practically feasible new algorithm is stated for the optimization of the index assignments of a multiple-description unconstrained vector quantizer with an arbitrary number of descriptions. In the simulations, the index-optimized multiple-description vector quantizer achieves significant gains in source signal-to-noise ratio over scalar multiple description schemes.

Abstract: The present invention provides a method and device of multi-resolution vector quantization (VQ) for audio encoding and decoding used to analyze the audio signal in multi-resolution and quantize the vectors of them. Said method for encoding audio comprises the steps of: adaptively filtering a input audio signal so as to gain a time-frequency filter coefficient and output a filtered signal; dividing vectors of the filtered signal in a time-frequency plane so as to gain a vector combination; selecting the vector to be quantized; quantizing the selected vectors and calculating a quantization residual error; and transmitting a quantized coding task information as a side-information of an encoder to an audio decoder to quantize and encode the quantization residual error. The invention can adaptively filter the audio signal, and adjust the resolutions of time and frequency. The hereinafter result of multi-resolution time-frequency analysis can be utilized effectively through reorganizing the filter coefficient by selecting different organizing policies. VQ may improve encoding efficiency as well as control quantizing precision simply and optimize it.

Abstract: This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on a weighted norm to measure the distance between the feature vectors and their prototypes. The development of LVQ and clustering algorithms is based on the minimization of a reformulation function under the constraint that the generalized mean of the norm weights be constant. According to the proposed formulation, the norm weights can be computed from the data in an iterative fashion together with the prototypes. An error analysis provides some guidelines for selecting the parameter involved in the definition of the generalized mean in terms of the feature variances. The algorithms produced from this formulation are easy to implement and they are almost as fast as clustering algorithms relying on the Euclidean norm. An experimental evaluation on four data sets indicates that the proposed algorithms outperform consistently clustering algorithms relying on the Euclidean norm and they are strong competitors to non-Euclidean algorithms which are computationally more demanding.

Abstract: Minimum classification error (MCE) training is proposed to improve performance of a discrete hidden Markov model (DHMM)-based landmine detection system. The system (baseline) was proposed previously for detection of both metal and nonmetal mines from ground-penetrating radar signatures collected by moving vehicles. An initial DHMM model is trained by conventional methods of vector quantization and the Baum-Welch algorithm. A sequential generalized probabilistic descent (GPD) algorithm that minimizes an empirical loss function is then used to estimate the landmine/background DHMM parameters, and an evolutionary algorithm (EA) based on fitness score of classification accuracy is used to generate and select codebooks. The landmine data of one geographical site was used for model training, and those of two different sites were used for evaluation of system performance. Three scenarios were studied: 1) apply MCE/GPD alone to DHMM estimation, 2) apply EA alone to codebook generation, and 3) first apply EA to codebook generation and then apply MCE/GPD to DHMM estimation. Overall, the combined EA and MCE/GPD training led to the best performance. At the same level of detection rate as the baseline DHMM system, the false-alarm rate was reduced by a factor of two, indicating significant performance improvement.

Abstract: In this paper, we analyse the effect of the quantization of signals used for system identification and show an optimal quantization scheme for minimizing estimation errors under a constraint on the number of subsections of the quantized signals. The optimal quantization scheme has the property that it is coarse near the origin and dense at a distance from it in the definition area of the signals. We also evaluate the estimated parameters and show a trade-off between the quantization error and the noise error under the constraint on the amount of information in the output data.

Abstract: A new scheme that aims to cut down on the computational cost of the vector quantization (VQ) encoding procedure is proposed in this paper. In this scheme, the correlation between the codewords in t...

Abstract: A number of methods and systems for efficiently storing defective-memory-location tables. A asymmetrical-distortion-model vector quantization method and a run-length quantization method for compressing a defective-memory-location bit map that identifies defective memory locations within a memory are provided. In addition, because various different compression/decompression methods may be suitable for different types of defect distributions within a memory, a method is provided to select the most appropriate compression/decompression method from among a number of compression/decompression methods as most appropriate for a particular defect probability distribution. Finally, bit-map compression and the figure-of-merit metric for selecting an appropriate compression technique may enable global optimization of error-correcting codes and defective memory-location identification.

Abstract: A new approach to the design of optimised codebooks using vector quantisation (VQ) is presented. A strategy of reinforced learning (RL) is proposed which exploits the advantages offered by fuzzy clustering algorithms, competitive learning and knowledge of training vector and codevector configurations. Results are compared with the performance of the generalised Lloyd algorithm (GLA) and the fuzzy K-means (FKM) algorithm. It has been found that the proposed algorithm, fuzzy reinforced learning vector quantisation (FRLVQ), yields an improved quality of codebook design in an image compression application when FRLVQ is used as a pre-process. The investigations have also indicated that RL is insensitive to the selection of both the initial codebook and a learning rate control parameter, which is the only additional parameter introduced by RL from the standard FKM.

Abstract: Topological simplification techniques and topology preserving compression approaches for 2D vector fields have been developed quite independently of each other. In this paper we propose a combination of both approaches: a vector field should be compressed in such a way that its important topological features (both critical points and separatrices) are preserved while its unimportant features are allowed to collapse and disappear. To do so, a number of new solutions and modifications of pre-existing algorithms are presented. We apply the approach to a flow data set, which is both large and topologically complex, and achieve significant compression ratios there.

Abstract: The main objective of this work is to determine a numerical measure that can judge block-based compressed images, namely vector quantization (VQ), JPEG, and JPEG2000 encoded images. Various objective measures are investigated and new measures are developed, then all measures are classified into groups. The analysis of normalized versions of the measures within each group has led to the design of a new measure, denoted as the normalized perceptual image quality measure (NPIQM), that better matches the psychophysical evaluation of the image quality than using the peak signal-to-noise ratio (PSNR) criterion. NPIQM can also be plotted graphically in a radar-fashion form, which gives the ability to compare different images at different bit rates, and give a detailed judgment based on the proposed graphical interpretation. A perceptual examination is carried out and proves the superiority of the NPIQM measure over PSNR.

Abstract: We propose two techniques that are applicable to any adaptive vector quantization (AVQ) systems. The first one is called the locality-based codebook updating: when performing a codebook updating, we update the operational codebook using not only the current input vector but also the codewords at all positions within a selected neighboring area (called the locality), while the operational codebook is organized in a "cache" manner. This technique is rationalized by the high correlation cross neighboring vectors that facilitates a more efficient coding of the indices of the codewords chosen from the codebook. The second technique is called the history aid, which makes use of the information of previously coded vectors to quantize the current input vector if it is used to update the operational codebook. A more effective AVQ system is obtained by combining together the history aid and the locality-based updating. Extensive simulations are carried out to demonstrate the improved results achieved by our AVQ systems. Particularly, when the operational codebook size is relatively small, the improvement over a benchmark AVQ system - the generalized threshold replenishment (GTR) - is drastic. For example, when the size is 32, testing on a nonstationary signal (containing frames from different video sequences, ordered in the concatenating or interleaving format) shows that the combination of history aid and locality-based updating offers more than 4 dB gain over GTR at 0.5 bpp.

Abstract: It is possible to prevent deterioration of image quality by selecting with priority an code amount from a DPCM path so as to satisfy the target code amount. Either a code amount obtained by a first compression method for quantizing input image signals by different quantization steps or a code amount obtained by a second compression method having a lower compression ratio and smaller loss than the first compression method is added for each encoding method selection unit, thereby calculating the total code amount of the length-equalizing unit. The total code amount calculated is compared to the target code amount in the length-equalizing unit. According to the comparison result, the quantization step in the first compression method is decided so that quantization is performed by the quantization step decided or the second compression method is selected for each encoding method selection unit.