Universal code

Abstract: Countable prefix codeword sets are constructed with the universal property that assigning messages in order of decreasing probability to codewords in order of increasing length gives an average code-word length, for any message set with positive entropy, less than a constant times the optimal average codeword length for that source. Some of the sets also have the asymptotically optimal property that the ratio of average codeword length to entropy approaches one uniformly as entropy increases. An application is the construction of a uniformly universal sequence of codes for countable memoryless sources, in which the n th code has a ratio of average codeword length to source rate bounded by a function of n for all sources with positive rate; the bound is less than two for n = 0 and approaches one as n increases.

Abstract: It is well known that if the data rate is chosen below the available channel capacity, error-free communication is possible. Furthermore, numerous practical error-correction coding techniques exist which can be chosen to meet the user's reliability constraints. However, a basic problem in designing a reliable digital communication system is still the choice of the actual code rate. While the popular rate-1/2 code rate is a reasonable, but not optimum, choice for additive Gaussian noise channels, its selection is far from optimum for channels where a high percentage of the transmitted bits are destroyed by interference. Code combining represents a technique of matching the code rate to the prevailing channel conditions. Information is transmitted in packet formats which are encoded with a relatively high-rate code, e.g., rate 1/2, which can be repeated to Obtain reliable communications when the redundancy in a rate-1/2 code is not sufficient to overcome the channel interference. The receiver combines noisy packets (code combining) to obtain a packet with a code rate which is low enough such that reliable communication is possible even for channels with extremely high error rates. By combining the minimum number of packets needed to overcome the channel conditions, the receiver optimizes the code rate and minimizes the delay required to decode a given packet. Thus, the receiver adapts to the actual jammer-to-signal (J/S) ratio which is critical when the level of interference J is not known a priori.

Abstract: We investigate a type of lossless source code called a grammar-based code, which, in response to any input data string x over a fixed finite alphabet, selects a context-free grammar G/sub x/ representing x in the sense that x is the unique string belonging to the language generated by G/sub x/. Lossless compression of x takes place indirectly via compression of the production rules of the grammar G/sub x/. It is shown that, subject to some mild restrictions, a grammar-based code is a universal code with respect to the family of finite-state information sources over the finite alphabet. Redundancy bounds for grammar-based codes are established. Reduction rules for designing grammar-based codes are presented.