Efficient Variable-to-Fixe d Length Coding Algorithms for Text Compression

TL;DR: This thesis focuses on lossless compression for text data, that is, text compression, and Variable-to-Fixed-length coding, a coding scheme that segments an input text into a consecutive sequence of substrings and then assigns a fixed length codeword to each substring.

Abstract: Data compression is a technique for reducing the storage space and the cost of transferring a large amount of data, using redundancy hidden in the data. We focus on lossless compression for text data, that is, text compression, in this thesis. To reuse a huge amount of data stored in secondary storage, I/O speeds are bottlenecks. Such a communication-speed problem can be relieved if we transfer only compressed data through the communication channel and furthermore can perform every necessary processes, such as string search, on the compressed data itself without decompression. Therefore, a new criterion “ease of processing the compressed data” is required in the field of data compression. Development of compression algorithms is currently in the mainstream of data compression field but many of them are not adequate for that criterion. The algorithms employing variable length codewords succeeded to achieve an extremely good compression ratio, but the boundaries between codewords are not obvious without a special processing. Such an “unclear boundary problem” prevents us from direct accessing to the compressed data. On the contrary, Variable-to-Fixed-length coding , which is referred to as VF coding, is promising for our demand. VF coding is a coding scheme that segments an input text into a consecutive sequence of substrings (called phrases) and then assigns a fixed length codeword to each substring. Boundaries between codewords of VF coding are obvious because all of them have the same length. Therefore, we can realize “accessible data compression” by VF coding. Nevertheless, VF coding was not paid much attention