無損數(shù)據(jù)編碼領(lǐng)域應(yīng)用較少。針對(duì)這種現(xiàn)狀，該文詳細(xì)地研究了最大熵統(tǒng)計(jì)模型和神經(jīng)網(wǎng)絡(luò)算法各自的特點(diǎn)，提出了一種基于最大熵原理的神經(jīng)網(wǎng)絡(luò)概率預(yù)測(cè)模型并結(jié)合自適應(yīng)算術(shù)編碼來進(jìn)行數(shù)據(jù)壓縮，具有精簡的網(wǎng)絡(luò)結(jié)構(gòu)的自適應(yīng)在線學(xué)習(xí)算法。試驗(yàn)表明，該算法在壓縮率上可以優(yōu)于目前流行的壓縮算法Limpel-Zip(zip,gzip)，并且在運(yùn)行時(shí)間和所需空間性能上同PPM和Burrows Wheeler算法相比也是頗具競(jìng)爭力的。該算法實(shí)現(xiàn)為多輸入和單輸出的兩層神經(jīng)網(wǎng)絡(luò)，用已編碼比特的學(xué)習(xí)結(jié)果作為待編碼比特的工作參數(shù)，符合數(shù)據(jù)上下文相關(guān)約束的特點(diǎn)，提高了預(yù)測(cè)精度，并節(jié)約了編碼時(shí)間。
關(guān) 鍵 詞 算術(shù)編碼; 數(shù)據(jù)壓縮; 最大熵; 神經(jīng)網(wǎng)絡(luò)
Lossless Data Compression with Neural Network Based on Maximum Entropy Theory
FU Yan，ZHOU Jun-lin，WU Yue
Neural networks are used more frequently in lossy data coding domains such as audio, image, etc than in general lossless data coding, because standard neural networks must be trained off-line and they are too slow to be practical. In this paper, an adaptive arithmetic coding algorithm based on maximum entropy and neural networks are proposed for data compression. This adaptive algorithm with simply structure can do on-line learning and does not need to be trained off-line. The experiments show that this algorithm surpasses those traditional coding method, such as Limper-Ziv compressors (zip, gzip), in compressing rate and is competitive in speed and time with those traditional coding method such as PPM and Burrows-Wheeler algorithms. The compressor is a bit-level predictive arithmetic which using a 2 layer network with muti-input and one output. The arithmetic, according with the context constriction, improves the precision of prediction and reduces the coding time.
Key words arithmetic encoding; data compression; maximum entropy; neural network