GitHub - flanglet/kanzi-go: Lossless data compression in Go
source link: https://github.com/flanglet/kanzi-go
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
README.md
kanzi
Lossless data compression in Go. The goal is to provide clean APIs and really fast implementation. It includes compression codecs (Run Length coding, Exp Golomb coding, Huffman, Range, LZ4, Snappy, ANS, Context Mixers, PAQ derivatives), bit stream manipulation, and transforms such as Burrows-Wheeler (BWT) and Move-To-Front, etc ...
For more details, check https://github.com/flanglet/kanzi/wiki.
Credits
Matt Mahoney, Yann Collet, Jan Ondrus, Yuta Mori, Ilya Muravyov, Neal Burns, Fabian Giesen, Jarek Duda
Disclaimer
Use at your own risk. Always keep a backup of your files.
Silesia corpus benchmark
i7-7700K @4.20GHz, 32GB RAM, Ubuntu 18.04
go 1.10.1
Kanzi version 1.4 Go implementation. Block size is 100 MB.
Compressor Encoding (sec) Decoding (sec) Size Original
211,938,580 Kanzi -l 1 2.7 1.5 80,790,093 Kanzi -l 1 -j 12 1.3 0.5 80,790,093 Gzip 1.6 6.0 1.0 68,227,965 Gzip 1.6 -9 14.3 1.0 67,631,990 Zstd 1.3.3 -13 11.9 0.3 58,789,182 Lzma 5.2.2 -3 24.3 2.4 55,743,540 Kanzi -l 2 15.5 11.5 55,728,473 Kanzi -l 2 -j 12 6.5 4.1 55,728,473 Bzip2 1.0.6 -9 14.1 4.8 54,506,769 Zstd 1.3.3 -19 45.2 0.4 53,977,895 Kanzi -l 3 19.0 13.3 51,781,285 Kanzi -l 3 -j 12 6.8 4.8 51,781,285 Kanzi -l 4 23.0 19.3 49,460,294 Kanzi -l 4 -j 12 8.2 6.4 49,460,294 Lzma 5.2.2 -9 65.0 2.4 48,780,457 Kanzi -l 5 41.2 37.7 46,485,064 Kanzi -l 5 -j 12 13.9 12.8 46,485,064 Tangelo 2.4 83.2 85.9 44,862,127 zpaq v7.14 m4 t1 107.3 112.2 42,628,166 zpaq v7.14 m4 t12 108.1 111.5 42,628,166 Tangelo 2.0 302.0 310.9 41,267,068 Kanzi -l 6 100.9 104.3 41,144,431 Kanzi -l 6 -j 12 39.7 40.0 41,144,431 zpaq v7.14 m5 t1 343.1 352.0 39,112,924 zpaq v7.14 m5 t12 344.3 350.4 39,112,924
enwik8
i7-7700K @4.20GHz, 32GB RAM, Ubuntu 18.04
go 1.10.1
Kanzi version 1.4 Go implementation. Block size is 100 MB. 1 thread
Compressor Encoding (sec) Decoding (sec) Size Original
100,000,000 Kanzi -l 1 2.1 1.6 37,389,414 Kanzi -l 2 8.0 5.6 25,439,827 Kanzi -l 3 11.6 7.5 22,744,813 Kanzi -l 4 13.9 10.6 22,096,910 Kanzi -l 5 22.0 18.7 20,791,382 Kanzi -l 6 42.4 42.8 19,464,686
Build
Option 1: go get
cd $GOPATH
go get github.com/flanglet/kanzi-go
cd src/github.com/flanglet/kanzi-go/app
go build -gcflags=-B Kanzi.go BlockCompressor.go BlockDecompressor.go InfoPrinter.go
Option 2: git clone
cd $GOPATH/src
mkdir github.com; cd github.com
mkdir flanglet; cd flanglet
git clone https://github.com/flanglet/kanzi-go.git
cd kanzi-go/app
go build -gcflags=-B Kanzi.go BlockCompressor.go BlockDecompressor.go InfoPrinter.go
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK