苹果开源了LZFSE无损压缩

3 years ago

source link: https://changkun.de/blog/posts/apple-released-lzfse/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

苹果开源了LZFSE无损压缩

Published at:

2016-07-11

Reading: 560 words ~2min

PV/UV: 17/16

开源地址：https://github.com/lzfse/lzfse

苹果开源了新的无损压缩算法 LZFSE ，该算法是去年在iOS 9和OS X 10.10中引入的。按照苹果公司的说法，LZFE的压缩增益和ZLib level 5相同，但速度要快2~3倍，能源效率也更高。

所以，马上找来测试与 zip 的情况 PK 一下，感觉无论从压缩速度还是压缩率来说都有点差强人意，说好的速度更快能源效率利用率更高呢？

文件1: 2014data.csv

文件大小: 83.5 MB

zip 压缩大小: 7.3 MB

lzfse 压缩大小: 9.2 MB

$ time zip test.zip 2014data.csv -v
updating: 2014data.csv .......	(in=83523458) (out=7286164) (deflated 91%)
total bytes=83523458, compressed=7286164 -> 91% savings
zip test.zip 2014data.csv -v  1.73s user 0.06s system 99% cpu 1.808 total

$ time ./lzfse -encode -i 2014data.csv -o test.lzfse -v
LZFSE encode
Input: 2014data.csv
Output: test.lzfse
Input size: 83523458 B
Output size: 9239873 B
Compression ratio: 9.039
Speed: 15.05 ns/B, 63.36 MB/s
./lzfse -encode -i 2014data.csv -o test.lzfse -v  1.22s user 0.15s system 98% cpu 1.393 total

文件1是一个 csv 格式的数据集，本质上是文本，压缩率不能看，完全比不了 zip，压缩速度还行，不过说好的两倍到三倍的提升呢?

文件2: movie.mkv

文件大小: 1.37 GB

zip 压缩大小: 1.37GB

lzfse 压缩大小: 1.38GB

$ time zip movie.zip movie.mkv -v                  
  adding: movie.mkv ...................................................................................................................................	(in=1371915092) (out=1368868136) (deflated 0%)
total bytes=1371915092, compressed=1368868136 -> 0% savings
zip movie.zip movie.mkv -v  39.73s user 1.52s system 99% cpu 41.332 total

$ time ./lzfse -encode -i movie.mkv -o movie.lzfse -v
LZFSE encode
Input: movie.mkv
Output: movie.lzfse
Input size: 1371915092 B
Output buffer was too small, increasing size...
Output size: 1380865044 B
Compression ratio: 0.994
Speed: 35.61 ns/B, 26.78 MB/s
./lzfse -encode -i movie.mkv -o movie.lzfse -v  47.82s user 2.96s system 95% cpu 53.343 total

文件2是一个视频文件，考虑到 MKV 的压缩率已经很高了，所以暂且不比较这个，但是这个速度有点感人? zip 只需40s 不到，但 lzfse 接近 50s 的压缩时间可是反复测试了多次。

Author: Changkun Ou
Link: https://changkun.de/blog/posts/apple-released-lzfse/
License: All articles in this blog are licensed under CC BY-NC-ND 4.0 unless stating additionally.

Recommend