Rivoreo Source Code Repositories
src.rivoreo.one / lz4 / 550b40849f6b77850453a1190b97430b121802bc
commit550b40849f6b77850453a1190b97430b121802bc[log] [download]
authorYann Collet <cyan@fb.com>Sun Feb 25 00:32:09 2018 -0800
committerYann Collet <cyan@fb.com>Sun Feb 25 00:32:09 2018 -0800
tree5205cc18b67ab826da2017ca90df2957e0f53d16
parent7173a631db61ab9535bd0d6e5e00e9dc081d4df3 [diff]
merge lz4opt.h into lz4hc.c

Having a dedicated file for optimal parser
made sense during its creation,
it allowed Przemyslaw to work more freely on lz4opt, with less dependency on lz4hc,
moreover, the optimal parser was more complex, with its own search functions.

Since the optimal was rewritten last year, it's now a lot lighter.
It makes more sense now to integrate it directly inside lz4hc.c,
making it easier to edit (editors are a bit "lost" inside a `*.h` dependent on its #include position),
it also reduces the number of files in the project,
which fits pretty well with lz4 objectives.
(adding lz4hc requires "just" lz4hc.h and lz4hc.c).
4 files changed
tree: 5205cc18b67ab826da2017ca90df2957e0f53d16
  1. .gitattributes
  2. .gitignore
  3. .travis.yml
  4. INSTALL
  5. LICENSE
  6. Makefile
  7. NEWS
  8. README.md
  9. appveyor.yml
  10. circle.yml
  11. contrib/
  12. doc/
  13. examples/
  14. lib/
  15. programs/
  16. tests/
  17. visual/
README.md

LZ4 - Extremely fast compression

LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.

Speed can be tuned dynamically, selecting an “acceleration” factor which trades compression ratio for more speed up. On the other end, a high compression derivative, LZ4_HC, is also provided, trading CPU time for improved compression ratio. All versions feature the same decompression speed.

LZ4 library is provided as open-source software using BSD 2-Clause license.

BranchStatus
masterBuild Status Build status coverity
devBuild Status Build status

Branch Policy:

  • The “master” branch is considered stable, at all times.
  • The “dev” branch is the one where all contributions must be merged before being promoted to master.
    • If you plan to propose a patch, please commit into the “dev” branch, or its own feature branch. Direct commit to “master” are not permitted.

Benchmarks

The benchmark uses lzbench, from @inikep compiled with GCC v6.2.0 on Linux 64-bits. The reference system uses a Core i7-3930K CPU @ 4.5GHz. Benchmark evaluates the compression of reference Silesia Corpus in single-thread mode.

CompressorRatioCompressionDecompression
memcpy1.0007300 MB/s7300 MB/s
LZ4 fast 8 (v1.7.3)1.799911 MB/s3360 MB/s
LZ4 default (v1.7.3)2.101625 MB/s3220 MB/s
LZO 2.092.108620 MB/s845 MB/s
QuickLZ 1.5.02.238510 MB/s600 MB/s
Snappy 1.1.32.091450 MB/s1550 MB/s
LZF v3.62.073365 MB/s820 MB/s
Zstandard 1.1.1 -12.876330 MB/s930 MB/s
Zstandard 1.1.1 -33.164200 MB/s810 MB/s
zlib deflate 1.2.8 -12.730100 MB/s370 MB/s
LZ4 HC -9 (v1.7.3)2.72034 MB/s3240 MB/s
zlib deflate 1.2.8 -63.09933 MB/s390 MB/s

LZ4 is also compatible and well optimized for x32 mode, for which it provides an additional +10% speed performance.

Installation

make
make install     # this command may require root access

LZ4's Makefile supports standard Makefile conventions, including staged installs, redirection, or command redefinition. It is compatible with parallel builds (-j#).

Documentation

The raw LZ4 block compression format is detailed within lz4_Block_format.

To compress an arbitrarily long file or data stream, multiple blocks are required. Organizing these blocks and providing a common header format to handle their content is the purpose of the Frame format, defined into lz4_Frame_format. Interoperable versions of LZ4 must respect this frame format.

Other source versions

Beyond the C reference source, many contributors have created versions of lz4 in multiple languages (Java, C#, Python, Perl, Ruby, etc.). A list of known source ports is maintained on the LZ4 Homepage.