| commit | dc868cd5b1bc2c68405732fe080b7b0e5dff0cca | [log] [download] |
|---|---|---|
| author | Julius Werner <jwerner@chromium.org> | Fri Feb 12 15:35:41 2016 -0800 |
| committer | Julius Werner <jwerner@chromium.org> | Fri Feb 12 22:52:51 2016 -0800 |
| tree | 4f81fe61be4cb253455dd4190b07daa03840bd7c | |
| parent | 4fcb2e17fb065f1d061a9c72066dd4dc1eb94a3f [diff] |
Allow for safe in-place decoding Add a check to allow safe "in-place" decoding (meaning that the beginning of the source buffer partially overlaps the end of the destination buffer). This is usually possible as long as the output stops at least 15 bytes before the end of the input buffer (7 to account for the extra spill from LZ4_wildCopy, 4 for a possible block checksum, and 4 for the terminating block header), but in some pathological edge cases it could be possible for the output stream to overwrite a byte in the input stream before it gets decoded. With this patch the decoder will reliably detect those cases and return a decoding error. Signed-off-by: Julius Werner <jwerner@chromium.org>
- .gitattributes
- .gitignore
- .travis.yml
- Makefile
- NEWS
- README.md
- cmake_unofficial/
- contrib/djgpp/
- debian/
- examples/
- lib/
- lz4_Block_format.md
- lz4_Frame_format.md
- programs/
- versionsTest/
- visual/2012/
LZ4 - Extremely fast compression
LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.
Speed can be tuned dynamically, selecting an “acceleration” factor which trades compression ratio for more speed up. On the other end, a high compression derivative, LZ4_HC, is also provided, trading CPU time for improved compression ratio. All versions feature the same decompression speed.
LZ4 library is provided as open-source software using BSD license.s
| Branch | Status |
|---|---|
| master |   |
| dev |  |
Branch Policy:
- The “master” branch is considered stable, at all times.
- The “dev” branch is the one where all contributions must be merged before being promoted to master.
- If you plan to propose a patch, please commit into the “dev” branch, or its own feature branch. Direct commit to “master” are not permitted.
Benchmarks
The benchmark uses the Open-Source Benchmark program by m^2 (v0.14.3) compiled with GCC v4.8.2 on Linux Mint 64-bits v17. The reference system uses a Core i5-4300U @1.9GHz. Benchmark evaluates the compression of reference Silesia Corpus in single-thread mode.
| Compressor | Ratio | Compression | Decompression |
|---|---|---|---|
| memcpy | 1.000 | 4200 MB/s | 4200 MB/s |
| LZ4 fast 17 (r129) | 1.607 | 690 MB/s | 2220 MB/s |
| LZ4 default (r129) | 2.101 | 385 MB/s | 1850 MB/s |
| LZO 2.06 | 2.108 | 350 MB/s | 510 MB/s |
| QuickLZ 1.5.1.b6 | 2.238 | 320 MB/s | 380 MB/s |
| Snappy 1.1.0 | 2.091 | 250 MB/s | 960 MB/s |
| LZF v3.6 | 2.073 | 175 MB/s | 500 MB/s |
| zlib 1.2.8 -1 | 2.730 | 59 MB/s | 250 MB/s |
| LZ4 HC (r129) | 2.720 | 22 MB/s | 1830 MB/s |
| zlib 1.2.8 -6 | 3.099 | 18 MB/s | 270 MB/s |
Documentation
The raw LZ4 block compression format is detailed within lz4_Block_format.
To compress an arbitrarily long file or data stream, multiple blocks are required. Organizing these blocks and providing a common header format to handle their content is the purpose of the Frame format, defined into lz4_Frame_format. Interoperable versions of LZ4 must respect this frame format.
Other source versions
Beyond the C reference source, many contributors have created versions of lz4 in multiple languages (Java, C#, Python, Perl, Ruby, etc.). A list of known source ports is maintained on the LZ4 Homepage.