Compress::Raw::Zlib#

📦 std

Low-level interface to the zlib compression library.

Compress::Raw::Zlib is the layer underneath Compress::Zlib and the whole IO::Compress::* / IO::Uncompress::* family. It gives direct access to zlib’s streaming compressor and decompressor, the crc32 and adler32 checksums, and the zlib constants. Most programs are better served by IO::Compress::Gzip and friends; this module is for code that needs the streams themselves.

use Compress::Raw::Zlib;

my $data = "hello, world\n" x 100;

# Compress: one deflate call per chunk, flush at the end.
my ($d, $status) = Compress::Raw::Zlib::Deflate->new(
    -Level => Z_BEST_COMPRESSION, -AppendOutput => 1, -CRC32 => 1);
$status == Z_OK or die "deflateInit: $status";
my $packed = '';
$d->deflate($data, $packed) == Z_OK or die $d->msg;
$d->flush($packed) == Z_OK or die $d->msg;
printf "%d -> %d bytes, crc32 %08x\n", $d->total_in, $d->total_out, $d->crc32;
# 1300 -> 32 bytes, crc32 6d00e0f5

# Uncompress: inflate eats its input and says Z_STREAM_END at the end.
my $i = Compress::Raw::Zlib::Inflate->new(-AppendOutput => 1);
my $out = '';
$status = $i->inflate($packed, $out);
print "status: $status, round trip ", ($out eq $data ? "ok" : "BAD"), "\n";
# status: stream end, round trip ok

# The checksums, one-off or running, and merged.
my $crc = crc32("hello, ");
$crc = crc32("world", $crc);
printf "%08x %08x\n", $crc,
    Compress::Raw::Zlib::crc32_combine(crc32("hello, "), crc32("world"), 5);
# ffab723a ffab723a

This is a native implementation of Compress::Raw::Zlib 2.222; its Compress/Raw/Zlib.pm (the option parser, the constructors, deflateParams) is used unchanged. use Compress::Raw::Zlib exports crc32, adler32 and every constant: the Z_* status, flush, level and strategy values, MAX_WBITS, MAX_MEM_LEVEL, WANT_GZIP, WANT_GZIP_OR_ZLIB, OS_CODE, ZLIB_VERSION, ZLIB_VERNUM and the ZLIBNG_* version parts.

Status values#

The constructors and the stream methods that do work (deflate, flush, deflateParams, inflate, inflateSync, the resets) return a dual value (deflateTune returns a plain number): it compares numerically with the Z_* constants (Z_OK, Z_STREAM_END, Z_BUF_ERROR, Z_DATA_ERROR, …) and prints as zlib’s text for it (“stream end”, “data error”). $stream->status repeats the last one as a plain number, and $stream->msg returns zlib’s message for the last error.

Compressing: Compress::Raw::Zlib::Deflate#

my ($d, $status) = Compress::Raw::Zlib::Deflate->new(OPTIONS) returns the stream and Z_OK, or undef and the zlib error code. In scalar context it returns the stream only. The options, all optional, are given as -Name => value pairs (or one hash reference):

  • -Level: 0 to 9, or Z_NO_COMPRESSION, Z_BEST_SPEED, Z_BEST_COMPRESSION, Z_DEFAULT_COMPRESSION (the default).

  • -Method: Z_DEFLATED, the only method and the default.

  • -WindowBits: 8 to 15 for a zlib (RFC 1950) stream, -MAX_WBITS for raw deflate (RFC 1951), WANT_GZIP for gzip (RFC 1952). The default is MAX_WBITS.

  • -MemLevel: zlib’s memLevel, default MAX_MEM_LEVEL.

  • -Strategy: Z_DEFAULT_STRATEGY (the default), Z_FILTERED, Z_RLE, Z_FIXED or Z_HUFFMAN_ONLY.

  • -Dictionary: a preset dictionary; $d->dict_adler is its Adler-32.

  • -Bufsize: the initial output buffer size and its growth step, default 4096.

  • -AppendOutput: when true, output is appended to the output argument; when false (the default) the argument is overwritten.

  • -CRC32, -ADLER32: keep a running checksum of the uncompressed data, read with $d->crc32 / $d->adler32.

The methods:

  • $d->deflate($input, $output) compresses all of $input (a scalar or a scalar reference) into $output and returns Z_OK. zlib may hold data back, so an empty $output after a successful call is not an error.

  • $d->flush($output [, $flush_type]) writes out what is pending. The default Z_FINISH ends the stream; Z_SYNC_FLUSH, Z_FULL_FLUSH, Z_PARTIAL_FLUSH and Z_NO_FLUSH keep it open.

  • $d->deflateReset starts a new stream on the same object, after a finishing flush.

  • $d->deflateParams(-Level => ..., -Strategy => ..., -BufSize => ...) changes those settings between calls; options not given stay.

  • $d->deflateTune($good_length, $max_lazy, $nice_length, $max_chain) is zlib’s deflateTune.

  • $d->get_Level, $d->get_Strategy, $d->get_Bufsize return the settings in force.

  • $d->total_in and $d->total_out count the bytes taken and given by zlib for the current stream; $d->uncompressedBytes and $d->compressedBytes are the same counts kept by the binding.

  • $d->crc32, $d->adler32, $d->dict_adler return the checksums described above (0 when the option is off); $d->status the status of the last call; $d->msg zlib’s last error message.

Uncompressing: Compress::Raw::Zlib::Inflate#

my ($i, $status) = Compress::Raw::Zlib::Inflate->new(OPTIONS) takes -WindowBits (as for deflate, plus WANT_GZIP_OR_ZLIB to accept either header), -Bufsize, -Dictionary, -AppendOutput, -CRC32, -ADLER32, and:

  • -ConsumeInput (default true): inflate removes the compressed data it used from its input argument. What is left after Z_STREAM_END is whatever followed the stream - the trailer of a container format, for example.

  • -LimitOutput: inflate stops when the output buffer is full instead of uncompressing all of its input, and returns Z_BUF_ERROR or Z_OK with input left over; call it again until the input is used up. It turns -ConsumeInput on.

The methods:

  • $i->inflate($input, $output [, $eof]) uncompresses $input into $output and returns Z_OK, or Z_STREAM_END at the end of the compressed data. $eof matters only to zlib older than 1.2.0.

  • $i->inflateSync($input) skips forward to the next full flush point (made by flush(Z_FULL_FLUSH)), to resume after corrupt data.

  • $i->inflateReset starts a new stream on the same object.

  • $i->set_Append($bool) switches -AppendOutput and returns the old setting.

  • $i->total_in, $i->total_out, $i->compressedBytes, $i->uncompressedBytes, $i->inflateCount, $i->get_Bufsize, $i->crc32, $i->adler32, $i->dict_adler, $i->status and $i->msg report as their deflate counterparts do.

Compress::Raw::Zlib::InflateScan->new makes an inflateScanStream, which IO::Compress::Zip uses to append to an existing deflate stream: scan, getEndOffset, getLastBlockOffset, getLastBufferOffset, resetLastBlockByte, inflateReset, inflateCount, createDeflateStream, and the same counters as above. Upstream does not document it, and neither does this page. Every stream class also has DispStream, which prints the stream’s internal state for debugging.

The zlib it uses#

zlib is part of the binary, as upstream builds its own copy into the module: no library is loaded, and the module works wherever pperl runs. The implementation is zlib-rs, which has the zlib 1.3 API. ZLIB_VERSION reads 1.3.0-zlib-rs- and the zlib-rs version; ZLIB_VERNUM is 0x1300.

Differences from upstream#

  • Compressed bytes. A stream deflated here is a valid stream that any zlib inflates, and any zlib’s stream inflates here, but it is not byte-for-byte the stream perl5 writes for the same input (level 0 excepted), and its size differs slightly. Upstream promises no particular bytes either: they change with the zlib it is built on.

  • Version. ZLIB_VERSION and ZLIB_VERNUM are as above, where perl 5.44 with its bundled zlib reports 1.3.2 and 0x1320.

  • There is no zlib-ng mode: the four is_zlib* functions answer as for plain zlib.

Functions#

Checksums#

adler32#

adler32($buffer [, $adler]): the Adler-32 checksum of $buffer. Pass the previous result as $adler to continue a running checksum; without it, or with undef, the checksum starts afresh. $buffer may be a scalar reference. Exported by default.

crc32#

crc32($buffer [, $crc [, $offset]]): the CRC-32 checksum of $buffer, as gzip and zip use it. Pass the previous result as $crc to continue a running checksum; without it, or with undef, it starts afresh. $offset skips that many leading bytes; an offset past the end dies “Offset out of range in Compress::Raw::Zlib::crc32”. $buffer may be a scalar reference. Exported by default.

crc32_combine#

Compress::Raw::Zlib::crc32_combine($crc1, $crc2, $len2): the CRC-32 of two pieces joined, from the CRC-32 of each and the length of the second - without the data. Not exported.

adler32_combine#

Compress::Raw::Zlib::adler32_combine($adler1, $adler2, $len2): as crc32_combine, for Adler-32. Not exported.

Library info#

zlib_version#

Compress::Raw::Zlib::zlib_version(): the version string of the zlib in use: 1.3.0-zlib-rs- and the zlib-rs version.

zlibng_version#

Compress::Raw::Zlib::zlibng_version(): the zlib-ng version string when the library is zlib-ng, and an empty string for zlib.

is_zlib_native#

Compress::Raw::Zlib::is_zlib_native(): true when the library is zlib itself rather than zlib-ng. With is_zlibng, is_zlibng_compat and is_zlibng_native it tells which library the streams run on.

is_zlibng_native#

Compress::Raw::Zlib::is_zlibng_native(): true when the binding uses zlib-ng’s own API. Always false here: the library is reached through the zlib API.

is_zlibng_compat#

Compress::Raw::Zlib::is_zlibng_compat(): true when the library is a zlib-ng built with the zlib-compatible API. Always false here.

is_zlibng#

Compress::Raw::Zlib::is_zlibng(): true when the library is zlib-ng, in either mode.

ZLIB_VERNUM#

ZLIB_VERNUM: zlib’s version as a number, one hex digit per part: 0x1320 for 1.3.2. Exported by default.

zlibCompileFlags#

Compress::Raw::Zlib::zlibCompileFlags(): zlib’s zlibCompileFlags, the bit set describing how the library was built (type sizes, debug, assembler code); zlib.h documents the bits. 0 for zlib before 1.2.1.

ZLIBNG_VER_STATUS#

ZLIBNG_VER_STATUS: the release status part of the zlib-ng version as a string; "0" when the library is zlib. Exported by default.