File formats — a GIF header#

By the end of this chapter you will be able to identify a binary file by its magic number, parse its fixed-layout header, and decode bit-packed flag bytes. The example is the GIF image format, but the approach transfers directly to PNG, WAV, BMP, ELF, ZIP, and every other format built around a fixed header.

The GIF header and logical screen descriptor#

A GIF file starts with two fixed-size blocks, thirteen bytes in total. From the GIF89a specification:

Header (6 bytes)#

Offset

Bytes

Field

0

3

Signature — "GIF"

3

3

Version — "87a" or "89a"

Logical Screen Descriptor (7 bytes)#

Offset

Bytes

Field

Byte order

6

2

Logical screen width

little-endian

8

2

Logical screen height

little-endian

10

1

Packed flags

—

11

1

Background colour index

—

12

1

Pixel aspect ratio

—

The packed flags byte has five sub-fields:

Bit:  7  6 5 4  3  2 1 0
     [GCT][CR  ][Sort][GCT size]

  • bit 7 — global colour table flag

  • bits 6-4 — colour resolution (minus 1)

  • bit 3 — sort flag

  • bits 2-0 — global colour table size (actual size = 2^(value+1))

Every format specification looks roughly like this: fixed-width fields in a known order, with the occasional packed-bits byte.

Reading it in#

The template writes itself field by field:

sub parse_gif_header {
    my ($fh) = @_;
    binmode $fh;

    my $header;
    read $fh, $header, 13
        or die "short read: $!";

    my ($sig, $ver, $w, $h, $flags, $bg, $aspect) =
        unpack "A3 A3 v v C C C", $header;

    die "not a GIF" unless $sig eq "GIF";
    die "unknown version: $ver" unless $ver eq "87a" or $ver eq "89a";

    return {
        version => $ver,
        width   => $w,
        height  => $h,
        flags   => $flags,
        bg      => $bg,
        aspect  => $aspect,
    };
}

Directive by directive:

  • A3 A3 — six bytes of ASCII, split into signature and version. A (not a or Z) because the spec guarantees exactly three characters with no padding — A returns them unchanged and has the safer strip-on-unpack behaviour.

  • v v — two little-endian 16-bit unsigned integers. The GIF format is little-endian throughout; v is the exact match.

  • C C C — three unsigned bytes.

Total width: 3 + 3 + 2 + 2 + 1 + 1 + 1 = 13 bytes. We read exactly that many before unpacking; any less is an error.

The magic-number check#

Two lines above illustrate a pattern you will repeat for every file format:

die "not a GIF" unless $sig eq "GIF";
die "unknown version: $ver" unless $ver eq "87a" or $ver eq "89a";

Check the magic first, unpack the rest second. Reversing the order lets nonsense data through into later stages where the error messages are less clear. For formats with a longer magic (PNG is 8 bytes, ELF is 4, WAV’s “RIFF…WAVE” sandwich is 12), the same idea applies: unpack enough to identify the file, abort on mismatch, then unpack the rest.

Decoding the packed flags byte#

pack and unpack cannot decode sub-byte bit fields directly (with the exception of the string-shaped b / B directives, see the strings chapter). The pragmatic approach is shifts and masks in plain Perl on the byte unpack returned:

sub decode_gif_flags {
    my ($flags) = @_;
    return {
        gct_present => ($flags >> 7) & 0x01,
        color_res   => (($flags >> 4) & 0x07) + 1,
        sort_flag   => ($flags >> 3) & 0x01,
        gct_size    => 2 ** ((($flags) & 0x07) + 1),
    };
}

The pattern — “unpack the byte, pick the bits apart with shifts” — is so common it deserves to be named. Anything smaller than a byte belongs outside the template.

Worked run#

open my $fh, "<:raw", "sample.gif" or die $!;
my $hdr = parse_gif_header($fh);
my $gct = decode_gif_flags($hdr->{flags});

printf "GIF%s %dx%d\n", $hdr->{version}, $hdr->{width}, $hdr->{height};
printf "global colour table: %s (%d entries)\n",
       $gct->{gct_present} ? "yes" : "no",
       $gct->{gct_size};

On a real GIF89a file of dimensions 640×480, this prints:

GIF89a 640x480
global colour table: yes (256 entries)

Writing a header#

The inverse is shorter still — every directive we unpacked, we can pack. One detail: the flags byte has to be re-assembled from its sub-fields first.

sub encode_gif_flags {
    my (%f) = @_;
    return (($f{gct_present} & 0x01) << 7)
         | (((log2($f{color_res})) & 0x07) << 4)
         | (($f{sort_flag}   & 0x01) << 3)
         | (log2($f{gct_size}) - 1);
}

sub write_gif_header {
    my ($fh, $hdr, $gct) = @_;
    my $flags = encode_gif_flags(%$gct);
    my $bytes = pack "A3 A3 v v C C C",
        "GIF", $hdr->{version},
        $hdr->{width}, $hdr->{height},
        $flags, $hdr->{bg}, $hdr->{aspect};
    print {$fh} $bytes;
}

Same template on the pack side, same 13 bytes out, matches the GIF reader on the other end. Round-tripping with a test vector is the quickest way to gain confidence you have the template right: unpack an existing file, re-pack the result, compare byte-for-byte.

Scaling up to larger formats#

The approach is compositional. A format like WAV has multiple chunks, each with its own header (id + size + data). You write:

  • A low-level “read one chunk header” helper — two unpack calls on 8 bytes: 4-byte FourCC and 4-byte little-endian size.

  • A dispatcher that looks at the FourCC and calls a per-chunk parser.

  • Per-chunk parsers that unpack the chunk body according to that chunk’s own byte layout.

Every layer is a small unpack against a tiny template. The moment a field’s width depends on a value you just read, you stop, compose the new template from that value, and call unpack again — the same staged pattern we used for DNS names.

What you have learned#

  • Magic number first, body second. Fail fast on the wrong format.

  • Little-endian file formats use v / V, big-endian use n / N; < / > handle the signed or 64-bit cases.

  • Sub-byte fields are plain-Perl shifts, not template directives.

  • Larger formats decompose into many small templates. The work of the programmer is to keep each template small and obvious.

With the reference pages as your lookup and these seven chapters under your belt, the next binary format that crosses your desk should read as a solved problem. The pack and unpack pages hold the full directive table when you need to confirm a width or a modifier.