File::Map#

📦 std

Memory-mapped scalars: make a file look like an ordinary string.

File::Map attaches a file - or a fresh anonymous region of memory - to a Perl scalar. Reads and writes to the scalar go straight to the kernel’s mapping: the file appears in your variable as a string, and changes propagate back without you ever calling read or write. A multi-gigabyte file costs almost nothing to open this way, because pages are faulted in lazily as you touch them.

Once a scalar is mapped it behaves like a regular string for the operations that matter: substr, regex match and substitution, index, length, and slicing. Direct assignment ($map = "...") is caught and fixed up so you never corrupt the mapping, though substr is the recommended way to write. The mapping is released automatically when the scalar goes out of scope; unmap is available for explicit control.

Synopsis#

use File::Map qw(map_file map_anonymous unmap);

map_file my $map, 'data.bin', '+<';   # read/write
$map =~ s/foo/bar/g;                  # edits go to disk
substr $map, 0, 4, "HEAD";            # in-place write

map_anonymous my $buf, 4096, 'shared'; # scratch region
unmap $map;                            # optional

Choosing a mapping function#

  • map_file - give a filename and a mode, get a mapped scalar. The common case.

  • map_handle - same idea, but you already have an open filehandle.

  • map_anonymous - no file; a scratch region shared with forked children or private to the process.

  • sys_map - low-level wrapper around raw mmap(2) flags for when the convenience functions don’t fit.

Managing a live mapping#

  • sync - flush pending writes back to disk.

  • advise - hint the kernel about access patterns ("sequential", "random", "willneed", and so on).

  • protect - change read/write protection at runtime.

  • remap - resize an existing mapping (Linux only).

  • unmap - drop the mapping explicitly.

  • pin / unpin - keep the mapping in physical memory.

This is a native implementation of the XS half of File::Map 0.71; its File/Map.pm is used unchanged. Nothing is exported by default. Name the functions, or take them by tag: :map (map_handle, map_file, map_anonymous, sys_map, unmap), :extra (remap, sync, pin, unpin, advise, protect), :lock (lock_map), :constants (PROT_*, MAP_*), or :all.

Differences from upstream#

None. It is File::Map as a perl without threads builds it, so the thread functions wait_until, notify and broadcast are absent and lock_map locks nothing.

Functions#

Mapping#

map_file#

Open a file by name and attach it to a scalar as a memory map.

map_handle#

Attach an already-open filehandle to a scalar as a memory map.

map_anonymous#

Allocate a fresh memory region backed by no file and attach it to a scalar.

sys_map#

Low-level mapping with raw mmap(2) protection and flag constants.

remap#

Resize an existing mapping in place.

Unmapping#

unmap#

Release a mapping explicitly before the scalar goes out of scope.

Synchronisation#

sync#

Flush pending writes from a mapped scalar back to disk.

Locking#

pin#

pin $map: lock the mapping in physical memory, so it is never paged out (mlock(2)). The system limits how much a process may pin (ulimit -l); beyond it pin dies with the error.

unpin#

unpin $map: undo pin; the mapping may be paged out again.

lock_map#

lock_map $map: take the mapping’s lock until the end of the enclosing scope - the lock that serialises threads using the same mapping. Perl here has no threads, so it only checks that $map is a mapped scalar; the thread-only condition functions wait_until, notify and broadcast do not exist, as in any unthreaded perl.

Advice#

advise#

Tell the kernel how the mapped region will be accessed.

protect#

Change the read/write/execute protection of a mapped region.