GDBM_File#

📦 std

Tie a Perl hash to a database file with GNU dbm’s interface: its open modes, error codes, options, dumps and recovery calls.

GDBM_File binds a hash to a file with tie; from then on $h{key}, delete, exists, keys and each read and write the file. The object tie returns has gdbm’s own methods besides: count the keys, sync, reorganize, dump and load the data in gdbm’s dump formats, read the last error, and read or set options.

Synopsis#

use GDBM_File;

my $db = tie my %h, 'GDBM_File', 'data.gdbm', GDBM_WRCREAT, 0640
    or die "$GDBM_File::gdbm_errno";
$h{greeting} = 'hello';
print $db->count, "\n";          # 1
$db->dump('data.dump');
undef $db;
untie %h;

Opening a database#

my $db = tie %h, 'GDBM_File', $filename, $flags, $mode;

$flags is one access mode, OR’d with modifiers:

  • GDBM_READER - open an existing file to read.

  • GDBM_WRITER - open an existing file to read and write.

  • GDBM_WRCREAT - the same, creating the file if it does not exist.

  • GDBM_NEWDB - create the file, emptying it if it exists.

  • GDBM_SYNC - commit every write to disk as it happens (see sync).

  • GDBM_FAST, GDBM_NOLOCK, GDBM_NOMMAP, GDBM_CLOEXEC, GDBM_BSEXACT, GDBM_XVERIFY, GDBM_PREREAD, GDBM_NUMSYNC - accepted and without effect.

$mode is the permission of a file it creates. tie returns the database object, or undef when the file cannot be opened: $GDBM_File::gdbm_errno then says why. Keep the object to call methods on it, and undef it (or call close) before untie: the file stays open as long as the object lives.

Errors#

$GDBM_File::gdbm_errno holds the code of the last failure: compare it with the GDBM_*_ERROR constants as a number, or print it for gdbm’s message (with $! appended when the system’s error says more). A failed method croaks with that message; errno, syserrno and strerror give the last error of one database, and clear_error clears it.

Options#

These methods read an option and, given a value, set it:

  • $db->sync_mode, $db->sync_mode($bool) - whether every write is committed to disk as it happens. Off unless the database was tied with GDBM_SYNC.

  • $db->cache_size, $db->cache_size($n) - the cache size. It reads only once set: before, it croaks GDBM_File::cache_size not implemented.

  • $db->centfree, $db->centfree($bool) and $db->coalesce, $db->coalesce($bool) - gdbm’s free-space settings, both off until set. As with gdbm 1.26, centfree reads back inverted: 1 while off, 0 once set on.

  • $db->mmapsize, $db->mmapsize($n) - the largest memory map.

  • $db->mmap - true: the file is always memory-mapped.

  • $db->block_size - the file’s page size, typically 4096.

  • $db->dbname - the file name the database was tied with.

  • $db->flags - the $flags it was tied with, GDBM_SYNC included while the sync mode is on.

Only sync_mode changes how the database works. The cache size, free space settings and memory map size keep the value they are given and change nothing: they tune gdbm’s own file layout, which pperl’s files do not have. setopt reaches the same options by their GDBM_SET* codes.

DBM filters#

$db->filter_store_key(sub {...}), filter_store_value, filter_fetch_key and filter_fetch_value install a sub that rewrites every key or value on its way into, or out of, the database. The sub changes $_ in place; a store filter works on a copy, so your own variable keeps its value. Each method returns the filter it replaces (or undef), and undef as the argument removes the filter. The DBM_Filter module offers ready-made ones (utf8, null, int32 and others). A filter that uses the same database croaks recursion detected in filter_store_key (or the filter’s own name).

Files#

pperl keeps a GDBM_File database in an LMDB file: the file named, with LMDB’s lock file, NAME-lock, beside it. The gdbm library is not used, and its files cannot be read: tying a real gdbm file (or a Berkeley DB file) croaks, naming what the file is, and the file is left as it was, even with GDBM_NEWDB. To bring a gdbm database over, dump it with gdbm’s own tools (gdbm_dump, or dump under perl 5) and load the dump. A key may be at most 511 bytes, 510 if it starts with a NUL byte; a longer one croaks.

Writes reach the file at sync and when the database is untied: until then the tie holds the file’s one writer slot, and another process that writes the file waits for it (readers never wait), as with gdbm’s own writer lock. GDBM_SYNC, sync_mode(1) or $Peta::LMDB::DBM_WRITE = 'through' commit every write as it happens; see Peta::LMDB.

Constants#

GDBM_File exports gdbm’s constants: the open modes and modifiers above, GDBM_INSERT and GDBM_REPLACE (the flag of STORE), the GDBM_*_ERROR and other error codes $GDBM_File::gdbm_errno holds, the GDBM_SET* option codes of setopt, and the GDBM_SNAPSHOT_* codes of latest_snapshot.

Differences from upstream#

  • The files are LMDB files, not gdbm files: real gdbm files are refused, and pperl’s files cannot be read by gdbm. Data moves between them through dump and load, in either direction. Covered by t/82-native/Peta-LMDB/090-gdbm.t and t/81-xs-native/GDBM_File/010-phenotype.t.

  • Keys are limited to 511 bytes; gdbm has no limit. Covered by t/82-native/Peta-LMDB/090-gdbm.t.

  • Writes reach the file at sync and untie unless the sync mode is on, as with gdbm, and a writer holds the file until then. Covered by t/82-native/Peta-LMDB/090-gdbm.t.

  • load reads dumps with empty keys or values, which gdbm itself writes but cannot load. Covered by t/82-native/Peta-LMDB/090-gdbm.t.

  • Deleting the key each just returned is safe: the walk goes on with the next key, where gdbm loses its place and ends the walk early. Covered by t/82-native/Peta-LMDB/090-gdbm.t.

  • Crash tolerance (failure_atomic, latest_snapshot) is reported unsupported, as by a gdbm built without it: pperl’s files survive a crash by themselves. Covered by t/81-xs-native/GDBM_File/150-upstream-snapshot.t.

  • The cache size, free-space and memory-map options have no effect. cache_size reads back a size once set, where perl 5 with gdbm 1.26 croaks not implemented on every read. Covered by t/82-native/Peta-LMDB/090-gdbm.t.

Everything else - error codes and messages, options, dump formats - behaves as with gdbm. Covered by t/81-xs-native/GDBM_File/010-phenotype.t and GDBM_File’s own test suite, t/81-xs-native/GDBM_File/100-upstream-count.t to 150-upstream-snapshot.t.

Functions#

Errors#

gdbm_check_syserr#

True when an error code is one that $! explains further - a failed open, read or write - as in gdbm_check_syserr($GDBM_File::gdbm_errno).

errno#

The last error of this database: as a number one of the GDBM_* error codes (GDBM_NO_ERROR, 0, when none), as a string gdbm’s message for it, such as Item not found.

syserrno#

The system error number (the $! value) behind this database’s last error, when that error is one the system’s error explains; 0 otherwise.

strerror#

gdbm’s message for this database’s last error, with the system’s message appended when that says more.

clear_error#

Forget this database’s last error: errno reads GDBM_NO_ERROR again.

Maintenance#

close#

Close the database now, writing what is pending, even while copies of the database object exist. Needed when the object was kept from tie: untie alone leaves the file open as long as the object lives. Returns 0. Methods called afterwards croak database was closed.

needs_recovery#

Whether the database needs recover: always 0, as an interrupted write leaves pperl’s file as it was before the write.

reorganize#

Shrink the file after many deletes: the data is copied, without the free space, into a new file that takes the old one’s place and mode.

recover#

Recover a damaged database. pperl’s files are never left damaged, so recovery finds every key intact; it still makes the backup copy and fills in the statistics asked for.

sync#

Write everything stored so far to the file and to disk, so that other processes - and the file after a crash - have it. Returns 0; a failure croaks. Without it, writes reach the file at untie, or as they happen in sync mode (GDBM_SYNC, sync_mode(1)).

count#

The number of keys in the database, without walking them.

dump#

Write the database’s keys and values to a dump file in gdbm’s format, which load - here or in perl 5 with gdbm - reads back.

load#

Read a dump file - ASCII or binary, from dump here or from gdbm’s own tools - into the database.

Options#

block_size#

The $flags the database was tied with, GDBM_SYNC included while the sync mode is on. The same method, under the option’s name, reads and sets the options listed under Options above (sync_mode, cache_size, block_size and the others); with a value too many it croaks NAME: too many arguments.

Also available as GDBM_File::cache_size, GDBM_File::centfree, GDBM_File::coalesce, GDBM_File::dbname, GDBM_File::flags, GDBM_File::mmap, GDBM_File::mmapsize, GDBM_File::sync_mode.

setopt#

$db->setopt($code, $value, $length): gdbm’s raw option call. $code is a GDBM_SET* (or GDBM_GET*) constant, $value an integer and $length its size in bytes, 4. Returns 0; an unknown option or a bad value croaks. The named methods under Options are easier to use.

Crash tolerance#

convert#

$db->convert($format): change gdbm’s file format, GDBM_NUMSYNC for its extended format or 0 for its standard one. pperl’s files have one format, so it does nothing and returns 0.

failure_atomic#

$db->failure_atomic($even, $odd): gdbm’s crash tolerance, which keeps two snapshot files of the database. Not supported: it croaks gdbm_failure_atomic: Function usage error: Function not implemented. pperl’s files survive a crash without snapshots, keeping the writes made up to the last sync.

latest_snapshot#

GDBM_File->latest_snapshot($even, $odd): which of crash tolerance’s two snapshot files to recover from. Not supported: it returns undef, in list context (undef, GDBM_SNAPSHOT_ERR, ENOSYS).

crash_tolerance_status#

GDBM_File->crash_tolerance_status: whether failure_atomic is supported - 0.

Library version#

GDBM_version#

The version of gdbm whose interface the module follows: in scalar context a string such as 1.26, in list context its numbers, such as (1, 26, 0). Call it on the class: GDBM_File->GDBM_version.