Peta::LMDB#

📦 std

Lightning memory-mapped databases: transactional key-value stores in a file, over the system’s liblmdb.

An environment (Peta::LMDB::Env) is a file, or a directory, holding one unnamed database and, with maxdbs, named ones. Everything happens in a transaction (Peta::LMDB::Txn): read transactions see a snapshot and never block; one write transaction at a time commits atomically, and nothing it wrote survives an abort or a crash before the commit. A database handle (Peta::LMDB::DB) names one database; a cursor (Peta::LMDB::Cursor) walks one in key order.

liblmdb is the system’s library, loaded when first needed; a call without it croaks naming the library it looked for.

Synopsis#

use Peta::LMDB qw(:flags);
my $env = Peta::LMDB::Env->new('data.lmdb', flags => MDB_NOSUBDIR,
                               mapsize => 1 << 30, maxdbs => 4);
my $txn = $env->begin;
my $db  = $txn->open_db('users', MDB_CREATE);
$txn->put($db, alice => 'admin');
$txn->commit;

my $r = $env->begin(MDB_RDONLY);
print $r->get($db, 'alice'), "\n";
my $c = $r->cursor($db);
while (my ($k, $v) = $c->next) { print "$k=$v\n" }

Keys and values#

Keys and values are byte strings; a string with characters over 255 croaks Wide character. In an MDB_INTEGERKEY database the keys are unsigned integers, and in an MDB_DUPSORT|MDB_INTEGERDUP one the values. A key may be at most $env->maxkeysize bytes, 511 with the usual liblmdb. A missing key is undef, not an error. $txn->get_view($db, $key, $var) makes $var the value without copying it: a read-only scalar that becomes undef when its transaction ends or, in a write transaction, at the next write.

Errors#

LMDB’s errors croak with a Peta::LMDB::Error object: $@->code is LMDB’s code, to compare with the MDB_* constants of the :errors tag, $@->message its text, which the object also stringifies to.

eval { $txn->put($db, $key, $value) };
if (ref $@ && $@->code == MDB_MAP_FULL) { ... }

Misuse croaks with a plain message naming the problem: using a transaction that has ended, a closed environment or cursor, a database of another environment, an option new does not know.

Ordering#

A database keeps its keys in byte order (integer order with MDB_INTEGERKEY) unless $db->set_compare gives it a sort sub; an MDB_DUPSORT database orders a key’s values the same way, or by $db->set_dupsort’s sub. The sub is called as a sort block is, with the two items in $a and $b of its package. The file does not remember it: set it every time the environment is opened.

Constants#

use Peta::LMDB qw(:flags) imports the flags of new, begin, open_db, put, del and copy (MDB_NOSUBDIR, MDB_RDONLY, MDB_CREATE, MDB_DUPSORT, MDB_NOOVERWRITE, MDB_CP_COMPACT and the rest); :errors the error codes (MDB_NOTFOUND, MDB_MAP_FULL …); :cursor_ops the operations of $cursor->get (MDB_FIRST, MDB_SET_RANGE …); :all all of them. They are also callable as Peta::LMDB::MDB_CREATE without importing.

The DBM modules’ write policy#

NDBM_File, ODBM_File, GDBM_File and DB_File (its Hash and Btree databases) keep their files with LMDB too, without transactions of their own. $Peta::LMDB::DBM_WRITE, read when a file is tied, says when their writes reach the file:

  • 'back' (the default): the writes of a tie collect in one LMDB transaction, committed at sync and when the hash is untied. Fast, as gdbm is; other processes see the writes after the commit, and a crash loses what was not committed. Like a gdbm writer, the tie holds the file’s one writer slot until then: another process that writes the file waits. Readers never wait.

  • 'through': every write is committed and synced at once - visible to other processes immediately, and a disk sync per write. Use it for files that several processes write.

GDBM_File’s own sync request, GDBM_SYNC or sync_mode(1), makes its tie write-through.

The DBM modules’ files#

A DBM module’s file is an LMDB environment of its own (one file, MDB_NOSUBDIR, with LMDB’s lock file NAME-lock beside it), which Peta::LMDB can open, with maxdbs => 1 or more, when the module has it closed. Its main database holds two records: the named database data, with the keys and values, and pperl-store, whose value says what they are - version=1 dups=0, or dups=1 when a key may hold several values (DB_File’s R_DUP). Such a key is stored as the key followed by an 8-byte big-endian sequence number, the order its values were stored in. A key that is empty or starts with a NUL byte is stored behind one more NUL byte. The modules refuse an LMDB file without the pperl-store record.

Modules#

  • Peta::LMDB::Cursor - A cursor: a position in a database that walks its keys in order, finds the first key at or after a given one, and writes or deletes where it stands.

  • Peta::LMDB::DB - A database handle: one database of an LMDB environment, as $txn->open_db returns it, to pass to the transaction’s and cursor’s methods.

  • Peta::LMDB::Env - An LMDB environment: the file (or directory) a set of databases lives in, and what every transaction starts from.

  • Peta::LMDB::Txn - A transaction: everything read from or written to an LMDB environment happens in one.

Functions#

Errors#

strerror#

LMDB’s message for an error code: Peta::LMDB::strerror(MDB_MAP_FULL) is MDB_MAP_FULL: Environment mapsize limit reached. A positive code is a system error, and gives the system’s message.

Library#

version#

The version of the system’s liblmdb: Peta::LMDB::version() gives its version string, such as LMDB 0.9.33: (May 21, 2024); in list context the string and its major, minor and patch numbers.