```{index} single: Peta::LMDB ``` # Peta::LMDB ```{pperl-module-badges} Peta::LMDB ``` Lightning memory-mapped databases: transactional key-value stores in a file, over the system's liblmdb. An environment (`Peta::LMDB::Env`) is a file, or a directory, holding one unnamed database and, with `maxdbs`, named ones. Everything happens in a transaction (`Peta::LMDB::Txn`): read transactions see a snapshot and never block; one write transaction at a time commits atomically, and nothing it wrote survives an abort or a crash before the commit. A database handle (`Peta::LMDB::DB`) names one database; a cursor (`Peta::LMDB::Cursor`) walks one in key order. liblmdb is the system's library, loaded when first needed; a call without it croaks naming the library it looked for. ## Synopsis ```perl use Peta::LMDB qw(:flags); my $env = Peta::LMDB::Env->new('data.lmdb', flags => MDB_NOSUBDIR, mapsize => 1 << 30, maxdbs => 4); my $txn = $env->begin; my $db = $txn->open_db('users', MDB_CREATE); $txn->put($db, alice => 'admin'); $txn->commit; my $r = $env->begin(MDB_RDONLY); print $r->get($db, 'alice'), "\n"; my $c = $r->cursor($db); while (my ($k, $v) = $c->next) { print "$k=$v\n" } ``` ## Keys and values Keys and values are byte strings; a string with characters over 255 croaks `Wide character`. In an `MDB_INTEGERKEY` database the keys are unsigned integers, and in an `MDB_DUPSORT|MDB_INTEGERDUP` one the values. A key may be at most `$env->maxkeysize` bytes, 511 with the usual liblmdb. A missing key is undef, not an error. `$txn->get_view($db, $key, $var)` makes `$var` the value without copying it: a read-only scalar that becomes undef when its transaction ends or, in a write transaction, at the next write. ## Errors LMDB's errors croak with a `Peta::LMDB::Error` object: `$@->code` is LMDB's code, to compare with the `MDB_*` constants of the `:errors` tag, `$@->message` its text, which the object also stringifies to. ```perl eval { $txn->put($db, $key, $value) }; if (ref $@ && $@->code == MDB_MAP_FULL) { ... } ``` Misuse croaks with a plain message naming the problem: using a transaction that has ended, a closed environment or cursor, a database of another environment, an option `new` does not know. ## Ordering A database keeps its keys in byte order (integer order with `MDB_INTEGERKEY`) unless `$db->set_compare` gives it a sort sub; an `MDB_DUPSORT` database orders a key's values the same way, or by `$db->set_dupsort`'s sub. The sub is called as a `sort` block is, with the two items in `$a` and `$b` of its package. The file does not remember it: set it every time the environment is opened. ## Constants `use Peta::LMDB qw(:flags)` imports the flags of `new`, `begin`, `open_db`, `put`, `del` and `copy` (`MDB_NOSUBDIR`, `MDB_RDONLY`, `MDB_CREATE`, `MDB_DUPSORT`, `MDB_NOOVERWRITE`, `MDB_CP_COMPACT` and the rest); `:errors` the error codes (`MDB_NOTFOUND`, `MDB_MAP_FULL` ...); `:cursor_ops` the operations of `$cursor->get` (`MDB_FIRST`, `MDB_SET_RANGE` ...); `:all` all of them. They are also callable as `Peta::LMDB::MDB_CREATE` without importing. ## The DBM modules' write policy NDBM_File, ODBM_File, GDBM_File and DB_File (its Hash and Btree databases) keep their files with LMDB too, without transactions of their own. `$Peta::LMDB::DBM_WRITE`, read when a file is tied, says when their writes reach the file: - `'back'` (the default): the writes of a tie collect in one LMDB transaction, committed at `sync` and when the hash is untied. Fast, as gdbm is; other processes see the writes after the commit, and a crash loses what was not committed. Like a gdbm writer, the tie holds the file's one writer slot until then: another process that writes the file waits. Readers never wait. - `'through'`: every write is committed and synced at once - visible to other processes immediately, and a disk sync per write. Use it for files that several processes write. GDBM_File's own sync request, `GDBM_SYNC` or `sync_mode(1)`, makes its tie write-through. ## The DBM modules' files A DBM module's file is an LMDB environment of its own (one file, `MDB_NOSUBDIR`, with LMDB's lock file `NAME-lock` beside it), which Peta::LMDB can open, with `maxdbs => 1` or more, when the module has it closed. Its main database holds two records: the named database `data`, with the keys and values, and `pperl-store`, whose value says what they are - `version=1 dups=0`, or `dups=1` when a key may hold several values (DB_File's `R_DUP`). Such a key is stored as the key followed by an 8-byte big-endian sequence number, the order its values were stored in. A key that is empty or starts with a NUL byte is stored behind one more NUL byte. The modules refuse an LMDB file without the `pperl-store` record. ## Modules - [`Peta::LMDB::Cursor`](LMDB/Cursor) - A cursor: a position in a database that walks its keys in order, finds the first key at or after a given one, and writes or deletes where it stands. - [`Peta::LMDB::DB`](LMDB/DB) - A database handle: one database of an LMDB environment, as `$txn->open_db` returns it, to pass to the transaction's and cursor's methods. - [`Peta::LMDB::Env`](LMDB/Env) - An LMDB environment: the file (or directory) a set of databases lives in, and what every transaction starts from. - [`Peta::LMDB::Txn`](LMDB/Txn) - A transaction: everything read from or written to an LMDB environment happens in one. ## Functions ### Errors #### `strerror` LMDB's message for an error code: `Peta::LMDB::strerror(MDB_MAP_FULL)` is `MDB_MAP_FULL: Environment mapsize limit reached`. A positive code is a system error, and gives the system's message. ### Library #### `version` The version of the system's liblmdb: `Peta::LMDB::version()` gives its version string, such as `LMDB 0.9.33: (May 21, 2024)`; in list context the string and its major, minor and patch numbers. ```{toctree} :hidden: :maxdepth: 1 LMDB/Cursor LMDB/DB LMDB/Env LMDB/Txn ```