Fast start-up: the daemon#

Every pperl invocation normally pays the full start-up bill: process creation, interpreter initialisation, and - the expensive part for real programs - compiling your script and every module it uses. The daemon lets repeated invocations of the same script skip that bill: a resident process keeps a fully compiled image of the script, and each new run is a fork of that image that goes straight to execution.

The feature is strictly opt-in, per invocation. Nothing changes for plain pperl script.pl, and whenever the daemon cannot serve a request exactly, pperl silently falls back to a normal cold run. Correctness never depends on the daemon; only speed does.

Quick start#

pperl --daemon=start                 # start the daemon (background)

pperl --via-daemon report.pl         # first run: compiles, caches
pperl --via-daemon report.pl         # every further run: fork + run

pperl --daemon=status                # what is cached, what is live
pperl --daemon=stop                  # shut it down

PPERL_DAEMON=2 pperl report.pl is the same request without the flag, which is the more convenient form when the command line is not yours to change.

The first opted-in run of a script builds its template: the daemon compiles the script (including all used modules and BEGIN blocks) and keeps the result resident. Every later run of the same script forks the template and only executes.

When it pays#

The saving is your script’s compile time: parsing plus module loading plus BEGIN work. For a script that loads a handful of modules this is routinely the dominant part of start-up, and a template run can be several times faster end to end. For a trivial script that loads nothing, the saving roughly equals the daemon’s own overhead and the result is a wash - never a meaningful loss.

As everywhere in this guide: measure your own script, on your own machine, before drawing conclusions - see Measuring.

A worked example#

Here is a script of the kind that benefits: it does almost no work, but it loads seven modules before it can start.

#!/usr/bin/env pperl
use strict;
use warnings;
use Getopt::Long;
use List::Util qw(sum max);
use POSIX qw(strftime);
use File::Spec;
use Time::HiRes qw(time);
use Data::Dumper;

# The contract, honoured in one line: INIT runs per invocation, so a
# run from a template gets its own start time and its own random seed.
INIT { $^T = time; srand() }

my $top = 3;
GetOptions('top=i' => \$top) or die "usage: report.pl [--top N]\n";

my %sales = (north => [10, 42, 7], south => [3, 19, 88], west => [5, 5, 5]);
my @ranked = sort { sum(@{$sales{$b}}) <=> sum(@{$sales{$a}}) } keys %sales;

printf "%s  top %d of %d regions\n",
    strftime('%F %T', localtime), $top, scalar @ranked;
printf "  %-6s total %3d  peak %3d\n",
    $_, sum(@{$sales{$_}}), max(@{$sales{$_}})
    for @ranked[0 .. $top - 1];

Start a daemon, then time the same twenty runs with and without it:

pperl --daemon=start

time (for i in $(seq 20); do pperl report.pl >/dev/null; done)

pperl --via-daemon report.pl >/dev/null        # builds the template
time (for i in $(seq 20); do pperl --via-daemon report.pl >/dev/null; done)

On the machine this page was written on, that is 0.47 s against 0.08 s: about 24 ms per cold run, about 4 ms per run from the template. Nothing in the program changed; what disappeared is the compile of the script and of the seven modules, which the first run paid once.

Two things about those numbers are worth internalising. The saving is a fixed amount of start-up time, not a percentage of your program: a script that then works for a minute saves the same 20 ms as this one. And the saving scales with what you use, not with how long the script is - the modules are almost always the expensive part.

Checking whether a run was served#

Because a decline is silent, “it got faster” is the wrong way to tell whether the daemon is doing anything. Ask the daemon:

pperl --daemon=status

build 0.6.8-10307.2a84282.6a670d70.10d8b48
uptime 42s
served 21
live 0
templates 1
template /home/you/report.pl pid 38109 served 21 age 39s ready

served counts requests the daemon handled; a run that fell back cold never appears there. templates lists one line per compiled script, with the process holding the image, how often it has been forked, and whether it is ready or still parsing.

For a single run, ask the program itself. A served run is a child of the daemon (or of its template), never of your shell:

pperl -e 'print getppid, "\n"'                # your shell
pperl --via-daemon -e 'print getppid, "\n"'   # the daemon

If both print the same pid, the request was declined and pperl ran the program itself - which is always correct, just not faster.

What running from a template means#

A template run executes your script’s run phase only. Compilation already happened, once, when the template was built. pperl guarantees that the following are fresh and correct for every run:

  • @ARGV, %ENV, the current directory, umask

  • standard input, output and error (your redirections and pipes)

  • the process id $$

  • exit codes, and death by signal (a killed script kills the invoking pperl the same way)

  • the __DATA__ handle - each run reads it from the start, independently of other runs

What is not re-done is anything your code did at compile time. BEGIN blocks, and module code executed at use time, ran once when the template was built; their side effects are part of the image every run inherits. Consequences you own when you opt a script in:

  • $^T (the script start time) and the random-seed state date from template creation. If your script cares, re-stamp them at the start of the run phase:

    INIT { $^T = time; srand(); }

  • Values read from %ENV at compile time (inside BEGIN or at use time) were the values of the run that built the template. Read configuration at run time, or in an INIT block, if it must track the invoking environment.

  • File handles, sockets or database connections opened at compile time are shared with every run. Open connections at run time - the same rule long established for preloading application servers.

If a script cannot satisfy this contract, simply do not opt it in; PPERL_DAEMON=2 is per invocation, not global.

Editing the script, and the cache#

The daemon notices when the script file itself changes and rebuilds the template automatically on the next run - editing your script never serves you a stale image.

Edits to modules the script loads are not detected automatically. After changing a module, drop the affected template by hand:

pperl --daemon-reset=report.pl   # forget this script's template
pperl --daemon-reset             # forget all templates

Templates are a bounded cache: at most --daemon-max-scripts of them (default 100). When the limit is reached, the least recently used template is dropped to make room.

--daemon-idle (default 900 seconds) is a single value that two independent clocks use. A template exits when it has served nothing for that long, releasing its memory while the daemon keeps running and its other templates stay cached. The daemon’s own clock only starts once nothing of its own is left alive - no request in flight, no template - so it never shuts down out from under a cached script. With the defaults, a script you stop using loses its template about fifteen minutes later, and a daemon with nothing left to hold exits about fifteen minutes after that.

Neither exit costs you anything but time: with the template gone the next run compiles the script again, and with the daemon gone the run is an ordinary cold start. A daemon is never restarted for you, so if a machine runs opted-in scripts around the clock, either raise --daemon-idle or run the daemon as a service (below).

When pperl falls back to a cold run#

The daemon refuses - and the client falls back to a normal run, silently and correctly - whenever it could not reproduce the invocation exactly:

  • no daemon is running for this pperl binary

  • environment variables that shape interpreter start-up differ from the daemon’s (PERL_HASH_SEED, PERL_PERTURB_KEYS, PERL_RAND_SEED, PERL_INTERNAL_RAND_SEED, PERL_UNICODE, LANG, any LC_*, any MALLOC_*)

  • the command-line flags differ from the ones the template was built with (a template stores one configuration)

  • the template is still being built by a concurrent first run

  • the invocation is a -e one-liner (templates cache script files)

  • --interactive or --stats is in effect

Opting a script in without typing it every time#

PPERL_DAEMON=2 in front of every invocation is fine for a measurement and tedious for daily use. Two ways to make the opt-in part of the script or the command instead.

In the shebang, which travels with the script and makes the preload-safety promise visible to whoever reads it next:

#!/usr/bin/env -S pperl --via-daemon

Or as a shell function, when the script is not yours to edit:

report() { pperl --via-daemon "$HOME/bin/report.pl" "$@"; }

--via-daemon and PPERL_DAEMON=2 are the same request and share one template, so a script with the shebang above still hits the cache when you run it once by hand with the environment variable, and the other way round.

The flag names an intent, not a boundary: given a script file it uses that script’s template, and the script itself stays an ordinary argument. pperl --via-daemon=report.pl is not a thing - that value slot takes trunk, nothing else (see below), and pperl will tell you so.

Running the daemon as a service#

pperl --daemon runs in the foreground and logs to standard error, which is what a service manager wants. A systemd user unit:

# ~/.config/systemd/user/pperl-daemon.service
[Unit]
Description=pperl start-up daemon

[Service]
Type=simple
ExecStart=/usr/local/bin/pperl --daemon --daemon-idle=86400
Restart=on-failure

[Install]
WantedBy=default.target

systemctl --user enable --now pperl-daemon.service

A long --daemon-idle is the point of running it this way: the unit is meant to be there when the next invocation arrives, whenever that is. The value must be a positive number of seconds; there is no “never”.

One thing to remember after upgrading pperl: the daemon is keyed to the binary it was started from, so the new binary will not talk to the old daemon. Nothing breaks - runs are simply cold again until you systemctl --user restart pperl-daemon.service.

Security and identity#

The daemon serves exactly one user: its socket lives in a directory with owner-only permissions and every connection’s peer user id is verified. It also serves exactly one build of pperl: the socket is keyed to the binary’s identity (pperl --build-id prints it), so after a pperl upgrade the old daemon is simply never contacted again and exits on its idle timeout.

Limitations#

  • Job control is approximate: the running script is a child of the daemon, not of your shell. Signals you send to pperl (including Ctrl-C) are forwarded to the script, but Ctrl-Z suspends only the foreground pperl, not the script process.

  • One template per script path, holding one flag configuration. Invoking the same script with different flags falls back to a cold run rather than rebuilding the template.

  • A one-liner cannot be cached. A template is keyed on a script file, which is also what tells pperl the cached image has gone stale; -e and -E have neither, so they only ever get the reduced mode below. This is not a -E limitation - -E is -e with features enabled and takes exactly the same path.

  • --via-daemon=trunk (or PPERL_DAEMON=1) is that reduced mode: it forks a ready interpreter but still compiles the script on every run. It saves almost nothing, and it exists so the daemon’s own tests can exercise that boundary. There is no reason to ask for it in normal use.