In-memory handles#

The third thing open can attach a handle to is a plain scalar variable. The scalar is used as the “file”: reads pull bytes out of it, writes push bytes into it. Nothing touches the filesystem, nothing forks.

This is the cleanest way to make a function that normally writes to a filehandle also produce an in-memory string. Any code that accepts “a filehandle” continues to work.

The signature#

open my $fh, MODE, \$scalar  or die;

The third argument is a reference to a scalar. That reference is what distinguishes an in-memory handle from a pathname — a plain string "$scalar" would be a filename.

Modes work as they do for files:

  • "<" — read from the contents of the scalar.

  • ">" — write into the scalar, clobbering any existing content.

  • ">>" — append to the scalar.

Writing into a buffer#

my $buf = "";
open my $fh, ">", \$buf   or die "open buffer: $!";

print $fh "hello, ";
print $fh "world\n";
printf $fh "%d items\n", 42;

close $fh                 or die "close buffer: $!";

print $buf;               # hello, world\n42 items\n

After close, $buf holds everything that was printed. Before close, the data may still be sitting in PerlIO’s buffer — treat it as not-yet-final, just as you would with a real file.

The typical use is capturing output from a function that expects to print to a handle:

sub render {
    my ($fh, @rows) = @_;
    for my $row (@rows) {
        print $fh join("\t", @$row), "\n";
    }
}

my $buf = "";
open my $fh, ">", \$buf or die;
render($fh, @rows);
close $fh               or die;

No temporary file, no tmpfile, no cleanup.

Reading from a buffer#

my $source = "one\ntwo\nthree\n";
open my $fh, "<", \$source  or die "open buffer: $!";

while (my $line = <$fh>) {
    chomp $line;
    # ... work with $line ...
}

close $fh                   or die;

This turns any string-processing task that “would be nicer with a filehandle” into one. Test data in particular reads cleanly:

my $fixture = <<'END';
name,count
apples,3
pears,7
END

open my $fh, "<", \$fixture or die;
parse_csv($fh);
close $fh                   or die;

parse_csv sees a filehandle, does not know it is a string in disguise, and the test is entirely self-contained.

Layers work the same way#

In-memory handles accept PerlIO layers:

my $buf = "";
open my $fh, ">:encoding(UTF-8)", \$buf   or die;
print $fh "café\n";
close $fh                                 or die;

# $buf now contains the UTF-8 bytes: c a f 0xC3 0xA9 0x0A

Without the layer, $buf would contain the character string "café\n". With the layer, the write step encodes to bytes on the way into the buffer. Which one you want depends on whether the buffer is destined for something that expects characters (another Perl function) or bytes (a socket, a hash function, a file on disk).

When not to use this#

  • When you want capture with fork and exec. In-memory handles are pure Perl; a child process cannot see them. If you want to capture the output of an external command, use a read-pipe as described in Pipes.

  • For bulk data that would not fit in RAM. The scalar grows as you write. A one-gigabyte log is better served by a real tempfile.

  • For seekable random-access editing. seek works on in-memory handles, but the semantics around growing the buffer past its current end are subtle. For random access, use a real file.

Next#

Every example on every page in this chapter has been shouting or die "...: $!". It is time to earn that idiom. See Handling errors.