Subprocesses and co-processes#

The most common IPC task is the simplest: run an external command and talk to it. This page covers the spectrum, from a one-way pipe open to a two-way co-process - a child you both write to and read from.

The mechanics of a one-direction pipe open ("-|" and "|-") are covered in full on the Pipes page. This page picks up where that one stops: when one direction is not enough, and when you need a raw pipe and fork of your own.

One direction: the pipe open#

To read a command’s output, or to feed a command its input, a pipe open is all you need:

open my $fh, "-|", "sort", "-u", "names.txt" or die "pipe: $!";
while (my $line = <$fh>) { ... }
close $fh;

That is the whole story for one-way work, and it is told on the Pipes page - including the list-versus-string form, why list form sidesteps the shell, and how to decode the child’s exit status. Start there if one direction is enough.

Both directions: IPC::Open2#

When you need to write to a command and read its reply - a filter you drive interactively, a co-process - one pipe is not enough. IPC::Open2 opens two: one into the child’s stdin, one out of its stdout.

use IPC::Open2;

my $pid = open2(my $from_child, my $to_child, "tr", "a-z", "A-Z");

print $to_child "hello\n";
close $to_child;                     # signal end-of-input to the child

my $reply = <$from_child>;
print "got: $reply";                 # got: HELLO

waitpid($pid, 0);

open2 returns the child’s process id. The argument order is the trap worth memorising: reader handle first, writer handle second, then the command. open2(OUT, IN, ...) - the handle you read from comes before the handle you write to.

The deadlock you must design around#

A co-process can deadlock, and IPC::Open2 does nothing to prevent it. The shape of the trap:

  • You write a large block to the child.

  • The child reads some, produces output, and that output fills the pipe back to you.

  • The child blocks writing to a full pipe; you block writing to a child that has stopped reading. Neither moves.

The recipe above sidesteps it by closing $to_child before reading - sending the whole input, then signalling end-of-file, then draining the reply. That works when the input fits comfortably in the pipe buffer. For unbounded streaming in both directions at once, you need non-blocking I/O or a select loop; the two-pipe shape alone is not enough.

Capturing standard error too: IPC::Open3#

IPC::Open2 joins you to stdin and stdout. When you also need the child’s stderr on its own handle, IPC::Open3 adds a third:

use IPC::Open3;
use Symbol qw(gensym);

my $err = gensym;                    # a fresh anonymous handle for stderr
my $pid = open3(my $to_child, my $from_child, $err, "sort");

print $to_child "banana\napple\ncherry\n";
close $to_child;

my @sorted = <$from_child>;          # ("apple\n", "banana\n", "cherry\n")
print @sorted;                       # apple / banana / cherry, one per line

waitpid($pid, 0);

Two differences from open2 to note:

  • The argument order flips. open3 takes the writer handle first, then the reader, then the error handle: open3(IN, OUT, ERR, ...). This is the opposite of open2’s reader-first order - a long-standing inconsistency that bites everyone once.

  • The error handle needs gensym. Pass a fresh anonymous handle from Symbol::gensym for stderr; a bareword or an undefined lexical will not collect it cleanly.

Rolling your own: pipe and fork#

When the command is not an external program but Perl code you want to run in a separate process, build the channel yourself. A raw pipe gives a connected reader/writer pair before the fork; after fork each side keeps the end it needs and closes the other:

pipe(my $reader, my $writer) or die "pipe: $!";

my $kid = fork() // die "fork: $!";

if ($kid == 0) {
    # Child writes, then exits.
    close $reader;                   # child has no use for the read end
    $writer->autoflush(1);
    print $writer "down-the-pipe\n";
    close $writer;
    exit 0;
}

# Parent reads.
close $writer;                       # parent has no use for the write end
my $line = <$reader>;
print "parent read: $line";          # parent read: down-the-pipe
close $reader;

waitpid($kid, 0);

The discipline is the same as the forking server on the TCP sockets page: each side closes the end it does not use. If the parent keeps the write end open, the child’s close never produces an end-of-file on the read end, and the parent’s read blocks forever.

Explicit fork() creates a separate operating-system process; PetaPerl’s auto-parallelism is an in-process thread pool. A forked child runs its own auto-parallel numeric reductions normally within itself, and the parent’s and child’s parallelism do not interfere - they are independent processes with independent thread pools. For running work across CPU cores inside one process rather than across separate processes, see the Concurrent Execution guide.

Next#

For two unrelated programs that meet on a filesystem path rather than through a shared parent, see FIFOs.