---
name: subprocesses and co-processes tutorial
---
# Subprocesses and co-processes

The most common IPC task is the simplest: run an external command and
talk to it. This page covers the spectrum, from a one-way pipe open to
a two-way co-process - a child you both write to and read from.

The mechanics of a one-direction pipe open (`"-|"` and `"|-"`) are
covered in full on the [Pipes](../io-streams/pipes) page. This page
picks up where that one stops: when one direction is not enough, and
when you need a raw [`pipe`](../../p5/core/perlfunc/pipe) and
[`fork`](../../p5/core/perlfunc/fork) of your own.

## One direction: the pipe open

To read a command's output, or to feed a command its input, a pipe
open is all you need:

```perl
open my $fh, "-|", "sort", "-u", "names.txt" or die "pipe: $!";
while (my $line = <$fh>) { ... }
close $fh;
```

That is the whole story for one-way work, and it is told on the
[Pipes](../io-streams/pipes) page - including the list-versus-string
form, why list form sidesteps the shell, and how to decode the
child's exit status. Start there if one direction is enough.

## Both directions: `IPC::Open2`

When you need to write to a command *and* read its reply - a filter
you drive interactively, a co-process - one pipe is not enough.
`IPC::Open2` opens two: one into the child's `stdin`, one out of its
`stdout`.

```perl
use IPC::Open2;

my $pid = open2(my $from_child, my $to_child, "tr", "a-z", "A-Z");

print $to_child "hello\n";
close $to_child;                     # signal end-of-input to the child

my $reply = <$from_child>;
print "got: $reply";                 # got: HELLO

waitpid($pid, 0);
```

`open2` returns the child's process id. The argument order is the trap
worth memorising: **reader handle first, writer handle second**, then
the command. `open2(OUT, IN, ...)` - the handle you read *from* comes
before the handle you write *to*.

### The deadlock you must design around

A co-process can deadlock, and `IPC::Open2` does nothing to prevent
it. The shape of the trap:

- You write a large block to the child.
- The child reads some, produces output, and that output fills the
  pipe back to you.
- The child blocks writing to a full pipe; you block writing to a
  child that has stopped reading. Neither moves.

The recipe above sidesteps it by closing `$to_child` before reading -
sending the whole input, then signalling end-of-file, then draining
the reply. That works when the input fits comfortably in the pipe
buffer. For unbounded streaming in both directions at once, you need
non-blocking I/O or a `select` loop; the two-pipe shape alone is not
enough.

## Capturing standard error too: `IPC::Open3`

`IPC::Open2` joins you to `stdin` and `stdout`. When you also need the
child's `stderr` on its own handle, `IPC::Open3` adds a third:

```perl
use IPC::Open3;
use Symbol qw(gensym);

my $err = gensym;                    # a fresh anonymous handle for stderr
my $pid = open3(my $to_child, my $from_child, $err, "sort");

print $to_child "banana\napple\ncherry\n";
close $to_child;

my @sorted = <$from_child>;          # ("apple\n", "banana\n", "cherry\n")
print @sorted;                       # apple / banana / cherry, one per line

waitpid($pid, 0);
```

Two differences from `open2` to note:

- **The argument order flips.** `open3` takes the writer handle
  first, then the reader, then the error handle:
  `open3(IN, OUT, ERR, ...)`. This is the opposite of `open2`'s
  reader-first order - a long-standing inconsistency that bites
  everyone once.
- **The error handle needs `gensym`.** Pass a fresh anonymous handle
  from `Symbol::gensym` for `stderr`; a bareword or an undefined
  lexical will not collect it cleanly.

## Rolling your own: `pipe` and `fork`

When the command is not an external program but Perl code you want to
run in a separate process, build the channel yourself. A raw
[`pipe`](../../p5/core/perlfunc/pipe) gives a connected reader/writer
pair before the fork; after [`fork`](../../p5/core/perlfunc/fork) each
side keeps the end it needs and closes the other:

```perl
pipe(my $reader, my $writer) or die "pipe: $!";

my $kid = fork() // die "fork: $!";

if ($kid == 0) {
    # Child writes, then exits.
    close $reader;                   # child has no use for the read end
    $writer->autoflush(1);
    print $writer "down-the-pipe\n";
    close $writer;
    exit 0;
}

# Parent reads.
close $writer;                       # parent has no use for the write end
my $line = <$reader>;
print "parent read: $line";          # parent read: down-the-pipe
close $reader;

waitpid($kid, 0);
```

The discipline is the same as the forking server on the
[TCP sockets](sockets-tcp) page: **each side closes the end it does
not use.** If the parent keeps the write end open, the child's `close`
never produces an end-of-file on the read end, and the parent's read
blocks forever.

Explicit `fork()` creates a separate operating-system process;
PetaPerl's auto-parallelism is an in-process thread pool. A forked
child runs its own auto-parallel numeric reductions normally within
itself, and the parent's and child's parallelism do not interfere -
they are independent processes with independent thread pools. For
running work across CPU cores *inside* one process rather than across
separate processes, see the
[Concurrent Execution](../../guide/concurrent-execution/index) guide.

## Reference cross-links

- [`open`](../../p5/core/perlfunc/open) - the one-way pipe open (`"-|"`, `"|-"`)
- [`pipe`](../../p5/core/perlfunc/pipe) - a raw connected handle pair
- [`fork`](../../p5/core/perlfunc/fork) - split into two processes
- [`waitpid`](../../p5/core/perlfunc/waitpid) - reap the child and recover its status
- [`exec`](../../p5/core/perlfunc/exec) - replace the child image with a command

## Next

For two unrelated programs that meet on a filesystem path rather than
through a shared parent, see [FIFOs](fifos).