# Progressive recipes

This chapter works from trivial substitutions up through reductions,
extraction, and structured output. Each section reuses the switches
introduced in the previous one, so a reader going straight through
builds up the idiom vocabulary without ever meeting a new flag mid-
recipe.

Readers coming from `awk`/`sed` should recognise the first sections;
the interesting material starts around
[Reductions across lines](#reductions) and
[Context-windowed filters](#context-windows).

Every recipe is copy-pasteable. When a sample input matters, it is
shown inline so the expected output is unambiguous.

<a id="baseline"></a>

## The baseline: `-e` and `-E`

The shortest one-liner. No I/O loop, no field splitting — just a
program that runs once.

```bash
pperl -e 'print 2 ** 32, "\n"'                    # 4294967296
pperl -E 'say 2 ** 32'                            # same, with -E/say
pperl -E 'say for 1 .. 5'                         # 1\n2\n3\n4\n5
```

The `,` in `print` separates arguments; the output field separator
`$,` (default empty) goes between them.

<a id="filtering"></a>

## Filtering: `-n` + `print if`

`-n` wraps the program in a read loop over every line of every input
file (or STDIN). Nothing is printed unless the program says so.

```bash
# Sample input
$ printf 'gate\napple\nwhat\nkite\n' > words.txt

pperl -ne 'print if /at/' words.txt               # gate, what
pperl -ne 'print unless /e/' words.txt            # what
pperl -ne 'print if /^[aeiou]/' words.txt         # apple
```

The regex match is against `$_` by default; `/at/` is short for
`$_ =~ m/at/`.

### Multiple conditions

```bash
# Lines containing both "ERROR" and "timeout"
pperl -ne 'print if /ERROR/ && /timeout/' app.log

# Lines containing neither
pperl -ne 'print if !/ERROR/ && !/timeout/' app.log

# Events A, B, C appearing in order on one line
pperl -ne 'print if /A.*B.*C/' events.log
```

### By line number

```bash
pperl -ne 'print if $. == 1'                      # head -n 1
pperl -ne 'print if $. <= 10'                     # head -n 10
pperl -ne 'print if 17 .. 30'                     # lines 17-30 (range op)
pperl -ne 'print if /START/ .. /END/'             # regex range, inclusive
pperl -ne 'print unless $. % 2'                   # even lines
```

The range operator `..` in scalar context is a flip-flop: compared
against `$.` when both endpoints are integers, against `$_` when
they are patterns. See [`perlop`](../../p5/core/perlop.md).

### By length

Use `-l` so the counted length excludes the trailing newline.

```bash
pperl -lne 'print if length >= 80' src.c          # long lines
pperl -lne 'print if length() < 5' words.txt      # short lines
```

Bare `length` with no argument applies to `$_`.

<a id="substitution"></a>

## Substitution: `-p` + `s///`

`-p` is `-n` plus an implicit [`print`](../../p5/core/perlfunc/print.md)
after each iteration. Use it when the output is the (possibly modified)
input.

```bash
# Replace every "foo" with "bar"
pperl -pe 's/foo/bar/g' input.txt

# First occurrence per line only
pperl -pe 's/foo/bar/' input.txt

# Conditional: replace only on lines that also match BAZ
pperl -pe 's/foo/bar/g if /BAZ/' input.txt
```

### Case transformations

```bash
pperl -nle 'print uc'                              # to uppercase
pperl -nle 'print lc'                              # to lowercase
pperl -nle 'print ucfirst lc'                      # Sentence case
pperl -ple 's/(\w+)/\u$1/g'                        # Title Case Per Word
```

[`uc`](../../p5/core/perlfunc/uc.md), [`lc`](../../p5/core/perlfunc/lc.md),
and [`ucfirst`](../../p5/core/perlfunc/ucfirst.md) with no argument
operate on `$_`.

### Whitespace normalisation

```bash
pperl -ple 's/^\s+//'                              # trim left
pperl -ple 's/\s+$//'                              # trim right
pperl -ple 's/^\s+|\s+$//g'                        # trim both
pperl -ple 's/\s+/ /g'                             # collapse runs
pperl -ne  'print if /\S/'                         # drop blank lines
```

### Line-ending conversion

```bash
pperl -pe 's/\r\n/\n/g' win.txt                    # DOS  → UNIX
pperl -pe 's/(?<!\r)\n/\r\n/g' unix.txt            # UNIX → DOS
```

The lookbehind `(?<!\r)` avoids producing `\r\r\n` when the input is
already DOS-encoded. See
[regex-recipes](regex-recipes.md#lookaround) for lookarounds in depth.

<a id="fields"></a>

## Field-oriented work: `-a` and `-F`

`-a` auto-splits each line into `@F`. `-F` changes the delimiter.

```bash
$ cat table.txt
brown bread mat hair 42
blue cake mug shirt -7
yellow banana window shoes 3.14

# Second field
pperl -lane 'print $F[1]' table.txt
# bread, cake, banana

# Last field
pperl -lane 'print $F[-1]' table.txt
# 42, -7, 3.14

# Count fields per line
pperl -lane 'print scalar @F' table.txt
# 5, 5, 5

# Lines whose last field is negative
pperl -lane 'print if $F[-1] < 0' table.txt
# blue cake mug shirt -7
```

### Custom delimiter

```bash
# /etc/passwd: print username (field 1) and shell (last field)
pperl -F: -lane 'print "$F[0] $F[-1]"' /etc/passwd

# Tab-separated
pperl -F'\t' -lane 'print $F[2]' metrics.tsv

# Comma or semicolon, one or more
pperl -F'[,;]+' -lane 'print scalar @F' mixed.csv
```

Proper CSV (quoted fields with embedded commas) is not a job for `-F`.
Use `-MText::CSV` — see [numeric](numeric.md#csv-basics) for the pattern.

### Rejoining fields

Interpolating `@F` inside double quotes uses `$"` (default: one space)
as the separator.

```bash
# Reverse field order, space-separated
pperl -lane 'print "@{[reverse @F]}"' table.txt

# Reverse field order, colon-separated
pperl -F: -lane 'local $" = ":"; print "@{[reverse @F]}"' /etc/passwd
```

`@{[ EXPR ]}` is the array-ref-then-deref idiom that lets arbitrary
expressions be interpolated into a double-quoted string.

<a id="reductions"></a>

## Reductions across lines

The move from per-line filters to cross-line computation.

### Counts

```bash
# wc -l
pperl -lne 'END { print $. }' big.txt

# grep -c pattern
pperl -lne '$n++ if /pattern/; END { print $n + 0 }' file

# Count blank lines
pperl -lne '$n++ if /^$/; END { print $n + 0 }' file

# Total fields across the file
pperl -lane '$t += @F; END { print $t }' table.txt
```

The `$n + 0` idiom prints `0` rather than empty when the counter was
never incremented.

### Sums, minima, maxima

Use `List::Util` via `-M`.

```bash
# Per line
pperl -MList::Util=sum -lane 'print sum @F' data.txt
pperl -MList::Util=min -lane 'print min @F' data.txt
pperl -MList::Util=max -lane 'print max @F' data.txt

# Across all lines
pperl -MList::Util=sum -lane '$s += sum @F; END { print $s }' data.txt
pperl -MList::Util=max -alne '$m = max($m // (), @F); END { print $m }' data.txt
```

`$m // ()` returns an empty list when `$m` is undefined, so the first
iteration does not pass `undef` to `max`. See
[numeric](numeric.md#sum-min-max) for the full arithmetic chapter.

### Longest and shortest line

```bash
pperl -ne '$l = $_ if length > length $l; END { print $l }' file
pperl -ne '$s = $_ if $. == 1 || length < length $s; END { print $s }' file
```

### Unique lines (first sighting only)

```bash
pperl -ne 'print unless $seen{$_}++' file

# Or the List::Util one-liner (slurps):
pperl -MList::Util=uniq -e 'print uniq <>' file
```

The first form streams — memory scales with distinct lines, not total
lines. The second slurps everything into memory, then emits. Prefer
the first for large inputs.

### Duplicates only

```bash
# Print each duplicate line once, when first seen for the second time
pperl -ne 'print if ++$seen{$_} == 2' file
```

<a id="context-windows"></a>

## Context-windowed filters

Printing the line *around* a match, not just the match itself.

```bash
# Line immediately before each match (grep -B 1, with caveats)
pperl -ne '/pattern/ && $prev && print $prev; $prev = $_' file

# Line immediately after each match (grep -A 1)
pperl -ne 'print if $p; $p = /pattern/' file

# tail -n 10
pperl -ne 'push @a, $_; shift @a if @a > 10; END { print @a }' file
```

For rich `-A` / `-B` / `-C` behaviour, `grep` is the right tool. One-
liners fit when the window rule is unusual (e.g. «print the line
matching X, plus every line until the next blank line»).

```bash
# Print from /SECTION/ to the next blank line
pperl -ne 'print if /SECTION/ .. /^$/' file
```

<a id="records"></a>

## Record mode: `-0`, `-00`, `-0777`

Switch away from line-based reading when the natural record is not a
line.

### Slurp with `-0777`

```bash
# Single-shot regex across the whole file
pperl -0777 -ne 'print "match\n" if /BEGIN.*?END/s' text.txt

# Replace only the first occurrence across the file
pperl -0777 -i.bak -pe 's/TODO/DONE/' notes.txt
```

`-0777` sets `$/ = undef`. The entire file comes in as one `$_`. The
`/s` modifier makes `.` match newlines. See
[regex-recipes](regex-recipes.md#multiline) for the full multi-line story.

### Paragraph mode with `-00`

`-00` sets `$/ = ""`. Each read returns everything up to the next
blank-line delimiter.

```bash
# Paragraphs containing "TODO"
pperl -00 -ne 'print if /TODO/' notes.md

# Reverse paragraph order
pperl -00 -e 'print reverse <>' notes.md
```

### NUL-delimited records with `-0`

```bash
# Count files produced by find -print0
find . -type f -print0 | pperl -0 -ne '$n++; END { print "$n\n" }'

# Convert NUL-delimited stream to newline-delimited
find . -type f -print0 | pperl -0 -pe 's/\0/\n/'
```

<a id="in-place"></a>

## In-place editing with `-i`

`-i` overwrites each input file with what `-p` prints. `-i.bak` keeps
a backup.

```bash
# Lowercase every occurrence of HOSTNAME in every .conf (keep backup)
pperl -i.bak -pe 's/HOSTNAME/\L$&/g' *.conf

# Delete every line matching DEPRECATED (no backup — only do this
# after you've validated the pattern on one file)
pperl -i -ne 'print unless /DEPRECATED/' file.txt
```

The backup files pile up. Clean them up with `rm *.bak` once verified.
See [gotchas](gotchas.md#in-place-edit) for the classic in-place traps.

## Structured output

### CSV-like (lightweight — commas are not quoted)

```bash
# Rebuild a TSV as CSV (naive, assumes no commas in fields)
pperl -F'\t' -lane 'print join ",", @F' data.tsv

# Add a column of line numbers
pperl -lane 'print join "\t", $., @F' data.tsv
```

### JSON (stdlib)

```bash
# Emit one JSON object per line (no commas in values assumed)
pperl -MJSON::PP -F: -lane '
    print JSON::PP->new->encode({
        user => $F[0], uid => 0+$F[2], shell => $F[-1]
    })
' /etc/passwd
```

Multi-line one-liners like this are where an
[alias](aliases.md#json-passwd) starts to pay for itself.

### Numbered input

```bash
pperl -pe '$_ = "$. $_"'                          # 1 gate, 2 apple, ...
pperl -pe 's/^/sprintf "%5d  ", $./e'             # padded width
```

The `/e` modifier on `s///` treats the replacement as Perl code. See
[regex-recipes](regex-recipes.md#e-modifier).

## Find out more

- [regex-recipes](regex-recipes.md) — the regex-dependent one-liners:
  captures, lookaround, named groups, `/e`, `/r`.
- [numeric](numeric.md) — arithmetic, statistics, randoms, date math.
- [aliases](aliases.md) — turn the recipes you use weekly into shell
  functions.
- [gotchas](gotchas.md) — quoting, encoding, `-i` backups, record-
  separator surprises.
- [`perlop`](../../p5/core/perlop.md) — the flip-flop operator, the
  diamond operator, `s///`, `tr///`, quoting forms.