# Awk to Perl You know `awk`. You think in `pattern { action }`, you split every line into `$1`, `$2`, `$3` without asking, you count things with `a[key]++`, and you frame the whole job with `BEGIN { }` and `END { }`. Perl was written partly as "awk, but bigger" - almost everything you do in awk has a direct Perl form, and then Perl keeps going: real data structures, full regex, modules, and a language that does not run out of room when the script grows. The one idea to carry in from the first line: **Perl is a near-superset of awk.** The record loop, the fields, the associative arrays, the `BEGIN`/`END` keywords, `print` and `printf`, `length`/`substr`/`split` - they all map across, often by the *same name*. What changes is that fields become a real array `@F` you can `push` and `pop`, the associative array becomes a Perl hash you can nest, and the regex engine is PCRE rather than POSIX ERE. This guide leads with the one-to-one mappings and points out the few places the analogy leaks. Every runnable example here was executed on `pperl`, a Rust reimplementation of Perl 5.42; the `# ...` comments show its real output. Start scripts with: ```perl use v5.42; ``` That one line turns on `strict`, `warnings`, and the `say` built-in (a `print` with a trailing newline), the modern feature set used throughout. :::{note} One-liner switches on this pperl build awk's natural Perl twin is the one-liner: `perl -ne`, `perl -ane`, and `perl -F, -ane`. Those switches are **correct perl5 5.42** and are shown throughout this guide as the idiomatic form. But the implicit-loop switch family (`-n`, `-p`, `-a`, `-F`, `-i`, `-l`) is **not yet recognised by this `pperl` build** - only `-e` is. So the one-liner examples below are shown *without* output, and each is paired with an explicit-loop full-script form (`while (<$fh>) { ... }`) that **does** run on `pperl` today and carries the verified output. Use the full-script form until the switches land. ::: ## How to read this - [The line/record loop](#the-linerecord-loop) - awk's implicit `{ ... }` becomes `while (<$fh>)` and `$_`. - [Fields and splitting](#fields-and-splitting) - `$1`/`$NF`/`NF`/`FS` become `@F`, `$F[0]`, `split`, `-F`. - [Regex and substitution](#regex-and-substitution) - `gsub`/`sub`/ `match` become `s///`/`=~`, a strict superset. - [Addresses and ranges](#addresses-and-ranges) - `/re/`, `cond { }`, and `/a/,/b/` become postfix `if` and the flip-flop. - [I/O, pipes, in-place edit](#io-pipes-in-place-edit) - `getline`, redirection, and `-i`. - [One-liners and switches](#one-liners-and-switches) - `-n`, `-a`, `-F`, `BEGIN`/`END`. - [Control flow](#control-flow) - `next`, `for`, the postfix forms. - [Variables and data](#variables-and-data) - associative arrays become hashes, the happy mapping. - [Gotchas coming from awk](#gotchas-coming-from-awk) - scan before you write. ## The line/record loop awk runs your program once per input record, with the record in `$0` and the action in `{ ... }`. Perl's equivalent is an explicit loop with the record in `$_`, the implicit topic variable. The `-n` switch reproduces awk's "run this block per line"; awk's `NR` (record number) is Perl's `$.`. ```awk awk '{ print }' input.txt ``` ```perl perl -ne 'print' input.txt ``` The runnable, full-script form of that same loop - the one that runs on `pperl` today - opens the file and iterates it explicitly. `$_` is the current record, exactly like awk's `$0`: ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { # each line lands in $_, like awk's $0 chomp; say "line $. : $_"; # $. is awk's NR } # line 1 : alice 85 90 # line 2 : bob 70 60 # line 3 : carol 95 100 # line 4 : alice 50 55 ``` (The input file held `alice 85 90`, `bob 70 60`, `carol 95 100`, `alice 50 55`.) The gotcha: a line read with `<$fh>` keeps its trailing newline, unlike awk's `$0`, which has the record separator stripped. Call `chomp` to strip it (chomp is the closest thing to "awk already removed the newline for you"). The record number is `$.` = awk's `NR`; when you read several files with `<>`, `$.` keeps counting across them, so the awk `FNR` (per-file counter) reset is something you arrange yourself by checking `eof`. ## Fields and splitting This is awk's heart, and Perl gives you the same picture with one twist: the fields live in a real array `@F`, so awk's `$1` is Perl's `$F[0]` (zero-based), `$0` is `$_`, and `NF` is the count `scalar @F`. Under the `-a` switch Perl autosplits each line into `@F` exactly as awk autosplits into `$1..$NF`; the runnable equivalent is one explicit `split`. ```awk awk '{ print $1, NF }' input.txt # first field, field count awk -F, '{ print $1 }' input.txt # comma separator (FS) ``` ```perl perl -ane 'print "$F[0] @{[scalar @F]}\n"' input.txt # -a fills @F perl -F, -ane 'print "$F[0]\n"' input.txt # -F sets the split pattern ``` The full-script form does the split itself. Bare `split` with no arguments splits `$_` on whitespace and discards leading blanks - exactly awk's default `FS`: ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { my @F = split; # awk's automatic field split say "$F[0] has ", scalar(@F) - 1, " scores"; # $F[0]=$1, @F=NF } # alice has 2 scores # bob has 2 scores # carol has 2 scores # alice has 2 scores ``` Arithmetic across fields is the everyday awk action `print $1, $2+$3`, and it reads almost identically: ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { my @F = split; say "$F[0]: ", $F[1] + $F[2]; # awk: print $1, $2+$3 } # alice: 175 # bob: 130 # carol: 195 # alice: 105 ``` awk's `$NF` (the last field) is Perl's negative index `$F[-1]`, and a non-default `FS` is the pattern argument to `split`: ```perl use v5.42; open my $fh, '<', '/tmp/data.csv' or die "open: $!"; # alice,85,90 ... while (<$fh>) { chomp; my @F = split /,/; # awk -F, say $F[-1]; # awk's $NF: last field } # 90 # 60 # 100 ``` To join fields back with a separator (awk's `OFS`), set `$,` (the output field separator that `print`/`say` inserts between its list arguments): ```perl use v5.42; open my $fh, '<', '/tmp/data.csv' or die "open: $!"; while (<$fh>) { chomp; my @F = split /,/; local $, = "|"; # OFS say @F; # join the list with OFS } # alice|85|90 # bob|70|60 # carol|95|100 ``` The gotcha: fields are **zero-based**. awk's `$1` is `$F[0]`, `$2` is `$F[1]`, and `$0` (the whole record) is `$_`, not an element of `@F`. `NF` is `scalar @F` (or `$#F + 1`); the last field is `$F[-1]`, not `$F[NF]`. And `split` with **one space as a string** (`split ' '`) is the special whitespace-and-strip mode; `split / /` (a regex with one space) is literal and does *not* strip leading blanks. Plain `split` with no argument is the awk-default mode you want most of the time. ## Regex and substitution awk's `sub`, `gsub`, `match`, and `~` all map onto Perl's single regex syntax: `=~` binds a pattern to a string, `s///` substitutes, `s///g` is global (awk's `gsub` vs `sub`). Perl's engine is PCRE, a strict superset of awk's POSIX ERE, so every awk pattern works and Perl adds backreferences, lookaround, named captures, and the `/e`, `/r`, `/x` flags awk has no equivalent for. ```awk awk '/^alice/ { sub(/alice/, "ALICE"); print }' input.txt ``` ```perl perl -ne 'if (/^alice/) { s/alice/ALICE/; print }' input.txt ``` The full-script form: ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { next unless /^alice/; # awk: /^alice/ { ... } s/alice/ALICE/; # awk: sub(/alice/, "ALICE") (first match) print; } # ALICE 85 90 # ALICE 50 55 ``` Use `s///g` for awk's `gsub` (every match), and capture groups land in `$1`, `$2` just as awk's `match` populates `RSTART`/`RLENGTH` plus the `gawk` array. The `/e` flag runs **arbitrary Perl** in the replacement, which awk cannot do: ```perl use v5.42; my $s = "items: 3 and 4"; (my $t = $s) =~ s/(\d+)/$1 * 10/ge; # /e: replacement is Perl code say $t; # items: 30 and 40 ``` `length`, `substr`, `index`, and the case functions map by name - watch the indexing: ```perl use v5.42; my $s = "alice"; say length $s; # 5 (awk length($s)) say substr $s, 0, 3; # ali (awk substr($s,1,3) - awk is 1-based!) say uc $s; # ALICE (awk toupper($s)) ``` The gotcha: `substr` is **zero-based** where awk's `substr` is one-based, so awk `substr($s,1,3)` is Perl `substr($s,0,3)`. And `s///` edits its target in place and returns the **count** of substitutions, not the new string - so `my $new = ($s =~ s/a/b/)` puts a number in `$new`. Copy first (`(my $t = $s) =~ s/.../.../`) or use the `/r` flag to get a fresh string. POSIX character classes (`[[:digit:]]`) work, but the Perl shorthands (`\d`, `\w`, `\s`) are shorter and what you will see in Perl code. ## Addresses and ranges An awk rule is `pattern { action }`: the action runs only on records matching the pattern. In Perl that pattern is just a condition on the statement - a postfix `if` testing `$_`. A bare `/re/` matches against `$_`, a numeric test uses `$.` (awk's `NR`), and awk's range `/start/,/end/` is Perl's **flip-flop** operator `/start/ .. /end/`. ```awk awk 'NR==2' input.txt # print record 2 awk '/banana/' input.txt # print matching records awk '/bob/,/carol/' input.txt # range, inclusive ``` ```perl perl -ne 'print if $. == 2' input.txt perl -ne 'print if /banana/' input.txt perl -ne 'print if /bob/ .. /carol/' input.txt ``` The full-script flip-flop, true from the line matching the left pattern through the line matching the right, inclusive - exactly awk's two-pattern range: ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { print if /bob/ .. /carol/; # awk: /bob/,/carol/ } # bob 70 60 # carol 95 100 ``` The gotcha: a bare `pattern` with no action in awk means "print the record"; in Perl the loop is explicit, so you write the `print` (or `say`) yourself. And in a boolean test `..` is the flip-flop (range) operator, **not** the list-range operator (`1 .. 10` builds a list). Perl tells them apart by context: in the scalar position of an `if`, `..` is the stateful line-range selector that mirrors awk's `addr1,addr2`. ## I/O, pipes, in-place edit awk reads every file named on its command line automatically and has `getline` for explicit reads. Perl's `<>` (the diamond operator) reads each file in `@ARGV` in turn - the same default - and `<$fh>` is the explicit `getline`. Writing to a command or reading from one (awk's `print | "cmd"` and `"cmd" | getline`) is a pipe filehandle. ```awk awk '{ print > "out.txt" }' input.txt # redirect output awk '{ print | "sort" }' input.txt # pipe to a command ``` ```perl use v5.42; open my $out, '>', '/tmp/out.txt' or die "write: $!"; open my $in, '<', '/tmp/scores.txt' or die "read: $!"; while (<$in>) { print {$out} $_; # awk: print > "out.txt" } close $out; ``` For pipes, `open my $fh, '|-', 'sort'` writes to a command's input and `open my $fh, '-|', 'ls', '-l'` reads a command's output - awk's `| "cmd"` in both directions. awk has no built-in in-place edit; you reach for a `tmp`-file shuffle or GNU `gawk -i inplace`. Perl spells it `-i` on a one-liner: ```perl perl -i -pe 's/red/crimson/' file.txt # edit in place ``` The runnable equivalent today is an explicit read-modify-write - slurp the lines, transform, write back - which is exactly what `-i` does under the hood: ```perl use v5.42; my $path = '/tmp/inplace.txt'; # holds: red green blue open my $in, '<', $path or die "read: $!"; my @lines = <$in>; # slurp all lines (list context) close $in; s/e/E/g for @lines; # transform each line's $_ open my $out, '>', $path or die "write: $!"; print {$out} @lines; # write them back close $out; # file now holds: rEd / grEEn / bluE ``` The gotcha: `open` returns false on failure rather than aborting, so the `or die "...: $!"` idiom is mandatory - `$!` is the system error string, awk's silent failure made explicit. And `print {$out} ...` needs the braces around the filehandle when it is a variable; the bare `print FH ...` form (no comma) is for named handles, which is a common first-day stumble. ## One-liners and switches The one-liner is awk's home turf and Perl's too. The switch family that reproduces awk: `-e` supplies the program, `-n` wraps it in a per-record loop (awk's implicit `{ }`), `-p` does the same with an auto-print at the end of each cycle, `-a` autosplits into `@F` (awk's automatic field split), `-F` sets the split pattern (awk's `FS`), and `-l` handles line endings (awk's `ORS`/`RS` convenience). `BEGIN { }` and `END { }` are the **same keywords** as awk, running before and after the loop. ```awk awk '/error/ { print }' log.txt # print error records awk '{ s += $NF } END { print s }' input.txt # sum the last field awk -F, '{ print $1 }' input.txt # first CSV field ``` ```perl perl -ne 'print if /error/' log.txt perl -ane 'END { print "$s\n" } $s += $F[-1]' input.txt perl -F, -ane 'print "$F[0]\n"' input.txt ``` These are shown without output because the implicit-loop switches are not recognised by this `pperl` build yet (see the note at the top). Each maps to a verified explicit-loop form: `perl -ne 'CODE'` is `while (<$fh>) { CODE }`, `-ane` adds `my @F = split` at the top, and `BEGIN`/`END` blocks work in a full script exactly as in awk. The runnable "sum a column with BEGIN/END" form: ```perl use v5.42; my $n = 0; my $sum = 0; BEGIN { say "start" } # runs before the loop, like awk BEGIN END { say "lines=$n sum=$sum" } # runs after, like awk END open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { my @F = split; $n++; $sum += $F[1]; # awk: sum += $2 } # start # lines=4 sum=300 ``` The gotcha: until the switches land on `pperl`, write the explicit loop (it runs today and is what the switches expand to anyway), and consult the [One-Liners guide](../oneliner/index) for the full switch catalogue and dozens of recipes - it is the highest-leverage thing to read coming from an awk background. Perl also has full subroutines, references, modules, and OO if your script outgrows a one-liner; see the [Object-oriented Programming guide](../oop/index) and the other coming-from guides. ## Control flow awk's control keywords map onto Perl's almost name-for-name. awk's `next` (skip to the next record) is Perl's `next`; `break`/`continue` in awk loops are Perl's `last`/`next`; awk's `for (k in arr)` becomes a Perl `for` over `keys`. The braces and `;` are the same; Perl adds the postfix forms that read like awk patterns. ```awk awk '{ if ($2 < 70) next; print }' input.txt ``` ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; while (<$fh>) { my @F = split; next if $F[1] < 70; # awk: if ($2 < 70) next print; } # alice 85 90 # carol 95 100 ``` The gotcha: Perl's comparison operators come in two families and you pick by the *operands' type*, where awk has only `<`/`==` for both. Numbers use `==`, `<`, `>`, `<=>`; strings use `eq`, `lt`, `gt`, `cmp`. awk decides numeric-vs-string comparison by the values at runtime; Perl makes you choose the operator. Using `==` on strings (or `eq` on numbers) is a frequent, quiet bug - `"abc" == "xyz"` is true in Perl (both numify to 0), which is almost never what you meant. ## Variables and data This is the happiest mapping in the guide: awk's associative array is a Perl **hash**, and the daily idioms are near-identical. awk's `count[$1]++` is Perl's `$count{$F[0]}++`; `for (k in count)` is `for my $k (keys %count)`; `if (k in count)` is `exists $count{$k}`. The one shift is the sigil - the whole hash is `%count`, but one value is `$count{$key}`. ```awk awk '{ total[$1] += $2 + $3 } END { for (k in total) print k, total[k] }' input.txt ``` ```perl use v5.42; open my $fh, '<', '/tmp/scores.txt' or die "open: $!"; my %total; while (<$fh>) { my @F = split; $total{$F[0]} += $F[1] + $F[2]; # awk: total[$1] += $2 + $3 } for my $name (sort keys %total) { # awk END loop over the array say "$name $total{$name}"; } # alice 280 # bob 130 # carol 195 ``` The two-file `NR==FNR` idiom - load keys from the first file, then test membership in the second - is one explicit loop per file in Perl, which is clearer than the awk one-liner it replaces: ```perl use v5.42; my %seen; open my $f1, '<', '/tmp/data.csv' or die "open: $!"; while (<$f1>) { my ($k) = split /,/; # first field only $seen{$k} = 1; } close $f1; open my $f2, '<', '/tmp/scores.txt' or die "open: $!"; while (<$f2>) { my ($k) = split; print if $seen{$k}; # awk: ($1 in seen) } # alice 85 90 # bob 70 60 # carol 95 100 # alice 50 55 ``` The gotcha: a hash has no inherent order (awk's `for (k in arr)` is also unordered, so this will not surprise you), so iterate `sort keys %h` when you want a stable order. Membership is `exists $h{$k}`, not awk's `k in arr` - and merely *reading* `$h{$k}` to test it will autovivify the key into existence in some contexts, where `exists` never does. Numbers and strings auto-coerce in Perl as in awk, but remember the `==` vs `eq` split from [Control flow](#control-flow). ## Gotchas coming from awk The traps, collected for a final scan before you write: - **Fields are zero-based.** awk's `$1` is `$F[0]`, `$2` is `$F[1]`, and the whole record `$0` is `$_`. `NF` is `scalar @F`; the last field is `$F[-1]`, not `$F[NF]`. - **`<$fh>` keeps the trailing newline,** unlike awk's `$0`. `chomp` to strip it. - **`substr` is zero-based,** where awk's is one-based: awk `substr($s,1,3)` is Perl `substr($s,0,3)`. - **`s///` returns a count, not the new string.** `my $x = $s =~ s/a/b/` puts the substitution *count* in `$x`. Copy first, or use the `/r` flag to get a new string. - **Two comparison families.** Numbers use `==`/`<`/`<=>`; strings use `eq`/`lt`/`cmp`. awk's single `==`/`<` is split in two, chosen by operator, not by value. `"abc" == "xyz"` is true (both numify to 0). - **Bare `split` is the awk-default mode.** `split` (no args) splits `$_` on whitespace and strips leading blanks. `split / /` (a regex with one space) is literal and does *not* strip - use plain `split` for awk's default `FS`. - **Membership is `exists $h{$k}`, not `k in arr`.** Testing a key by reading `$h{$k}` can autovivify it; `exists` never does. - **`open` does not abort on failure.** Use `open my $fh, '<', $path or die "...: $!"`. `$!` is the system error string. - **The implicit-loop switches (`-n`/`-p`/`-a`/`-F`/`-i`/`-l`) are not yet recognised by this `pperl` build.** Only `-e` works today. Write the explicit `while (<$fh>) { my @F = split; ... }` loop, which is exactly what the switches expand to, until they land.