--- name: regex groups and captures --- ```{index} single: grouping single: capture group ``` # Groups and captures Grouping does two things, which are easy to confuse: it turns a sequence of pattern elements into a single unit for quantification and alternation, and it saves what that unit matched for later use. ```perl /house(cat|keeper)/; # 'house' followed by 'cat' or 'keeper' /(ab){3}/; # 'ababab' /(\d{3})-(\d{4})/; # capture two groups separated by '-' ``` Beyond the plain `(…)` capturing form, Perl provides a small zoo of parenthesised constructs: non-capturing groups for clean grouping, named captures for self-documenting patterns, atomic groups for controlling backtracking, recursive subpatterns for nested structures, conditional patterns for branching on prior captures, and the branch-reset construct for parallel alternatives. This chapter covers all of them. ```{index} single: $1 single: capture group; numbering single: match; in list context ``` ## Capturing groups: `$1`, `$2`, … Every pair of unescaped parentheses in a pattern opens a capturing group. After a successful match the matched text of the nth group is in `$n`: ```perl if ($time =~ /(\d\d):(\d\d):(\d\d)/) { my ($hours, $minutes, $seconds) = ($1, $2, $3); } ``` In list context, a match returns the list of captured strings directly: ```perl my ($h, $m, $s) = $time =~ /(\d\d):(\d\d):(\d\d)/; ``` If the pattern fails, the list is empty - a useful idiom for "parse or give up": ```perl my ($h, $m, $s) = $time =~ /(\d\d):(\d\d):(\d\d)/ or die "not a time: $time"; ``` There is no upper limit on the number of capture groups. Groups are numbered with the leftmost open parenthesis as group 1, the next as group 2, and so on: ``` /(ab(cd|ef)((gi)|j))/ 1 2 34 ``` `$1` captures the outer group, `$2` the first inner, `$3` the next, `$4` the innermost. ```{index} single: capture group; undefined versus empty ``` ### Captured the empty string vs. did not match at all A capture group that did not participate in the match has `$n` *undefined*. A group that participated and matched an empty string has `$n` defined and equal to `""`. The distinction matters: ```perl "aba" =~ / a (x)* b \g1 a /x; # does NOT match "aba" =~ / a (x)? b \g1 a /x; # does NOT match "aba" =~ / a (x*) b \g1 a /x; # matches; $1 = "" "aba" =~ / a (x?) b \g1 a /x; # matches; $1 = "" ``` In the first two cases, the quantifier is *outside* the group, so the group itself never closed (the engine matched zero iterations - the group was not entered at all). The backreference `\g1` therefore has no value to compare against and fails. In the second two cases, the quantifier is *inside* the group, so the group ran exactly once and captured an empty string; `\g1` matches the same empty string at that position. The lesson: when a group is optional, put the quantifier inside if you want the empty-match-counts-as-match semantics. Always check captures with `defined`, not for truth - an empty capture is defined but false under string semantics: ```perl if ("x" =~ /(a)?(x)/) { print "1 is $1\n" if defined $1; # $1 is undef here print "2 is $2\n" if defined $2; } ``` ```{index} single: $1; after a failed match ``` ### Failed matches do not reset capture variables If a match fails, `$1`, `$2`, … keep their previous values from the last *successful* match in the same scope. This is a feature: it lets you write a series of more specific patterns and refer to the best match's captures afterwards. ```perl "foo" =~ /(\w+)/ and "" =~ /(\d+)/; # second fails print $1; # "foo" - first match's capture survives ``` It is also a common source of confusion. Always check the match's return value before reading the capture variables. ```{index} single: (?:...) single: non-capturing group ``` ## Non-capturing groups: `(?:…)` If you only need the grouping for quantification or alternation, and don't want the capture, use `(?:…)`: ```perl /(?:ab){3}/; # 'ababab', no capture /(?:\d+\.)*\d+/; # a dotted decimal, no captures at all ``` Non-capturing groups are a small speed win and a larger clarity win. They signal "this grouping is for syntax, not data". They also prevent renumbering of the capturing groups you do care about: ```perl # match a number - $1 = whole, $2 = optional exponent value /([+-]?\ *(?:\d+(?:\.\d*)?|\.\d+)(?:[eE]([+-]?\d+))?)/; ``` Without the `(?:…)` wrappings, `$2`, `$3`, `$4` would all be set and the intended `$2` (the exponent) would shift to `$5`. The scoped form `(?flags:…)` (e.g. `(?i:cat)`, `(?xms:…)`) attaches modifiers to the inner pattern only and is also non-capturing - see the [modifiers](modifiers) chapter. `split` also benefits from `(?:…)`. `split /(?:\s+)/` separates on runs of whitespace without inserting the separators into the output; `split /(\s+)/` leaves them in alternating positions. ```{index} single: named capture single: (?...) single: %+ single: \k ``` ## Named captures `(?…)` or `(?'name'…)` names a group. Its match is accessible through the hash `%+`: ```perl if ("2026-04-23" =~ /(?\d{4})-(?\d{2})-(?\d{2})/) { print "year = $+{year}\n"; # 2026 print "month = $+{month}\n"; # 04 print "day = $+{day}\n"; # 23 } ``` Named groups also populate `$1`, `$2`, … in the usual left-to-right order, so code that uses both conventions works. Inside the pattern, reference a named group with `\k` (or any of the brace/quote forms below): ```perl /(?["'])(.*?)\k/; # same quote at start and end ``` Names follow Perl-identifier rules (`[_A-Za-z][_A-Za-z0-9]*`); they cannot begin with a digit and cannot contain hyphens. If two distinct groups share a name, `$+{name}` refers to the *leftmost defined* group in the match. This is unusual to want *outside* a branch reset (`(?|…)`), where shared names are idiomatic. ```{index} single: (?P...) single: named capture; Python syntax ``` ### `(?P…)` - Python/PCRE compatibility For programmers porting from Python's `re` module or PCRE, Perl accepts the Python-style spellings: | Python form | Perl equivalent | |-----------------|---------------------| | `(?P...)` | `(?...)` | | `(?P=NAME)` | `\k` | | `(?P>NAME)` | `(?&NAME)` | The Python forms work but are not idiomatic Perl; prefer the native forms in new code. ```{index} single: backreference single: \1 single: \g{...} ``` ## Backreferences A backreference in a pattern demands that a later position match the *same text* an earlier group captured - not the same pattern, the same actual characters. | Form | Refers to | |--------------|--------------------------------------------------------| | `\1` … `\9` | first through ninth capturing group (legacy form) | | `\g1`, `\g2` | numbered capture; equivalent to `\1`, `\2` | | `\g{1}` | brace form; required when digits would otherwise follow | | `\g-1` | most-recently-opened capturing group (relative) | | `\g{-2}` | second-most-recently-opened group | | `\k` | named capture | | `\k'name'` | same, single-quote form | | `\k{name}` | same, brace form (allows surrounding spaces) | | `\g{name}` | named, alternative spelling | Examples: ```perl # Match a three-letter word followed by a space and the same word. "the the other day" =~ /\b(\w{3})\s\1\b/; # $1 eq 'the' # Match a four-letter, three-letter, two-letter, or one-letter # run followed by itself. /^(\w{1,4})\1$/; # 'beriberi', 'booboo', 'coco', 'mama', 'papa' ``` ```{index} single: backreference; versus octal escape ``` ### `\g{…}` versus `\1` for disambiguation Use `\g{…}` (or `\k<…>`) when digits or octal-looking characters follow the reference, to avoid ambiguity: ```perl /(\d)abc\g{1}23/; # the '1' refers to group 1, '23' is literal /(\d)abc\123/; # '\123' is octal 0x53 ('S'), not group 1 ``` Perl's rule for the bare numbered form: `\1` through `\9` always mean backreferences. `\10`, `\11`, … mean a backreference *only if* that many capture groups have opened earlier in the pattern; otherwise they are octal character literals. This is why `\g{...}` is the safer form when the pattern is built by concatenation. If you use the brace form, optional surrounding spaces are permitted - `\g{ -1 }` and `\k{ name }` are valid. ```{index} single: backreference; relative single: \g{-1} ``` ### Relative backreferences `\g-1`, `\g{-1}` refer to the *immediately previous* capturing group; `\g{-2}` refers to the one before that; and so on. The distance is counted by *opened* parens, including unclosed ones. This matters when a pattern fragment is interpolated inside another: ```perl my $pair = '([a-z])(\d)\g{-1}\g{-2}'; # a11a, g22g, x33x, ... # Embed it: outer group shifts numbering by 1, but relative # backreferences still work: "code=e99e" =~ /^(\w+)=$pair$/; # matches ``` Without relative backreferences, this would require knowing how many groups precede `$pair` at every interpolation site. Named and relative references make long patterns robust against copy-paste. ```{index} single: atomic group single: (?>...) single: (*atomic:...) ``` ## Atomic groups: `(?>…)` `(?>…)` is a non-capturing group whose contents, once matched, are *committed*. The engine cannot backtrack into the group on failure - only past it as a whole. ```perl "aaab" =~ /a*ab/; # matches: a* gives back one 'a' "aaab" =~ /(?>a*)ab/; # does not match: a* takes all, refuses to give ``` The full treatment lives in the [performance](performance) chapter. The construct is documented here because its syntax is a parenthesised group; structurally it belongs in the captures chapter alongside `(?:…)`. Possessive quantifiers (`*+`, `++`, `?+`, `{n,m}+`) are exact syntactic sugar for `(?>…)` around the quantified atom. The following are equivalent: | Possessive | Atomic-group form | |------------|-------------------| | `PAT*+` | `(?>PAT*)` | | `PAT++` | `(?>PAT+)` | | `PAT?+` | `(?>PAT?)` | | `PAT{n,m}+`| `(?>PAT{n,m})` | The long-form spelling `(*atomic:…)` is also accepted. ```{index} single: recursive subpattern single: (?R) single: (?1) single: (?&NAME) single: balanced parentheses; matching ``` ## Recursive subpatterns Perl regexps can refer back to a capture group as if it were a subroutine call. The construct re-runs the captured pattern at the current position. This makes genuinely recursive structures matchable - balanced parentheses, S-expressions, nested brackets - without falling out of the regex DSL. | Form | Recurses to | |-------------|-------------------------------------------| | `(?R)` | the entire pattern | | `(?0)` | same as `(?R)` | | `(?1)` | group 1 | | `(?2)` | group 2 (and so on) | | `(?-1)` | most-recently-opened group (relative) | | `(?+1)` | next group to be opened (forward relative) | | `(?&NAME)` | named group | | `(?P>NAME)` | same as `(?&NAME)` (Python-compatible) | Note: relative recursion counts unclosed groups, just as relative backreferences do. `(?-1)` always means the most recently opened group whether or not it has closed. A balanced-paren matcher: ```perl my $bal = qr/ (?(DEFINE) (? \( # opening paren (?: [^()]++ # non-paren run, possessive | (?&paren) # or a balanced sub-group, recursively )*+ \) # closing paren ) ) (?&paren) /x; "((a)(b(c)))" =~ /^$bal$/; # matches ``` The `(?(DEFINE)...)` block declares a named subpattern without matching anything itself; the `(?&paren)` after it invokes that subpattern, and the recursive `(?&paren)` inside the body is what gives the unbounded depth. `(?R)` would not work here - it recurses the *whole* enclosing pattern, including the `^` and `$` anchors that get added at the call site, which forces every level of recursion to demand start- and end-of-string. A more nuanced recursive pattern: match a function `foo(...)` where the argument may itself contain balanced parens. ```perl my $re = qr/( # group 1: full function call foo ( # group 2: parens with content \( ( # group 3: contents of parens (?: (?> [^()]+ ) # non-paren without backtracking | (?2) # recurse to group 2 )* ) \) ) )/x; 'foo(bar(baz)+baz(bop))' =~ /$re/ and print "1: $1\n2: $2\n3: $3\n"; # 1: foo(bar(baz)+baz(bop)) # 2: (bar(baz)+baz(bop)) # 3: bar(baz)+baz(bop) ``` ```{index} single: recursive subpattern; captures ``` ### Capture state inside recursion When a group recurses into itself, captures set inside the recursion are *not* visible to the caller after the recursion returns. The recursive call has its own capture state, which is discarded on return. This is why most recursive patterns wrap a secondary capture group around the recursive call when the matched text is needed: ```perl /(?(?&NAME_PAT))(?(?&ADDRESS_PAT)) (?(DEFINE) (?....) (?....) )/x ``` Here `$+{NAME}` is the outer capture; `$+{NAME_PAT}` is undefined because it lived only inside the recursion. The recursion-depth ceiling is compiled into the engine. Patterns that recurse without consuming input fail immediately rather than running forever - the engine detects the cycle. ```{index} single: (?(DEFINE)...) single: regular expression; grammar ``` ### `(DEFINE)` blocks The `(?(DEFINE)…)` block holds named subpatterns that never run on their own - they are only invoked through `(?&NAME)`. This is how to write a regex that looks like a small grammar: ```perl my $email = qr/ \A (?&LOCAL) @ (?&DOMAIN) \z (?(DEFINE) (? [\w.+-]+ ) (? (?&LABEL) (?: \. (?&LABEL) )+ ) (?