Parallel Execution#

PetaPerl automatically parallelizes eligible loops using Rayon, a work-stealing thread pool. Combined with JIT compilation, this enables dramatic speedups for compute-heavy workloads.

How It Works#

When PetaPerl encounters a parallelizable loop, it:

  1. Analyzes the loop body for side effects and shared mutable state

  2. Identifies reduction variables (accumulators like $sum += ...)

  3. Distributes iterations across threads using Rayon’s work-stealing scheduler

  4. Combines results using the detected reduction operations

Each thread gets its own copy of loop-local variables. Reduction variables are combined after all threads complete.

What Gets Parallelized#

Parallel dispatch applies to JIT-compiled loops whose iterations are provably independent except for sum-style reductions:

  • for over an integer range - the primary parallel shape. The range is chunked across threads; each thread accumulates private reduction copies that merge at the end.

  • while loops whose condition is a counter bounded by a constant or read-only variable, with the counter incremented exactly once per iteration.

One gate is stricter than the side-effect analysis and deserves its own sentence: bit-exactness. Integer reductions always qualify. Floating-point reductions qualify only when reassociating the sum provably cannot change the result (for example when every contribution is integral and within the exact range of a double). An inexact float reduction runs compiled but sequential - pperl never trades determinism for cores.

Loops that touch strings, arrays, hashes, or call subs compile to native code but run on one core; see Writing fast pperl for how to split such loops.

CLI Control#

# Default: parallelization enabled
pperl script.pl

# Disable parallelization
pperl --no-parallel script.pl

# Explicitly enable (default)
pperl --parallel script.pl

# Set thread count (default: number of CPU cores)
pperl --threads=4 script.pl

# Set minimum collection size for parallelization
pperl --parallel-threshold=1000 script.pl

The test harness runs with --no-parallel by default to ensure deterministic test output.

Performance#

Mandelbrot Set (1000x1000, mid-2026)#

Mode

Time

vs perl5

perl5

~12,500ms

1.0x

pperl JIT (sequential)

~160ms

~75x faster

pperl JIT + parallel (8 threads)

~30ms

~400x faster

Scaling#

The work-stealing scheduler provides near-linear scaling for embarrassingly parallel workloads:

Threads

Mandelbrot 4000x4000

Scaling

1

baseline

1.0x

2

~50% time

~1.9x

4

~25% time

~3.8x

8

~13% time

~5.2x

Scaling is sub-linear due to memory bandwidth, cache effects, and reduction overhead.

Limitations#

String Operations#

String operations (.= concat, string building) are not parallelized. The JIT’s string support uses extern calls back to the Rust runtime, which requires mutable access to shared state. When the JIT detects string variables in a loop, parallel dispatch is disabled.

Side-Effect Detection#

The parallelization analyzer is conservative. Any of these disqualify a loop:

  • I/O operations (print, open, file reads)

  • Global variable writes

  • Subroutine calls (unless proven pure)

  • Regex operations with side effects (s///)

False negatives (missed parallelization opportunities) are safe - the loop simply runs sequentially. False positives (incorrect parallelization) would be bugs.

Determinism#

Parallel execution may change the order of side effects. For this reason, parallelization is only applied when the analysis proves the loop body is free of observable side effects.

map and grep compile to sequential native loops (array state gates parallel dispatch off), so their output order is trivially the sequential one.

How Reduction Detection Works#

The analyzer identifies reduction variables by scanning for accumulation patterns and subtracting reset patterns:

# Detected as reduction: $sum accumulates, never reset in loop
my $sum = 0;
for my $x (@data) {
    $sum += $x;
}

# NOT a reduction: $temp is reset each iteration
for my $x (@data) {
    my $temp = $x * 2;  # reset (my declaration)
    $sum += $temp;       # $sum is still a reduction
}

The formula: reductions = accumulations - resets. This prevents false positives where a variable is both accumulated and reset within the loop body.