Perl5 Comparison Testing

Methodology

PetaPerl’s test methodology is conformance testing: every test runs under both perl5 and pperl, and pperl passes only when its output matches perl5 exactly. This ensures behavioral compatibility without encoding assumptions about “correct” behavior — perl5 is the authoritative reference.

How It Works

For each test.t:
  1. Run under perl5 → capture stdout, stderr, exit code
  2. Run under pperl → capture stdout, stderr, exit code
  3. Compare all three
  4. PASS if identical, FAIL if any differ

This approach has several advantages:

• No false positives: If perl5 and pperl agree, the behavior is correct by definition
• Self-documenting: The expected output doesn’t need to be hardcoded
• Catches subtle differences: Numeric formatting, warning text, edge cases
• Easy to extend: Write a Perl script, run it — no test framework needed

System Perl Requirement

The test harness requires a system perl5 (5.42+ recommended) to generate reference output. This is the only runtime dependency on perl5 — pperl itself runs independently.

Edge Cases

Warnings

Perl5’s warning messages include specific text that may differ between versions. Tests should be written to avoid triggering version-dependent warnings where possible.

Random Output

Tests involving rand, time, or other non-deterministic functions need special handling. Either seed the RNG (srand 42) or structure the test to compare properties rather than exact values.

Platform-Dependent Behavior

Some perl5 behaviors vary by platform. Since pperl targets Linux only, ensure the reference perl5 also runs on Linux to avoid false failures.

Numeric Precision

Floating-point output formatting may differ at the last decimal place. PetaPerl aims to match perl5’s sprintf formatting exactly, but edge cases exist.

Version Targeting

PetaPerl targets perl 5.42 compatibility. The reference perl5 used for comparison testing should be version 5.42.x. Using a different perl5 version may produce false failures due to behavioral differences between perl versions.

PetaPerl does not attempt backward compatibility with older Perl versions. use v5.18 and similar declarations are parsed but do not change behavior — pperl always behaves like 5.42.