Test Structure
Directory Layout
Test directories are numbered and run in order. The numbering provides natural ordering and allows insertion of new categories.
| Directory | Purpose | Speed |
|---|---|---|
00-base/ | Gate tests (must all pass) | Fast |
01-parsing/ | Parser tests | Fast |
02-builtins/ | Builtins registry | Fast |
05-op/ | Operators | Fast |
10-cmd/ | Control flow | Fast |
15-sub/ | Subroutines | Fast |
15-modules/ | Module system | Fast |
20-data/ | Data structures | Fast |
22-unicode/ | Unicode handling | Fast |
25-regex/ | Regular expressions | Fast |
30-special/ | Special variables | Fast |
40-package/ | Packages and OOP | Fast |
45-builtin/ | Built-in functions | Fast |
50-context/ | Context handling | Fast |
55-io/ | I/O operations | Fast |
60-parallel/ | Auto-parallelization | Medium |
62-jit/ | JIT compilation | Medium |
64-auto-ffi/ | Auto FFI | Medium |
66-bytecode/ | Bytecode tests | Slow |
68-daemon/ | Daemon mode | Medium |
70-core-tier0/ | Core module integration (tier 0) | Slow |
71-core-tier1/ | Core module integration (tier 1) | Slow |
72-core-tier2/ | Core module integration (tier 2) | Slow |
80-complex/ | Integration tests | Medium |
81-xs-native/ | XS-native module replacements | Medium |
82-native/ | Native module tests | Medium |
99-debug/ | Bug regression / active debugging | Fast |
benchmarks/ | Performance benchmarks (not TAP) | Varies |
Gate Tests (00-base)
The 00-base/ directory contains fundamental tests. If any fail, the harness stops (bail-out). These verify:
- Basic parsing works
printproduces output- Variables and assignment work
- Basic operators function
These must always pass before any other testing is meaningful.
Tier 0 Core Modules (70-core-tier0/)
Tier 0 tests exercise core CPAN modules. They are subdivided:
| Range | Purpose | Count |
|---|---|---|
| 005-030 | Native pass modules | ~55 modules |
| 035-050 | Fallback pass modules | ~20 modules |
| 055 | TAP::* modules | ~12 modules |
| 057 | Newly passing misc | ~3 modules |
| 060 | Remaining failures | ~4 modules |
Naming Conventions
- Directories:
NN-name/— numbered with gaps (00, 01, 02, 05, 10, 15, …) - Test files:
NNN-name.t— numbered within directory (005, 010, 015, …) - Perl5 ports:
*-perl5.tsuffix indicates a test ported from perl5-upstream - Gaps: Leave room for insertion (use increments of 5)
The 99-debug Directory
99-debug/ is a workspace for active bug investigation. Tests here follow the “complex case” debugging workflow:
- Create synthetic tests isolating a hypothesis
- Fix the bug
- Verify the fix
- Move passing tests to the appropriate category directory
- Clean up 99-debug
Tests should not accumulate in 99-debug permanently.
Benchmarks
The benchmarks/ directory contains performance tests. These are not TAP tests — they measure execution time and are not run by the standard harness. Key benchmarks:
006-mandelbrot-bench.pl— JIT + parallelization scaling test- Various arithmetic and string operation benchmarks