```{index} single: HTTP::Parser::XS ``` # HTTP::Parser::XS ```{pperl-module-badges} HTTP::Parser::XS ``` Parse an HTTP request or response head out of a byte buffer - the parser Plack, Starman and Twiggy sit on. Two entry points, both non-blocking and both operating on a buffer the caller owns: - `parse_http_request($buf, \%env)` fills `%env` with a PSGI environment and returns the number of bytes the request head occupies, `-2` if the head is not complete yet, `-1` if it is malformed. A server calls it again with more bytes on `-2`. - `parse_http_response($buf, $format, \%special)` returns `($ret, $minor_version, $status, $message, $headers)` for a client. ```perl use HTTP::Parser::XS qw(parse_http_request parse_http_response HEADERS_AS_HASHREF); my $req = "GET /search?q=perl%20glob HTTP/1.1\r\nHost: example.com\r\n" . "X-Trace: 1\r\nX-Trace: 2\r\n\r\nbody"; my %env; my $len = parse_http_request($req, \%env); print "$len bytes of head; $env{REQUEST_METHOD} $env{PATH_INFO} ? $env{QUERY_STRING}\n"; # 81 bytes of head; GET /search ? q=perl%20glob print "host $env{HTTP_HOST}, trace '$env{HTTP_X_TRACE}'\n"; # host example.com, trace '1, 2' print parse_http_request("GET / HTTP/1.1\r\nHost:", {}), "\n"; # -2: not complete yet my ($ret, $minor, $status, $msg, $headers) = parse_http_response("HTTP/1.0 404 Not Found\r\nContent-Length: 0\r\n\r\n", HEADERS_AS_HASHREF); print "$ret: HTTP/1.$minor $status $msg, length $headers->{'content-length'}\n"; # 45: HTTP/1.0 404 Not Found, length 0 ``` `$format` is `HEADERS_AS_HASHREF` (a hash of lower-cased names; a repeated header's values in an array reference), `HEADERS_AS_ARRAYREF` (name, value pairs in order) or `HEADERS_NONE`. This is a native implementation of HTTP::Parser::XS 0.17; its `XS.pm` is used unchanged, and nothing is exported by default. ## The PSGI environment `parse_http_request` writes `REQUEST_METHOD`, `REQUEST_URI`, `SCRIPT_NAME` (always `""`), `PATH_INFO`, `QUERY_STRING` and `SERVER_PROTOCOL`, then one key per header. Header names are upper-cased, `-` becomes `_`, and everything except `Content-Type` and `Content-Length` is prefixed `HTTP_`; those two land as `CONTENT_TYPE` and `CONTENT_LENGTH` with no prefix, as CGI and PSGI require. A header sent twice is joined with `", "`. `REQUEST_URI` keeps the request target verbatim. `PATH_INFO` is the part before `?` with `%XX` escapes decoded, `QUERY_STRING` the part after; a `#` fragment and everything after it is dropped from both but not from `REQUEST_URI`. ## Limits At most 128 headers, and a header name of at most 1024 bytes. The two parsers differ on what happens past the name limit: the request parser fails the whole request (`-1`, and `%env` is cleared), the response parser silently skips the offending header and carries on. ## `$BACKEND` `'xs'`, set by XS.pm once XSLoader::load has loaded this native code. `PERL_ONLY=1` or `PERL_HTTP_PARSER_XS=pp` make XS.pm load the distribution's pure-Perl `HTTP::Parser::XS::PP` instead, and then it is `'pp'`. ## Vectorised tokenizer The byte scanners run AVX2 (32B/iteration) or SSE2 (16B) blocks before falling back to picohttpparser's own eight-way unrolled loop and then a byte tail. Upstream drives PCMPESTRI instead, but only under `-msse4.2`, which its Makefile.PL never passes - the shipped `.so` has no vector instruction in it at all - so every CPAN install runs the scalar loop. Speed relative to the reference XS on a Skylake Xeon E3-1505M (`--release`, processes interleaved, min of 5 - sequential runs drift enough between processes to invent a 10% result): | workload | speedup vs perl5 XS (>1 = pperl faster) | |---|---| | 64 KB header value, `HEADERS_NONE` | 6.01x | | 64 KB header value | 4.23x | | 4 KB header value | 2.15x | | 1 KB header name | 1.95x | | 4 KB request target | 1.25x | | **realistic 10-header GET** | **1.00x** | | response, `HEADERS_AS_HASHREF` | 0.88x | The long-value rows are where the vector scan pays. A realistic request is not bound by scanning: most of its cost is creating the hash entry and the string for each PSGI key, so there the two run at the same speed. `PPERL_HTTP_PARSER_SIMD=avx2|sse2|scalar` forces a kernel, for example to compare them on one machine; all three give identical results. ## Differences from upstream None in what is parsed; only the speed differs. ## Functions ### Request parsing #### `parse_http_request` `parse_http_request($buf, \%env)` - XS.xs `parse_http_request`. ### Response parsing #### `parse_http_response` `parse_http_response($buf, $header_format, \%special_headers)` - XS.xs `parse_http_response`.