```{index} single: GPU single: WebGPU single: wgpu single: Peta::WebGPU; device and surface ``` # The GPU: a device, a window, and the frame loop Everything so far computed pixels in Perl. From here the graphics card computes them. pperl's GPU binding is [`Peta::WebGPU`](../../p5/Peta/WebGPU): WebGPU as implemented by the `wgpu` library, which drives Vulkan, Metal, Direct3D 12 or OpenGL underneath, whichever the machine has. The programs that run on the card are written in WGSL, WebGPU's shading language. The [world generator](capstone-world-generator) used the binding's compute half; this arc uses the half that draws. The window, the events and the clock come from the native [`SDL2`](../../p5/SDL2) binding, as in the [2D guide's toolkit chapter](../graphics/gui-toolkits). The two meet at exactly one point: wgpu builds its surface on the handles of the SDL2 window. There is no context to create and no swap call. This chapter covers what takes their place. ```{index} single: GPU; detecting single: GPU adapter ``` ## Is there a GPU at all? Two different things can be missing, and they fail differently. The pperl binary may be built without the GPU option. Loading the module then dies, naming the fix: ```text Peta::WebGPU requires the 'webgpu' capability, which this pperl binary was built without. Rebuild with: cargo build --features webgpu ``` Or the build has it and the machine has no usable adapter, as in a container with no GPU driver. `have_gpu` answers that without dying, so one line covers both cases and a program can fall back to the [software rasteriser](software-rasteriser): ```perl my $gpu_ok = eval { require Peta::WebGPU; 1 } && Peta::WebGPU::have_gpu(); ``` `adapters` lists what the machine offers: ```perl for my $a (Peta::WebGPU::adapters()) { printf "%-8s %-14s %s\n", $a->{backend}, $a->{device_type}, $a->{name}; } ``` ```text Vulkan IntegratedGpu Intel(R) HD Graphics P530 (SKL GT2) Vulkan DiscreteGpu Quadro M2000M Gl Other Quadro M2000M/PCIe/SSE2 ``` One card appears twice, once per backend that can reach it. The list is for reporting; `Peta::WebGPU::Device->new` makes the choice, steered by its `power` option. On this machine `low-power` opens the Intel chip and `high-performance` opens the Quadro. `$gpu->info` says which one you got, with the same keys. ```{index} single: headless rendering single: render pass single: readback single: PNG; from GPU pixels ``` ## Drawing without a window Start with the render target that needs no display at all, because it is how every example in this arc is checked, and how you check yours. ```perl use strict; use warnings; use Peta::WebGPU; my $gpu = Peta::WebGPU::Device->new; # no window my ($W, $H) = (320, 240); my $img = $gpu->texture(width => $W, height => $H); $gpu->render_pass($img, clear => [0.2, 0.4, 0.6, 1])->end; my $rgba = $img->read; # $W * $H * 4 bytes sub pixel { my ($x, $y) = @_; return unpack 'C4', substr($rgba, ($y * $W + $x) * 4, 4); } printf "%d bytes, centre pixel %s\n", length $rgba, join ' ', pixel($W / 2, $H / 2); # 307200 bytes, centre pixel 51 102 153 255 ``` `texture` makes an image on the card; its format defaults to `rgba8unorm`, four bytes per pixel. `render_pass` starts recording into it, clearing to the given colour, components from 0 to 1. **Nothing happens until `end`**: a pass records, and `end` hands the recording to the card. `read` waits for the card to finish and copies the image back, rows top first, bytes in R, G, B, A order: `0.2, 0.4, 0.6` arrive as `51 102 153`. Reading crosses the bus and waits, so it belongs at the end of a job, not inside a loop. The same program run with `DISPLAY` and `WAYLAND_DISPLAY` removed from the environment prints the same line. It needs no display server, no window manager and no compositor, so it runs on a server or in a test suite. To look at the result, write it out as a PNG through [`GD`](../../p5/GD), which the [2D guide](../graphics/raster-gd-magick) covers: ```perl use GD; sub save_png { my ($file, $rgba, $w, $h) = @_; my $im = GD::Image->new($w, $h, 1); # truecolor for my $y (0 .. $h - 1) { for my $x (0 .. $w - 1) { my ($r, $g, $b) = unpack 'C3', substr($rgba, ($y * $w + $x) * 4, 3); $im->setPixel($x, $y, $im->colorAllocate($r, $g, $b)); } } open my $fh, '>', $file or die "$file: $!\n"; binmode $fh; print $fh $im->png; close $fh; } save_png('clear.png', $rgba, $W, $H); ``` A loop over every pixel is slow in Perl, but this is a checking tool, run once, and it keeps the picture exactly as the card produced it. ```{index} single: surface; WebGPU single: SDL2; with WebGPU ``` ## A window, and a device opened for it ```perl use strict; use warnings; use SDL2 qw(:all); use Peta::WebGPU; my $win = SDL2::Window->new(title => 'pperl gpu', w => 800, h => 600, resizable => 1); my $gpu = Peta::WebGPU::Device->new(window => $win); my $surf = $gpu->surface; ``` **The window goes to `Device->new`, not to a later call.** The adapter has to be one that can present to that window, and wgpu can only answer that question for a real surface, so the device is chosen with the window in hand. A machine with two cards could otherwise open the one that does not drive the screen. `Peta::WebGPU` opens no windows itself. It asks the window object two questions, `$win->wm_info` for the window system's own handles and `$win->size`, and draws on anything that answers them. `SDL2::Window` does; under X11 its `wm_info` carries `driver`, `display` and `window`, under Wayland `display` and `surface`. Those two window systems are the ones a surface can be made on. `$gpu->surface` is the window's chain of images to draw into, configured at the window's size. On the machine these examples were checked on: ```perl print $surf->format, ' ', join('x', $surf->size), "\n"; # bgra8unorm-srgb 800x600 ``` The format is the compositor's preference. `surface(format => ...)` can ask for another the window offers, and one it does not offer is refused with the list of those it does. Whichever it is, anything that draws into the window has to be built for it, which the [next chapter](gpu-pipeline) returns to. ```{index} single: import; compile time versus runtime single: SDL2; import trap single: bareword; constant not yet imported single: strict; barewords ``` ## The import trap This one costs an afternoon, and it produces a wrong result rather than an error. ```perl use SDL2; # no import list SDL2->import(qw(:all)); # WRONG: runs at runtime my $win = SDL2::Window->new(title => 'trap', w => 160, h => 120); SDL_PushEvent({ type => SDL_QUIT() }); # parentheses: resolved at runtime my $quit = 0; while (my $ev = SDL_PollEvent()) { $quit = 1 if $ev->{type} == SDL_QUIT; } print "SDL_QUIT is: ", SDL_QUIT, "\n"; print "quit noticed: ", ($quit ? "yes" : "no"), "\n"; ``` `use SDL2 qw(:all)` imports at **compile time**, so the constants exist before the rest of the file is parsed. A separate `SDL2->import` runs after the whole file has been compiled, so while it was being compiled `SDL_QUIT` without parentheses was not a known function. It parsed as a bareword and became the string `"SDL_QUIT"`: ```text SDL_QUIT is: SDL_QUIT quit noticed: no ``` The quit event was on the queue and the comparison, against a string that numifies to 0, never matched it. In a real program the window's close button does nothing. `use warnings` adds one line, `Argument "SDL_QUIT" isn't numeric in numeric eq (==)`, and the program still runs on. **`use strict` converts this from a silent wrong answer into a compile error** naming the constant. The same file with `use strict;` as its first line: ```text Bareword "SDL_QUIT" not allowed while "strict subs" in use at trap.pl line 9. Bareword "SDL_QUIT" not allowed while "strict subs" in use at trap.pl line 12. Execution of trap.pl aborted due to compilation errors. ``` Line 9 is the statement modifier inside the `while` body, reported at the line where that statement starts, as perl5 reports it. That is the whole argument for `use strict` in one example. Write `use SDL2 qw(:all);` and keep `use strict` on. ```{index} single: frame loop single: present ``` ## The frame loop A frame is three steps: take an image from the surface, draw into it, present it. ```perl my $frame = $surf->next_frame or next; # no image this time: skip $gpu->render_pass($frame, clear => [0.2, 0.3, 0.5, 1])->end; $frame->present; ``` A frame is a render target exactly as a texture is: the same `render_pass`, the same recording calls. That is what lets one drawing routine serve both the screen and the checking path above. `next_frame` returns undef rather than dying when wgpu reports the surface occluded or timed out. That is not an error; skip the frame and come back. When wgpu reports the surface out of date, `next_frame` configures it again with the size it was last given and retries once, which is why a missed resize, below, produces no error at all. **End the pass before presenting.** Presenting first is accepted, and the `end` that follows dies, because the image it recorded into is gone: ```text Peta::WebGPU::Pass::end: Validation Error Caused by: In Queue::submit In a pass parameter Texture with '' label has been destroyed ``` Presenting the same frame twice is refused with `this frame has already been presented`. ```{index} single: present mode single: vertical sync ``` ### Pacing A surface's present mode decides how a frame reaches the screen. The default is `fifo`: it queues each frame for the display's next refresh and never tears, so the loop runs at the display's rate with no sleep anywhere in the program. It is also the one mode WebGPU guarantees every surface supports, so the default never fails. Measured over 120 frames on a 60 Hz display, a surface made as above averaged 16.68 ms per frame, and one made with `present_mode => 'fifo'` spelled out 16.67 ms. Another mode is asked for by name when the surface is made: ```perl my $surf = $gpu->surface(present_mode => 'mailbox'); ``` `mailbox` keeps only the newest frame and `immediate` does not wait at all, which can tear. Both returned as fast as the loop could go, under a millisecond per frame. `fifo-relaxed` waits like `fifo` but lets a late frame through at once, at the risk of a tear. A mode the window does not offer is refused, with the ones it does offer in the message. ```{index} single: sRGB single: linear light single: colour; sRGB decoding ``` ## Colour arrives in sRGB The surface format above ends in `-srgb`, and that suffix changes the numbers. An `-srgb` target treats what is written to it as linear light and encodes it on the way in, the encoding image files and displays use. The same clear into two formats: ```text format clear [0.2, 0.4, 0.6] stored as rgba8unorm written as given 51 102 153 255 rgba8unorm-srgb encoded 124 170 203 255 ``` Lighting arithmetic is only correct in linear light, so rendering into an `-srgb` target is the right thing. The consequence is for colours you pick by eye, from an image editor or a web page: those are already encoded, and fed straight in they come out washed out. Decode them first. Raising each component to the power 2.2 is close enough for picking colours: the clear `[ map { $_ ** 2.2 } 0.2, 0.4, 0.6 ]` stores as `48 102 154`. For the checking path, render into `rgba8unorm-srgb` whenever the image should match what the window shows. `bgra8unorm-srgb` would match too, but `read` returns its bytes in B, G, R, A order. ```{index} single: window resize single: SDL2; window events single: SDL_PushEvent ``` ## Window events The event loop from the [2D guide](../graphics/gui-toolkits) is unchanged. Three events matter from the first program on: ```perl while (my $ev = SDL_PollEvent()) { if ($ev->{type} == SDL_QUIT) { $running = 0 } elsif ($ev->{type} == SDL_KEYDOWN && $ev->{sym} == SDLK_ESCAPE) { $running = 0 } elsif ($ev->{type} == SDL_WINDOWEVENT && $ev->{event} == SDL_WINDOWEVENT_SIZE_CHANGED) { $surf->configure($ev->{data1}, $ev->{data2}); } } ``` **Symptom: after a resize the picture is the wrong size, and nothing reported an error.** Cause: the surface was never reconfigured. A window resized from 320x240 to 400x300 kept a 320x240 surface, every frame presented without complaint, and `$surf->size` said `320x240` against the window's `400x300`. Key the reconfigure on `SDL_WINDOWEVENT_SIZE_CHANGED`, not `SDL_WINDOWEVENT_RESIZED`. A resize made by the program with `$win->set_size(400, 300)` delivered `SIZE_CHANGED` with `data1 = 400`, `data2 = 300`, and no `RESIZED` at all. `SDL_PushEvent` puts an event on the queue exactly as if a person had caused it, which is how a test drives a program that normally waits for input. Pushed and read back, an Escape key press arrives as `sym = 27, scancode = 41, key_name = Escape`, and a mouse drag as `SDL_MOUSEMOTION` with `x`, `y`, the movement since the last event in `xrel`, `yrel`, and the held buttons in `state`, which is what an orbit camera reads. ```{index} single: fixed timestep single: fixed timestep; step cap ``` ## Timing: a fixed step `fifo` paces the frames, but a simulation must not depend on the frame rate: a faster display would make everything move faster. Accumulate the real elapsed time and advance the simulation in fixed steps: ```perl use constant { STEP => 16, MAX_STEPS => 5 }; # milliseconds, steps per frame my $now = SDL_GetTicks(); $acc += $now - $last; $last = $now; my $steps = 0; while ($acc >= STEP) { update(); $acc -= STEP; if (++$steps >= MAX_STEPS) { $acc = 0; last } } ``` In a window paced by `fifo`, this ran 205 updates in 3307 ms of real time, and 3307 / 16 is 206.7: the simulation keeps to the clock whatever the frame rate. **The step cap is not optional**, and neither is dropping the backlog when the cap is hit. What each part prevents is clearest on a simulated clock, where every frame costs 6 ms of rendering plus the time its updates take. The figures are updates run per frame, in consecutive frames: ```text each update costs 20 ms, more than the 16 ms it simulates: no cap 7 9 12 15 19 24 31 39 49 62 77 97 122 153 ... cap, keep backlog 6 6 6 6 ... owed 22, 52, 82 ... 412 ms, and growing cap, drop backlog 6 6 6 6 ... owed 0 a 500 ms stall, then updates costing 12 ms: no cap 33 25 19 15 11 9 7 6 5 4 3 3 2 ... cap, keep backlog 6 6 6 6 ... owed 436 ms, still 220 ms 12 frames later cap, drop backlog 6 4 4 3 3 2 2 2 2 ... ``` Without a cap, an update slower than its step is a **spiral**: each frame takes longer, owes more steps, and takes longer still, until the program stops responding. A stall on its own is survivable when updates are cheaper than their step, but the first frame after it runs 33 updates, which is a second freeze of almost 400 ms. With a cap but keeping the backlog, frame time is bounded but the simulation fast-forwards for seconds afterwards, working off time that has already passed. Dropping the backlog is the honest answer: after a stall, a breakpoint or a window drag, the simulation loses that time and carries on at normal speed. When every update is slower than its step, it runs slower than real time, which is the only possible outcome, and the window keeps responding. ## The whole program Put together, with an optional frame count so it can run unattended: ```perl use strict; use warnings; use SDL2 qw(:all); use Peta::WebGPU; use constant { STEP => 16, MAX_STEPS => 5 }; my $max_frames = shift // 0; # 0: run until the window is closed my $win = SDL2::Window->new(title => 'pperl gpu', w => 800, h => 600, resizable => 1); my $gpu = Peta::WebGPU::Device->new(window => $win); my $surf = $gpu->surface; printf "%s via %s, surface %s %s\n", $gpu->info->{name}, $gpu->info->{backend}, $surf->format, join('x', $surf->size); my $phase = 0; # the whole simulation state sub update { $phase += 0.001 * STEP } # radians per step: one cycle in 6.3 s sub colour { return [ 0.2 + 0.2 * sin($phase), 0.3, 0.5 + 0.2 * cos($phase), 1 ]; } my ($last, $acc, $frames, $running) = (SDL_GetTicks(), 0, 0, 1); while ($running) { while (my $ev = SDL_PollEvent()) { if ($ev->{type} == SDL_QUIT) { $running = 0 } elsif ($ev->{type} == SDL_KEYDOWN && $ev->{sym} == SDLK_ESCAPE) { $running = 0 } elsif ($ev->{type} == SDL_WINDOWEVENT && $ev->{event} == SDL_WINDOWEVENT_SIZE_CHANGED) { $surf->configure($ev->{data1}, $ev->{data2}); } } my $now = SDL_GetTicks(); $acc += $now - $last; $last = $now; my $steps = 0; while ($acc >= STEP) { update(); $acc -= STEP; if (++$steps >= MAX_STEPS) { $acc = 0; last } } my $frame = $surf->next_frame or next; # no image this time: skip $gpu->render_pass($frame, clear => colour())->end; $frame->present; $running = 0 if $max_frames && ++$frames >= $max_frames; } printf "%d frames, phase %.2f\n", $frames, $phase; ``` ```text Intel(R) HD Graphics P530 (SKL GT2) via Vulkan, surface bgra8unorm-srgb 800x600 120 frames, phase 1.94 ``` A window whose colour drifts slowly, closing on Escape or the close button. 1.94 radians is 121 steps of 0.016, which is 1936 ms: two seconds of clock for 120 frames at 60 Hz. The simulation followed the clock, not the frame count. ## Next The loop runs and the checking path works. The [next chapter](gpu-pipeline) puts geometry through it: a shader, a vertex buffer, a pipeline and a draw call.