# A 3D scene on the GPU: matrices, depth, and light The [previous chapter](gpu-pipeline) handed the card positions already in clip space. This one hands it model coordinates and the matrices from [foundations](foundations), and brings back, on the card, everything the software arc built by hand: the [depth buffer](software-rasteriser), back-face culling, and the lighting model from [shading](shading), evaluated per pixel. The examples use the vector and matrix subs from foundations unchanged: `v_add`, `v_sub`, `v_scale`, `v_dot`, `v_cross`, `v_norm`, `m_mul`, `m_translate`, `m_scale`, `m_rotate_y`, `m_look_at`, `m_perspective` and `xform`. The complete program at the end loads them from a file. ```{index} single: uniform single: uniform buffer single: column-major order; WGSL ``` ## Matrices as uniforms A value that is the same for every vertex of a draw is a **uniform**. The shader declares it, and a uniform buffer supplies it: ```wgsl @group(0) @binding(0) var mvp: mat4x4; @vertex fn vs(@location(0) p: vec3) -> @builtin(position) vec4 { return mvp * vec4(p, 1.0); } ``` ```perl my $mvp = m_mul($proj, m_mul($view, $model)); my $ub = $gpu->uniform_buffer(pack 'f16', @$mvp); $gpu->render_pass($img, clear => [0, 0, 0, 1]) ->pipeline($pipe)->bind([$ub])->vertices($vb)->draw($n)->end; ``` `bind` takes a list of buffers for `@group(0)`, in binding order: `[$a, $b]` supplies `@binding(0)` and `@binding(1)`. It comes after `pipeline`, because the pipeline read the shader and knows what each binding has to be. **No transposing.** Foundations stores matrices column by column, element `m[col * 4 + row]`, and WGSL's `mat4x4` is column-major too, so `pack 'f16', @$m` is already the layout the shader reads, and `mvp * v` in WGSL is `xform($mvp, $v)` in Perl. That is checkable to the pixel. Five points drawn with `topology => 'point-list'`, each lighting one pixel of an 800x600 target, against foundations' own screen mapping of the same points, with the camera at `[0, 0, 5]`: ```text point Perl screen position pixel the card lit [0,0,0] (400.00, 300.00) (399,299) [1,0,-1] (486.60, 300.00) (486,299) [1,0,-8] (439.97, 300.00) (439,299) [-1.5,1,0] (244.12, 196.08) (244,196) [0.7,-1.2,2] (521.24, 507.85) (521,507) ``` Every coordinate strictly inside a pixel matches. The three points whose predicted y is exactly 300.00 sit on the boundary between rows 299 and 300, and the card chose 299: a tie on a pixel edge, not an error. The first point is foundations' own "dead centre of an 800x600 window". That check needed one more matrix than foundations used, for the reason below. ```{index} single: depth range single: clip space; WebGPU versus OpenGL single: projection matrix; WebGPU depth range ``` ## WebGPU's depth range OpenGL, whose conventions foundations follows, keeps a point when `-w <= z <= w` in clip space, so normalised depth runs from -1 to +1. WebGPU keeps it when `0 <= z <= w`: depth runs from 0 to 1. Everything foundations' projection sends to negative depth, the card throws away. **Symptom: objects close to the camera vanish, well before they reach the near plane.** Cause: an OpenGL-style projection matrix on WebGPU. With `near = 0.1` and `far = 100`, a small square drawn at decreasing distances, its size on screen held constant: ```text distance foundations' ndc.z pixels drawn remapped ndc.z pixels drawn 5 0.962 100 0.981 100 0.5 0.602 100 0.801 100 0.21 0.049 100 0.524 100 0.19 -0.052 0 0.474 100 0.15 -0.333 0 0.334 100 0.12 -0.666 0 0.167 100 ``` The cut-off is where `ndc.z` crosses zero, at `2 * far * near / (far + near)`, which is 0.1998: about **twice** the near distance. Geometry between the near plane and there is silently lost, and for a camera walking through a scene that is everything within arm's reach. Keep foundations' projection and convert its output with one more matrix, which maps `z` to `(z + w) / 2`: ```perl # OpenGL clip space keeps -w <= z <= w, WebGPU keeps 0 <= z <= w. my $gl_to_wgpu = [1,0,0,0, 0,1,0,0, 0,0,0.5,0, 0,0,0.5,1]; my $proj = m_mul($gl_to_wgpu, m_perspective(60, 4/3, 0.1, 100)); ``` Only depth changes. `x`, `y` and `w` are untouched, so every screen position in foundations still holds, as the point table above shows. Foundations' example point at depth 5 goes from `ndc.z = 0.962` to `0.981`. ```{index} single: uniform buffer; updating single: uniform buffer; alignment ``` ## Changing uniforms `$buf->write($bytes)` replaces a buffer's contents, and the next pass sees the new values. Writing once per frame, before the pass, is how a camera moves. **Symptom: several objects drawn in one pass all appear where the last one should be.** Cause: one uniform buffer rewritten between the draws. The writes are applied before the pass runs, so every draw in it sees the last value. Two squares, one uniform, rewritten between two draws: ```text one buffer rewritten: left half 0 px, right half 144 px one buffer per object: left half 144 px, right half 144 px ``` Give each object its own buffer and `bind` it before its draw, or use instancing, below, which is one draw for all of them. **Symptom: a uniform value is shifted, or reads as zero.** Cause: WGSL aligns a `vec3` to 16 bytes, not 12. In `struct U { a: vec3, b: vec3 }`, `b` starts at byte 16, so `pack 'f6'` puts `b`'s first component into padding: ```text packed tight pack 'f8', a, b, 0, 0 b reads as 0 0 0 padded pack 'f8', a, 0, b, 0 b reads as 255 0 0 ``` A buffer smaller than the struct is refused at `bind`, with `Binding size 24 of Buffer ... is less than minimum 32`, but a misaligned one of the right size is accepted. The reliable rule is to use `vec4` in uniform structs and leave the fourth component unused, as every struct in this chapter does. ```{index} single: depth buffer; on the GPU ``` ## The depth buffer The [rasteriser chapter](software-rasteriser) kept the nearest depth per pixel and skipped anything further. On the card that is two settings, and both are needed: ```perl my $pipe = $gpu->render_pipeline($shader, depth => 1, ...); my $depth = $gpu->depth_buffer($W, $H); $gpu->render_pass($target, clear => [0.05, 0.05, 0.08, 1], depth => $depth) ->pipeline($pipe) ... ->end; ``` `depth => 1` on the pipeline makes it test and write depth; `depth => $depth` on the pass supplies the buffer. The pass clears it to 1.0, the far end of WebGPU's range, and the test keeps a fragment only when it is nearer than what is there: the rasteriser's `next if $z >= $t->{depth}[$y][$x]`, in hardware. Either half alone is refused at `end`, naming the mismatch: ```text Incompatible depth-stencil attachment format: the RenderPass uses a texture with format None but the RenderPipeline with '' label uses an attachment with format Some(Depth32Float) ``` The depth buffer must be the size of the target. After a resize, make a new one; the old one is refused with `Attachments have differing sizes`. Nine cubes in three rows, rendered into a 320x240 texture, show what it buys. Drawn back row first, the scene comes out identical with the depth buffer and without it, because that order happens to be the painter's algorithm done right. Drawn front row first: ```text front row first, depth buffer on identical to back row first front row first, depth buffer off 3007 pixels differ ``` The far cubes paint over the near ones, the [rasteriser chapter's](software-rasteriser) second figure in full colour. With the depth buffer the order of the draws stops mattering, which is the whole point. ## A mesh: the cube A cube has 8 corners but needs 24 vertices here: each face has its own normal, and a vertex carries one normal, so a corner shared by three faces is three vertices. That is the flat-faces case from [shading](shading); a smooth surface shares its vertices instead. Each face is described by its normal and two in-plane axes, chosen so that `u x v` is the normal. Then its corners, taken in the order below, wind counter-clockwise seen from outside, and the cross product checks it while building: ```perl my @FACES = ( # normal u (right) v (up) u x v == normal [[ 1, 0, 0], [ 0, 0,-1], [0, 1, 0]], [[-1, 0, 0], [ 0, 0, 1], [0, 1, 0]], [[ 0, 1, 0], [ 1, 0, 0], [0, 0,-1]], [[ 0,-1, 0], [ 1, 0, 0], [0, 0, 1]], [[ 0, 0, 1], [ 1, 0, 0], [0, 1, 0]], [[ 0, 0,-1], [-1, 0, 0], [0, 1, 0]], ); my (@vert, @idx); for my $f (@FACES) { my ($n, $u, $v) = @$f; die "face winds the wrong way\n" unless v_dot(v_cross($u, $v), $n) > 0; my $base = @vert / 6; for my $c ([-1,-1], [1,-1], [1,1], [-1,1]) { my $p = v_add(v_scale($n, 0.5), v_add(v_scale($u, 0.5 * $c->[0]), v_scale($v, 0.5 * $c->[1]))); push @vert, @$p, @$n; } push @idx, map { $base + $_ } 0, 1, 2, 0, 2, 3; } # 24 vertices of six floats, 36 indices my $verts = $gpu->vertex_buffer(pack 'f*', @vert); my $index = $gpu->index_buffer(pack 'L*', @idx); ``` The vertex layout is `['float32x3', 'float32x3']`: position at `@location(0)`, normal at `@location(1)`. ```{index} single: back-face culling; on the GPU single: front face single: winding order; reversed by reflection ``` ## Back-face culling ```perl my $pipe = $gpu->render_pipeline($shader, cull => 'back', ...); ``` `cull => 'back'` discards every triangle whose front faces away. Which side is the front is `front_face`: `ccw`, the default, means counter-clockwise as the viewer sees it, which is how the cube above was built. This is the rasteriser chapter's signed-area test, done by the card before any pixel work. On the nine-cube scene: ```text cull => 'none' identical to cull => 'back' cull => 'back', front_face => 'cw' 22508 of 76800 pixels differ ``` On a closed mesh behind a depth buffer, culling changes nothing visible: the back faces it removes would have lost the depth test anyway. That identity is the test the [world generator](capstone-world-generator) applies with its `--no-cull` option. It is also why a wrong winding convention is so visible: with `front_face => 'cw'` the card keeps exactly the faces that point away, and each cube renders as its own dark interior. **Symptom: a model renders dark and inside out.** Cause: its winding and the pipeline's `front_face` disagree. The model may have been authored the other way round, or its model matrix may contain a reflection: the same scene with every cube mirrored by `m_scale(-1, 1, 1)` differed in the same 22508 pixels under `cull => 'back'`, and `front_face => 'cw'` restored it exactly. A reflection reverses winding, the same fact the world generator's y-flip ran into. ```{index} single: instancing single: storage buffer ``` ## Instancing Nine cubes are the same 36 indices nine times with a different model matrix. Drawing them as nine draws works, but one draw can do it, with an instance count after the index count: ```perl $gpu->render_pass($target, clear => [0.05, 0.05, 0.08, 1], depth => $depth) ->pipeline($pipe)->bind([$scene, $instances]) ->vertices($verts)->indices($index)->draw_indexed(36, 9)->end; ``` The shader tells the instances apart by `@builtin(instance_index)`, 0 to 8, and reads per-instance data from a storage buffer: ```wgsl struct Instance { model: mat4x4, colour: vec4, }; @group(0) @binding(1) var inst: array; @vertex fn vs(@location(0) p: vec3, @location(1) n: vec3, @builtin(instance_index) i: u32) -> VOut { let m = inst[i].model; ... } ``` ```perl my @inst; for my $k (0 .. 8) { my ($gx, $gz) = ($k % 3 - 1, int($k / 3) - 1); my $model = m_mul(m_translate(1.8 * $gx, 0, 1.8 * $gz), m_rotate_y(0.35 * $k)); push @inst, @$model, @{ $COLOUR[$k % 3] }, 1; } my $instances = $gpu->storage_buffer(pack 'f*', @inst); # 9 x 80 bytes ``` Each instance is 20 floats, a matrix and a `vec4` colour, 80 bytes, with no padding to think about because both members are multiples of 16 bytes. Moving the cubes is one `write` of 720 bytes per frame. Declare the storage buffer `read`. A vertex shader may not write to storage, and `read_write` there is refused when the pipeline is built: `Features ... (VERTEX_WRITABLE_STORAGE) are required but not enabled`. ```{index} single: per-pixel shading; on the GPU single: Blinn-Phong; in WGSL single: floating-point render target ``` ## Lighting per fragment [Shading](shading) ended on its rule: interpolate the inputs, not the answer. The vertex shader passes the fragment shader a world position and a normal, the card interpolates both, and the lighting runs once per pixel: ```wgsl @fragment fn fs(v: VOut) -> @location(0) vec4 { let n = normalize(v.normal); let l = normalize(scene.light_dir.xyz); let e = normalize(scene.eye.xyz - v.world); let lambert = max(dot(n, l), 0.0); let h = normalize(l + e); let spec = pow(max(dot(n, h), 0.0), 32.0); let c = v.colour * (0.1 + 0.9 * lambert) + vec3(0.4) * spec; return vec4(min(c, vec3(1.0)), 1.0); } ``` Line for line it is shading's model: the Lambert term clamped at zero, the ambient `0.1 + 0.9 * lambert`, the Blinn-Phong halfway vector with the dot product clamped before the power. `light_dir` points from the surface toward the light and `e` from the surface toward the eye, the conventions shading fixed. **The first `normalize` is not optional.** Interpolating between two unit vectors gives a vector shorter than 1 in between, and an unnormalised normal scales the lighting silently. To check the arithmetic rather than the picture, render into a floating-point target and read the numbers themselves. A pipeline and texture with `format => 'rgba32float'` store four `f32` per pixel, and `unpack 'f4'` reads them. A fragment shader writing `vec4(lambert, spec, 0.0, 1.0)` for a surface facing the viewer reproduces shading's tables: ```text light off the normal lambert shininess specular Perl, double 0 deg 1.000 1 0.965926 0.965926 30 deg 0.866 8 0.757793 0.757792 60 deg 0.500 32 0.329763 0.329762 90 deg 0.000 128 0.011825 0.011825 180 deg 0.000 ``` The specular column has the light 30 degrees off and the viewer straight on, so the halfway vector is 15 degrees off the normal. The card and Perl's `cos(15 deg) ** $shininess` agree to the sixth decimal, and differ there because WGSL computes in `f32`. ```{index} single: normal; non-uniform scale single: inverse transpose matrix ``` ### Normals and the model matrix The vertex shader turns the normal with `(m * vec4(n, 0.0)).xyz`. The `0.0` makes it a direction: the translation column is multiplied by zero, so moving an object does not bend its normals. That is correct for rotation, translation and uniform scale, which is all this chapter uses. Under a **non-uniform** scale it is wrong. A face with normal `(1, 1, 0)` and in-plane direction `(1, -1, 0)`, scaled by `(1, 3, 1)`: the in-plane direction becomes `(1, -3, 0)` and the transformed normal `(1, 3, 0)`, and their dot product is -8, so the "normal" no longer stands at right angles to its own surface. Turned by the inverse transpose of the scale instead, the normal is `(1, 1/3, 0)`, and the dot product is 0. A model matrix with non-uniform scale needs that inverse-transpose matrix passed alongside it. ```{index} single: linear light; colours picked by eye ``` ### Colours in linear light The window's surface is an `-srgb` format, as the [device chapter](gpu-device-and-window) measured, so what the fragment shader writes is encoded on the way to the screen. The lighting above is therefore computed in linear light, which is correct, and the base colours, picked by eye, are decoded before upload: ```perl my @COLOUR = map { [ map { $_ ** 2.2 } @$_ ] } # picked by eye: decode [0.9, 0.3, 0.2], [0.3, 0.8, 0.3], [0.3, 0.4, 0.9]; ``` Left encoded, the same scene renders pale and washed out. The headless renders in this chapter use an `rgba8unorm-srgb` texture for the same reason, so the image read back is the image the window shows. ## The whole scene Nine lit cubes turning on a grid, in a resizable window, with the fixed step from the [device chapter](gpu-device-and-window). The subs from foundations are in `foundations.pl` beside it, followed by a line `1;`. ```perl use strict; use warnings; use SDL2 qw(:all); use Peta::WebGPU; BEGIN { require './foundations.pl' } # the v_* and m_* subs, then 1; use constant { STEP => 16, MAX_STEPS => 5 }; my $max_frames = shift // 0; my $WGSL = <<'WGSL'; struct Scene { view_proj: mat4x4, light_dir: vec4, // xyz: toward the light, world space eye: vec4, // xyz: camera position, world space }; struct Instance { model: mat4x4, colour: vec4, }; @group(0) @binding(0) var scene: Scene; @group(0) @binding(1) var inst: array; struct VOut { @builtin(position) pos: vec4, @location(0) world: vec3, @location(1) normal: vec3, @location(2) colour: vec3, }; @vertex fn vs(@location(0) p: vec3, @location(1) n: vec3, @builtin(instance_index) i: u32) -> VOut { let m = inst[i].model; let w = m * vec4(p, 1.0); var o: VOut; o.pos = scene.view_proj * w; o.world = w.xyz; o.normal = (m * vec4(n, 0.0)).xyz; o.colour = inst[i].colour.rgb; return o; } @fragment fn fs(v: VOut) -> @location(0) vec4 { let n = normalize(v.normal); let l = normalize(scene.light_dir.xyz); let e = normalize(scene.eye.xyz - v.world); let lambert = max(dot(n, l), 0.0); let h = normalize(l + e); let spec = pow(max(dot(n, h), 0.0), 32.0); let c = v.colour * (0.1 + 0.9 * lambert) + vec3(0.4) * spec; return vec4(min(c, vec3(1.0)), 1.0); } WGSL my @FACES = ( # normal u (right) v (up) u x v == normal [[ 1, 0, 0], [ 0, 0,-1], [0, 1, 0]], [[-1, 0, 0], [ 0, 0, 1], [0, 1, 0]], [[ 0, 1, 0], [ 1, 0, 0], [0, 0,-1]], [[ 0,-1, 0], [ 1, 0, 0], [0, 0, 1]], [[ 0, 0, 1], [ 1, 0, 0], [0, 1, 0]], [[ 0, 0,-1], [-1, 0, 0], [0, 1, 0]], ); my (@vert, @idx); for my $f (@FACES) { my ($n, $u, $v) = @$f; die "face winds the wrong way\n" unless v_dot(v_cross($u, $v), $n) > 0; my $base = @vert / 6; for my $c ([-1,-1], [1,-1], [1,1], [-1,1]) { my $p = v_add(v_scale($n, 0.5), v_add(v_scale($u, 0.5 * $c->[0]), v_scale($v, 0.5 * $c->[1]))); push @vert, @$p, @$n; } push @idx, map { $base + $_ } 0, 1, 2, 0, 2, 3; } my $win = SDL2::Window->new(title => 'cubes', w => 800, h => 600, resizable => 1); my $gpu = Peta::WebGPU::Device->new(window => $win); my $surf = $gpu->surface; my ($W, $H) = $surf->size; my $pipe = $gpu->render_pipeline($gpu->shader($WGSL), format => $surf->format, depth => 1, cull => 'back', vertex_buffers => [ { attributes => ['float32x3', 'float32x3'] } ]); my $verts = $gpu->vertex_buffer(pack 'f*', @vert); my $index = $gpu->index_buffer(pack 'L*', @idx); my $depth = $gpu->depth_buffer($W, $H); my @COLOUR = map { [ map { $_ ** 2.2 } @$_ ] } # picked by eye: decode [0.9, 0.3, 0.2], [0.3, 0.8, 0.3], [0.3, 0.4, 0.9]; my $instances = $gpu->storage_buffer(9 * 20 * 4); my $scene = $gpu->uniform_buffer(24 * 4); my $gl_to_wgpu = [1,0,0,0, 0,1,0,0, 0,0,0.5,0, 0,0,0.5,1]; my $angle = 0; sub update { $angle += 0.0015 * STEP } sub upload { my @inst; for my $k (0 .. 8) { my ($gx, $gz) = ($k % 3 - 1, int($k / 3) - 1); my $model = m_mul(m_translate(1.8 * $gx, 0, 1.8 * $gz), m_rotate_y($angle + 0.35 * $k)); push @inst, @$model, @{ $COLOUR[$k % 3] }, 1; } $instances->write(pack 'f*', @inst); my $eye = [0, 4, 6]; my $proj = m_mul($gl_to_wgpu, m_perspective(45, $W / $H, 0.1, 100)); my $vp = m_mul($proj, m_look_at($eye, [0, 0, 0], [0, 1, 0])); $scene->write(pack 'f*', @$vp, @{ v_norm([0.6, 1.0, 0.8]) }, 0, @$eye, 1); } sub draw { my ($target) = @_; $gpu->render_pass($target, clear => [0.05, 0.05, 0.08, 1], depth => $depth) ->pipeline($pipe)->bind([$scene, $instances]) ->vertices($verts)->indices($index)->draw_indexed(scalar @idx, 9)->end; } my ($last, $acc, $frames, $running) = (SDL_GetTicks(), 0, 0, 1); while ($running) { while (my $ev = SDL_PollEvent()) { if ($ev->{type} == SDL_QUIT) { $running = 0 } elsif ($ev->{type} == SDL_KEYDOWN && $ev->{sym} == SDLK_ESCAPE) { $running = 0 } elsif ($ev->{type} == SDL_WINDOWEVENT && $ev->{event} == SDL_WINDOWEVENT_SIZE_CHANGED) { ($W, $H) = @$ev{qw(data1 data2)}; $surf->configure($W, $H); $depth = $gpu->depth_buffer($W, $H); # must match the frame } } my $now = SDL_GetTicks(); $acc += $now - $last; $last = $now; my $steps = 0; while ($acc >= STEP) { update(); $acc -= STEP; if (++$steps >= MAX_STEPS) { $acc = 0; last } } my $frame = $surf->next_frame or next; upload(); draw($frame); $frame->present; $running = 0 if $max_frames && ++$frames >= $max_frames; } printf "%d frames at %dx%d\n", $frames, $W, $H; ``` `draw` takes its target as an argument, so the same routine renders into a texture for checking. A texture with `format => $surf->format` and the same size as the depth buffer is all it needs; with `bgra8unorm-srgb`, remember that `read` returns B, G, R, A. Run in a window that was resized from 800x600 to 640x480 while it ran, the depth buffer was recreated at the new size and the frames carried on. ## What this arc covers The card receives meshes as vertex and index buffers, transforms them with foundations' matrices, resolves visibility with a depth buffer, culls back faces, draws many objects in one call, and lights every pixel with shading's model. Every stage of the software arc has its hardware counterpart, and each was checked against the software version: the same 54 pixels, the same screen positions, the same lighting numbers. The capstones put it to work on complete programs: an [interactive modeller](capstone-modeller), which adds the orbit camera and picking from [scene graphs](scene-graphs), and a [small 3D game](capstone-3d-game).