A 3D scene on the GPU: matrices, depth, and light#
The previous chapter handed the card positions already in clip space. This one hands it model coordinates and the matrices from foundations, and brings back, on the card, everything the software arc built by hand: the depth buffer, back-face culling, and the lighting model from shading, evaluated per pixel.
The examples use the vector and matrix subs from foundations unchanged: v_add, v_sub, v_scale, v_dot, v_cross, v_norm, m_mul, m_translate, m_scale, m_rotate_y, m_look_at, m_perspective and xform. The complete program at the end loads them from a file.
Matrices as uniforms#
A value that is the same for every vertex of a draw is a uniform. The shader declares it, and a uniform buffer supplies it:
@group(0) @binding(0) var<uniform> mvp: mat4x4<f32>;
@vertex fn vs(@location(0) p: vec3<f32>) -> @builtin(position) vec4<f32> {
return mvp * vec4<f32>(p, 1.0);
}
my $mvp = m_mul($proj, m_mul($view, $model));
my $ub = $gpu->uniform_buffer(pack 'f16', @$mvp);
$gpu->render_pass($img, clear => [0, 0, 0, 1])
->pipeline($pipe)->bind([$ub])->vertices($vb)->draw($n)->end;
bind takes a list of buffers for @group(0), in binding order: [$a, $b] supplies @binding(0) and @binding(1). It comes after pipeline, because the pipeline read the shader and knows what each binding has to be.
No transposing. Foundations stores matrices column by column, element m[col * 4 + row], and WGSL’s mat4x4<f32> is column-major too, so pack 'f16', @$m is already the layout the shader reads, and mvp * v in WGSL is xform($mvp, $v) in Perl.
That is checkable to the pixel. Five points drawn with topology => 'point-list', each lighting one pixel of an 800x600 target, against foundations’ own screen mapping of the same points, with the camera at [0, 0, 5]:
point Perl screen position pixel the card lit
[0,0,0] (400.00, 300.00) (399,299)
[1,0,-1] (486.60, 300.00) (486,299)
[1,0,-8] (439.97, 300.00) (439,299)
[-1.5,1,0] (244.12, 196.08) (244,196)
[0.7,-1.2,2] (521.24, 507.85) (521,507)
Every coordinate strictly inside a pixel matches. The three points whose predicted y is exactly 300.00 sit on the boundary between rows 299 and 300, and the card chose 299: a tie on a pixel edge, not an error. The first point is foundations’ own “dead centre of an 800x600 window”.
That check needed one more matrix than foundations used, for the reason below.
WebGPU’s depth range#
OpenGL, whose conventions foundations follows, keeps a point when -w <= z <= w in clip space, so normalised depth runs from -1 to +1. WebGPU keeps it when 0 <= z <= w: depth runs from 0 to 1. Everything foundations’ projection sends to negative depth, the card throws away.
Symptom: objects close to the camera vanish, well before they reach the near plane. Cause: an OpenGL-style projection matrix on WebGPU. With near = 0.1 and far = 100, a small square drawn at decreasing distances, its size on screen held constant:
distance foundations' ndc.z pixels drawn remapped ndc.z pixels drawn
5 0.962 100 0.981 100
0.5 0.602 100 0.801 100
0.21 0.049 100 0.524 100
0.19 -0.052 0 0.474 100
0.15 -0.333 0 0.334 100
0.12 -0.666 0 0.167 100
The cut-off is where ndc.z crosses zero, at 2 * far * near / (far + near), which is 0.1998: about twice the near distance. Geometry between the near plane and there is silently lost, and for a camera walking through a scene that is everything within arm’s reach.
Keep foundations’ projection and convert its output with one more matrix, which maps z to (z + w) / 2:
# OpenGL clip space keeps -w <= z <= w, WebGPU keeps 0 <= z <= w.
my $gl_to_wgpu = [1,0,0,0, 0,1,0,0, 0,0,0.5,0, 0,0,0.5,1];
my $proj = m_mul($gl_to_wgpu, m_perspective(60, 4/3, 0.1, 100));
Only depth changes. x, y and w are untouched, so every screen position in foundations still holds, as the point table above shows. Foundations’ example point at depth 5 goes from ndc.z = 0.962 to 0.981.
Changing uniforms#
$buf->write($bytes) replaces a buffer’s contents, and the next pass sees the new values. Writing once per frame, before the pass, is how a camera moves.
Symptom: several objects drawn in one pass all appear where the last one should be. Cause: one uniform buffer rewritten between the draws. The writes are applied before the pass runs, so every draw in it sees the last value. Two squares, one uniform, rewritten between two draws:
one buffer rewritten: left half 0 px, right half 144 px
one buffer per object: left half 144 px, right half 144 px
Give each object its own buffer and bind it before its draw, or use instancing, below, which is one draw for all of them.
Symptom: a uniform value is shifted, or reads as zero. Cause: WGSL aligns a vec3<f32> to 16 bytes, not 12. In struct U { a: vec3<f32>, b: vec3<f32> }, b starts at byte 16, so pack 'f6' puts b’s first component into padding:
packed tight pack 'f8', a, b, 0, 0 b reads as 0 0 0
padded pack 'f8', a, 0, b, 0 b reads as 255 0 0
A buffer smaller than the struct is refused at bind, with Binding size 24 of Buffer ... is less than minimum 32, but a misaligned one of the right size is accepted. The reliable rule is to use vec4<f32> in uniform structs and leave the fourth component unused, as every struct in this chapter does.
The depth buffer#
The rasteriser chapter kept the nearest depth per pixel and skipped anything further. On the card that is two settings, and both are needed:
my $pipe = $gpu->render_pipeline($shader, depth => 1, ...);
my $depth = $gpu->depth_buffer($W, $H);
$gpu->render_pass($target, clear => [0.05, 0.05, 0.08, 1], depth => $depth)
->pipeline($pipe) ... ->end;
depth => 1 on the pipeline makes it test and write depth; depth => $depth on the pass supplies the buffer. The pass clears it to 1.0, the far end of WebGPU’s range, and the test keeps a fragment only when it is nearer than what is there: the rasteriser’s next if $z >= $t->{depth}[$y][$x], in hardware.
Either half alone is refused at end, naming the mismatch:
Incompatible depth-stencil attachment format: the RenderPass uses a
texture with format None but the RenderPipeline with '' label uses an
attachment with format Some(Depth32Float)
The depth buffer must be the size of the target. After a resize, make a new one; the old one is refused with Attachments have differing sizes.
Nine cubes in three rows, rendered into a 320x240 texture, show what it buys. Drawn back row first, the scene comes out identical with the depth buffer and without it, because that order happens to be the painter’s algorithm done right. Drawn front row first:
front row first, depth buffer on identical to back row first
front row first, depth buffer off 3007 pixels differ
The far cubes paint over the near ones, the rasteriser chapter’s second figure in full colour. With the depth buffer the order of the draws stops mattering, which is the whole point.
A mesh: the cube#
A cube has 8 corners but needs 24 vertices here: each face has its own normal, and a vertex carries one normal, so a corner shared by three faces is three vertices. That is the flat-faces case from shading; a smooth surface shares its vertices instead.
Each face is described by its normal and two in-plane axes, chosen so that u x v is the normal. Then its corners, taken in the order below, wind counter-clockwise seen from outside, and the cross product checks it while building:
my @FACES = (
# normal u (right) v (up) u x v == normal
[[ 1, 0, 0], [ 0, 0,-1], [0, 1, 0]],
[[-1, 0, 0], [ 0, 0, 1], [0, 1, 0]],
[[ 0, 1, 0], [ 1, 0, 0], [0, 0,-1]],
[[ 0,-1, 0], [ 1, 0, 0], [0, 0, 1]],
[[ 0, 0, 1], [ 1, 0, 0], [0, 1, 0]],
[[ 0, 0,-1], [-1, 0, 0], [0, 1, 0]],
);
my (@vert, @idx);
for my $f (@FACES) {
my ($n, $u, $v) = @$f;
die "face winds the wrong way\n" unless v_dot(v_cross($u, $v), $n) > 0;
my $base = @vert / 6;
for my $c ([-1,-1], [1,-1], [1,1], [-1,1]) {
my $p = v_add(v_scale($n, 0.5),
v_add(v_scale($u, 0.5 * $c->[0]), v_scale($v, 0.5 * $c->[1])));
push @vert, @$p, @$n;
}
push @idx, map { $base + $_ } 0, 1, 2, 0, 2, 3;
}
# 24 vertices of six floats, 36 indices
my $verts = $gpu->vertex_buffer(pack 'f*', @vert);
my $index = $gpu->index_buffer(pack 'L*', @idx);
The vertex layout is ['float32x3', 'float32x3']: position at @location(0), normal at @location(1).
Back-face culling#
my $pipe = $gpu->render_pipeline($shader, cull => 'back', ...);
cull => 'back' discards every triangle whose front faces away. Which side is the front is front_face: ccw, the default, means counter-clockwise as the viewer sees it, which is how the cube above was built. This is the rasteriser chapter’s signed-area test, done by the card before any pixel work.
On the nine-cube scene:
cull => 'none' identical to cull => 'back'
cull => 'back', front_face => 'cw' 22508 of 76800 pixels differ
On a closed mesh behind a depth buffer, culling changes nothing visible: the back faces it removes would have lost the depth test anyway. That identity is the test the world generator applies with its --no-cull option. It is also why a wrong winding convention is so visible: with front_face => 'cw' the card keeps exactly the faces that point away, and each cube renders as its own dark interior.
Symptom: a model renders dark and inside out. Cause: its winding and the pipeline’s front_face disagree. The model may have been authored the other way round, or its model matrix may contain a reflection: the same scene with every cube mirrored by m_scale(-1, 1, 1) differed in the same 22508 pixels under cull => 'back', and front_face => 'cw' restored it exactly. A reflection reverses winding, the same fact the world generator’s y-flip ran into.
Instancing#
Nine cubes are the same 36 indices nine times with a different model matrix. Drawing them as nine draws works, but one draw can do it, with an instance count after the index count:
$gpu->render_pass($target, clear => [0.05, 0.05, 0.08, 1], depth => $depth)
->pipeline($pipe)->bind([$scene, $instances])
->vertices($verts)->indices($index)->draw_indexed(36, 9)->end;
The shader tells the instances apart by @builtin(instance_index), 0 to 8, and reads per-instance data from a storage buffer:
struct Instance {
model: mat4x4<f32>,
colour: vec4<f32>,
};
@group(0) @binding(1) var<storage, read> inst: array<Instance>;
@vertex
fn vs(@location(0) p: vec3<f32>, @location(1) n: vec3<f32>,
@builtin(instance_index) i: u32) -> VOut {
let m = inst[i].model;
...
}
my @inst;
for my $k (0 .. 8) {
my ($gx, $gz) = ($k % 3 - 1, int($k / 3) - 1);
my $model = m_mul(m_translate(1.8 * $gx, 0, 1.8 * $gz), m_rotate_y(0.35 * $k));
push @inst, @$model, @{ $COLOUR[$k % 3] }, 1;
}
my $instances = $gpu->storage_buffer(pack 'f*', @inst); # 9 x 80 bytes
Each instance is 20 floats, a matrix and a vec4 colour, 80 bytes, with no padding to think about because both members are multiples of 16 bytes. Moving the cubes is one write of 720 bytes per frame.
Declare the storage buffer read. A vertex shader may not write to storage, and read_write there is refused when the pipeline is built: Features ... (VERTEX_WRITABLE_STORAGE) are required but not enabled.
Lighting per fragment#
Shading ended on its rule: interpolate the inputs, not the answer. The vertex shader passes the fragment shader a world position and a normal, the card interpolates both, and the lighting runs once per pixel:
@fragment
fn fs(v: VOut) -> @location(0) vec4<f32> {
let n = normalize(v.normal);
let l = normalize(scene.light_dir.xyz);
let e = normalize(scene.eye.xyz - v.world);
let lambert = max(dot(n, l), 0.0);
let h = normalize(l + e);
let spec = pow(max(dot(n, h), 0.0), 32.0);
let c = v.colour * (0.1 + 0.9 * lambert) + vec3<f32>(0.4) * spec;
return vec4<f32>(min(c, vec3<f32>(1.0)), 1.0);
}
Line for line it is shading’s model: the Lambert term clamped at zero, the ambient 0.1 + 0.9 * lambert, the Blinn-Phong halfway vector with the dot product clamped before the power. light_dir points from the surface toward the light and e from the surface toward the eye, the conventions shading fixed.
The first normalize is not optional. Interpolating between two unit vectors gives a vector shorter than 1 in between, and an unnormalised normal scales the lighting silently.
To check the arithmetic rather than the picture, render into a floating-point target and read the numbers themselves. A pipeline and texture with format => 'rgba32float' store four f32 per pixel, and unpack 'f4' reads them. A fragment shader writing vec4<f32>(lambert, spec, 0.0, 1.0) for a surface facing the viewer reproduces shading’s tables:
light off the normal lambert shininess specular Perl, double
0 deg 1.000 1 0.965926 0.965926
30 deg 0.866 8 0.757793 0.757792
60 deg 0.500 32 0.329763 0.329762
90 deg 0.000 128 0.011825 0.011825
180 deg 0.000
The specular column has the light 30 degrees off and the viewer straight on, so the halfway vector is 15 degrees off the normal. The card and Perl’s cos(15 deg) ** $shininess agree to the sixth decimal, and differ there because WGSL computes in f32.
Normals and the model matrix#
The vertex shader turns the normal with (m * vec4<f32>(n, 0.0)).xyz. The 0.0 makes it a direction: the translation column is multiplied by zero, so moving an object does not bend its normals.
That is correct for rotation, translation and uniform scale, which is all this chapter uses. Under a non-uniform scale it is wrong. A face with normal (1, 1, 0) and in-plane direction (1, -1, 0), scaled by (1, 3, 1): the in-plane direction becomes (1, -3, 0) and the transformed normal (1, 3, 0), and their dot product is -8, so the “normal” no longer stands at right angles to its own surface. Turned by the inverse transpose of the scale instead, the normal is (1, 1/3, 0), and the dot product is 0. A model matrix with non-uniform scale needs that inverse-transpose matrix passed alongside it.
Colours in linear light#
The window’s surface is an -srgb format, as the device chapter measured, so what the fragment shader writes is encoded on the way to the screen. The lighting above is therefore computed in linear light, which is correct, and the base colours, picked by eye, are decoded before upload:
my @COLOUR = map { [ map { $_ ** 2.2 } @$_ ] } # picked by eye: decode
[0.9, 0.3, 0.2], [0.3, 0.8, 0.3], [0.3, 0.4, 0.9];
Left encoded, the same scene renders pale and washed out. The headless renders in this chapter use an rgba8unorm-srgb texture for the same reason, so the image read back is the image the window shows.
The whole scene#
Nine lit cubes turning on a grid, in a resizable window, with the fixed step from the device chapter. The subs from foundations are in foundations.pl beside it, followed by a line 1;.
use strict;
use warnings;
use SDL2 qw(:all);
use Peta::WebGPU;
BEGIN { require './foundations.pl' } # the v_* and m_* subs, then 1;
use constant { STEP => 16, MAX_STEPS => 5 };
my $max_frames = shift // 0;
my $WGSL = <<'WGSL';
struct Scene {
view_proj: mat4x4<f32>,
light_dir: vec4<f32>, // xyz: toward the light, world space
eye: vec4<f32>, // xyz: camera position, world space
};
struct Instance {
model: mat4x4<f32>,
colour: vec4<f32>,
};
@group(0) @binding(0) var<uniform> scene: Scene;
@group(0) @binding(1) var<storage, read> inst: array<Instance>;
struct VOut {
@builtin(position) pos: vec4<f32>,
@location(0) world: vec3<f32>,
@location(1) normal: vec3<f32>,
@location(2) colour: vec3<f32>,
};
@vertex
fn vs(@location(0) p: vec3<f32>, @location(1) n: vec3<f32>,
@builtin(instance_index) i: u32) -> VOut {
let m = inst[i].model;
let w = m * vec4<f32>(p, 1.0);
var o: VOut;
o.pos = scene.view_proj * w;
o.world = w.xyz;
o.normal = (m * vec4<f32>(n, 0.0)).xyz;
o.colour = inst[i].colour.rgb;
return o;
}
@fragment
fn fs(v: VOut) -> @location(0) vec4<f32> {
let n = normalize(v.normal);
let l = normalize(scene.light_dir.xyz);
let e = normalize(scene.eye.xyz - v.world);
let lambert = max(dot(n, l), 0.0);
let h = normalize(l + e);
let spec = pow(max(dot(n, h), 0.0), 32.0);
let c = v.colour * (0.1 + 0.9 * lambert) + vec3<f32>(0.4) * spec;
return vec4<f32>(min(c, vec3<f32>(1.0)), 1.0);
}
WGSL
my @FACES = (
# normal u (right) v (up) u x v == normal
[[ 1, 0, 0], [ 0, 0,-1], [0, 1, 0]],
[[-1, 0, 0], [ 0, 0, 1], [0, 1, 0]],
[[ 0, 1, 0], [ 1, 0, 0], [0, 0,-1]],
[[ 0,-1, 0], [ 1, 0, 0], [0, 0, 1]],
[[ 0, 0, 1], [ 1, 0, 0], [0, 1, 0]],
[[ 0, 0,-1], [-1, 0, 0], [0, 1, 0]],
);
my (@vert, @idx);
for my $f (@FACES) {
my ($n, $u, $v) = @$f;
die "face winds the wrong way\n" unless v_dot(v_cross($u, $v), $n) > 0;
my $base = @vert / 6;
for my $c ([-1,-1], [1,-1], [1,1], [-1,1]) {
my $p = v_add(v_scale($n, 0.5),
v_add(v_scale($u, 0.5 * $c->[0]), v_scale($v, 0.5 * $c->[1])));
push @vert, @$p, @$n;
}
push @idx, map { $base + $_ } 0, 1, 2, 0, 2, 3;
}
my $win = SDL2::Window->new(title => 'cubes', w => 800, h => 600, resizable => 1);
my $gpu = Peta::WebGPU::Device->new(window => $win);
my $surf = $gpu->surface;
my ($W, $H) = $surf->size;
my $pipe = $gpu->render_pipeline($gpu->shader($WGSL),
format => $surf->format, depth => 1, cull => 'back',
vertex_buffers => [ { attributes => ['float32x3', 'float32x3'] } ]);
my $verts = $gpu->vertex_buffer(pack 'f*', @vert);
my $index = $gpu->index_buffer(pack 'L*', @idx);
my $depth = $gpu->depth_buffer($W, $H);
my @COLOUR = map { [ map { $_ ** 2.2 } @$_ ] } # picked by eye: decode
[0.9, 0.3, 0.2], [0.3, 0.8, 0.3], [0.3, 0.4, 0.9];
my $instances = $gpu->storage_buffer(9 * 20 * 4);
my $scene = $gpu->uniform_buffer(24 * 4);
my $gl_to_wgpu = [1,0,0,0, 0,1,0,0, 0,0,0.5,0, 0,0,0.5,1];
my $angle = 0;
sub update { $angle += 0.0015 * STEP }
sub upload {
my @inst;
for my $k (0 .. 8) {
my ($gx, $gz) = ($k % 3 - 1, int($k / 3) - 1);
my $model = m_mul(m_translate(1.8 * $gx, 0, 1.8 * $gz),
m_rotate_y($angle + 0.35 * $k));
push @inst, @$model, @{ $COLOUR[$k % 3] }, 1;
}
$instances->write(pack 'f*', @inst);
my $eye = [0, 4, 6];
my $proj = m_mul($gl_to_wgpu, m_perspective(45, $W / $H, 0.1, 100));
my $vp = m_mul($proj, m_look_at($eye, [0, 0, 0], [0, 1, 0]));
$scene->write(pack 'f*', @$vp, @{ v_norm([0.6, 1.0, 0.8]) }, 0, @$eye, 1);
}
sub draw {
my ($target) = @_;
$gpu->render_pass($target, clear => [0.05, 0.05, 0.08, 1], depth => $depth)
->pipeline($pipe)->bind([$scene, $instances])
->vertices($verts)->indices($index)->draw_indexed(scalar @idx, 9)->end;
}
my ($last, $acc, $frames, $running) = (SDL_GetTicks(), 0, 0, 1);
while ($running) {
while (my $ev = SDL_PollEvent()) {
if ($ev->{type} == SDL_QUIT) { $running = 0 }
elsif ($ev->{type} == SDL_KEYDOWN && $ev->{sym} == SDLK_ESCAPE) { $running = 0 }
elsif ($ev->{type} == SDL_WINDOWEVENT
&& $ev->{event} == SDL_WINDOWEVENT_SIZE_CHANGED) {
($W, $H) = @$ev{qw(data1 data2)};
$surf->configure($W, $H);
$depth = $gpu->depth_buffer($W, $H); # must match the frame
}
}
my $now = SDL_GetTicks();
$acc += $now - $last;
$last = $now;
my $steps = 0;
while ($acc >= STEP) {
update();
$acc -= STEP;
if (++$steps >= MAX_STEPS) { $acc = 0; last }
}
my $frame = $surf->next_frame or next;
upload();
draw($frame);
$frame->present;
$running = 0 if $max_frames && ++$frames >= $max_frames;
}
printf "%d frames at %dx%d\n", $frames, $W, $H;
draw takes its target as an argument, so the same routine renders into a texture for checking. A texture with format => $surf->format and the same size as the depth buffer is all it needs; with bgra8unorm-srgb, remember that read returns B, G, R, A. Run in a window that was resized from 800x600 to 640x480 while it ran, the depth buffer was recreated at the new size and the frames carried on.
What this arc covers#
The card receives meshes as vertex and index buffers, transforms them with foundations’ matrices, resolves visibility with a depth buffer, culls back faces, draws many objects in one call, and lights every pixel with shading’s model. Every stage of the software arc has its hardware counterpart, and each was checked against the software version: the same 54 pixels, the same screen positions, the same lighting numbers.
The capstones put it to work on complete programs: an interactive modeller, which adds the orbit camera and picking from scene graphs, and a small 3D game.