Tutorial 3 — Deferred Rendering

Prerequisites

Make sure you have completed Tutorial 2 — Sample Object.

Setting up the project

In MTMetalTutorialsApp.swift set MT3ContentView() and run:

@main
struct MetalTutorialsApp: App {
    var body: some Scene {
        WindowGroup {
            MT3ContentView()
        }
    }
}

New concepts in this tutorial

In Tutorial 2 we shaded each fragment immediately as the geometry was rasterized — this is forward rendering. The problem: every fragment runs the full lighting computation, even fragments that will later be overdrawn by closer geometry.

Deferred rendering splits the work into two passes:

1. Geometry pass (GBuffer pass) — rasterize the scene and write surface data (albedo, normal, position) into off-screen textures called the GBuffer.
2. Lighting pass — draw a full-screen quad and compute lighting for each pixel once, reading from the GBuffer textures.

This decouples the cost of geometry from the cost of lighting.

flowchart LR
    Scene["Scene\n(meshes)"] --> GP["Geometry Pass\ngbuffer_fragment"]
    GP --> A["albedoSpecular\nbgra8Unorm"]
    GP --> N["normal\nrgba16Float"]
    GP --> P["position\nrgba16Float"]
    A & N & P --> LP["Lighting Pass\ndeferred_lighting_fragment"]
    LP --> D["Drawable\n(screen)"]

Files

The GBuffer

The GBuffer is a set of textures that store per-pixel surface properties. Tutorial 3 uses three:

They are created (and recreated on resize) in mtkView(_:drawableSizeWillChange:):

let albedoDesc = MTLTextureDescriptor
    .texture2DDescriptor(pixelFormat: .bgra8Unorm,
                         width: Int(size.width), height: Int(size.height),
                         mipmapped: false)
albedoDesc.usage       = [.shaderRead, .renderTarget]   // written as RT, read as texture
albedoDesc.storageMode = .private                       // GPU-only memory

let gBufferDesc = MTLTextureDescriptor
    .texture2DDescriptor(pixelFormat: .rgba16Float, …)
gBufferDesc.usage       = [.shaderRead, .renderTarget]
gBufferDesc.storageMode = .private

_gBuffer = GBuffer(
    albedoSpecular: device.makeTexture(descriptor: albedoDesc)!,
    normal:         device.makeTexture(descriptor: gBufferDesc)!,
    position:       device.makeTexture(descriptor: gBufferDesc)!)

The render target indices are shared between Swift and Metal via a C enum:

// MT3GBuffer.h
typedef enum MT3RenderTargetIndices {
    MT3RenderTargetAlbedo   = 1,
    MT3RenderTargetNormal   = 2,
    MT3RenderTargetPosition = 3,
} MT3RenderTargetIndices;

The pipelines

Two separate pipelines are built whenever the size changes.

GBuffer pipeline

Writes to three color attachments simultaneously (MRT):

_gBuffPipelineState = _buildPipeline(
    vertexFunction:   "MT3::vertex_main",
    fragmentFunction: "MT3::gbuffer_fragment",
    label: "GBufferPSO"
) { descriptor in
    descriptor.colorAttachments[MT3RenderTargetAlbedo].pixelFormat   = .bgra8Unorm
    descriptor.colorAttachments[MT3RenderTargetNormal].pixelFormat   = .rgba16Float
    descriptor.colorAttachments[MT3RenderTargetPosition].pixelFormat = .rgba16Float
    descriptor.depthAttachmentPixelFormat = metalView.depthStencilPixelFormat
}

Lighting (display) pipeline

Reads from the GBuffer and writes to the final drawable:

_displayPipelineState = _buildPipeline(
    vertexFunction:   "MT3::display_vertex",
    fragmentFunction: "MT3::deferred_lighting_fragment",
    label: "DeferredLightingPSO"
) { descriptor in
    descriptor.colorAttachments[0].pixelFormat   = metalView.colorPixelFormat
    descriptor.depthAttachmentPixelFormat        = metalView.depthStencilPixelFormat
}

The render function

Both passes share a single MTLCommandBuffer within draw(in:):

let commandBuffer = _commandQueue.makeCommandBuffer()!

Pass 1 — GBuffer

let gBufferPassDescriptor: MTLRenderPassDescriptor = {
    let desc = MTLRenderPassDescriptor()
    desc.colorAttachments[MT3RenderTargetAlbedo].texture     = _gBuffer.albedoSpecular
    desc.colorAttachments[MT3RenderTargetAlbedo].loadAction  = .clear
    desc.colorAttachments[MT3RenderTargetAlbedo].storeAction = .store

    desc.colorAttachments[MT3RenderTargetNormal].texture     = _gBuffer.normal
    desc.colorAttachments[MT3RenderTargetNormal].loadAction  = .clear
    desc.colorAttachments[MT3RenderTargetNormal].storeAction = .store

    desc.colorAttachments[MT3RenderTargetPosition].texture     = _gBuffer.position
    desc.colorAttachments[MT3RenderTargetPosition].storeAction = .store

    desc.depthAttachment.texture     = view.depthStencilTexture
    desc.depthAttachment.loadAction  = .clear
    desc.depthAttachment.storeAction = .dontCare   // depth not needed after this pass
    return desc
}()

_encodePass(into: commandBuffer, using: gBufferPassDescriptor, label: "GBuffer Pass") { enc in
    enc.setVertexBytes(&uniforms.0, …, index: 1)
    enc.setFragmentBytes(&uniforms.1, …, index: 1)
    enc.setViewport(_buildViewport())
    enc.setRenderPipelineState(_gBuffPipelineState)
    enc.setDepthStencilState(_depthStencilState)

    for mesh in _meshes {
        enc.setVertexBuffer(mesh.vertexBuffers.first!.buffer, …, index: 0)
        for submesh in mesh.submeshes {
            enc.drawIndexedPrimitives(…)
        }
    }
}

GBuffer two-pass walkthrough

Step through the two passes below. The geometry pass writes albedo, normal, and position into separate textures (the GBuffer). The lighting pass then reads all three to compute one lit output per pixel — it never re-touches the geometry. Click any pixel after both passes to see its exact values and how they combine.

Pass 2 — Deferred lighting

_encodePass(into: commandBuffer, using: view.currentRenderPassDescriptor!, label: "Deferred Lighting Pass") { enc in
    enc.setRenderPipelineState(_displayPipelineState)

    // bind GBuffer textures for the fragment shader
    enc.setFragmentTexture(_gBuffer.albedoSpecular, index: MT3RenderTargetAlbedo)
    enc.setFragmentTexture(_gBuffer.normal,         index: MT3RenderTargetNormal)
    enc.setFragmentTexture(_gBuffer.position,       index: MT3RenderTargetPosition)
    enc.setFragmentBytes(&uniforms.1, …, index: 1)

    enc.setVertexBuffer(quadVertexBuffer, offset: 0, index: MT3BufferIndexMeshPositions)
    enc.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 6)  // full-screen quad
}

commandBuffer.present(view.currentDrawable!)
commandBuffer.commit()

The full-screen quad is a pre-built BufferView<MT3BasicVertex> with 6 vertices covering the clip-space rectangle:

let quadVertices: [MT3BasicVertex] = [
    .init(position: .init(x: -1, y: -1)),
    .init(position: .init(x: -1, y:  1)),
    .init(position: .init(x:  1, y: -1)),
    .init(position: .init(x:  1, y: -1)),
    .init(position: .init(x: -1, y:  1)),
    .init(position: .init(x:  1, y:  1)),
]

Metal shaders

GBuffer fragment shader

Writes surface data to all three GBuffer render targets at once. The [[color(n)]] attributes on the struct members route each field to the corresponding render target:

struct GBuffer {
    float4 albedo   [[color(MT3RenderTargetAlbedo)]];
    float4 normal   [[color(MT3RenderTargetNormal)]];
    float4 position [[color(MT3RenderTargetPosition)]];
};

fragment GBuffer gbuffer_fragment(VertexOut in [[stage_in]],
                                  constant MT3FragmentUniforms &uniforms [[buffer(1)]])
{
    GBuffer out;
    out.albedo   = float4(1, 1, 1, 1);                      // white for now
    out.normal   = float4(normalize(in.viewNormal), 1);
    out.position = normalize(in.viewPosition);
    return out;
}

The [[color(n)]] attributes route each struct field to the corresponding color attachment. Metal writes all three GBuffer textures in a single fragment shader invocation.

Display vertex shader

Simply passes through the full-screen quad positions (already in clip space):

vertex QuadInOut
display_vertex(constant MT3BasicVertex *vertices [[buffer(MT3BufferIndexMeshPositions)]],
               uint vid [[vertex_id]])
{
    QuadInOut out;
    out.position = float4(vertices[vid].position, 0, 1);
    return out;
}

Deferred lighting fragment shader

Reads one pixel from each GBuffer texture using texture.read(pixel_pos) and runs the same GGX lighting as Tutorial 2 via calculate_out_radiance:

fragment float4
deferred_lighting_fragment(
                               QuadInOut             in        [[ stage_in ]],
                               texture2d<float>      albedo    [[ texture(MT3RenderTargetAlbedo) ]],
                               texture2d<float>      normal    [[ texture(MT3RenderTargetNormal) ]],
                               texture2d<float>      position  [[ texture(MT3RenderTargetPosition) ]],
                               constant MT3FragmentUniforms &uniforms [[buffer(1)]])
{
    uint2 pixel_pos = uint2(in.position.xy);
    float4 albedo_specular_at_pix = albedo.read(pixel_pos);
    float4 normal_at_pix          = normal.read(pixel_pos);
    float4 position_at_pix        = position.read(pixel_pos);

    const float3 V = normalize(-float3(position_at_pix));
    const float3 N = normalize(normal_at_pix.xyz);
    const float3 L = normalize(float3(uniforms.viewLightPosition));

    return calculate_out_radiance(albedo_specular_at_pix, L, N, V);
}

Note texture.read(pixel_pos) instead of texture.sample(sampler, uv) — GBuffer textures are full-resolution off-screen targets, so there is a 1:1 mapping between screen pixel and texel. No sampler or UV interpolation is needed.

Key concepts recap

Next: Tutorial 4 — Shadows — add a shadow map pass to the deferred pipeline.