The GPU ReSTIR pass uses the same uploaded scene and spectral transport functions as the ordinary GPU path tracer. Its additional state is a device-local reservoir per pixel and the logic required to validate, reconnect, reweight, and merge previous history.

Commit contents Files discussed in this chapter gpu_ris_path_trace_pass.cpp · ris_path_trace_core.slangh · reservoir.slangh · resampling.slangh
Complete commit diff ↗

Persistent reservoir allocation

The pass distinguishes the current color sample from cross-frame history:

ctx.buffers.AddOrGetReusableGPU(kGpuRestirColorOutputName, sizeof(GfVec4f), count,
                                GPUFrameBufferMemoryType::DeviceLocal, "HdRestir.ReSTIR.colorOutput");
auto &fb{ctx.buffers.GetGPUFrameBuffer(kGpuRestirColorOutputName)};

// ...

ctx.buffers.AddOrGetPersistentGPU(kGpuReservoirBufferName, sizeof(Gpu::ReservoirData), count,
                                  GPUFrameBufferMemoryType::DeviceLocal, "HdRestir.ReSTIR.reservoirs");
auto &reservoirs{ctx.buffers.GetGPUFrameBuffer(kGpuReservoirBufferName)};

Reusable storage may be overwritten each frame. Persistent storage is injected back into the next frame’s pass graph. Both remain device-local.

The dispatch makes history optional:

_kernel.RunFrame(ctx.services.Get<IDynamicLibProvider>(), input, params, fb,
                 _enableTemporalReuse ? &reservoirs : nullptr);

Passing nullptr selects the no-history control experiment. The candidate generation and current-frame reservoir remain available; only temporal merge is disabled.

The complete direct-light path for one shading point is:

flowchart TD
  C["current NEE, sky, and BSDF candidates"] --> R["stream into current reservoir"]
  R --> F["finalize current stream"]
  F --> V["test preliminary winner visibility"]
  V --> H{"compatible previous reservoir?"}
  H -- no --> S["shade selected contribution and persist finite reservoir"]
  H -- yes --> J["reconnect endpoint and compute Jacobian"]
  J --> M["resample current and history with confidences 1 and m"]
  M --> E["evaluate both source targets and form generalized-balance weight"]
  E --> S

Only one representative from each source stream enters the temporal merge. The weights attached to those representatives must therefore reconstruct the mass of all candidates represented by each source. The boxes after reconnection form one generalized RIS estimator and must be read together; its source-mixture weights are derived below.

Finite-safe streaming update

The shader implementation follows the weighted reservoir proof from step 2:

void ReservoirUpdate(inout ReservoirData r, NeeCandidateData s, float weight, inout Pcg32Rng rng)
{
    r.CandidateCount += 1;
    if (!IsFinitePositiveWeight(weight))
    {
        return;
    }
    float updatedWeightSum = r.WSum + weight;
    if (!IsFinitePositiveWeight(updatedWeightSum))
    {
        return;
    }
    r.WSum = updatedWeightSum;
    if (rng.NextFloat() <= weight / updatedWeightSum)
    {
        r.ChosenSample = s;
        SetFlag(r.Flags, kReservoirFlagHasSample, true);
    }
}

CandidateCount increases for every represented proposal, including a non-positive or non-finite weight. WSum includes only accepted finite positive mass. If the updated sum overflows to infinity, the update is rejected before it corrupts the stored normalization.

Candidate construction in the kernel

The current-frame NEE loop connects light sampling, PDF conversion, target construction, MIS, and the streaming reservoir. This abridged excerpt removes only the separate dome and invalid-sample branches:

uint nNee  = settings.NeeCandidateCount;
uint nBsdf = settings.BsdfCandidateCount;
ReservoirData reservoir = MakeEmptyReservoirData();

for (uint i = 0; i < nNee; ++i)
{
    uint slotIdx = rng.NextUintBounded(totalLightSlots);
    LightData light = lights[slotIdx];
    GpuLightSample ls = SampleLightByStrategy(
        light, hitPos, rng);
    bool lsIsRayHittable = LightIsRayHittable(light);

    float dist2 = ls.Dist * ls.Dist;
    float cosY  = max(0.0, dot(-ls.Dir, ls.LightNormal));
    float solidAnglePdf =
        ls.PdfSpace == kPdfSpaceArea
            ? (cosY > 0.0
                ? ConvertPdfToSolidAngle(
                    ls.Pdf, ls.PdfSpace, dist2, cosY)
                : 0.0)
            : ls.Pdf;
    float pNee  = lightSelectPdf * solidAnglePdf;
    float pBsdf = GGXBsdfPdf(shadingNormal, wo, ls.Dir, c);

    float nDotL = dot(shadingNormal, ls.Dir);
    float3 scatteringResponse = float3(0.0);
    float4 contribution       = float4(0.0);
    bool geometryOk =
        nDotL > 0.0 && pNee > 0.0 &&
        !(cosY <= 0.0 && lsIsRayHittable);
    if (geometryOk)
    {
        scatteringResponse =
            GGXEval(shadingNormal, wo, ls.Dir, c) * nDotL;
        contribution =
            RGBToSpectrum(ls.Color, wavelengths) *
            RGBToSpectrum(scatteringResponse, wavelengths);
    }

    NeeCandidateData candidate;
    candidate.LightDir       = ls.Dir;
    candidate.LightDist      = ls.Dist;
    candidate.Contribution   = contribution;
    candidate.TargetFunction =
        ResamplingTargetFunction(ls.Color, scatteringResponse);

    float risWeight = ResamplingPowerHeuristicWeight(
        candidate.TargetFunction, pNee, int(nNee), pBsdf, int(nBsdf));
    ReservoirUpdate(reservoir, candidate, risWeight, rng);
}

For an area-density light sample, the conversion is

\[p_\omega(\omega) =p_A(y)\frac{\lVert y-x\rVert^2}{|\mathbf n_y\cdot(-\omega)|}.\]

The uniform light-slot probability is then multiplied into pNee. Omitting either factor would make the proposal density in the RIS weight inconsistent with the process that generated the endpoint. BSDF candidates are constructed in the following loop and enter the same reservoir with the complementary MIS weight. A failed proposal still increments CandidateCount; it represents one attempt with zero mass.

The persistent target cannot be reconstructed from the current frame’s random wavelength packet, because another packet will be drawn next frame. The shader therefore builds it from RGB quantities:

float matchedLuminance =
    dot(radiance * scatteringResponse, kLuminanceWeights);
float supportLuminance =
    dot(abs(radiance), kLuminanceWeights) *
    dot(abs(scatteringResponse), kLuminanceWeights);
float target = max(matchedLuminance, supportLuminance);

matchedLuminance tracks the ordinary RGB contribution. The product of absolute luminances preserves nonzero support when signed intermediate values would cancel component-wise. The full spectral contribution remains stored in the candidate; this scalar changes the resampling probabilities and the normalization carried by W, not the wavelength-domain quantity ultimately accumulated.

Current-stream contribution weight

Before temporal reuse, the current candidate stream is reduced to one sample and its complete RIS contribution weight. The function is named Finalize, but the result is final only for this source stream; a later temporal merge can use it as an input. Finalization succeeds only when the reservoir contains a sample, a positive candidate count, a finite weight sum, and a finite positive target:

int   frameCount  = max(1, r.CandidateCount / max(1, candidatesPerFrame));
float denominator = float(frameCount) * phat;
r.W               = IsFinitePositiveWeight(denominator) ? r.WSum / denominator : 0.0;
if (!IsFinitePositiveWeight(r.W))
{
    r.W = 0.0;
}

This project convention differs from the generic \(S/(M\hat p)\) notation. Candidate weights already contain the per-technique sample-count normalization, so finalization divides by the number of represented frames. The identity

\[\texttt{frameCount} =\frac{\texttt{CandidateCount}}{\texttt{candidatesPerFrame}}\]

is valid only while every represented frame uses the recorded candidate budget and candidate construction retains the same normalization convention.

History compatibility

The previous reservoir is indexed by the same pixel because a camera or scene change resets persistent state. Subpixel jitter can still move a primary ray across an edge, so the shader validates the stored surface:

if (previous.MaterialId != current.MaterialId)
{
    return false;
}
if (previous.ObjectId != 0 && current.ObjectId != 0 && previous.ObjectId != current.ObjectId)
{
    return false;
}

// ...

if (dot(previousNormal, currentNormal) < 0.95)
{
    return false;
}

// ...

float depthTolerance = max(1e-3, 0.02 * max(abs(previous.Depth), abs(current.Depth)));
float planeDistance  = abs(dot(current.Position - previous.Position, previousNormal));
return !isnan(planeDistance) && !isinf(planeDistance) && planeDistance <= depthTolerance;

Material and object identity reject unrelated surfaces. The normal threshold requires an angle smaller than \(\arccos(0.95)\), approximately \(18.2^\circ\). The final plane-distance check scales with depth but has a minimum absolute tolerance of \(10^{-3}\) scene units.

These are acceptance heuristics, not a proof that the two shading points represent the same path-space domain. They reduce invalid reuse at edges and discontinuities while motion-vector reprojection remains unimplemented.

Reconnection and Jacobian

For a finite light sample, the stored endpoint is reconstructed from the previous surface, direction, and distance. A new direction and distance are then computed from the current surface. Infinite-light samples retain their direction.

Changing the receiving surface changes the solid angle subtended by a finite area-light endpoint. The shader applies

\[J= \frac{\cos\theta_{\mathrm{current}}} {\cos\theta_{\mathrm{previous}}} \left( \frac{d_{\mathrm{previous}}} {d_{\mathrm{current}}} \right)^2.\]

This follows from \(d\omega/dA=\cos\theta/d^2\). The implementation returns a zero Jacobian when distances, cosines, or the result are not finite and positive.

More explicitly, keep the area endpoint \(y\) fixed and move only the receiving point from \(x_{\mathrm{previous}}\) to \(x_{\mathrm{current}}\). For the same differential patch \(dA_y\),

\[\frac{d\omega_{\mathrm{current}}}{dA_y} =\frac{\cos\theta_{\mathrm{current}}}{d_{\mathrm{current}}^2}, \qquad \frac{d\omega_{\mathrm{previous}}}{dA_y} =\frac{\cos\theta_{\mathrm{previous}}}{d_{\mathrm{previous}}^2}.\]

Their ratio is exactly the shader’s \(J\). The direction matters: this implementation maps a sample represented in the previous receiver’s solid-angle measure into the current receiver’s measure.

Temporal merge, line by line

The current candidate stream is finalized before temporal reuse. Its representative enters a second reservoir with mass \(W\hat p\):

ReservoirFinalize(reservoir, candidatesPerFrame);

float currentMergeWeight =
    HasFlag(reservoir.Flags, kReservoirFlagHasSample)
        ? reservoir.W * reservoir.ChosenSample.TargetFunction
        : 0.0;
ReservoirMerge(
    temporalReservoir, reservoir, currentMergeWeight, rng);

For a valid previous sample, the code reconnects and re-evaluates it in the current context, limits the represented history to 20 frames, and adds its source mass:

NeeCandidateData reconnected = ReconnectTemporalCandidate(
    previousState.ChosenSample,
    previousState.Surface.Hit, hit, wo, c, wavelengths);
reconnected = EvaluateCandidateVisibility(accel, hit, reconnected);
float shiftJacobian =
    TemporalShiftJacobian(previousState.ChosenSample, reconnected);

int maxPreviousCandidates = max(1, candidatesPerFrame * 20);
previousState.CandidateCount =
    min(previousState.CandidateCount, maxPreviousCandidates);
previousFrameCount =
    max(1, previousState.CandidateCount / max(1, candidatesPerFrame));

float mergeWeight =
    previousState.W *
    float(previousFrameCount) *
    reconnected.TargetFunction *
    shiftJacobian;
ReservoirMerge(
    temporalReservoir, previousState, mergeWeight, rng);

The factors answer separate questions:

  • previousState.W is the stored normalization of the represented source;
  • previousFrameCount restores how many frame streams that representative stands for after clamping;
  • reconnected.TargetFunction evaluates its importance at the receiver;
  • shiftJacobian changes the receiver’s solid-angle measure.

The cap sets the history confidence used by the source mixture to at most 20 represented frames. It does not inspect the retained light sample before choosing that confidence. Its purpose is to keep repeatedly reused, correlated history from receiving an ever-growing confidence relative to the new frame. This is a variance-and-correlation policy, and the resulting confidence is part of the mixture denominator from the outset.

The temporal merge is a generalized MIS estimator

Let the two source streams be \(s\in\{c,h\}\): the current frame \(c\) and history \(h\). Their confidence weights are

\[c_c=1,\qquad c_h=m,\]

where \(m=\texttt{previousFrameCount}\). For a retained light endpoint \(y\), let \(\bar p_c(y)\) and \(\bar p_h(y)\) be the target evaluated at the current and previous shading contexts in one common endpoint measure. For a finite area-light endpoint the shader constructs that canonical target as

\[\bar p_s(y)=p_s^{\omega}(y) \frac{\cos\theta_s}{d_s^2};\]

for infinite and discrete lights the stored target already uses an invariant direction or discrete measure. The weighted source-mixture denominator is

\[D(y)=c_c\bar p_c(y)+c_h\bar p_h(y) =\bar p_c(y)+m\bar p_h(y).\]

The contribution assigned to source \(s\) uses the generalized balance weight

\[b_s(y)=\frac{c_s\bar p_s(y)}{D(y)}, \qquad b_c(y)+b_h(y)=1\]

whenever \(D(y)>0\). A zero target in one context simply gives that source zero balance weight at that endpoint; proposal support still determines whether the overall estimator covers the integral. This is weighted multiple importance sampling over the source streams and is part of the estimator definition. It is the two-source form of the generalized balance heuristic described in the ReSTIR course notes and formalized for shifted, reused samples by Generalized Resampled Importance Sampling.

The shader distributes this balance weight across reservoir selection and output normalization. Let \(q_c(y)\) denote the current-context target in the receiving solid-angle measure used by the second reservoir, let \(W_s\) be the incoming reservoir contribution weight, and let \(J_s\) be the shift Jacobian, with \(J_c=1\). The merge masses are

\[a_s=c_s W_s q_c(y_s)J_s, \qquad A=a_c+a_h.\]

ReservoirMerge selects source \(s\) with conditional probability \(a_s/A\). Because the merged candidate count represents \(1+m\) frame streams, ReservoirFinalize first produces

\[W_0(Y)=\frac{A}{(1+m)q_c(Y)}.\]

The target evaluations already computed by the shader supply the remaining source-mixture factor

\[F_s(Y)=(1+m)\frac{\bar p_s(Y)}{D(Y)}.\]

Thus the stored weight is one expression,

\[W_Y=W_0(Y)F_s(Y) =\frac{A\,\bar p_s(Y)}{q_c(Y)D(Y)}.\]

The cancellation becomes explicit by conditioning on the two incoming representatives:

\[\begin{aligned} \mathbb E[\widehat I\mid y_c,y_h,W_c,W_h] &=\sum_{s\in\{c,h\}} \frac{a_s}{A} f_c(y_s) \frac{A}{(1+m)q_c(y_s)} (1+m)\frac{\bar p_s(y_s)}{D(y_s)}\\ &=\sum_{s\in\{c,h\}} \frac{c_s\bar p_s(y_s)}{D(y_s)} f_c(y_s)W_sJ_s\\ &=\sum_{s\in\{c,h\}} b_s(y_s)f_c(y_s)W_sJ_s. \end{aligned}\]

The implementation has factored one generalized MIS coefficient so that the reservoir can sample with the inexpensive masses \(a_s\) and evaluate the cross-context denominator only for the selected endpoint. If the current representative wins, its target must be evaluated at the previous shading point to obtain \(\bar p_h(Y)\); if history wins, the stored previous-context target supplies that term. This is why material parameters, UV coordinates, incoming direction, and visibility are part of the stored or reconstructed context.

The displayed equality is conditional on the two incoming representatives. Taking its outer expectation requires each incoming \(W_s\) to be a valid contribution weight for its source stream, the history shift to be bijective on the reused support with the stated Jacobian, and source/confidence decisions to be made without conditioning on the retained sample value. Iterative temporal reuse also introduces correlation, so those one-step identities do not by themselves prove convergence of an arbitrarily long history chain.

Visibility and wavelength packets

After reconnection, the target is re-evaluated at the current surface. Visibility is traced for the preliminary winner before it becomes valid persistent history. An occluded winner has both contribution and target set to zero.

The scalar resampling target is derived from RGB quantities and does not depend on the current four-wavelength packet. Once a sample is retained, its spectral contribution is evaluated for the current packet. This avoids storing spectral values tied to a previous packet, while wavelength-aware target construction remains a separate research question.

Persistent state is written only when the reservoir has a selected sample, finite positive W, and a finite positive visible target. Otherwise the slot is replaced by an empty reservoir. This prevents an occluded winner with old positive weight from returning as a black temporal sample.

What the production test establishes

The frame test reads the device reservoir back and checks both representation and reuse invariants:

DBG_ASSERT(reservoir.CandidateCount >= 2,
           "primary reservoir did not record current candidates");
DBG_ASSERT(std::isfinite(reservoir.WSum) &&
               std::isfinite(reservoir.W),
           "primary reservoir produced non-finite weights");
DBG_ASSERT(
    std::abs(reservoir.ChosenSample.TargetFunction - expectedTarget)
        <= targetTolerance,
    "GPU ReSTIR target depends on the frame's wavelength packet");
DBG_ASSERT(HasFlag(
               reservoir.ChosenSample.Flags,
               kNeeCandidateFlagVisibilityTested),
           "persistent reservoir stored an untested temporal winner");

if (frame > 0)
    DBG_ASSERT(foundTemporalHistory,
               "compatible primary reservoir history was not merged");

These checks establish that current proposals are counted, weights stay finite, the stored target is wavelength-packet independent, visibility is resolved before persistence, and compatible history is actually merged after the first frame. They do not establish unbiasedness for arbitrary motion or occlusion; those properties depend on proposal support, the shift mapping, the Jacobian, and evaluation of every source target in the common measure. See the complete test_gpu_ris_path_trace_frame.cpp.

Remaining reuse limits

This checkpoint implements same-pixel temporal reuse with geometric validation. It does not implement:

  • motion-vector reprojection;
  • spatial neighbor reuse;
  • disocclusion filling;
  • a general many-to-one visibility cache;
  • wavelength-conditioned targets.

Those omissions define the algorithm being measured. They should not be described as full spatiotemporal ReSTIR.

The stationary, denoiser-free comparison in Reservoirs and temporal reuse uses the final GPU path with one candidate per pixel and reports the internal variance estimate with history disabled and enabled. It is not repeated here. An acceptance/rejection visualization would require a dedicated debug AOV, which revision dbca10b does not expose; an ordinary color render cannot show which individual history tests accepted or rejected a reservoir.

Code map