Frustum Culling: Rejecting Geometry Before It Costs You a Draw Call

The core idea and its invariant

A perspective camera sees a view frustum: a truncated rectangular pyramid bounded by six planes — near, far, left, right, top, bottom. Every point the camera can render lies inside all six. Frustum culling exploits the contrapositive: if an object lies entirely outside even one of the six planes, it is invisible and can be skipped.

The key design choice is conservatism. We test a cheap bounding volume — a sphere or axis-aligned bounding box (AABB) — not the real mesh. That guarantees no false negatives: we never cull something that is actually visible. We accept false positives (an object whose bounding volume clips a plane but whose triangles are all off-screen still passes), because the GPU's later stages — clipping and the z-buffer — handle those correctly and cheaply. The invariant is simply:

• Rejection is sound: if the test says "outside plane P", the object contributes zero visible pixels.
• Acceptance is permissive: "inside/intersecting" may still be off-screen; that is fine.

Each plane is stored as (n, d) with unit normal n pointing into the frustum, so the signed distance from a point p is n·p + d. Positive means inside the half-space, negative means outside.

How the sphere and AABB tests work

The workhorse is the sphere-vs-frustum test. Give each object a bounding sphere (center c, radius r). For every plane, compute the signed distance from c. If that distance is less than −r for any plane, the whole sphere sits fully outside that plane and we reject immediately.

bool sphereInFrustum(Sphere s, Plane planes[6]):
  for i in 0..5:
    dist = dot(planes[i].n, s.center) + planes[i].d
    if dist < -s.radius:
      return false          // fully outside plane i
  return true               // inside or intersecting all 6

Sphere tests are branch-light and order-independent, but spheres over-estimate volume for long thin meshes. The AABB test is tighter. The classic trick (Greene, 1994) avoids testing all 8 corners: for each plane, pick the box corner farthest along the plane's normal — the positive vertex (p-vertex) — by choosing max or min per axis according to the sign of each normal component. If even that best-case corner is outside, the whole box is outside.

bool aabbInFrustum(AABB b, Plane planes[6]):
  for i in 0..5:
    n = planes[i].n
    // p-vertex: the corner most aligned with n
    px = n.x >= 0 ? b.max.x : b.min.x
    py = n.y >= 0 ? b.max.y : b.min.y
    pz = n.z >= 0 ? b.max.z : b.min.z
    if dot(n, (px,py,pz)) + planes[i].d < 0:
      return false          // even nearest-in corner is outside
  return true

Both run in constant time — at most six planes, each a dot product plus a compare — regardless of mesh complexity. That decoupling from triangle count is the whole point.

Extracting the six planes from the matrix

You rarely build the planes by hand. The standard method, published by Gil Gribb and Klaus Hartmann (2001), extracts all six planes directly from the combined view-projection matrix M = P·V. Each frustum plane is a sum or difference of two rows of M. Writing the rows as m₀…m₃:

• Left = m₃ + m₀, Right = m₃ − m₀
• Bottom = m₃ + m₁, Top = m₃ − m₁
• Near = m₃ + m₂ (or m₂ for a [0,1] depth range like Direct3D/Vulkan), Far = m₃ − m₂

Each result is a 4-vector (a, b, c, d) giving the plane ax + by + cz + d = 0. Normalize by dividing all four components by ‖(a, b, c)‖ so signed distances come out in world units — essential for the sphere test, where you compare against a radius. Extraction is O(1): a fixed number of adds and one √ per plane. Recompute it only when the camera moves, then reuse it across the entire scene traversal. Because M already folds in view and projection, the same six numbers work in world space, saving you from transforming every object into view space.

Complexity: from O(n) to hierarchical O(log n)

Flat culling tests every object independently: n objects × O(1) per object = Θ(n) time, O(1) extra space. With a few million dot products per frame that is often fine on modern CPUs, and it vectorizes beautifully with SIMD — pack 4 or 8 spheres per instruction.

But Θ(n) still touches every object even when 99% share the same fate. A spatial hierarchy — an octree, BVH, or quadtree — fixes this. Test a node's bounding volume first; if the node is fully outside the frustum, prune the entire subtree in one test. If fully inside, accept the whole subtree with no further plane tests (all descendants trivially pass). Only nodes that straddle a plane need recursion.

• Best case: camera looks at empty space — root is culled in one test, O(1).
• Typical case: the visible set has size k; traversal touches O(k + log n) nodes for a balanced tree.
• Worst case: every leaf straddles a plane (pathological geometry), degrading to O(n) — no better than flat, plus tree overhead.

The hierarchy costs O(n) space and O(n log n) to build (amortized across many frames, or rebuilt incrementally for dynamic objects). The engineering rule of thumb: below ~10⁴ objects a SIMD flat pass often beats tree traversal on constant factors; above that, hierarchies win decisively.

Where it runs in real engines

Frustum culling is one of the oldest and most universal optimizations, and it sits early in every render pipeline — after scene update, before draw submission.

• Three.js gives each Object3D a frustumCulled = true flag and a bounding sphere; the renderer builds a Frustum from the camera's projection-screen matrix and calls frustum.intersectsObject() per mesh each frame.
• Unity runs frustum culling on the main thread and via its Burst-compiled jobs system, and layers occlusion culling (precomputed PVS) on top.
• Unreal Engine combines distance culling, frustum culling, precomputed visibility volumes, and hardware/software occlusion; frustum rejection happens before the visible-mesh list is assembled.
• Godot uses a BVH over its spatial partition for frustum queries.

On GPU-driven pipelines (Nanite in UE5, DrawIndirect-based renderers), culling migrates to a compute shader: thousands of instances tested in parallel, survivors compacted into an indirect draw buffer — the same six-plane math, now data-parallel. The canonical academic reference is Akenine-Möller, Haines & Hoffman, Real-Time Rendering, whose culling chapter formalizes all of the above.

Trade-offs, pitfalls, and edge cases

Frustum culling is nearly free and always worth doing, but it has sharp edges:

• It does not do occlusion. A wall directly in front of a mountain will still let the mountain pass the frustum test even though it is invisible. Frustum culling handles off-screen, not hidden-behind — that is what occlusion culling and the z-buffer are for.
• Plane normalization matters. Forget to normalize the extracted planes and the sphere test's dist < -r compares distances in the wrong units, silently culling visible objects or keeping off-screen ones. Symptom: geometry popping at the screen edges.
• Stale bounding volumes. A skinned or animated mesh that swings a limb outside its cached AABB gets clipped when the arm should be visible. Recompute or pad bounds for dynamic geometry.
• Shadow casters. An object behind the camera can still cast a shadow into view. Cull shadow-map rendering against the light's frustum, not the camera's, or you erase shadows.
• Reversed-Z and clip ranges. OpenGL uses NDC z ∈ [−1, 1] while D3D/Vulkan use [0, 1]; the near-plane extraction differs (m₃ + m₂ vs m₂). Mixing conventions puts the near plane in the wrong place.
• Cost floor. With very few, very heavy meshes the per-object test is noise; with millions of trivial objects the culling loop itself can dominate — that is when SIMD and hierarchies pay off.

Visibility culling techniques compared: cost, what they reject, and where they run.

Technique	Rejects	Per-object cost	False negatives?
Frustum culling	Off-screen (outside the pyramid)	O(1), ≤6 plane tests	None (conservative)
Backface culling	Away-facing triangles	O(1) per triangle	None
Occlusion culling	Hidden behind other geometry	O(k) queries / depth reprojection	None (conservative)
Z-buffer (per-pixel)	Fragments behind others	O(1) per fragment	None, but pays fragment cost
LOD selection	Nothing — swaps mesh detail	O(1) distance test	N/A