Physically based rendering comprises a set of mathematical approximations tuned to look realistic and run fast. Often these approximations are tuned or tweaked but often land in the same or similar places across projects in pursuit of these two goals. What happens if we tune them for different goals; to build worlds where light behaves differently to our own?

PBR, a recap

Physically based rendering (PBR) treats every surface as a field whose light interaction behaviour can be dialled up or down from conceptual ‘roughness’ and ‘metallic’ properties.  These two dials, along with a few other rules (energy conservation, reflection at glancing angles) are enough to convincingly produce an enormous range of real materials from brushed steel to wet stone.

Bending the rules

Bending these approximations of reality sometimes becomes a practical decision made with the Creative Team. For example, in one XR project, we found the real rules of light were standing in the way of success. The project involved a movement exercise set in a dreamy luminous space inspired by James Turrell’s light installations.

Turrell notably works with light as a medium. Our equivalent was re-writing the rules that govern it.

The settting was a vast dome in which orbs could be steered to pick up collectible objects. Each orb carried a point light and illuminating the far walls was a problem. The inverse square law determines the brightness of light over distance and the source needed to be very strong to cover large distances but this also blew out anything that was too close. Also widening the range flattened the distant surfaces. The result was washed out. 

We needed light that was very intense close up but held across a large distance and faded off very gradually. So we re-wrote the falloff (the rate of light fading over distance), then used calculations regarding the normals (surface direction) interaction with light to create a translucent quality to surfaces. This approach of bending light physics gradually became a design principle rather than a patch.

Prior works

Others have bent the rules in different ways. Portal and Antichamber bend space, not light. Stylised worlds (Borderlands, Ōkami), draw on the tradition of non-photorealistic rendering. In those cases, often the underlying motive is to imitate other mediums such as ink, dithering and line art rather than specifically re-imagine light physics.

A Slower Speed Of Light

A good case study to draw on is  MIT’s A Slower Speed of Light, a project which demonstrates what vision would be like if the speed of light was slower. However, this works on how light reaches the viewer rather than how light meets a surface. The viewer’s physics is changed rather than materials. The effect, however, is truly mind-bending, with the player seeing doppler colour shifts, aberrations and brightening unlike anything experienced in the real world.

There are also examples of non-physical techniques designed purely to solve perceptual problems. In Half-Life 2, Valve documented a half-Lambert technique which was used to shift the diffuse term. This improved the visual perception of the shaded regions of characters lit by weak directional light.

Journey

Another example is thatgamecompany’s Journey. John Edwards’ three year effort to refine the sand is covered in his 2013 GDC talk. Based on the Pismo dunes of the Californian coast, he is upfront about intentionally leaving reality behind. In the slide notes, he coins the terms ‘glitter specular’ and ‘ocean specular’, admitting the equations for the former ‘may or may not mean anything’, though they solved a perceptual aliasing issue with the standard specular approach. For the ocean specular he mentions “we added an ocean-style big column of specular…I’ve never seen any sand, … behave this way, but it ended up feeling right”. Notable was the absence of any claim this was how light really behaved only that the effect worked in context.

Our perceptions

Why should invented algorithms with no theoretical basis work (when they do)?

One possibility is they simply chance upon a new approximation that holds a kernel of truth from real light. The viewer recognises the characteristics but is unable to join the dots to those similar experiences. Another possibility is when something alien yet seems to follow coherent internal rules invites curiosity, we are led to want to master our understanding of that light/material interaction. Either way, these examples show the strangeness must be tolerable and coherent to remain appealing.

What this means for a team

For a studio at the early-idea stage, how ‘real’ something should look can be the wrong question. Physical realism and visual quality bar are not the same. A better question might be; what aspects of realism do we want to keep and what do we want to break to emphasise our visual identity? Early prototypes can be where that gets tuned into something distinct enough to really stand out. 

John Edwards, “Sand Rendering in Journey”, GDC 2013 (link)

Jason Mitchell, Shading in Valve’s Source Engine, Siggraph 2006 (link)

MIT Game Lab, A Slower Speed of Light. 2013 (link)

Alexander (Demruth) Bruce, Antichamber 2018

Clover Studio, Ōkami 2006

Gearbox Software, Borderlands 2009