This session is about some important aspects of the rendering pipeline of the upcoming Action-RPG "Lords of the Fallen", developed by Deck13 Interactive and CI Games for PS4, Xbox One, and PC. The topic covers several closely related areas like the deferred rendering system, image-based lighting using deferred cubemaps, deferred decals, and an approach for transparent object lighting and shadowing. More specifically, the lecture will cover several strategies to keep the G-Buffer as small and efficient as possible. This includes the description of a G-Buffer attribute-packing scheme and how per-material attributes can be exposed using special parameter lookup tables. Furthermore, a traditional problem of most deferred rendering systems is the seamless integration of transparent objects into the lighting. The lecture will present several ways to approach this problem, for example multi-pass deferred rendering, coloured transparent shadows, and a novel method for deferred particle lighting.
The rendering technology of 'lords of the fallen' philip hammer
1. The Rendering Technology
of 'Lords of the Fallen'
Philip Hammer (Deck13 Interactive)
@philiphammer0
phammer@deck13.com
GameConnection Europe 2014
October 29-31, Porte de Versailles, Paris
2. Agenda
● Introduction
● Current-Generation Deferred Rendering
● Deferred Transparent Lighting
● Transparent Shadowcasting
● Volumetric Lighting
● Q&A
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
3. Intro: Who is Deck13?
● Deck13 Interactive is one of the leading game developers in Germany
○ ~50 people in Frankfurt office + 6 people in Hamburg office
○ In-house multiplatform technology FLEDGE
(PS4, Xbox One, PC, PS3, Xbox 360, iOS)
○ Previously: Venetica, Jack Keane, Blood Knights, Tiger & Chicken, etc.
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
4. Intro: Who am I?
● Senior Engine/Graphics Programmer
○ Working since >8 Years @ Deck13 Interactive
○ Part of an amazing team of 9 full-time programmers
● Responsible for the overall rendering system, lighting, materials, etc.
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
5. Intro: What is ‘Lords of the Fallen’?
● Deck13’s largest title so far
● 3rd-person challenging Action-RPG
● Dark, demonic fantasy setting
● Platforms: PS4, Xbox One, PC (DX11)
● Release this week
○ First PS4 release from a german developer
○ Some reviews are already out
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
6. Current Generation Deferred Rendering
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
7. Current Generation Deferred Rendering
● We’re not doing super-duper funky stuff..
○ .. just combining some well-known techniques in the right way
● We’re not covering the basics of deferred rendering..
○ .. but our specific differences to the textbook
● Game-production started as a PS3/X360 title
○ Switched early in development after new platforms got introduced
○ All last-gen assets replaced
○ We had to find “next gen” for ourselves and adapt the tech
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
8. Current Generation Deferred Rendering
● Major advancements in FLEDGE
○ Switched from DX9 to DX11 only
○ Switched from Light-Prepass to more traditional Deferred Rendering
○ Support for deferred transparent object lighting (incl. particles)
○ Support for Volumetric Lighting
○ Support of parallax-corrected cubemap reflections [Lagarde12]
○ Massively multithreaded rendering for consoles
○ Massive utilization of hardware-instanced geometry
○ Global Illumination (using Geomerics Enlighten)
○ Advanced physics simulations for Clothing, GPU-Particles, etc.
(using NVIDIA PhysX)
○ Support a lot more objects, lights, shadows, decals, etc.
○ etc.
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
9. Current Generation Deferred Rendering
● Switch from Light-Prepass to Deferred Rendering
○ Second geometry-pass was too costly
○ Way more bandwidth and memory available in this HW generation
(+ more flexible ESRAM on XB1)
○ Challenge: Move a lot of rendering effects to deferred stages
(e.g. Fog, Reflections, etc.)
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
10. G-Buffer Layout
● G-Buffer should be as small as possible
○ Parameter encoding
■ 2-channel normals
■ 2-channel albedo + specular-color
(chroma subsampling, YCoCg, inspired by [Mavridis12])
○ Parameter packing
■ Pack single bits and use lower precision for some attributes
○ Share channels for mutually exclusive parameters
■ Material-ID/-Index for referencing per-material parameters
○ 128 bpp (31,64 MB @ 1080p) even fits fully into XB1 ESRAM
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
11. G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
12. G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● Colored Specular and Translucency are mutually exclusive!
○ Determined by Material-ID
13. G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● 1 Bit Material-ID determines whether MRT1.a contains
translucency or specular chroma.
● 7 Bit Material-Index used to reference
Material Parameter Buffers (more on this later)
14. G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil
● Stencil used for masking
○ 3 Bits for Decals, Reflections, Skin/SSSSS
○ 2 Bits double-sided Light-Volume Stenciling
15. G-Buffer Layout
● We needed more attributes for special effects
○ Tangents for smooth alpha-test
○ Per-object rimlights, emissive/glow & other FX
○ Motion-Vectors of moving/dynamic objects for motionblur
○ Distortion-Vectors for Post-Distort
● Not enough space in G-Buffer, but only needed for ~10% of objects
○ Render them in a separate pass and blend (glow) or
use immediately in subsequent pass (motion-/distortion-vectors)
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
16. Material Parameter Buffer
● G-Buffer stores surface-parameters ..
○ .. but some surface-parameters are not needed per-pixel
○ Store certain surface parameters indirectly to avoid
wasting precious G-Buffer space
● Introduce a “Material Parameter Buffer”
○ Generate a LUT which contains material parameters once per frame
○ Store the LUT index in the G-Buffer to gain access to the parameters later
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
17. Material Parameter Buffer
● “Sparse” parameters would unnecessarily bloat G-Buffer
○ We don’t want to afford more G-Buffer space
○ We have quite a lot artist-tweakable parameters
● Material-based Parameter examples
○ Reflection-Strength
○ Fresnel
○ Subsurface-scattering parameters
○ etc.
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
18. Material Parameter Buffer (Example)
● Example: Implementation of
“Approximating Translucency for a Fast, Cheap and
Convincing Subsurface Scattering Look” [Brisebois12]
in a deferred scenario without compromising the parameters
● Problem:
A lot of properties varying per material, but effect is applied during in
deferred light-pass
● Solution:
Store scalar translucency values per-pixel in G-Buffer and all other
properties in Material Parameter Buffer
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
19. Material Parameter Buffer (Example)
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
Material-Index sssDistortion sssPower sssMaterialColor …
0 4.5 2.4 0.43, 0.39, 0.84 …
1 8.0 3.5 0.22, 0.51, 0.52 …
2 5.6 4.1 0.41, 0.43, 0.11 …
... ... ... ... ...
Parameters per Light
Parameters per Material
Parameters per Pixel
// backscattering, according to [Brisebois12]
float3 sssLightVector = L + N * sssDistortion ;
float sssDot = exp2 ( saturate (
dot ( V, -sssLightVector ) ) *
sssPower - sssPower ) * sssScale ;
float3 sssColor = ( sssDot *
sssMaterialColor * sssLightColor ) * translucency ;
20. Material Parameter Buffer (Example)
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
Material-Index sssDistortion sssPower sssMaterialColor …
0 4.5 2.4 0.43, 0.39, 0.84 …
1 8.0 3.5 0.22, 0.51, 0.52 …
2 5.6 4.1 0.41, 0.43, 0.11 …
... ... ... ... ...
Find implementation details in
Appendix 1 or ask me in Q&A.
22. Deferred Transparent Lighting
● Most engines use Forward Rendering for transparents in Deferred
Rendering scenarios
○ Typical drawbacks: light-count, shadowing, etc.
● We wanted a deferred solution
● And we ended up with two different kinds of it
○ Deferred Per-Pixel
■ “expensive” - used for selected objects
○ Deferred Per-Vertex
■ “cheap” - used mostly for particles
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
23. Deferred Per-Pixel Transparent Lighting
● Gives you one additional layer of lighting
● Straight-forward implementation
(with similarities to Light-Prepass Rendering)
○ 1. Render a second G-Buffer
■ Only with lit transparent objects
■ Advise the artists to flag only a few transparent objects as lit
○ 2. Render a second L-Buffer
■ Render all light volumes also on the second layer
○ 3. Blend the transparent objects onto final scene while
sampling the light of the second L-Buffer
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
24. Deferred Per-Pixel Transparent Lighting
● Caveats
○ You *can* easily destroy your performance ,,
■ .. but it won’t if used wisely
○ Memory consumption is higher, but not a big deal on current-gen console ..
■ .. and you can reuse rendertargets in different rendering stages
● Optimizations
○ Mask rendered pixels in stencil for culling
○ Rough Scene-AABB culling
■ Build AABB of all lit transparent objects once per frame
■ Check against light-volume AABBs
■ Omit a lot of lights cheaply
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
25. Deferred Particle Lighting
● “We want lit particles!”
○ Particles fitting better to the scene
○ Particles can be re-used under different lighting conditions
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
26. Deferred Particle Lighting
● Common type of particles: atmospheric, smoke, dust & fog effects
○ Usually transparent, dense ... and cause a lot of overdraw
○ Costs can easily explode when lighting every fragment
○ Non-directional, low-frequency information
○ Perfect for vertex-lighting
● Deferred vertex-lighting
○ No compromise in number of affecting lights and shadows
○ Fits perfectly into deferred pipeline
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
27. Deferred Particle Lighting
● Again three steps to success
○ Step 1: Render Vertex G-Buffer
■ Write particle vertex positions sequentially into a texture
○ Step 2: Render Vertex L-Buffer
■ Accumulate light volumes
○ Step 3: Particle Rendering
■ Sample L-Buffer and apply lighting
● Find implementation details in Appendix 2 and ask in Q&A.
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
28. Deferred Particle Lighting
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
29. Deferred Particle Lighting
● Great for many types of particles
○ Cheap and effective
○ Apply all light types and features like shadows, gobos, IES, etc.
● Inherits problems of all per-vertex techniques
○ Low spatial resolution
○ Works best with small particles
○ Very small and bright lights can lead to flickering if
isolated corners of particles getting suddenly lit
● Be wary of buffer overflow with many lit particles
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
30. Transparent Shadows
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
31. Transparent Shadows
● All lights can cast colored, dynamic, transparent shadows
● Simple and effective approach
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
32. Transparent Shadows
● All lights can cast colored, dynamic, transparent shadows
● Simple and effective approach
○ Render solid shadows as usual into shadowmap
○ Render/Blend transparent objects into shadowmap-sized RGBA8 buffer
○ Use shadowmap as depth-buffer
○ Sample transparent shadowbuffer with shadow UVs and multiply with light
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
33. Transparent Shadows
● Works also with volumetric lights, transparent particles, etc.
● Individual features really shine when everything works together
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
34. Volumetric Lighting
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
35. Volumetric Lighting
● One of the major effect for “Next Gen”-Look
● Raymarching-based Approach
○ Inspired by “Real-time Volumetric Lighting in participating media” [Toth09]
○ Raymarching in light’s view space while evaluating the shadowmap
○ Interleaved Sampling and Temporal Reprojection for Performance
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
36. Volumetric Lighting
● Multiple light-types and -features
○ Point, Spot, Box
○ Also evaluate shadowmap, projector-texture(s), transparent shadows, IES light-profiles,
animated noise volume-texture, etc.
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
37. Volumetric Lighting
● Optimizations
○ Interleaved Sampling
○ Low-resolution rendering with depth-aware upsampling
○ Temporal Reprojection
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
38. Volumetric Lighting
● Performance
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
Pass PC / DX11 (GTX
700 GPU)
PS4 / GNM
Accumulation 0.362 ms 0.161 ms
Gather 0.223 ms 0.375 ms
Upscale 0.127 ms 0.321 ms
= 0.712 = 0.857 ms
39. Volumetric Lighting
● Attention: Artists love it and therefore tend to overuse it
● See for the gritty details:
“Volumetric Lighting for Many Lights in Lords of the Fallen” by Benjamin
Glatzel
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
40. Thanks for listening!
@philiphammer0
phammer@deck13.com
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
41. Thanks for listening!
@philiphammer0
phammer@deck13.com
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
Questions?
42. Thanks for listening!
.. and thanks to the Deck13 development team, making all this possible.
.. and very special thanks to the Tech-Team which contributed a lot to this talk (alphabetical)
● Arturo Cepeda
● Holger Durach
● Michele Giacalone (@miccalone)
● Benjamin Glatzel (@begla)
● Robert Hallinger (@robih29)
● Philip Hammer (@philiphammer0)
● Thorsten Lange
● Dominik Lazarek (@Omme)
● David Reinig (@D13_Dreinig)
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
43. References
[Glatzel14] “Volumetric Lighting for Many Lights in Lords of the Fallen”, Digital Dragons 2014,
http://de.slideshare.net/BenjaminGlatzel/volumetric-lighting-for-many-lights-in-lords-of-the-fallen
[Mavridis12] "The Compact YCoCg Frame Buffer", Journal of Computer Graphics Techniques, Vol. 1, No. 1, 2012
[Lagarde12] “Local Image-based Lighting With Parallax-corrected Cubemap”, Siggraph, 2012
[Brisebois12] “Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look”, GDC 2011
http://dice.se/wp-content/uploads/Colin_BarreBrisebois_Programming_ApproximatingTranslucency.pdf
[Persson07] “Deep Deferred Shading”, Blog Post 2007
http://www.humus.name/index.php?page=3D&ID=75
[Drobot12] "Lighting of Killzone: Shadow Fall"
http://de.slideshare.net/guerrillagames/lighting-of-killzone-shadow-fall
[Toth09] “Real-time Volumetric Lighting in Participating Media”, Eurographics 2009
http://sirkan.iit.bme.hu/~szirmay/lightshaft.pdf
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
44. Appendix 1 - Material Parameter Buffer
● Step 1/3: Buffer generation
○ Generate a buffer / LUT (array of structs) each frame on the CPU
■ Determine for each drawcall a set of material parameters
■ Smart value quantization and interleaving
○ LUT contains selected data of the currently rendered materials
■ DX11/GNM: structured buffer
■ DX9: multiple 1D textures
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
45. Appendix 1 - Material Parameter Buffer
● Step 2/3: G-Buffer stage
○ Write the LUT-index to the G-Buffer
■ Pass the index as shader constant
(index is associated with the material during (1) )
○ can index up to 128 different materials per frame
● 7 bit, because 1 bit reserved for Material-ID
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
46. Appendix 1 - Material Parameter Buffer
● Step 3/3: Lighting/Combine stage(s)
○ Bind the Material Parameter Buffer(s) to the shader
○ Sample/unpack the Material-Index from G-Buffer
○ Grab and use the parameter values when needed like
value = buffer[index].x
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
47. Appendix 1 - Material Parameter Buffer
Pixel-Shader
// fragment shader for indexing deferred material-parameter Buffer
Buffer <float4 > parameterBuffer ;
uint materialIndex = unpackMaterialIndex(gbuffer0Value . a);
float4 matParamBuffer0 = parameterBuffer[materialIndex];
float4 matParamBuffer1 = parameterBuffer[materialIndex + 1];
float sssDistortion = matParamBuffer0 . x ;
// backscattering according to [Brisebois12]
float3 sssLightVector = L + N * sssDistortion ;
float sssDot = exp2 ( saturate ( dot ( V, -sssLightVector ) ) * sssPower - sssPower ) * sssScale ;
float3 sssColor = ( sssDot * sssMaterialColor * sssLightColor ) * backscattering_mask ;
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
48. Appendix 2 - Deferred Particle Lighting
● Step 1/3: Particle Vertex G-Buffer
○ Render a non-screenspace particle “G-Buffer”
■ Basically a sequential list
○ Render particle vertex-buffer as point-list
(D3D11_PRIMITIVE_TOPOLOGY_POINTLIST)
○ Rasterize viewspace-positions of particles subsequently into a buffer
■ Currently a 1024x512 RGBA16F texture for all visible lit particles
○ G-Buffer currently contains only vertex-positions
■ Could be extended with more surface attributes like
translucency or normals
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
50. Appendix 2 - Deferred Particle Lighting
● Step 2/3: Particle Vertex L-Buffer
○ Apply lighting similar to normal deferred lighting
○ Use fullscreen-quads instead of light-volumes, because the fragments
have no spatial relationship (non-screenspace fragment list)
■ Optimization: Stencil-Mask each fragment in Per-Vertex G-Buffer
○ Different light-types, shadows, projected textures, etc. “just works”
○ Very simple “lighting model”: just add up light colors
○ Shadowing: PCF with large kernel width reduces flickering
in shadow/light transitions
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
51. Appendix 2 - Deferred Particle Lighting
● Step 2/3: Particle Vertex L-Buffer
// pixel shader
// trivial implementation (different for each light type):
// - sample g-buffer containing the viewspace positions of particle vertices
// - compute light attenuation base on position and light parameters (...)
float3 positionVS = SAMPLE ( perVertexGBufferSampler, screenUV . xy ) . rgb ;
col0 . rgb = light_color . rgb * getLightAttenuation ( positionVS ) * getShadowing ( positionVS ) ;
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
52. Appendix 2 - Deferred Particle Lighting
● Step 3/3: Particle Rendering
○ Render particles as you would normally render them
○ Apply lighting
■ Compute UVs according to the rasterized position from Step 1
■ Sample the Particle L-Buffer in Vertex-Shader
■ Interpolate lighting to fragment shader and modulate with particle texture
○ Take care of exact rendering order
(must be the same as in Step 1/3)
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014
53. Appendix 2 - Deferred Particle Lighting
● Step 3/3: Particle Rendering
// vertex-shader
// vertexId = SV_VertexID, instanceId = SV_InstanceID
// param_vertexcount = shader constant holding the currently rendered number of vertices
float vertexIndex = vertexId + ( instanceId * vertexCountPerInstance ) + param_vertexcount ;
float3 perVertexLightBufferUV = 0;
perVertexLightBufferUV . y = trunc ( vertexIndex / textureWidth ) ;
perVertexLightBufferUV . x = trunc ( vertexIndex - perVertexLightBufferUV . y * textureWidth ) ;
perVertexLightBufferUV . y = ( textureHeight - 1.0 ) - perVertexLightBufferUV . y ;
float4 lbufferValue = perVertexLightBufferTexture . Load ( (int3)perVertexLightBufferUV . xyz ) ;
// pixel-shader
col0 . rgb = particleTextureColor . rgb * f_in . lbufferValue . rgb ;
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014