Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019

Unity Technologies
21. Oct 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019
1 von 47

Más contenido relacionado

Was ist angesagt?

Unreal Summit 2016 Seoul Lighting the Planetary World of Project A1Unreal Summit 2016 Seoul Lighting the Planetary World of Project A1
Unreal Summit 2016 Seoul Lighting the Planetary World of Project A1Ki Hyunwoo
The Rendering Technology of 'Lords of the Fallen' (Game Connection Europe 2014)The Rendering Technology of 'Lords of the Fallen' (Game Connection Europe 2014)
The Rendering Technology of 'Lords of the Fallen' (Game Connection Europe 2014)Philip Hammer
Screen Space Reflections in The SurgeScreen Space Reflections in The Surge
Screen Space Reflections in The SurgeMichele Giacalone
Low-level Shader Optimization for Next-Gen and DX11 by Emil PerssonLow-level Shader Optimization for Next-Gen and DX11 by Emil Persson
Low-level Shader Optimization for Next-Gen and DX11 by Emil PerssonAMD Developer Central
Killzone Shadow Fall Demo PostmortemKillzone Shadow Fall Demo Postmortem
Killzone Shadow Fall Demo PostmortemGuerrilla
Rendering AAA-Quality Characters of Project A1Rendering AAA-Quality Characters of Project A1
Rendering AAA-Quality Characters of Project A1Ki Hyunwoo

Was ist angesagt?(20)

Similar a Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019

Unity - Internals: memory and performanceUnity - Internals: memory and performance
Unity - Internals: memory and performanceCodemotion
Developing applications and games in Unity engine - Matej Jariabka, Rudolf Ka...Developing applications and games in Unity engine - Matej Jariabka, Rudolf Ka...
Developing applications and games in Unity engine - Matej Jariabka, Rudolf Ka...gamifi.cc
「原神」におけるコンソールプラットフォーム開発「原神」におけるコンソールプラットフォーム開発
「原神」におけるコンソールプラットフォーム開発Unity Technologies Japan K.K.
【Unite 2017 Tokyo】インスタンシングを用いた美麗なグラフィックの実現方法【Unite 2017 Tokyo】インスタンシングを用いた美麗なグラフィックの実現方法
【Unite 2017 Tokyo】インスタンシングを用いた美麗なグラフィックの実現方法Unity Technologies Japan K.K.
【Unite 2017 Tokyo】インスタンシングを用いた美麗なグラフィックの実現方法【Unite 2017 Tokyo】インスタンシングを用いた美麗なグラフィックの実現方法
【Unite 2017 Tokyo】インスタンシングを用いた美麗なグラフィックの実現方法Unite2017Tokyo
High resolution animated scenes from stillsHigh resolution animated scenes from stills
High resolution animated scenes from stillsCarolyn Rose

Similar a Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019 (20)

Más de Unity Technologies

Build Immersive Worlds  in Virtual RealityBuild Immersive Worlds  in Virtual Reality
Build Immersive Worlds in Virtual RealityUnity Technologies
Augmenting reality: Bring digital objects into the real worldAugmenting reality: Bring digital objects into the real world
Augmenting reality: Bring digital objects into the real worldUnity Technologies
Using synthetic data for computer vision model trainingUsing synthetic data for computer vision model training
Using synthetic data for computer vision model trainingUnity Technologies
The Tipping Point: How Virtual Experiences Are Transforming Global IndustriesThe Tipping Point: How Virtual Experiences Are Transforming Global Industries
The Tipping Point: How Virtual Experiences Are Transforming Global IndustriesUnity Technologies
Unity Roadmap 2020: Live games Unity Roadmap 2020: Live games
Unity Roadmap 2020: Live games Unity Technologies
Unity Roadmap 2020: Core Engine & Creator ToolsUnity Roadmap 2020: Core Engine & Creator Tools
Unity Roadmap 2020: Core Engine & Creator ToolsUnity Technologies

Más de Unity Technologies(20)

Último

EuroBSDCon 2023 - (auto)Installing BSD Systems - Cases using pfSense, TrueNAS...EuroBSDCon 2023 - (auto)Installing BSD Systems - Cases using pfSense, TrueNAS...
EuroBSDCon 2023 - (auto)Installing BSD Systems - Cases using pfSense, TrueNAS...Vinícius Zavam
Product Research Presentation-Maidy Veloso.pptxProduct Research Presentation-Maidy Veloso.pptx
Product Research Presentation-Maidy Veloso.pptxMaidyVeloso
Google cloud Study Jam 2023.pptxGoogle cloud Study Jam 2023.pptx
Google cloud Study Jam 2023.pptxGDSCNiT
Uber Clone Script - Keys to Understanding the Ride Hailing IndustryUber Clone Script - Keys to Understanding the Ride Hailing Industry
Uber Clone Script - Keys to Understanding the Ride Hailing IndustryeSiteWorld TechnoLabs Pvt. Ltd.
Privacy in the era of quantum computersPrivacy in the era of quantum computers
Privacy in the era of quantum computersSpeck&Tech
Knowledge Graphs and Generative AI_GraphSummit Minneapolis Sept 20.pptxKnowledge Graphs and Generative AI_GraphSummit Minneapolis Sept 20.pptx
Knowledge Graphs and Generative AI_GraphSummit Minneapolis Sept 20.pptxNeo4j

Último(20)

Developing and optimizing a procedural game: The Elder Scrolls Blades- Unite 2019

Hinweis der Redaktion

  1. Intros
  2. How difficult it is to build a procedural game with advanced visuals for mobile platforms? We will try to offer some insights Our experience is from working on The Elder Scrolls Blades – it reimagined TES for modern mobile platforms Specifically we will talk about: precomputed lighting in procedural context  some aspects of performance/optimization that we dealt with
  3. This is an example of what a level looks like from an editor point of view Made of standalone blocks called rooms The red lines represent connections between rooms Different shapes and size but they all fit together Any room can be attached to any other rooms for the same theme. All fit on a grid that acts as a framework for the level generator To support longer levels without using too much memory rooms are streamed in while the player progresses through the level
  4. Example of a single rooms Rooms are prefab that contain all the data we need to use that piece in a level
  5. Unity render pipeline Scriptable render pipeline was still experimental when we started the project Realtime lighting Looks great with the PBR shader, especially with high definition normal maps Allows dynamic shadow in key areas like this window Baked global illumination Both lightmaps and light probes Add depth to scenes with indirect lighting Can add lot of baked lights without affecting performance
  6. On this screenshot what you see is a harsh cut produced by lightmaps along the floor and wall were two different rooms connect. The problem here is that since lightmaps are generated separately for each room, we can get end up with dramatically different colors and brightness around room connection. Because of the way we bake global illumination per room, our global illumination isn’t really global. We also can’t predict which rooms will get connected together because they’re assembled randomly at runtime. Bake time solutions… We concluded that we needed a runtime solution to this
  7. That runtime solution is lightmap blending. The concept here is rather simple, what if we could blend the lightmap on each side of a connection to make the seam disappear? Then we could start with a 50-50 blend at the connection and the fade it out based on distance. Well you need a few things to achieve this. One, we need to add a secondary lightmap for blending to the materials on each side. Each side needs the other side’s lightmap And since a blend like this needs to be applied per pixel, we need to assign a blend factor to each vertex. And we need a uv set to sample the secondary lightmap texture. Once we have that we simply edit the shader to sample both lightmaps and blend them based on the blending factor. All of this is setup can be done at runtime when attaching two rooms together
  8. With this technique we only end up using the lightmap color at the seam and stretching it, so any local detail can get blow out of proportion which doesn’t look good. The problem is that we don’t have enough to blend that shadow correctly, we don’t know what that shadow in the blue room would have looked like if it had continued naturally. Well what if we could create that information?
  9. This is exactly what we did! We’ve created a mesh that is precisely the shape of the connection but stretched based on the blending distance. We make it so that it gets lightmapped by Unity at bake time, but we never actually render it in game. We only use it’s lightmap data. With this we have more accurate data for blending so we don’t need to stretch anything anymore.
  10. The harsh cut is gone and we don’t have any noticeable artifact. The light on the floor from that torch does fade pretty quickly at the connection. It’s not 100% accurate, but still an acceptable compromise.
  11. Because of the blending we do have an additional texture fetch per pixel, but we only do that for static renderers that are near a connection. So it’s not very significant. The worst performance problem comes from computing the uv set for the secondary lightmap. It involves finding the closest point on the extension mesh for each vertex of the blended renderers. This can be quite heavy calculations but what we did is run all of that asynchronously in a background thread. When streaming a level, the new rooms are spawned outside of the view of the player so we don’t need to apply lightmap blending immediately. And because we modify meshes at runtime by adding an extra uv set this doesn’t work with Unity’s static batching. You’ll have to either remove the static batching option on meshes that are blended or disable the feature globally in your project settings.
  12. So this takes care of our biggest lightmap problem, but I mentioned earlier that we also use light probes as part of our lighting pipeline.
  13. Light probes are object that contain directional lighting information for a specific point in space. They’re a built-in feature in Unity. If you setup a LightProbeGroup component in your scene, Unity will generate light probe information when you bake the lighting for that scene. At runtime they allow the engine to sample baked lighting information for dynamic objects. They’re a complementary feature to lightmaps. Lightmap are for applying baked lightins to static object and light probes are for applying baked lighting to dynamic objects. The problem with light probes is that Unity’s implementation doesn’t support our procedural pipeline. The first reason is that light probe data is stored in scenes only, but we use prefabs for our rooms. The second and main reason is that probe positions are set at bake time and cannot be moved. A specific room’s position in world space is not fixed in our case. We have to be able to spawn that room wherever our level generator tell us to.
  14. At a high level this is how the light probes pipeline is separated in Unity. We have the generation part that runs in the editor when baking lighting. There’s the renderer that samples light probes at runtime for each dynamic object. It finds the probes that surround the object’s position and interpolate their values and then there’s the shader that uses that information to render the object. The only part of this that’s not compatible with our procedural pipeline is the renderer. So what if we just replaced with our own custom implementation?
  15. So how would do that? We write the light probes data generated by Unity to a binary file that we attach to each room prefab. When a new room is attached at runtime we deserialize the probes data and attach the probe to a global probes network for the level. Then every frame we sample the probes network to find probe information for each dynamic object based on their position. Then we send over that data to the shader.
  16. How do we manage that data? Well once lighting an been baked the LightPorbes class contains a SphericalHarmonicsL2 object for each probe in your scene. Under the hood this is just an array of floats so it’s easy to serialize. The main problem we faced here is that the shader expects a different format and we didn’t know how to translate a SphericalHarmonicsL2 object into the format the shader expects. Well it turns out there’s a very important line about this in the Unity documentation. It says: “The Unity shader code for reconstruction is found in UnityCG.cginc and is using the method from Appendix A10 Shader/CPU code for Irradiance Environment Maps from Peter-Pikes paper.” That links to a paper called Stupid Spherical Harmonics tricks by Peter-Pike Sloan. Appendix 10 of that paper contains the precise formulas that you need to feed a SphericalHarmonicsL2 object to Unty’s Standard shader.
  17. But how exactly do you feed that data to the shader. The standard way of filling in shader properties is through materials. But probe values could be different for every single object even if they share the same material. We don’t want to create a unique material for every object, that would prevent batching. Unity has a feature for cases like this called MaterialPropertyBlocks. This allows you to fill in shader properties per object without having to go through a material. When using this you also need to set the [PerRendererData] tag on you shader property so that the engine knows your property will be filled in through MaterialPropertyBlocks. For probes data specifically you should set the lightProbeUsage value of all your renderers to “Custom Provided” This tells Unity that you’re responsible for filling in spherical harmonics. That way Unity won’t override your values.
  18. Overall replacing that system worked well for us, but here are a few considerations. Sampling and interpolating probes for each dynamic object each frame can be pretty expensive so you need a way to know which renderers will or won’t move. Objects that don’t move should sample the probes once when spawned and then never again. Objects that do move should not sample probes if they have not moved since the last sampling. In our case we fill rooms with probe based on a 3D grid which means we a regular layout for our probes. This is very helpful because it allows us to find the closest probes to any point in space in constant time. Without a regular layout it becomes a much more complex problem to find the surrounding probes for any position. Unity’s default system supports that with by tetrahedral tessellation, but we didn’t need to reimplement that on our side. Another thing to keep in mind is that the shader properties used for spherical harmonics also contain the scene’s ambient color, so you’ll need to handle that yourself by adding the ambient color with the probe’s values before sending them to the shader.
  19. Not running at high enough frame rate Android might be more affected
  20. CPU side rendering is costly and could often be a bottleneck While GPU side could be as well but Tile based GPU deal well with overdraw Generally shader performance is easier to deal with The number of draw calls is a big performance driver for both CPU and often GPU Control the number of passes Shadows Depth pass Pixel lights etc. Efficiency of batching: static and dynamic Efficiency of graphics API used…
  21. Culling is significant in reducing the number of draw calls A procedural game cannot use precomputed visibility culling Dynamic visibility culling  interior strategy exterior strategy Dynamic occlusion culling for town environment Buildings are natural occluders for other buildings, vegetations, decorations, NPCs etc.
  22. Graphics API is a huge performance driver What is faster OpenGL or Vulkan? OpenGL is a traditional API maintaining internal state Vulkan is much more modern with many facilities to reference states and rendering data
  23. Performance depends on graphics drivers… Loading times and FPS We discovered shader cache on some OpenGL drivers Vulkan is generally faster… But could be slower compared to OpenGL on other devices
  24. Dynamic graphics API selection to attempt the best of both world Unity cannot change graphics API in the run time But there is a provision for auto selection and fall back  Can also be initialized with a different API Requires custom Unity build To implement -- needs custom player Activity (java) Checking device or SOC name to pass a flag to unity
  25. Vulkan is viable Not many graphics artefacts Faster overall Beware validation layer Helps shader compilation caching Test in your specific case Both frame performance And loading times Dynamic API selection when beneficial Works in debug out of the box May need a custom Unity build May make it into main line Thanks to Unity for their help working with us to implement
  26. Continuing with Android:  How Unity distributes threads to different cores  Most modern Android devices are 8-cores There are big cores that are clocked higher And small cores that are slower but consume less power Unity game can easily have 30-40 threads running Most important ones are: UnityMain UnityGFXDeviceW – multithreaded rendering is presumed to be used Worker threads These will be used more and more with DOTs and Jobs system Some systems use these already  E.g.: skinning, physics, particles Use Systrace to find out how threads are distributed
  27. Core jumping Contention between workers and main/rendering threads Little cores not used…
  28. Putting everything on big cores might actually be ok These are faster Workers and not uber busy We experimented with alternative affinities
  29. Reserve cores for UnityMain and UnityGFXDeviceW Allocate workers (and choreographer and audio etc.) to other cores big and small
  30. This is what the result looks like Generally found this to be measurably faster (5-10%) Only 4 workers We asked Unity for a feature to allocate more workers
  31. Nav mesh generation benefits from workers on little cores Need to code a custom plugin to discover thread ids and adjust affinities Plugins are very easy with IL2CPP Setaffinity syscall to set core mask Not a major performance improvement Yet measurable We asked Unity for more workers Systrace is very useful iOS has much better single core performance (fewer cores though) Thanks to Google for their help
  32. Running out of memory iOS is more affected Generally less memory on iOS devices, many with only 2G
  33. Tools give different reading Xcode widget is the most important reading – OS jettisons apps Other tools give smaller estimates  Modern mobile devices memory system is complex
  34. Native memory for asset data: textures, meshes etc – Reference counted Managed memory for Unity Objects and C# objects – GC collected Doesn’t account for everything in a Unity app though Virtual memory space – large, exceeds physical space Physical wired memory is only a small part Consists of wired dirty memory, Reusable (GPU uses this for storage during rendering) External, encrypted code Some read-only assets can be unloaded Some dirty memory can be compressed
  35. Use mach functionality to get detailed view
  36. A practical view of memory types evolving over time Overall use of internal memory is important Managed heap state is important Compressed memory utilization is important
  37. Xcode widget reflects what OS is using for decision making There are many memory types and several levels of allocators Tools compute memory differently and often conservatively Use mach functionality to get details
  38. Heap growth is sizable steps Sometimes we are short of memory and have MM of space in unused managed heap slack
  39. Unity uses Boehm-Demers GC Easy enough to write a plugin that externs some GC statics One to note is GC_free_space_divisor that controls how heap will expand Can increase the divisor to have heap expanding less
  40. Unity is great of exposing GameObject hierarchy Tools can be used to automate finding GameObject leaks It is harder for native and managed memory There is a Unity memory profiler which is improving Allows to compare captures and drill down Custom tools can be build using low level memory API The API provides a snapshot of all memory blocks in native and managed memory
  41. We had to build a custom tool to help finding C# level leaks Can take a snapshot that compare with previous snapshot based on type names Also built a feature to see which types refer to which types Often times it is not drilling down into individual object but high level view that helps figuring out leaks Here is an example? Perhaps a leak?
  42. We have access to Boehm statics Exercise caution as with anything low level Thanks to the team