An overview of the differences between film and interactive game techniques and their use in virtual production.

1. Widescreen Test Pattern (16:9)
Aspect Ratio
Test
(Should appear
circular)
16x9
4x3

2. THE LONG ROAD TO
FILM / GAME CONVERGENCE

3. DISCLAIMER
I am going to make a lot of generalizations.
There are always exceptions.

4. ABOUT ME
Christopher Evans
• Technical Art Director
• Product Manger (CineBox)

5. BRIEF OUTLINE
Talk focuses on how film can
benefit from gaming technology.

6. BRIEF OUTLINE
 Conceptual and technological differences
 CONVERGENCE!
 Why the idea excites people, why it hasn’t happened
 Game tech applications in film
 Case studies
 CryENGINE Cinema Sandbox (CineBox)
 Issues that we have run into, solutions

7. THE
GAP

8. LINEAR
STATIC
2D
NON-LINEAR
DYNAMIC
3D

9. THE DETAILS
PRESENTATION
Single view, 2D images
Interactive view, 3D assets
TIME
Static, based on the current cut
Dynamic, based on player progression

10. THE DETAILS
IMAGE GENERATION
Not time dependent
Time dependent (‘interactive framerate’)
ANIMATION
Hand tailored to one specific view
Interactively blended from library, any view

11. THE DETAILS
PHYSICS
Static. Bake the hits, ignore the misses
Dynamic, based on physical attrs
LIGHTING / MATERIALS
Lit before render time, mats often for one env
Lit real-time, need to work in any env

12. THE DETAILS
AUDIO
To the cut
Generated by physical impacts and triggers
VFX / PARTICLES
As many attempts as you’d like. Bake. Record.
All live. Based on physics, physical materials

13. THE GAP IN UNDERSTANDING
Film is linear, static, 2D
Hand-crafted, directed, optimal
Games are non-linear, dynamic, 3D
Interactive, chaotic, user-controlled

14. Super-specialized for one thing

15. Super-specialized for one thing
General, but can handle anything

16. “REAL-TIME ALL THE TIME”

17. “REAL-TIME ALL THE TIME”
CAN BE PAINFUL!

18. Why are people excited about these
technologies converging? Who has tried?
© AWN, 2006

19. WHY IS THERE INTEREST?
 REALTIME creative iteration, brainstorming
 30-40% is ‘just finding your feet’
 Fast iteration = More time experimenting
 Everything realtime means realtime inter-
disiplinary collaboration
 Game engines visualize complexity well
 Game engines are world simulators

20. GAME TECH APPLICATIONS IN FILM
 Rendering
 Pre-vis, Post-vis, Pitch-vis
 Blocking / layout
 Lighting
 Virtual Production
 Performance capture
 Live collaboration
 Puppeting or ‘playing’ characters
 Utilization of same assets

21. THE
GAP

22. COMMON CONVERGENCE MYTHS
 Many games are created by the
same studios that do the films!
 Game assets are derived from
original film assets
 Game engines are used for film
previs

23. CASE STUDIES OF
GAME TECH USED IN FILM

24. PREVIOUS CASE STUDIES: THE ABYSS

25. PREVIOUS CASE STUDIES : THE ABYSS

26. PREVIOUS CASE STUDIES : THE ABYSS
 David A. Smith built ‘Deep Core’
in Colony Editor
 Team realized one wing of set
would not appear on camera
 Smith went on to create Virtus
Walkthrough

27. PREVIOUS CASE STUDIES: AI

28. PREVIOUS CASE STUDIES: AI

29. PREVIOUS CASE STUDIES : AI
 Modded engine to support
lenses
 Used for Rouge City location
scouting
 ILM went on to create virtual set
tracking/previs with Softimage
Realtime Viewer

30. OUR CASE STUDY:
CryENGINE CineBox

31. Video to keep interest

32. small team, big idea

33. ANOTHER DISCLAIMER:
We just left alpha!

34. ISSUES THAT WE HAD TO SOLVE
 Rendering
 Simple lights/shaders (one shading model)

35. SOLUTIONS: Approximated Area Lights
MENTAL RAY CINEBOX

36. SOLUTIONS: Improved Lighting MENTAL RAY
~40sec

37. SOLUTIONS: Improved Lighting CINEBOX
~24ms

38. ISSUES THAT WE HAD TO SOLVE
 Rendering
 Simple lights/shaders (one shading model)
 Area Lighting, Improved AO, SSS, physically accurate rendering,
multiple shading models (Ward anisotropic, Cook-Torrance specular, etc)
 All color and lighting calculations in linear space
 Support for custom IBLs (import/export)

39. CINEBOX
~28ms

40. Specialized for one thing

42. CINEBOX
~20ms

43. ISSUES THAT WE HAD TO SOLVE
 Rendering
 Simple lights/shaders (one shading model)
 Area Lighting, Improved AO, SSS, physically accurate rendering,
multiple shading models (Ward anisotropic, Cook-Torrance specular, etc)
 All color and rendering in linear space
 Support for custom IBLs (import/export)
 No support for standard rendering features

44. SOLUTIONS: Render Output Dialog

45. ISSUES THAT WE HAD TO SOLVE
 Rendering
 Simple lights/shaders (one shading model)
 Area Lighting, Improved AO, SSS, physically accurate rendering,
multiple shading models (Ward anisotropic, Cook-Torrance specular, etc)
 All color and rendering in linear space
 Support for custom IBLs (import/export)
 No support for standard rendering features
 Floating point render targets to channels in EXR
 Shot batching, rendering to system codecs
 Offscreen rendering, scene pre-roll
 In-viewport stereo

46. ISSUES THAT WE HAD TO SOLVE
 Cameras
 Game engines have no concept of ‘lenses’ or ‘filmback’
 Not an acurrate camera model

50. Stephen bokeh screen 3
CINEBOX
~30ms

51. ISSUES THAT WE HAD TO SOLVE
 Cameras
 Game engines have no concept of ‘lenses’ or ‘filmback’
 Not an accurate camera model
 Re-wrote camera model - filmback, shutter, aperture
 All attrs exposed to liveSync interface (Maya/Mocap sync)
 User editable tonemapper – OpenColorIO support

52. ISSUES THAT WE HAD TO SOLVE
 Characters / Animation
 Final anim comes from AI-pathfinding/state-
machine/blended clips/physics/etc
 No external data input (motion capture, DCC apps)
 Support for Vicon, Motion Analysis, Organic Motion,
PhaseSpace
 Display/record, piece together clips, shoot cameras on clips
 SDK / API
 [Vicon vid, 2009]

53. ISSUES THAT WE HAD TO SOLVE
 Characters / Animation
 No external data input (motion capture, DCC apps)
 Support for LiveSync to DCC apps (Maya)
 Characters, cameras, lights, etc…
 [liveSync Video]

54. ISSUES THAT WE HAD TO SOLVE
 Difficult to integrate into a pipeline
 Engines cannot handle mesh complexity and
displacement
 New formats are a pain, scene translation

55. SOLUTIONS: Catmull-Clark Subd

56. ISSUES THAT WE HAD TO SOLVE
 Difficult to integrate into a pipeline
 Engines cannot handle mesh complexity and
displacement
 32bit mesh support, Catmull-Clark subd [vid1, vid2]
 Displacement
 New formats are a pain, scene translation
 Alembic cache support [alembic_video]

57. SOLUTION: SUPPORT OPEN FORMATS

58. BIGGEST ISSUES REMAINING
 Pixel-Accurate Playback and Determinism
 Physics/AI/Particles/Sound
 CryENGINE is a dynamic world simulation
 Still not ideal to get complex things working (crowds,
etc)
 Playing nice with other devices
 HD SDI video, timecode, thin clients, cameras, etc..

59. INTERNAL PROJECTS:
[warface_trailer] [warface_making_of]

60. OUR VISION
 Paradigm shift in how linear content is made
 The power of games is REALTIME is ITERATION
 Live inter-departmental collaboration
 Physics/AI/Particles/Sound
 CryENGINE is a dynamic ‘multiplayer’ world simulation.
 Power experiment, mock up ideas fast
 Less risk

61. DO YOU BELIEVE?
Do you believe what we’re doing is important? Do you
want to be part of our team?
www.crytek.com
Or speak with me.
Want to be involved in the beta?

62. THANKS:
Interviews: David A Smith (The Colony), Wilson Tang (ILM/AI)
Code Monkeys
Stephen Clement
Serhat Eser Erdem
Vaclav Kyba
Minghao Pan
Xiaomao Wu
Support
Christopher Evans
Kirthy Iyer
Carl Jones
Leon Zeer
TEAM