Ross Kukulinski speaks about "Serious Communication for Serious Games" at Serious Play Conference 2012 ABSTRACT: Voice communication is a key component of military-unit based training. Soldiers rely on communications skills on the battlefield to share information within a unit and throughout the chain of command. Both observing proper radio protocol and verbally relaying information between unit members are essential in dismounted soldier and convoy training. State-of-the-art solutions for serious games have been inadequate for reinforcing these important skills. While some games do offer integrated voice solutions, the implementation often fails to resemble realistic battlefield scenarios. And most in-game solutions also do not provide interoperability with the existing deployed base of military simulators or instructor stations. In this session, the speaker will demonstrate a new communication product that provides these capabilities. Based on input from actual military training facilities, this solution is can be used to augment existing serious game training, raising the fidelity of the simulation.

1. Serious Communication for
Serious Games
Ross Kukulinski
ross.kukulinski@asti-usa.com
Serious Play Conference

2. Photo by Derek Jensen
2

3. Photo Courtesy of U.S. Military
3

4. Roadmap
1. Serious games and teamwork
2. Communications modeling
3. Fidelity
4. Game integration
5. Network interoperability
6. Final thoughts and future directions
4

5. Part 1
Serious Games are
Powerful Tools
5

6. Serious Games …
• Allow soldiers to experience situations that
are impossible in the real world1
• Provide improved hand-eye coordination,
multi-tasking, and teamwork2
• Are uniquely flexible to support varied training
needs
1
Corti, 2006; Squire & Jenkins, 2003
2
Michael & Chen, 2006
6

7. ‘Good’ Serious Games
Six Ingredients to a ‘good’ game1
1. Mechanics
2. Rules
3. Immersive Graphics
4. Interactivity
5. Challenge
6. Risks
7. What about communication?
1
Derryberry, 2007
7

8. Fundamentals of Teamwork
The Big Five Core Components of Teamwork1
1. Team Leadership
2. Performance Monitoring
3. Backup Behavior
4. Adaptability
5. Team/Collective Orientation
Hypothesis: Communication key element?
1
Salas, Sims, & Burke, 2004
8

9. Communication and Performance
• America’s Army experiments
– Researchers measured team communication
• Communication network level
• Number of report-ins
• Number of normal communications
• Teams with regular organized reports had:
– Higher performance
– Higher estimated situational awareness
Schneider & Carley, 2005
9

10. DARWARS Ambush!: Authoring
Lessons Learned in a Training Game1
• Communication skills are critical for success
– Requires effective communications training
• Communications capabilities differ widely
across varying military units
• Training system should be similar to real-
world communication system
1
Diller, Orberts, Blankenship, Nielsen, 2004
10

11. DARWARS Lessons Cont’d
• Primary functions of a convoy commander
– Establish and maintain communications within the
convoy
– Maintain communication with superordinate and
subordinate element commanders
Diller, Orberts, Blankenship, Nielsen, 2004
11

12. A Training Transfer Study of Serious
Games
“Our work in this project demonstrated
consistently through all five experiments
that communications is fundamental to the
training experience and one of the most
important aspects of the exercise.”
Major Ben Brown
MOVES Institute
Naval Postgraduate School
Brown, 2010
12

13. Game Communication Options
• Nothing
• Text-chat
• Game integrated voice communication
• Third-party voice communication
None simulate real-world communication!
13

14. Our Customer Feedback
• Some were content with what they had
• Some engineered custom solutions
• Many were frustrated
– Current game communication systems…
• are not robust
• are difficult to manage in large installations
• do not simulate real-world radio communication
• do not integrate well with other training systems
• lack live technical support and expertise
14

15. Brief History Lesson
• Full Spectrum Warrior – 2000-2003
• America’s Army – 2002
• DARWARS Ambush – 2005-2009
• ‘Game After Ambush’ (VBS2) – 2009-2012/13
• ‘Games For Training’ Recompete – Q4-2012
15

16. Communication Specifications
• Game After Ambush (2009)
– 8,118 words in technical specification
– 128 words for describing communication
• Games for Training (Draft – October 2011)
– 5,113 words in technical specification
– 128 words for describing communication
• Games for Training (Draft – April 2012)
– Finally requires high-fidelity voice communication
16

17. Part 2
Communications
Modeling
17

18. Modes of Voice Communication
• Intercoms
• Radios
• Earshot
18

19. Intercoms
• Full-duplex
• One channel per
‘wire’
• 1-to-1 or n-to-n
participants
• Phone/Conference
call
Photo by K!T
19

20. Radios
• Half-duplex (usually)
• 1-to-n participants
• Many configurations
possible (AM, FM,
PT/CT, freq, encoding,
etc.)
• Variable
communication link
quality
• Complex and hard to
simulate in real-time Photo courtesy of U.S. Military
• Noisy! 20

21. Earshot
• Simulated voice communication
– Full duplex
– Volume and quality degrades over distance
21

22. Simulating Real-world Communication
• Simulated radios behave like real-world radios
– AM, FM, Frequency Hopping
– Half-duplex radios
– Full-duplex intercoms
– Real-time dynamic radio noise
– Realistic propagation effects due to ranging,
occulting, radio power level, and terrain
– Crypto system sound effects
22

23. Simulating Real-world Communication
• Simulate voice communication (Earshot)
– Volume and quality degrades over distance
– Separate from radio simulation
• Trainees limited to channels they would have
in real-world
23

24. Part 3
Fidelity
24

25. Acceptable Fidelity
• What is the training goal?
• What is the real-world communication?
• Combined arms, convoy, and small unit
communications “must be correct and
effective”
• “One can debate the level of fidelity needed
for useful training, but fidelity must certainly
be high when it relates to the specific task
being trained”
Brown, 2010
1
25

26. Basic Intercom
26

27. Intercom and Individual Radios
27

28. Geolocated Individual Radios
28

29. Geolocated Vehicle & Individual Radios
29

30. Highest Fidelity
30

31. Part 4
Game Integration
31

32. “Quite simply, communications should be
as seamless as all other aspects of [the serious
game]. Communications should be internal to
[the game] with seams between vendor
production transparent to the user.”1
1
Brown, 2010
32

33. Administrative Interface
• In addition to personnel and weapons,
game scenarios should include
communication
• Radio/intercom configuration and
allocation
• DIS/HLA configuration
• Re-usable communication configuration
33

34. User Interface
• Two possible views
– Heads Up Display (HUD)
– In-game objects
• HUD
– Simple and intuitive
– Not realistic – does it break flow?
• In-game objects
– Higher realism – does it impede training?
34

35. User Interface Capabilities
• Regardless of view mode:
– Support for multiple radios and intercoms
– View radio channel and Tx/Rx status
– Support changing radio channels
– Dynamic vehicle communication systems
– Earshot voice communication
35

36. Required Simulation Information
• Radio location from game entities for realistic
radio effects like ranging and occulting
• Player location for Earshot voice
communication
• Assign radios and intercoms to vehicles for
mounted training
– Players acquire vehicle-based radios when
mounted, lose access when dismounted
36

37. After Action Review
• “Both simulation groups commented
extensively on the AAR tool. Both groups
believed the AAR tool was critical in providing
a big picture view of what happened during
the exercise.” 1
• Communication playback synced with visuals
• Seek, Pause, FF, RW, Bookmarks
• Export audio/visual for later analysis and
study Brown, 2010
1
37

38. Part 5
Network
Interoperability
38

39. Live-Virtual-Constructive Example
39

40. DIS & HLA
• Distributed Interactive Simulation
– Wire-level specification
– UDP
– Simple!
– But standard slow to evolve
• High-Level Architecture
– Run Time Infrastructure
– Set of API functionality
– Federation Object Model
– Very complex
40

41. Networked Voice
• Audio encoding and sample rate
• Dynamic packet sizes
• Latency
– Maximum 150ms one-way latency
– Latency <100 ideal
• Unreliable networks (jitter, lost packets)
41

42. Part 6
Final Thoughts
42

43. Summary
• Communication is critical for teamwork
• Real-time communication simulation is
computationally complex
• Serious games require high-fidelity
communication for effective training
• Requirement to network disparate training
systems into a common network
43

44. Resources
• Brown, B., (2010) A Training Transfer Study of Simulation Games
• Carpenter, R., White, C., (2005) Commercial Computer Games in the Australian
Department of Defense
• Corti, K. (2006) Games-based Learning; a serious business application.
• Derryberry, A. (2007) Serious Games: online games for learning
• Diller, D., Roberts, B., Blankenship, S., Nielsen, D. (2004) DARWARS Ambush!
Authoring Lessons Learned in a Training Game
• Hussain, T., etal (2010) Development of game-based training systems: Lessons
learned in an inter-disciplinary field in the making
• Hussain T. & Ferguson, W. (2005) Efficient Development of Large-Scale Military
Training Environments using a Multi-Player Game
• McGowan, C., Pecheux, B. (2007) Serious Games that Improve Performance
• Michael, D., & Chen, S. (2006) Serious games: Games that educate, train and inform
• Sims E., Salas E., Burke S. (2004) Is There a ‘Big Five’ in Teamwork
• Snider, M., Carley K., Moon, I. (2005) Detailed Comparison of America’s Army and
Unit of Action Experiments
• Squire, K. & Jenkins, H. (2003) Harnessing the power of games in education
44

45. Thank You!
Ross Kukulinski
ross.kukulinski@asti-usa.com
45

46. Useful Links:
www.seriousplayconference.com
www.seriousgamesdirectory.com
www.seriousgamesassociation.com
Contact:
sbohle@seriousgamesassociation.com