By Kristoffer Benjaminsson, CTO, Easy.
This talk presents the telemetry system used in Battlefield Heroes and how it helps the team make technical decisions in order to provide the best service possible. We will show real life examples of how telemetry helped improve matchmaking, reduce latency for players and help find false alarms from the cheat detection system. We will also discuss how telemetry can be used in development for catching bugs and support game designers in their work.
4. Telemetry - Wikipedia ” Telemetry (synonymous with telematics) is a technology that allows remote measurement and reporting of information. The word is derived from Greek roots tele = remote, and metron = measure.”
5. Telemetry - Easy Data that helps us make decisions to provide a better service for our players Provides answers to specific questions Multiple sources Financial data Game statistics Web behaviour data Game telemetry Vertical slice of the player population
6. Why bother? We prefer to know what is going on instead of guessing We can measure results from actions Data do not (usually) lie Forums not representative Less than 20% read forum posts Less than 5% post in forums
7. Game telemetry examples Latency data Punkbuster kicks Matchmaking Time to level Hardware profiles Play time
9. Implementation TelemetryManager sends data to our backend for database storage Hooks in the game code collects data Hooks are records packed as name value pairs Each record will create its own table in the database All telemtry includes a common identifer so we can cross reference data between telemetry hooks
10. Example record class PingRecord : public ITelemetryRecord { public: PingRecord(const std::string& clientIp, const std::string& serverIp, int ping, const std::string& dataCenter, const std::string& community); public: virtual boolhasType( inttypeId ) const { return typeId == RTPing; } virtual bool assimilate( const ITelemetryRecord* other ) { return false; } virtual IDataCollector::DataRef buildData() const; private: std::string m_clientIp; std::string m_serverIp; int m_ping; std::string m_dataCenter; std::string m_community; };
11. Example usage player->addTelemetry(new backend::telemetry::PingRecord(clientIp, serverAddressAndPort, conn->getAveragePing(), serverSettings->getProdDataCenter(), serverSettings->getServerCommunity()));
23. Asserts We collect all asserts fromboth client and server We differentiate between content asserts and code asserts We have leaderboards and summaries Top 10 asserts Total content/code asserts per branch Provides guidance to what we need to fix
24. Fragalyzer Level design helper tool Collects information about how a map is played Kill position Killed from position Player/Vehicle movement Etc Currently collected locally on a game server
25. Summary Telemetry is awesome! Collect data to answer specific queries Build a flexible system to make it super easy to add new telemetry hooks Chose the right reporting tools to mine and report on the data