Social Point Analytics in AWS summarizes Social Point's use of AWS services for analytics. Social Point is a top mobile and Facebook gaming developer. They use Redis to store analytics events, S3 for log file storage, SQS for event processing, and Redshift for analytics querying and transformations. This architecture provides scalability, flexibility and reliability for Social Point's analytics needs.
2. Social Games developer for Mobile & Facebook
Founded in 2008, offices in Barcelona(22@) and San Francisco
Top #20 mobile grossing games worldwide
Top #3 Facebook developer
3. Social Games: interaction
between friends; virality
Freemium model: Play for free
with in-app purchases
Midcore
Leader in Breeding & Collecting
strategy games
4. Top 20 Grossing in iOS
App Store worldwide
Recently launched for
Android, featured on
Google Play
6M DAU on Facebook
5. No hardware maintenance or planning: business benefits from increased
speed
Flexible: Pay for use
Facilitates scalability:
Auto Scaling
Facilitates high availability: multiple availability zones
Managed components: Load Balancers, Databases …
6. Analytics Driven. Vital for almost every team.
Engineers: realtime analytics, monitoring, detecting problems
Product: taking decisions, A/B testing, game balancing
Marketing: optimizing campaigns
Finance: monitoring the business
8. Backend writes events in Redis lists
Why Redis?
Cost and Performance: 10K events/second/server
Problem: it’s a memory-based database; queues have to be constantly
consumed
Scaled and HA: randomly distributed N servers
BACKEND
REDIS
REDIS
REDIS
9. Python processes continuously
consume queues and:
Calculate Real Time metrics
Store event logfiles to
upload to S3
GENERATION Of EVENTS
Redis Queue
LPOP event
Consumer
Redis
Real Time
write event
Event Log File
Enqueue S3 object URL to
SQS
INCR
counter
put object
Amazon S3
LOAD DATA
Amazon SQS
enqueue S3 object URL
10. Why these technologies?
GENERATION OF EVENTS
Python is well suited to developing
workers and dealing with data
Redis Queue
Redis: structures like counters, sets,
sorted sets for Real Time metrics
Consumer
LPOP event
put object
Amazon S3
SQS reliability and availability at a
higher cost than Redis
Redis
Real Time
write event
Event Log File
S3: virtually infinite space, scalable,
high availability
INCR
counter
LOAD DATA
Amazon SQS
enqueue S3 object URL
11. EVENT PROCESSING
A process called importer:
Amazon S3
Amazon SQS
Reads SQS URLs
Downloads S3 logfiles
Importer
It converts them to TSV
TSV
And imports multiple logfiles to Redshift
at the same time
RedShift
12. Allows flexibility-> schema changes with no downtime
Highly scalable (but you cannot write while scaling)
Low implementation risk
It is an offline system. Gameplay is unaffected by downtime.
We have backups. In the worst case, we restore a backup and reload.
Minimum maintenance: only vacuums, space monitoring
Good SQL support, unlike other columnar data bases
13. Daily transformations and calculations implemented in SQL
Example:
UPDATE USER SET total_revenues = (SELECT SUM(amount) FROM transaction t
WHERE t.user_id = user.user_id);
Why not hadoop?
Much slower and complex; for now SQL operations meet all of our
needs. In Redshift these SQL operations are very fast.
14. ¿Would you like to work in the video gaming sector?
Talent attracts talent. You have the talent, we have the
playground.
www.socialpoint.es/jobs
¡Thank you!