how to solve Fast Data Problem and reacting to the World faster

1. Reactive Data System
in Practice
Or how to solve Fast Data Problem
and reacting to the World faster
By Triều Nguyễn at http://trieu.xyz
http://www.rfxlab.com (Reactive Data System Lab)
λ(x)

2. 2008: Java Developer, develop Social Trading Network for a
startup (Yopco)
2011: joined FPT Online, software engineer, worked in FPT ID
project and VnExpress Mobile (Backend)
2012: backend engineer at Greengar (Scaling server and real-
time services)
12/2012 to 05/2015: back to FPT Online, lead engineer at
eClick Ad Platform
06/2015 to now: lead engineer at iTVad, a startup project at
FPT for OTT and TV Advertising Solutions
about me

3. 1. Just some concepts
2. Ideas in practice
3. What is RFX framework ?
4. Why is RFX ?
5. How to solve problems with RFX ?
a. Counting pageview of website
b. Counting unique user of website
c. Querying data more reactive and adaptive
6. Final ideas
Contents of this talk

4. 1 - Just some concepts

5. Fast Data vs Big Data

6. What is Reactive Data System ?
is a software design approach that uses the
principles of event driven architecture for the
design and implementation of data intensive
applications.
Event Driven Architecture
EDA is a framework that promotes production,
detection, consumption and reaction to the
events. Event based applications will run on
RDS framework.

7. Why is Reactive Data System
We want to ask bigger questions faster

8. 2 - Ideas in practice

9. Problem (User Story in Practice)
● User does login with FB Connect at
Ecommerce website
● User does a click on an item, then system
should do the following tasks:
1. Find best items to recommend
2. Find a best item to send email marketing
3. If there are more than 100 users, that click on a item,
run a marketing campaign on Facebook
automatically.

10. An example from E-Commerce

11. 3 - What is RFX framework ?

12. ● A framework for big fast data problems
● A collection of Open Source Tools from Netty to Spark
● The mission of RFX
1. Build data product quickly with design patterns
2. React to critical events in real-time
What is RFX or Reactive Function X ?

13. RFX is inspired by the Actor Model Theory

14. Paper: http://vialab.science.uoit.ca/textvis2011/papers/textvis%202011-rohrdantz.pdf

15. RFX is designed for startup business
I design RFX because I have to solve following problems both at
FPT and at RFXLab.com
● Marketing Automation
● Product UX personalization
● Real-time Advertising
● System monitoring (frontend and backend)
● Email marketing
● Product Analytics
● Customer Analytics
→ So I build a full-stack framework (a collection of tools for
both backend and frontend)

16. Philosophy of RFX

17. Sub modules in RFX framework

18. 4 - Why is RFX framework ?

19. Why Rfx ?
● Ideas since 2012 (from Actor model theory)
● R&D and Deployed in Production since 2013
● Open Source: Apache License, Version 2.0
● Full Stack: from Frontend to Backend
● Agile for Data Analytics
● Lightweight Lambda Architecture
● Really fast and near-real-time processing
● Tested with 1.000.000 events / second
● Simple development model for data engineer

20. Roadmap to the future

21. 5 - How to solve problems
with RFX ?

22. Problems
User story in Digital Marketing:
1. Counting pageview of website
2. Counting unique user of website
3. Real-time marketing

23. Find rush hour in your system

24. Which device user reach our content

25. Solution Architecture
Tracking
(RFX-track)
Event Queue
(Apache Kafka)
Event Processor
(RFX-stream)
RedisMongoDB
Synch Data Jobs
(RFX-job)
Report
(RFX-report) Event Actor
(RFX-actor)
Ad Server

27. An example topology

28. 6 - Final ideas

29. Summary
● Fast Data is cool
● Fast data: The next step after big data
● RFX is designed to solve common Fast Data
problems in Digital Marketing and Advertising

30. https://github.com/rfxlab/rfx

31. The end and thank you
https://github.com/rfxlab
http://rfxlab.com
λ