Big Data Expo

1. 8 learnings from
building a Data Platform
DIGITAL INTERNATIONAL

2. Goal
Of this talk

3. About
The org - me

4. Why
Did we embark on this anyway?

5. First, take a step back
Vision
Mission
Objectives
Key Results

6. We have a dream: Vision
It's our vision to be the engine of customer obsessed
intelligence, continuously driving agile fact based
decision making

7. Tangible: Objectives
We have a flat file data lake updated in daily batch with at
least 1 data source

8. Tangible: Key results
Tables ABC are from
source X offloaded in
landing zone
Staging, transformation
and harmonisation are
automated
1 2

9. Learning #1
Laser focus

10. Learn by trying

11. Learn by testing
Set criteria BEFORE
Maintainability Scalability Reliability Security Cost(ability?)
etc...

12. Learning #2
Do POCs

13. Are you ready to roll?
Right skills Budget Data access Infra freedom Sponsor

14. Learning #3
Remove dependencies

15. Hindsight engineering = Pain
GDPR

16. Learning #4
Actively involve privsec

17. Design Principles
● PaaS
● Automate all things
● Separate compute from storage
● Event driven
● Loosely coupled

18. Learning #5
Acceptance criteria

19. With the business for the business

20. Learning #6
Include your customer

21. Don’t make it a big project

22. Don’t make it perfect
Your goal

23. Learning #7
Be Agile - adopt

24. Data is an intangible asset - make it real

26. Learning #8
Share!

29. Learnings
- Learning 1: laser focus
- Learning 2: do POCs
- Learning 3: be independent
- Learning 4: involve privsec
- Learning 5: design principles
- Learning 6: include your customer
- Learning 7: be Agile
- Learning 8: share!

30. Thank you