Given at the European Trading Architecture Summit 2012

1. Welcome

2. It used to be easy…

3. they all looked pretty much alike

4. NoSQL BigData MapReduce Graph Document
Shared Column Eventual
BigTable CAP
Nothing Oriented Consistency
ACID BASE Mongo Coudera Hadoop
Voldemort Cassandra Dynamo Marklogic Redis
Velocity Hbase Hypertable Riak BDB

5. Now it’s downright
c0nfuZ1nG!

6. What Happened?

7. we changed scale

8. tack
ge d
we ch an

9. the
big data
conundrum

10. the
big data
conundrum ?

11. The Internet

12. Which isn’t mostly text
Everything (500,000)
Web Pages (40)
~0.01% is web pages
Words (0.6)
~1% of that is text
Sizes in Petabytes

13. And there is lots of other stuff out there
mobile
weather sensors
Social
Logs
data
audio video

14. Gartner
80% of business is conducted on
unstructured information

15. Big Data is now a new class of
economic asset*
*World economic forum 2012

16. Yet 80% Enterprise Databases < 1TB
Reference
from 2009

17. so what does Big Data mean for
the enterprise?

18. Insight α data
>
Data beats Algorithms

19. Backing up a bit…

20. We live in a world of, largely, private data
Where data is often changed
and forwarded on

21. Sometimes we’re a bit more
organized!

22. But most of our data is not generally
accessible
Core
Operational
Exposed
Data

23. Sharing is often an afterthought
Core
Operational
Exposed
Data

24. How do we process, acquire, reason about
and act upon information?

25. The Brain
Reptilian
Primitive operations:
balance, temperature
regulation, breathing
Mammalian
Emotion, short-term
memory, flee of fight
etc (limbic)
Neocortex
Plan, innovate,
problem solve etc

26. Our intelligence is segregated in
disparate worlds

27. could our corporations be
more intelligent?

28. Siloed, closed, bespoke data makes
our organisations opaque and
unresponsive

29. What if we exposed it all?

30. So what might that look like?
• Single data store
• Federated, homogenous stores
• Federated, heterogeneous stores

31. The Google Approach
MapReduce
Google Filesystem
BigTable
Tenzing
Megastore
F1
Dremel
Spanner

32. The Ebay Approach

33. so is one approach
better?

34. Data Volume?
TB
0 1 10 100 1000 10,000
We live well within the overlap region

35. Academic acumen?

36. Performance Trade-Off Curve
• Volume (pure physical size)
• Velocity (rate of change)
• Variety (number of different types of data,
formats and sources)
• Static & Dynamic Complexity (do you need to
interpret the affect one message has on
another)

37. Problem
Our ability to model data is much more of a
gating factor than raw size, particularly
when considering new forms of data
Dave Campbell
(Microsoft – VLDB Keynote)

38. Gravitate around a single data model
Globally Accessible
Application
Specific
Core Data Models
Core Data Model
Model
Views /
linkages

39. The data itself follows a similar pattern
Globally Accessible
Application
Specific
Core Data
Core Data Model data
Views /
linkages

40. Compose Solutions (for now)

41. Big Data is more than the opportunity for
better insight over new data sources

42. It is the opportunity to make the
organisation smarter, simply by
making data more accessible

43. But the harder job, for us, is unifying
the various domains to make all that
data intelligible

44. Thanks
http://www.benstopford.com
@benstopford