Data meshes are the latest data architecture trend. Really a paradigm shift. But what actually happens is just the natural evolution of technological decentralization. This is a very short crash course, focusing in on the basics of data meshes.

1. MarsMissions&DataMeshes
by Sven Balnojan; A short introduction to the topic

2. MarsClimateOrbiter,launchedDec.1998.
Mission-Studyclimate,atmosphere&surfacechanges.

3. Targetorbit(150kmalt.)
Possiblecorrections
bymultiplethrusters

4. Lotsofsensors,sending
data"0.42","1.002",...
Thrusterscanbe
controlledfromground
(timelapse:~30mins)

5. Sensors:Builtbysome
otherentity,inthiscasea
companyformilitarytech
called"Lockhead".
Thrusters:Controlledby
the"maincontrolling
entity"whichevalutesthe
data &thenprovidesitto
decisionmakers -
missioncontrol-touseit
todecideonproper
actions.

6. Decisionmakers
Soundsfamiliar?Thetypicaldataorg
Providers&analyzersof
data
Datasources Datasources
0.42
(1,2,20) inagreedupon
specification

7. BOOOMMMMM!!!!!!!!!!

8. https://www.simscale.com/blog/2017/12/nasa-mars-climate-orbiter-metric/

9. Domainboundaries....That'swhathappenstocentraldatateamsalldaylong.

10. Theonlydifference?NASAhasasimplefix:interfacebeforehand=>be
happy.
Datateamsontheotherhand=>notachance,becausethebasechanges,
andshouldchangeoften&frequently.

11. MarsMissions&DataMeshes
by Sven Balnojan; A shortintroduction to the topic

12. Agenda
• Theorganizationalproblem(Whoisresponsibleforwhat?)
• Thesolution(Youareresponsibleforthat.)
• Whenshouldweconsiderchanging?
• Threebuildingblocksofdatameshes
• Thisiscompletelynormal(Evolutionofdecentralizationintech.)
• The"light"variant(X-as-a-Service)
• Questions...

13. Theproblem?

15. Thesolution?

17. If/Whenshouldwechange?

18. Twoforcesdeterminewhetheryouneedtochange:
1)SourceCount+SourceComplexity
2)UseCases+(Quality)Demands
=>willleadtoa"pullofdecentralization"whichyou
shouldfeel.
The"PullofDecentralization"
Determiningforces

19. Whatisadatamesh?

20. "OldWineinNewBottles"(toquoteArifWider)
ProductThinking
• Data,notaby-productanymore,but
wrappedina"data-service"subjecttoa
productmanager.
• Applyusualproductmanagement
practicestoadataset(whoisthe
customer?...).
• Data-servicemaintainedinacross-
functionalteam(withitspipelines,docs,
interfaces,...).
Domain-DrivenDistributedArchitecture
• Thedataproducthasexactlyone
domain.
• Theteamcanbecomeadomainexpert.
• Data-servicesbecomenodesinamesh
=datamesh.
• Mightbereasonabletodistinguish
betweensource-orienteddataservices
&higher-orderdataservices(e.g.aCLV)
• Data-serviceonlyatrueone,iff....apply
Platforms(X-as-a-Service)
• Toboth,developdataservicesfastAND
• connectthemtogetherwewant
platformstoreduceduplication(justlike
indeployment,...)
• Animportantpointtofocuson:Domain-
agnosticplatform!

21. Whataboutgovernance?

22. Decisionmakers
Providers&analyzersof
data
Datasources Datasources
0.42
(1,2,20) inagreedupon
specification
Twosources,e.g.twosensorsmadebytwodifferentsuppliers,
yetthedatacanbecombinedbecauseatleasttheformat
(vectororscalar)isfixed.

23. EvolutionofDecentralizationinTech

24. Turnsout,we'vedonethisbefore!Thefourdecentralizationmovesintech.
Monoliths=>SOA/Micro-
Services
• 2014formalizedbyMartin
Fowleret.al.butgoingback
tosomewhat2011
• WHAT: The"code",the
"micro-service"
Ops+Dev=>DevOps
• "Phoenixproject"important
book(2018)butroots
sometime2007/2008.
• Verymuchparalleltothe
micro-servicesmove.
• WHAT:Operationofa
service.
Frontends=>Microfrontends
• FirstontheThoughtWorks
TechRadarin2016
• Somegreatexamplesdone
veryearlybySpotify
• CamJacksonsarticlein2019
withlotsofexamples+tech
implementationideas
• WHAT:Thefrontendpartof
aspecific
service/component.
Dataasby-products=>Data
Mesh
• Firstonstageby
ThoughtWorks2019.
• WHAT:Theraw&possibly
transformed"data".

25. The"light"Variant

26. X-as-a-Serviceisacollaborationmodelbetween
teams.Itaimstohelpteamsbuilddomainexpertise
andremovingfrictions.
HowdoesX-as-a-Servicework
TeamAprovidingX-as-a-ServicetoTeamB
means...
• TeamAandTeamBalmostnever"talk"
• TeamBcanusetheserviceasis,withoutasking
anyone(becauseofgreatdocs&properAPI
versioning)
• TeamAunderstandtheneedsoftheirend-users
andhelpsthemdeliverfasterbyprovidingX-as-
a-Service
X-as-a-Service
X-as-a-ServiceDefinition

27. ExamplesofX-as-a-Service
Models-as-a-Service GTM-as-a-Service Clusters-as-a-Service PushNServe-as-a-Service

28. Questions?

29. Ressources!
• StuffbytheoriginatorsatThoughtWorks
• 2019,https://martinfowler.com/articles/data-monolith-to-mesh.html
• 2020,https://martinfowler.com/articles/data-mesh-principles.html
• https://www.thoughtworks.com/de/webinar/data-mesh
• Stuffbyothers
• BarrMoses,https://towardsdatascience.com/data-mesh-101-everything-you-need-to-know-to-get-started-72087f5a7d91
• Stuffbyme
• 2019,https://towardsdatascience.com/data-mesh-applied-21bed87876f2
• 2020,https://towardsdatascience.com/theres-more-than-one-kind-of-data-mesh-three-types-of-data-meshes-7cb346dc2819
• 2020,https://medium.com/swlh/microservices-data-meshes-micro-frontends-and-the-timeless-principles-of-decentralization-2ac2516b2951