SlideShare a Scribd company logo
1 of 22
Download to read offline
Max Klymyshyn
       CTO at GVMachines




Data Driven Design
Build app design around data
flow rather than program flow



 tweet tweet @maxmaxmaxmax
What's the problem?




  Handling many of different data
    sources and keep context
Typical task for this approach




             Random picture from Google
Really?


   Yup, honestly it's depend on the task.

  There's a lot of examples of data-driven
           programming around.

     The most known example is
        Django Middleware
So, let's go deeper
Data Driven
programming
   definition
What is Data Driven programming



     Data driven programming is a
      programming model where
the data itself controls the flow of the
  program and not the program logic
Not so strict




     Data Driven definition is quite strict.

Typically it's mixed with other approaches
Real world task
   sample usage
Show only sunny days in the month




 I want to grab weather from different sources
   and display only sunny days of the month
Pipeline




To grab data we should define pipeline - the
way our data going to go
Pipeline



   Grab    Parse   Validate   Display
In details
    Grab

                      Fetch data from weather.com
                      Fetch data from pogoda.yandex.ua
                      Fetch data from weather.yahooapis.com




    Parse                    Parse weather.com
                             Parse pogoda.yandex.ua
                             Parase api.aerisapi.com



           Validate                  Pass sunny days only
Pipeline




At this moment we have only sunny days to
display
Approaches
when this may be be effective?
If this then that



ifttt.com
ifttt is a web service platform that connects
various web services together to automate
common tasks on the web
Various sources


When you have a lot of mostly similar data from
different sources

You need to keep only data which
matters for you
Data slices generation




In case you need to generate a lot of similar
data sets filtered by many of params
Pipes




In general, approach is the same as usage of
UNIX pipes:

cat file.dat | grep something | sed 's/xxx/yyy/g'
Summary



  Grab    Parse   Validate   Display
Example




https://github.com/joymax/data-driven-design
That's all, thank you.
     Questions?

tweet tweet @maxmaxmaxmax

      Github: joymax

More Related Content

Viewers also liked

KharkovPy #12: I/O in Python apps and smart logging (russian)
KharkovPy #12: I/O in Python apps and smart logging (russian)KharkovPy #12: I/O in Python apps and smart logging (russian)
KharkovPy #12: I/O in Python apps and smart logging (russian)Max Klymyshyn
 
Angular.js - JS Camp UKraine 2013
Angular.js - JS Camp UKraine 2013Angular.js - JS Camp UKraine 2013
Angular.js - JS Camp UKraine 2013Max Klymyshyn
 
Инновации и JavaScript
Инновации и JavaScriptИнновации и JavaScript
Инновации и JavaScriptMax Klymyshyn
 
Fighting async JavaScript (CSP)
Fighting async JavaScript (CSP)Fighting async JavaScript (CSP)
Fighting async JavaScript (CSP)Max Klymyshyn
 
PiterPy 2016: Parallelization, Aggregation and Validation of API in Python
PiterPy 2016: Parallelization, Aggregation and Validation of API in PythonPiterPy 2016: Parallelization, Aggregation and Validation of API in Python
PiterPy 2016: Parallelization, Aggregation and Validation of API in PythonMax Klymyshyn
 
Los Medios audiovisuales en los canales digitales
Los Medios audiovisuales en los canales digitalesLos Medios audiovisuales en los canales digitales
Los Medios audiovisuales en los canales digitalesAgencia Vertice
 
React.js: Ускоряем UX/UI
React.js: Ускоряем UX/UIReact.js: Ускоряем UX/UI
React.js: Ускоряем UX/UIMax Klymyshyn
 
AgileBaseCamp 2013 - Start Up and Get Done
AgileBaseCamp 2013 - Start Up and Get DoneAgileBaseCamp 2013 - Start Up and Get Done
AgileBaseCamp 2013 - Start Up and Get DoneMax Klymyshyn
 
LvivJS 2014 - Win-win c React.js
LvivJS 2014 - Win-win c React.jsLvivJS 2014 - Win-win c React.js
LvivJS 2014 - Win-win c React.jsMax Klymyshyn
 
Изоформные приложения на React.js
Изоформные приложения на React.jsИзоформные приложения на React.js
Изоформные приложения на React.jsMax Klymyshyn
 
Robust web apps with React.js
Robust web apps with React.jsRobust web apps with React.js
Robust web apps with React.jsMax Klymyshyn
 
Трансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScript
Трансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScriptТрансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScript
Трансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScriptMax Klymyshyn
 
Kharkivpy#3: Javascript and Python backend
Kharkivpy#3: Javascript and Python backendKharkivpy#3: Javascript and Python backend
Kharkivpy#3: Javascript and Python backendMax Klymyshyn
 
Communicating Sequential Processes (CSP) in JavaScript
Communicating Sequential Processes (CSP) in JavaScriptCommunicating Sequential Processes (CSP) in JavaScript
Communicating Sequential Processes (CSP) in JavaScriptMax Klymyshyn
 
LvivPy - Flask in details
LvivPy - Flask in detailsLvivPy - Flask in details
LvivPy - Flask in detailsMax Klymyshyn
 
Odessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and PythonOdessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and PythonMax Klymyshyn
 
Testing with Jenkins, Selenium and Continuous Deployment
Testing with Jenkins, Selenium and Continuous DeploymentTesting with Jenkins, Selenium and Continuous Deployment
Testing with Jenkins, Selenium and Continuous DeploymentMax Klymyshyn
 

Viewers also liked (17)

KharkovPy #12: I/O in Python apps and smart logging (russian)
KharkovPy #12: I/O in Python apps and smart logging (russian)KharkovPy #12: I/O in Python apps and smart logging (russian)
KharkovPy #12: I/O in Python apps and smart logging (russian)
 
Angular.js - JS Camp UKraine 2013
Angular.js - JS Camp UKraine 2013Angular.js - JS Camp UKraine 2013
Angular.js - JS Camp UKraine 2013
 
Инновации и JavaScript
Инновации и JavaScriptИнновации и JavaScript
Инновации и JavaScript
 
Fighting async JavaScript (CSP)
Fighting async JavaScript (CSP)Fighting async JavaScript (CSP)
Fighting async JavaScript (CSP)
 
PiterPy 2016: Parallelization, Aggregation and Validation of API in Python
PiterPy 2016: Parallelization, Aggregation and Validation of API in PythonPiterPy 2016: Parallelization, Aggregation and Validation of API in Python
PiterPy 2016: Parallelization, Aggregation and Validation of API in Python
 
Los Medios audiovisuales en los canales digitales
Los Medios audiovisuales en los canales digitalesLos Medios audiovisuales en los canales digitales
Los Medios audiovisuales en los canales digitales
 
React.js: Ускоряем UX/UI
React.js: Ускоряем UX/UIReact.js: Ускоряем UX/UI
React.js: Ускоряем UX/UI
 
AgileBaseCamp 2013 - Start Up and Get Done
AgileBaseCamp 2013 - Start Up and Get DoneAgileBaseCamp 2013 - Start Up and Get Done
AgileBaseCamp 2013 - Start Up and Get Done
 
LvivJS 2014 - Win-win c React.js
LvivJS 2014 - Win-win c React.jsLvivJS 2014 - Win-win c React.js
LvivJS 2014 - Win-win c React.js
 
Изоформные приложения на React.js
Изоформные приложения на React.jsИзоформные приложения на React.js
Изоформные приложения на React.js
 
Robust web apps with React.js
Robust web apps with React.jsRobust web apps with React.js
Robust web apps with React.js
 
Трансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScript
Трансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScriptТрансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScript
Трансдюсеры, CSP каналы, неизменяемые структуры данных в JavaScript
 
Kharkivpy#3: Javascript and Python backend
Kharkivpy#3: Javascript and Python backendKharkivpy#3: Javascript and Python backend
Kharkivpy#3: Javascript and Python backend
 
Communicating Sequential Processes (CSP) in JavaScript
Communicating Sequential Processes (CSP) in JavaScriptCommunicating Sequential Processes (CSP) in JavaScript
Communicating Sequential Processes (CSP) in JavaScript
 
LvivPy - Flask in details
LvivPy - Flask in detailsLvivPy - Flask in details
LvivPy - Flask in details
 
Odessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and PythonOdessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and Python
 
Testing with Jenkins, Selenium and Continuous Deployment
Testing with Jenkins, Selenium and Continuous DeploymentTesting with Jenkins, Selenium and Continuous Deployment
Testing with Jenkins, Selenium and Continuous Deployment
 

Similar to PyCon 2012 - Data Driven Design

Big data in action
Big data in actionBig data in action
Big data in actionTu Pham
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data AnalyticsOsman Ali
 
Just enough web ops for web developers
Just enough web ops for web developersJust enough web ops for web developers
Just enough web ops for web developersDatadog
 
GitHub Data and Insights
GitHub Data and InsightsGitHub Data and Insights
GitHub Data and InsightsJeff McAffer
 
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...VMware Tanzu
 
High Throughput Data Analysis
High Throughput Data AnalysisHigh Throughput Data Analysis
High Throughput Data AnalysisJ Singh
 
The Internet as a Single Database
The Internet as a Single DatabaseThe Internet as a Single Database
The Internet as a Single DatabaseDatafiniti
 
BigData Meets the Federal Data Center
BigData Meets the Federal Data CenterBigData Meets the Federal Data Center
BigData Meets the Federal Data CenterAbe Usher
 
Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...
Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...
Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...Amazon Web Services
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunk
 
Monitoring as Software Validation
Monitoring as Software ValidationMonitoring as Software Validation
Monitoring as Software ValidationBioDec
 
FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...
FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...
FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...Amazon Web Services
 
Snowplow presentation for Amsterdam Meetup #3
Snowplow presentation for Amsterdam Meetup #3Snowplow presentation for Amsterdam Meetup #3
Snowplow presentation for Amsterdam Meetup #3Snowplow Analytics
 
RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011Gerardo Pardo-Castellote
 
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunk
 
Amplitude wave architecture - Test
Amplitude wave architecture - TestAmplitude wave architecture - Test
Amplitude wave architecture - TestKiran Naiga
 
Brand Niemann12102009
Brand Niemann12102009Brand Niemann12102009
Brand Niemann12102009guest8c518a8
 

Similar to PyCon 2012 - Data Driven Design (20)

Big data in action
Big data in actionBig data in action
Big data in action
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
Just enough web ops for web developers
Just enough web ops for web developersJust enough web ops for web developers
Just enough web ops for web developers
 
GitHub Data and Insights
GitHub Data and InsightsGitHub Data and Insights
GitHub Data and Insights
 
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
 
High Throughput Data Analysis
High Throughput Data AnalysisHigh Throughput Data Analysis
High Throughput Data Analysis
 
Pratical Deep Dive into the Semantic Web - #smconnect
Pratical Deep Dive into the Semantic Web - #smconnectPratical Deep Dive into the Semantic Web - #smconnect
Pratical Deep Dive into the Semantic Web - #smconnect
 
The Internet as a Single Database
The Internet as a Single DatabaseThe Internet as a Single Database
The Internet as a Single Database
 
BigData Meets the Federal Data Center
BigData Meets the Federal Data CenterBigData Meets the Federal Data Center
BigData Meets the Federal Data Center
 
Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...
Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...
Keys to Successfully Monitoring and Optimizing Innovative and Sophisticated C...
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding Overview
 
Monitoring as Software Validation
Monitoring as Software ValidationMonitoring as Software Validation
Monitoring as Software Validation
 
FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...
FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...
FINRA's Managed Data Lake: Next-Gen Analytics in the Cloud - ENT328 - re:Inve...
 
TSE_Pres12.pptx
TSE_Pres12.pptxTSE_Pres12.pptx
TSE_Pres12.pptx
 
Snowplow presentation for Amsterdam Meetup #3
Snowplow presentation for Amsterdam Meetup #3Snowplow presentation for Amsterdam Meetup #3
Snowplow presentation for Amsterdam Meetup #3
 
RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011
 
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
 
Amplitude wave architecture - Test
Amplitude wave architecture - TestAmplitude wave architecture - Test
Amplitude wave architecture - Test
 
Build vs Migrate to PaaS
Build vs Migrate to PaaSBuild vs Migrate to PaaS
Build vs Migrate to PaaS
 
Brand Niemann12102009
Brand Niemann12102009Brand Niemann12102009
Brand Niemann12102009
 

More from Max Klymyshyn

Papers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON Datatype
Papers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON DatatypePapers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON Datatype
Papers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON DatatypeMax Klymyshyn
 
KharkivJS 2017: Коллаборативные системы и CRDT
KharkivJS 2017: Коллаборативные системы и CRDTKharkivJS 2017: Коллаборативные системы и CRDT
KharkivJS 2017: Коллаборативные системы и CRDTMax Klymyshyn
 
OdessaJS 2017: Groupware Systems for fun and profit
OdessaJS 2017: Groupware Systems for fun and profitOdessaJS 2017: Groupware Systems for fun and profit
OdessaJS 2017: Groupware Systems for fun and profitMax Klymyshyn
 
PyCon Ukraine 2017: Operational Transformation
PyCon Ukraine 2017: Operational Transformation PyCon Ukraine 2017: Operational Transformation
PyCon Ukraine 2017: Operational Transformation Max Klymyshyn
 
5 мифов о производительности баз данных и Python
5 мифов о производительности баз данных и Python5 мифов о производительности баз данных и Python
5 мифов о производительности баз данных и PythonMax Klymyshyn
 
PiterPy 2015 - Трансдюсеры и Python
PiterPy 2015 - Трансдюсеры и PythonPiterPy 2015 - Трансдюсеры и Python
PiterPy 2015 - Трансдюсеры и PythonMax Klymyshyn
 
Зачем читать чужой код?
Зачем читать чужой код?Зачем читать чужой код?
Зачем читать чужой код?Max Klymyshyn
 
Kyivpy#8 - Quality Driven Development with Python
Kyivpy#8   - Quality Driven Development with PythonKyivpy#8   - Quality Driven Development with Python
Kyivpy#8 - Quality Driven Development with PythonMax Klymyshyn
 

More from Max Klymyshyn (8)

Papers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON Datatype
Papers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON DatatypePapers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON Datatype
Papers We Love Kyiv, July 2018: A Conflict-Free Replicated JSON Datatype
 
KharkivJS 2017: Коллаборативные системы и CRDT
KharkivJS 2017: Коллаборативные системы и CRDTKharkivJS 2017: Коллаборативные системы и CRDT
KharkivJS 2017: Коллаборативные системы и CRDT
 
OdessaJS 2017: Groupware Systems for fun and profit
OdessaJS 2017: Groupware Systems for fun and profitOdessaJS 2017: Groupware Systems for fun and profit
OdessaJS 2017: Groupware Systems for fun and profit
 
PyCon Ukraine 2017: Operational Transformation
PyCon Ukraine 2017: Operational Transformation PyCon Ukraine 2017: Operational Transformation
PyCon Ukraine 2017: Operational Transformation
 
5 мифов о производительности баз данных и Python
5 мифов о производительности баз данных и Python5 мифов о производительности баз данных и Python
5 мифов о производительности баз данных и Python
 
PiterPy 2015 - Трансдюсеры и Python
PiterPy 2015 - Трансдюсеры и PythonPiterPy 2015 - Трансдюсеры и Python
PiterPy 2015 - Трансдюсеры и Python
 
Зачем читать чужой код?
Зачем читать чужой код?Зачем читать чужой код?
Зачем читать чужой код?
 
Kyivpy#8 - Quality Driven Development with Python
Kyivpy#8   - Quality Driven Development with PythonKyivpy#8   - Quality Driven Development with Python
Kyivpy#8 - Quality Driven Development with Python
 

PyCon 2012 - Data Driven Design