Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
eXtremeProgramming for ETL 
Agile Pune 2014 
_________________________ 
Prafulla Girgaonkar [@pgirgaonkar] 
Naresh Jain [@...
25YEARS 
6000+CLIENTS 
Nr 1M ROOMS 
94COUNTRIES 
95%+ 
ago we started helping clients manage their revenue 
later we’re st...
ETL 
DECISION 
IP 
PMS/CRS 
Competitor 
WebRate 
Social Media 
Periodically or 
Real Time 
Full 
or 
Delta 
Messages or 
F...
Concept 
Exploration 
Structuring 
Requirements 
Discovery 
Earlier ETL Practices
Concept 
DataExploration 
“Structurization” 
ETLDevelopment 
Certification 
GA 
Typical Development Cycle on an ETL Project
Vendor 
IDEAS 
Development 
Earlier ETL Practices
Challenges and Flaws 
Source : www.dogri.org/
Challenges and Flaws
Challenges and Flaws
Challenges and Flaws
Challenges and Flaws
Challenges and Flaws 
Time 
Discovery 
Development
Challenges and Flaws 
Source : http://roselawgroupreporter.com/
Challenges and Flaws
Requirement Conformity 
Requirement Misunderstanding 
Late Data Change 
Delays 
API Performance 
Priorities 
Challenges an...
Naresh Jain 
•nareshjain.com 
•@nashjain
PrafullaS Girgaonkar 
Product Development for 
Hospitality |Revenue Optimization |Finance 
Prafulla.Girgaonkar@ideas.com 
...
Journey 
Hilstar 
Opera 
Pegasus
Old Approach
Existing Approach -Iterative & Incremental Approach
Initial XP Practices -Hilstar
DBAQADEVPMUX 
Initial XP Practices –Whole Team 
Source: http://www.selfishprogramming.com/ BOKM
Iteration #1 
Iteration #2 
Iteration #3 
Initial XP Practices –Small Releases
Initial XP Practices -Continuous Integration
Initial XP Practices –User Story
Evolution of the Practices… OPERA 
Source: http://www.freedomchurchunited.org/
Evolution of the Practices -Pairing
Evolution of the Practices –Planning/Grooming 
Source: http://www.h4-entertainment.com/
Evolution of the Practices –Test Driven Development
Evolution of the Practices –Refactoring
Evolution of the Practices –Coding Standards
Evolution of the Practices –Collective Ownership
Evolution of the Practices –Simple Design
•The customer is always available 
•Code theunit testfirst 
•Leaveoptimizationfor later 
Evolution of the Practices –Optim...
Write a TEST before FIXING the BUG 
Evolution of the Practices –Test Driven Bug Fixing
Further…Evolution of the Practices… Pegasus
Data Management 4GL 
MACROS 
PROCS 
FUNCTIONS 
Learn SAS 4gl @ 
http://support.sas.com/software/ products/ondemand- academ...
Further…Evolution of the Practices –Earlier Code
No Unit test 
No Build Integration 
Language Unawareness 
Discover the Tool 
SASUNIT 
Further…Evolution of the Practices –...
Further…Evolution of the Practices –Code Later
Part of build Process 
Fail Fast 
Further…Evolution of the Practices –Build Process Correction
Integrated With Jenkins 
Fail Fast 
Fail Reports 
Further…Evolution of the Practices –Jenkins for SASUNIT
Engineering Practices are always at the core. 
Tools May not be there. 
There is WILL, there is WAY 
CRUX
Benefits 
Source: http://flickr.com
Source: http://docustar.com/blog/quality-check-is-99-9-good-enough/ 
Benefits –
Source: http://www.ftjco.com/wp-content/uploads/2012/10/diamonds.jpg 
Benefits
Benefits
Benefits
Benefits 
Relative Time 
Hilstar 
Opera 
Pegasus
UNIT Test … 
UNIT Test … 
UNIT Test … 
Source: www.topnews.in 
Benefits
Source: http://en.wikipedia.org/wiki/Cello 
Benefits
Change Management 
Source: http://www.animalnational.com/monarch-butterfly-migration/
Change Management –Usual Reactions 
Source: http://ericgerlach.com/introphilosophy8/ 
DEVs | QAs | POs | BOs
Change Management –Looking for Rituals 
Source: http://www.theguardian.com/books/2014/apr/27/reading-print-books-ereaders-...
Courtesy: Jeff Patton 
Change Management –Smaller Release/Stories were Shaker 
DEVs | QAs | POs | BOs
Change Management –Incremental Development 
DEVs | QAs | POs | BOs
Change Management –Story Map 
DEVs | QAs | POs | BOs
Change Management –Spikes were new 
Source: http://www.kitchology.com/ 
DEVs | QAs
Change Management –CI & Failing Builds 
DEVs | QAs
Change Management –Pairing 
Source: https://www.airpair.com/mean-stack/posts/2014-10-job-post-mean-stack-developer 
DEVs |...
Change Management –Team Ownership 
DEVs | QAs 
Source: https://dilbert.com
Change Management –Testers part of Design Discussion 
Source: http://www.mercatornet.com/articles/view/health_economics_10...
Change Management –Defect are discussed; not entered 
QAs 
Source: http://letstalkaboutdiscussion.ca/
Change Management –Test Automation 
QAs 
Source: http://www.mobileqazone.com/profiles/blogs/software-testing-by-human-or-m...
Road Ahead 
Source: http://www.gettyimages.in/
Engineering Practice -Refactoring / Managing Code Smells 
Source:www.123rf.com
•Engineering Practice -Simple Design & Solving critical challenges by taking them out of context 
Road Ahead -Engineering ...
•Engineering Practice -Simple Design & Solving critical challenges by taking them out of context 
Road Ahead -Engineering ...
Road Ahead -Engineering Practice -Evolutionary Design Extended 
Source: http://nordicedge.com/
Road Ahead -Engineering Practice –Programming skills and polyglot
Road Ahead -Real Time Integration
WEI JAD 
Road Ahead -Joint Application development with Vendor 
Source: http://www.p-r-i.org/nadcap/nadcap-meetings/
Take Away 
•Engineering Practices are universal 
–determination and perseverance 
•XP Practices lead to T-Shaped individua...
Prafulla Girgaonkar [@pgirgaonkar] Naresh Jain [@nashjain] Illustration: Umesh KaleSession Ends.. Efforts Continue…
Nächste SlideShare
Wird geladen in …5
×

Extreme Programming for ETL and Data Analytics

1.654 Aufrufe

Veröffentlicht am

Over the last decade, eXtreme Programming practices like User Stories, Evolutionary Design, Test-Driven Development (TDD), Behavior Driven Developer (BDD), Refactoring, Continuous Integration and Automation have fundamentally changed software development processes and inherently how engineers work.

Having experienced various benefits from XP practices on our J2EE stack, our team started to apply these practices to extract, transform, and load (ETL) and Data Analytics side of our product. Unfortunately, there is very little guidance available in this context, esp. for the SAS Platform. Right from finding the unit testing framework to structuring the code to designing our modules and setting up a Continuous Integration builds, our team had to figure out everything, the hard way.

Join us to understand the challenges we faced during this process and how we resolved these challenges.

Details: http://confengine.com/agile-pune-2014/proposal/535/extreme-programming-for-etl-and-data-analytics

Conference: http://pune.agileindia.org/

Veröffentlicht in: Software
  • Softcrylic’s digital analytics service focuses on implementing data tracking techniques and real-time engagement techniques to drive traffic, improve conversions and maximize ROI. http://bit.ly/2Daudsb
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Extreme Programming for ETL and Data Analytics

  1. 1. eXtremeProgramming for ETL Agile Pune 2014 _________________________ Prafulla Girgaonkar [@pgirgaonkar] Naresh Jain [@nashjain] Illustration: Umesh Kale
  2. 2. 25YEARS 6000+CLIENTS Nr 1M ROOMS 94COUNTRIES 95%+ ago we started helping clients manage their revenue later we’re still finding the revenue opportunity in every situation around the world are priced every single day using IDeaS on 6 continents have hotels that now run on IDeaS Revenue Management Systems More than 95 percent of all our clients are still working with us today. About IDeaS
  3. 3. ETL DECISION IP PMS/CRS Competitor WebRate Social Media Periodically or Real Time Full or Delta Messages or File * * * * ETL & IDEAS
  4. 4. Concept Exploration Structuring Requirements Discovery Earlier ETL Practices
  5. 5. Concept DataExploration “Structurization” ETLDevelopment Certification GA Typical Development Cycle on an ETL Project
  6. 6. Vendor IDEAS Development Earlier ETL Practices
  7. 7. Challenges and Flaws Source : www.dogri.org/
  8. 8. Challenges and Flaws
  9. 9. Challenges and Flaws
  10. 10. Challenges and Flaws
  11. 11. Challenges and Flaws
  12. 12. Challenges and Flaws Time Discovery Development
  13. 13. Challenges and Flaws Source : http://roselawgroupreporter.com/
  14. 14. Challenges and Flaws
  15. 15. Requirement Conformity Requirement Misunderstanding Late Data Change Delays API Performance Priorities Challenges and Flaws -Vendor
  16. 16. Naresh Jain •nareshjain.com •@nashjain
  17. 17. PrafullaS Girgaonkar Product Development for Hospitality |Revenue Optimization |Finance Prafulla.Girgaonkar@ideas.com @pgirgaonkar
  18. 18. Journey Hilstar Opera Pegasus
  19. 19. Old Approach
  20. 20. Existing Approach -Iterative & Incremental Approach
  21. 21. Initial XP Practices -Hilstar
  22. 22. DBAQADEVPMUX Initial XP Practices –Whole Team Source: http://www.selfishprogramming.com/ BOKM
  23. 23. Iteration #1 Iteration #2 Iteration #3 Initial XP Practices –Small Releases
  24. 24. Initial XP Practices -Continuous Integration
  25. 25. Initial XP Practices –User Story
  26. 26. Evolution of the Practices… OPERA Source: http://www.freedomchurchunited.org/
  27. 27. Evolution of the Practices -Pairing
  28. 28. Evolution of the Practices –Planning/Grooming Source: http://www.h4-entertainment.com/
  29. 29. Evolution of the Practices –Test Driven Development
  30. 30. Evolution of the Practices –Refactoring
  31. 31. Evolution of the Practices –Coding Standards
  32. 32. Evolution of the Practices –Collective Ownership
  33. 33. Evolution of the Practices –Simple Design
  34. 34. •The customer is always available •Code theunit testfirst •Leaveoptimizationfor later Evolution of the Practices –Optimization at the End
  35. 35. Write a TEST before FIXING the BUG Evolution of the Practices –Test Driven Bug Fixing
  36. 36. Further…Evolution of the Practices… Pegasus
  37. 37. Data Management 4GL MACROS PROCS FUNCTIONS Learn SAS 4gl @ http://support.sas.com/software/ products/ondemand- academics/ Further…Evolution of the Practices –SAS 4GL
  38. 38. Further…Evolution of the Practices –Earlier Code
  39. 39. No Unit test No Build Integration Language Unawareness Discover the Tool SASUNIT Further…Evolution of the Practices –Testability
  40. 40. Further…Evolution of the Practices –Code Later
  41. 41. Part of build Process Fail Fast Further…Evolution of the Practices –Build Process Correction
  42. 42. Integrated With Jenkins Fail Fast Fail Reports Further…Evolution of the Practices –Jenkins for SASUNIT
  43. 43. Engineering Practices are always at the core. Tools May not be there. There is WILL, there is WAY CRUX
  44. 44. Benefits Source: http://flickr.com
  45. 45. Source: http://docustar.com/blog/quality-check-is-99-9-good-enough/ Benefits –
  46. 46. Source: http://www.ftjco.com/wp-content/uploads/2012/10/diamonds.jpg Benefits
  47. 47. Benefits
  48. 48. Benefits
  49. 49. Benefits Relative Time Hilstar Opera Pegasus
  50. 50. UNIT Test … UNIT Test … UNIT Test … Source: www.topnews.in Benefits
  51. 51. Source: http://en.wikipedia.org/wiki/Cello Benefits
  52. 52. Change Management Source: http://www.animalnational.com/monarch-butterfly-migration/
  53. 53. Change Management –Usual Reactions Source: http://ericgerlach.com/introphilosophy8/ DEVs | QAs | POs | BOs
  54. 54. Change Management –Looking for Rituals Source: http://www.theguardian.com/books/2014/apr/27/reading-print-books-ereaders-technology DEVs | QAs | POs | BOs
  55. 55. Courtesy: Jeff Patton Change Management –Smaller Release/Stories were Shaker DEVs | QAs | POs | BOs
  56. 56. Change Management –Incremental Development DEVs | QAs | POs | BOs
  57. 57. Change Management –Story Map DEVs | QAs | POs | BOs
  58. 58. Change Management –Spikes were new Source: http://www.kitchology.com/ DEVs | QAs
  59. 59. Change Management –CI & Failing Builds DEVs | QAs
  60. 60. Change Management –Pairing Source: https://www.airpair.com/mean-stack/posts/2014-10-job-post-mean-stack-developer DEVs | QAs
  61. 61. Change Management –Team Ownership DEVs | QAs Source: https://dilbert.com
  62. 62. Change Management –Testers part of Design Discussion Source: http://www.mercatornet.com/articles/view/health_economics_101_contraceptives_pay_for_themselves_usually_babies_dont DEVs | QAs
  63. 63. Change Management –Defect are discussed; not entered QAs Source: http://letstalkaboutdiscussion.ca/
  64. 64. Change Management –Test Automation QAs Source: http://www.mobileqazone.com/profiles/blogs/software-testing-by-human-or-machine-what-should-shouldn-t-be
  65. 65. Road Ahead Source: http://www.gettyimages.in/
  66. 66. Engineering Practice -Refactoring / Managing Code Smells Source:www.123rf.com
  67. 67. •Engineering Practice -Simple Design & Solving critical challenges by taking them out of context Road Ahead -Engineering Practice –Simple Design Source: http://flickr.com/
  68. 68. •Engineering Practice -Simple Design & Solving critical challenges by taking them out of context Road Ahead -Engineering Practice –Sandbox Source: http://ifum.com/
  69. 69. Road Ahead -Engineering Practice -Evolutionary Design Extended Source: http://nordicedge.com/
  70. 70. Road Ahead -Engineering Practice –Programming skills and polyglot
  71. 71. Road Ahead -Real Time Integration
  72. 72. WEI JAD Road Ahead -Joint Application development with Vendor Source: http://www.p-r-i.org/nadcap/nadcap-meetings/
  73. 73. Take Away •Engineering Practices are universal –determination and perseverance •XP Practices lead to T-Shaped individual or Generalizing Specialist •Product Discovery & Story Mapping vital for Alignment.
  74. 74. Prafulla Girgaonkar [@pgirgaonkar] Naresh Jain [@nashjain] Illustration: Umesh KaleSession Ends.. Efforts Continue…

×