Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automated and Continuous Data
•
0 gefällt mir•11 views
Curtis ODell, Global Director Data Integrity at Tricentis Join me to learn about a new end-to-end data testing approach designed for modern data pipelines that fills dangerous gaps left by traditional data management tools—one designed to handle structured and unstructured data from any source. You'll hear how you can use unique automation technology to reach up to 90 percent test coverage rates and deliver trustworthy analytical and operational data at scale. Several real world use cases from major banks/finance, insurance, health analytics, and Snowflake examples will be presented. Key Learning Objective 1. Data journeys are complex and you have to ensure integrity of the data end to end across this journey from source to end reporting for compliance 2. Data Management tools do not test data, they profile and monitor at best, and leave serious gaps in your data testing coverage 3. Automation with integration to DevOps and DataOps' CI/CD processes are key to solving this. 4. How this approach has impact in your vertical
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automated and Continuous Data
Ähnlich wie Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automated and Continuous Data (20)
Mehr von Data Con LA
Mehr von Data Con LA (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automated and Continuous Data
- 1. 1 THE GLOBAL LEADER IN CONTINUOUS TEST AUTOMATION Data Integrity Automation Drive better business outcomes through data you can trust
- 2. 2 Data Integrity Automation Agenda Why? To reduce your risk of failures, you must test your processes for integrity. Data Integrity can ONLY be achieved by testing ALL your data processes and ensure integrity end-to-end, with automation, and do it continuously Problem to be Solved Why it is not Solved How to Solve it Benefits of doing so What are the Consequences
- 3. 3 Data Governance for Data Trust Metadata and Data Catalogs Thousands of Hours to Create Complex Reports People – Process – Technology Data Delivery Culture – Access – Stewardship – Quality - Utilization - Acquisition Data Quality Master Data Data Security Data Governance is the Epicenter of Data Disruption comes with Information Stewardship Data Lifecycle
- 4. 4 Change is constantly at work across your digital landscape Application Changes Data Changes Environment Changes On-Prem Cloud Partner ecosystem web apps AI/ML- driven initiatives
- 5. 5 Application Changes Data Changes Environment Changes On-Prem Cloud Partner ecosystem web apps Application problems: Technical/UI changes Business requirements Customizations Data problems: Incorrect data Duplicate data Missing data Environment problems: System / desktop updates Integrations Network changes Compliance reports Reports, dashboards, visualizations AI/ML- driven initiatives Change is constantly at work across your digital landscape
- 6. 6 Accelerated Digital Change *Sources: Mayfield CXO Survey – Post COVID-19 Impacts to IT, IDC FutureScape IT Industry 2021 Predictions, ASUG Tricentis Survey 2021 – Future of SAP Delivery CLOUD MIGRATION 85% plan to shift to cloud-centric infrastructure & applications twice as fast as before the pandemic APPLICATION MODERNIZATION 67% plan to migrate half of on-prem applications. 91% plan to upgrade to SAP S/4HANA in next 24 months RAPID AND REGULAR UPDATES released by enterprise apps like SAP, Salesforce, ServiceNow DATA MIGRATION 43.5% say data migration is the main challenge when moving to advanced versions/upgrades DIGITAL OPTIMIZATION 50% plan to digitize eCommerce, deliver new features to improve customer self-service & UX
- 7. 7 If you don’t deal with change, expect consequences… Delivering poor quality Being late Inefficient resource management Large multinational bank 24h downtime = $7 million loss and reputational damage Telecommunications provider Manual testing = 10K tests 3 releases/yr, delayed innovation Global oil & gas company Required highly technical skills, >$45 million maintenance costs
- 8. 8 If you don’t deal with change, expect consequences… Accounting Failures Costly Compliance Fines Major insurer ERP consolidation of data flows, Leads to business operations failures (40K Invoices) Major Bank >$50 million in due to bad data quality used for AML/KYC regulatory requirements Loss due to Data Analytics Platform Mistrust For WorldPay, we helped deliver decision-grade data to business teams from massive volumes of transactional payment data. Thousands of hours of manual effort saved per month AND Trust in the numbers regained
- 9. 9 Data can break anywhere in the process because of dangerous data management gaps DATA WAREHOUSE DATA ANALYTICS / BI ECOSYSTEM Information Steward Data Services Advanced Data Migration by Syniti MDG
- 10. 10 Why? - Data Gaps in Testing Coverage Gap Reasons • Point Solutions only check one point in the process. Examples, MDM, ETL Test Tools.. • MDM is a production problem catcher, not a fixer, and not a tester solution • Solutions like Snowflake are focused on the data processes themselves not guaranteeing the quality of the data OUTSIDE their processes. • ETL providers • Report Testing
- 11. 11 So, the race is on to find the data errors EXTRACT Enterprise data warehouse Data marts/ cubes Business data sources TRANSFORM LOAD AGGREGATE TRANSFORM REPORT ? Did the problem originate in the source data? Was there an issue with a data load? ? Did a transformation job go wrong? Did a job fail to run or run too many times? Were there issues with the transformation logic? ? Is the report pulling from the right data mart? Is there a problem with the report logic? Is the report rendering incorrectly? REFINE Reports, dashboards, visualizations
- 12. 12 Manual “stare and compare” is slow and doesn’t scale. And is not a great use of your team’s brainpower. So why isn’t your data better already?
- 13. 13 To Trust the Production Environment: ENTERPRISE DATA WAREHOUSE DATA LAKE DATA ANALYTICS / BI ECOSYSTEM Information Steward Data Services Advanced Data Migration by Syniti MDG ENTERPRISE DATA WAREHOUSE DATA LAKE DATA ANALYTICS / BI ECOSYSTEM APPLICATION ECOSYSTEM You must End to End Test in the Test Environment:
- 14. 14 It’s time to think differently about how you maintain the integrity of your data
- 15. 15 It’s time to bring the discipline of end-to-end testing to the world of data
- 16. 16 End-to-end Automated Continuous Implement a data testing solution that’s… Enterprise Data Integrity Testing
- 17. 17 Automated Continuous A data testing solution that’s… Includes data, UI, and API testing across your landscape. End-to-end Enterprise Data Integrity Testing
- 18. 18 Catch more data issues up front Get higher data quality Test at scale and at speed So, you can… Enterprise Data Integrity Testing
- 19. 19 Automation — the key to moving from data integrity to decision integrity EXTRACT Enterprise data warehouse Data marts/ cubes TRANSFORM LOAD AGGREGATE TRANSFORM REPORT REFINE Reports, dashboards, visualizations PRE- SCREENING Metadata checks Format checks VITAL CHECKS Completeness Uniqueness Nullness Referential integrity FIELD TESTS Aggregation Value range Transformation RECONCILIATION TESTS Detailed source to target comparison tests ETL validation REPORT & APP TESTING UI automation Visual checks Content checks Security checks CONTINUOUS MONITORING Row counts, Job run times, Data distribution
- 20. 20 Keep your automation easy, faster and at scale with model-based test automation Make quick tweaks to either layer as things change. Create Maintain Auto- generate tests. Auto- update tests. Business logic Template 2 1 3 4 Separate out what you need to test into layers for fast, easy creation and maintenance. Build a flexible testing model. Test case design sheets Test cases
- 21. 21 8 advancing opportunities Proven strategies to tackle test automation challenges Empower any user to contribute to automation through a codeless approach that removes programming resources. Test what matters Aligning testing with business risks delivers high ROI in the shortest time and with less effort. Cover all testing needs Seek a comprehensive toolset that is technology-agnostic to avoid relying on specialists as and to expand testing use cases. Eliminate maintenace The less maintenance is required, the lower your total costs. Seek resilient, codeless test automation approaches. Get on-demand test data Eliminate wait time and false postives by enabling testers to create and access test data when needed. Promote collaboration through reusable test artifacts that can be plugged into a central repository for end-to- end test automation. Shift left testing Test at the API layer and use AI-based technologies to create UI tests before UIs are completed. Simulate environments Get rid of access fees and ensure that testing can proceed even if test environments are unavailable or unstable. SUSTAINABLE AUTOMATION RESILIENCY AND REUSE SELF-SERVICE + SHIFT LEFT
- 22. 22 How CI/CD with CT works to continuously test across your digital enterprise FIX QA Collaborate across teams to build better tests—end-to-end Business Data No-code, low-code solution No expert programming skills required AUTOMATE Tests in parallel at scale and speed Get detailed, actionable reports. RUN requirements & test case design PLAN CREATE model-based tests. Pinpoint the root cause of problems REPORT
Hinweis der Redaktion
- Hello - Introduction
- We understand the need for enterprise data governance and can help in the process. This is the umbrella I will talk to with the “pillar” of data quality our focus today…
- Changes are constantly happening throughout your IT landscape One end-to-end business process within that landscape could touch several interconnected systems and technologies typical enterprise portfolio contains thousands of applications, (2000 – 3000) applications in production MuleSoft research - Single transaction touches an average of 83 different technologies from mainframes, legacy customs apps to microservices ad cloud native apps. Next slide: New and real challenge is not just dealing with application changes, but dealing with environment changes and data changes Examples of environment could be – as simple as a Microsoft update, or a deployment of a new browser version Due to the move to the cloud, we now have an extremely short cycle to keep up with those environment and data changes Three dimensions are happening in parallel: application, environment and data changes = complexity
- New and real challenge is not just dealing with application changes, but dealing with environment changes and data changes Examples of environment could be – as simple as a Microsoft update, or a deployment of a new browser version Due to the move to the cloud, we now have an extremely short cycle to keep up with those environment and data changes Three dimensions are happening in parallel: application, environment and data changes = complexity
- The global pandemic has accelerated change across the digital landscape (remote work, remote sales, remote everything) Companies need to go digital Mayfield CXO survey: 85% of organizations are planning to move to the cloud twice as fast Application Modernization – related to cloud transformation but could mean many different things: Migrating your applications to more modern ones in order to deliver more business value or enhance customer/employee experience (for example, shifting Namely and Expensify to Workday – for more scalability) Application modernization could also mean migrating your legacy applications to scalable, cloud-native app environments – 67% plan to migrate on-premise applications. Example: according to the ASUG Tricentis 2021 survey Future of SAP delivery, 91% plan to move to S4HANA. But SAP ECC is more popular. Organizations say cloud is great, but need to justify the cost and risk involved Challenges include rapid and regular updates: keeping up with the pace of updates release by SAP, Salesforce, ServiceNow Other challenges include data: 43% say data migration in cloud migration projects is a huge problem (for SAP – ASUG survey) It could also mean Digital optimization – delivering new business/digital functionality: 50% of companies said they are planning on moving their eCommerce, marketing and sales to digital platforms in order to improve customer self-service and experience
- if you don’t deal with this inevitable change, you will experience consequences Example 1: Large multinational bank (Barclays – not our customer)– has online banking down, leaving customers locked out of accounts as payments also delayed: The average cost of downtime is $5,600 per minute (according to Gartner research), So it is estimated this company suffered 7 million dollars in loss, loss of customers, reputational damage Barclays online banking goes down: https://www.thesun.co.uk/money/12788031/barclays-online-banking-down-2/ Average cost of downtime: https://www.atlassian.com/incident-management/kpis/cost-of-downtime Why the cost of network downtime is so high in the banking industry: https://www.garlandtechnology.com/blog/why-is-the-cost-of-network-downtime-so-high-in-the-banking-industry Barclays tops list of banks with most IT shutdowns: https://www.bbc.com/news/business-49412055 Example 2- One of the largest telecommunications providers in Europe (A1 – our customer, before scenario). They were doing manual testing which resulted in more than 10,000 tests performed. Business process was distributed across 60 critical systems and enterprise applications. As a result, they were getting only 3 to 4 major releases a year, delayed innovation. Additionally, if a program is late, it costs money to get people to deliver that program on time. Example 3: inefficient resource management. World’s most established oil and gas providers – Exxon Mobil (our customer, before scenario). They needed highly specialized people to develop and maintain their testing. This lead to high costs not only in technical IT staff but also high maintenance to keep up with those changes
- if you don’t deal with this inevitable change, you will experience consequences
- If you're like most enterprise organizations, you've already invested in lots of tools to get and keep your data in good shape. Fantastic, they do a great job at what they do — keep on using them. The issue is that they're designed with a particular task in mind — like governing your master data or profiling data at specific points in time, like after a transformation event. In complex enterprise landscapes where you're dealing with masses of structured and unstructured data from many different systems, these point solutions leave dangerous data management gaps.
- These gaps are risk in your data process. I listed some examples of the gaps they often don’t fill.
- We must find the problem ad fix it, but where in the process was the failure.
- At the end of the day all the testing of the gaps in the data process is a manual effort, we call stare and compare Points to make: Manual “stare and compare” methods that involve people spot checking data in source systems and the final data destination This doesn’t scale or get them the coverage they need It’s also a real waste of expensive human resources it’s estimated that all knowledge workers in an organization spend 50% of their day, fixing data, when automating this task makes much more sense
- TO be clear, to mitigate the risk of changes to the processes you must test ALL the processes End to END BEFORE they are in production. Not just a single point like an ETL change but preform true REGRESSION that ensures the process is golden end to end after any change. Also, you can’t rely on production testing or profiling or monitoring they only check a part of the process and if you are catching problems in production, it is TOO LATE! Point “testing” solution like Informatica’s IQA, their testing solution, only have coverage on their intersection points. You must look at all the integration points, you can now test all the intersection of the data, and much more that just a “sample” of the data. All the data at all the intersection points. .
- []
- []
- [Jeanette] - Talk Track – To solve this business problem every enterprise has you need an automated, end to end solution, that can run continuously and integrate with a company’s DevOps or DataOps CI/CD structure.
- [Jeanette] - Our major differentiator, is that SAP EDIT (Tosca DI) has unique ability to test across 160 UI and API technologies as well as across the over 250 different data technologies enterprises have deployed across the enterprise. An we do this testing in an automated, scriptless framework we call Model Based Test Automation. Truly our secret sauce for accomplishing the automation at scale.
- [Jeanette – handover to Curtis to walk through B of A examples] - Talk Track – Once we implement it we can catch the problem at the root cause. Finding issues early, identifying the root cause and remediating them is the technical purpose behind all testing. Doing it with automation ensures the coverage needed, and in the end you get higher data quality for better business decisions.
- Use this slide to show the value of model-based test automation (and test case design – turn one test model into dozens of tests, easy update them in this one model)