SlideShare ist ein Scribd-Unternehmen logo
1 von 15
Grid and Flow By Robert Betts robert.betts@scalefast.com
The Offering A distributed, stable and well synchronised platform for grid computing. A methodology, toolset and library for deploying an efficient, co-ordinated and parallel platform.
Current Operational Challenges Processing of large datasets open to high rates of failure Processing takes a long time to complete Processing is often executed sequentially Technology bottlenecks e.g. 32 bit software often only supports between 2 and 3 GB RAM 64 bit processes can hog available resources Most tools can’t exploit multi core configurations Dedicated hardware allocated to accommodate the maximum processing load Difficult to audit and trace processing problems
With ScaleFast you can ... Centralise all business processes Define hierarchical processes with step inter-dependencies Parallelise the running of processes and steps Re-run failed processes from any point. Automatically split large process steps Make use of multi core/processor computers Distribute jobs across multiple computers Make use of user workstations and other idle computing resources
ScaleFast Grid Distributed Computing Grid A distributor and worker nodes Implements map/reduce Workers can run on user workstations or dedicated infrastructure Can be easily deployed to a cloud platform Supports the native running of Python, Java and .Net
ScaleFast Flow Process Workflow Engine Processes are made up of individual jobs which have inter-dependencies The output of one job can be the input of the next job Processes have notifications based on success or failure Flow has a built in scheduler which can be triggered by: Time with multiple time zone support User Interface API Processes can be restarted from any point of failure Processes can be made up of sub processes
Common Use Cases Reporting and data processing Stabilising processes that fail due to resource constraints Speeding up processes that take a long time to run Improve and/or balance resource utilisation  Process orchestration and scheduling Co-ordinating processes with event based synchronisation Parameter and data flow between process steps Centralisation and versioning of processes Reducing support administration with full process auditing General processing and application development Any application/process that would benefit from parallelism Risk Management and PL Processing Distributed Computations
Case Study 1 – Hedge Fund After Before ,[object Object]
Reports continuously failing
Reporting taking longer to run each day
System support occupies a fulltime resource with additional assistance frequently required
Overnight failures push EOD processing to t+2 (SLA at t+1 am)
Fund considers:Adding head count with full time EOD support resources Purchasing additional hardware Purchasing a scheduling and automation product ,[object Object]
Fine grained audit trail of all processes
EOD processing time reduced from 8 hours to 50 minutes

Weitere ähnliche Inhalte

Was ist angesagt?

Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...
Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...
Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...WSO2
 
Digital Transformation for Karnataka Bank Through API-led Integration
Digital Transformation for Karnataka Bank Through API-led IntegrationDigital Transformation for Karnataka Bank Through API-led Integration
Digital Transformation for Karnataka Bank Through API-led IntegrationWSO2
 
[Webinar] WSO2 Enterprise Integrator 7.1.0 Release
[Webinar] WSO2 Enterprise Integrator 7.1.0 Release[Webinar] WSO2 Enterprise Integrator 7.1.0 Release
[Webinar] WSO2 Enterprise Integrator 7.1.0 ReleaseWSO2
 
Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)
Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)
Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)Doug Kern
 
Building an integration agile digital ecosystem
Building an integration agile digital ecosystemBuilding an integration agile digital ecosystem
Building an integration agile digital ecosystemRohitha Liyanagama
 
Real-Time ETL in Practice with WSO2 Enterprise Integrator
Real-Time ETL in Practice with WSO2 Enterprise IntegratorReal-Time ETL in Practice with WSO2 Enterprise Integrator
Real-Time ETL in Practice with WSO2 Enterprise IntegratorWSO2
 
[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...
[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...
[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...WSO2
 
[WSO2Con EU 2018] Up-Leveling Brownfield Integration
[WSO2Con EU 2018] Up-Leveling Brownfield Integration[WSO2Con EU 2018] Up-Leveling Brownfield Integration
[WSO2Con EU 2018] Up-Leveling Brownfield IntegrationWSO2
 
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...WSO2
 
Exposing Lambda Functions as Managed APIs
Exposing Lambda Functions as Managed APIsExposing Lambda Functions as Managed APIs
Exposing Lambda Functions as Managed APIsWSO2
 
OpenText Captiva - What's new in Release 16 (EP5)
OpenText Captiva - What's new in Release 16 (EP5)OpenText Captiva - What's new in Release 16 (EP5)
OpenText Captiva - What's new in Release 16 (EP5)OpenText
 
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...WSO2
 
WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...
WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...
WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...WSO2
 
[Workshop] API-driven Integration
[Workshop] API-driven Integration[Workshop] API-driven Integration
[Workshop] API-driven IntegrationWSO2
 
BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010
BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010
BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010Bala Subra
 
The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...
The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...
The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...Manuel Pais
 
Exposing GraphQLs as Managed APIs
Exposing GraphQLs as Managed APIsExposing GraphQLs as Managed APIs
Exposing GraphQLs as Managed APIsWSO2
 
The Role of Integration in Microservice Architecture (MSA)
The Role of Integration in Microservice Architecture (MSA)The Role of Integration in Microservice Architecture (MSA)
The Role of Integration in Microservice Architecture (MSA)Asanka Abeysinghe
 
What's New in Capture Overview - Release 16 EP4
What's New in Capture Overview - Release 16 EP4What's New in Capture Overview - Release 16 EP4
What's New in Capture Overview - Release 16 EP4OpenText
 

Was ist angesagt? (20)

Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...
Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...
Continuous Integration and Continuous Deployment (CI/CD) with WSO2 Enterprise...
 
Digital Transformation for Karnataka Bank Through API-led Integration
Digital Transformation for Karnataka Bank Through API-led IntegrationDigital Transformation for Karnataka Bank Through API-led Integration
Digital Transformation for Karnataka Bank Through API-led Integration
 
[Webinar] WSO2 Enterprise Integrator 7.1.0 Release
[Webinar] WSO2 Enterprise Integrator 7.1.0 Release[Webinar] WSO2 Enterprise Integrator 7.1.0 Release
[Webinar] WSO2 Enterprise Integrator 7.1.0 Release
 
Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)
Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)
Webcast: Inovis-Dell Case Study (B2B Cloud Integration Platforms)
 
Building an integration agile digital ecosystem
Building an integration agile digital ecosystemBuilding an integration agile digital ecosystem
Building an integration agile digital ecosystem
 
Websphere - overview and introduction
Websphere - overview and introduction Websphere - overview and introduction
Websphere - overview and introduction
 
Real-Time ETL in Practice with WSO2 Enterprise Integrator
Real-Time ETL in Practice with WSO2 Enterprise IntegratorReal-Time ETL in Practice with WSO2 Enterprise Integrator
Real-Time ETL in Practice with WSO2 Enterprise Integrator
 
[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...
[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...
[WSO2Con EU 2018] Adaptive and Iterative Integration for Microservices and Cl...
 
[WSO2Con EU 2018] Up-Leveling Brownfield Integration
[WSO2Con EU 2018] Up-Leveling Brownfield Integration[WSO2Con EU 2018] Up-Leveling Brownfield Integration
[WSO2Con EU 2018] Up-Leveling Brownfield Integration
 
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
[APIdays Paris 2019] API Management in Service Mesh Using Istio and WSO2 API ...
 
Exposing Lambda Functions as Managed APIs
Exposing Lambda Functions as Managed APIsExposing Lambda Functions as Managed APIs
Exposing Lambda Functions as Managed APIs
 
OpenText Captiva - What's new in Release 16 (EP5)
OpenText Captiva - What's new in Release 16 (EP5)OpenText Captiva - What's new in Release 16 (EP5)
OpenText Captiva - What's new in Release 16 (EP5)
 
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
 
WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...
WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...
WSO2 Guest Webinar: Building Enterprise Awareness with API Analytics in the A...
 
[Workshop] API-driven Integration
[Workshop] API-driven Integration[Workshop] API-driven Integration
[Workshop] API-driven Integration
 
BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010
BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010
BizTalk 2010 with Appfabric Hosting in the Cloud: WCF Services vs BT2010
 
The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...
The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...
The Elephant in the Kubernetes Room - Team Interactions at Scale @ KubeCon No...
 
Exposing GraphQLs as Managed APIs
Exposing GraphQLs as Managed APIsExposing GraphQLs as Managed APIs
Exposing GraphQLs as Managed APIs
 
The Role of Integration in Microservice Architecture (MSA)
The Role of Integration in Microservice Architecture (MSA)The Role of Integration in Microservice Architecture (MSA)
The Role of Integration in Microservice Architecture (MSA)
 
What's New in Capture Overview - Release 16 EP4
What's New in Capture Overview - Release 16 EP4What's New in Capture Overview - Release 16 EP4
What's New in Capture Overview - Release 16 EP4
 

Ähnlich wie ScaleFast Grid And Flow

Accounting System Design and Development - System Planning and Development
Accounting System Design and Development - System Planning and Development Accounting System Design and Development - System Planning and Development
Accounting System Design and Development - System Planning and Development HelpWithAssignment.com
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
informatica data replication (IDR)
informatica data replication (IDR)informatica data replication (IDR)
informatica data replication (IDR)MaxHung
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Phase Two: What’s Next for Life Sciences and Enterprise Content Management
Phase Two: What’s Next for Life Sciences and Enterprise Content ManagementPhase Two: What’s Next for Life Sciences and Enterprise Content Management
Phase Two: What’s Next for Life Sciences and Enterprise Content ManagementScott Abel
 
T3 Consortium's Performance Center of Excellence
T3 Consortium's Performance Center of ExcellenceT3 Consortium's Performance Center of Excellence
T3 Consortium's Performance Center of Excellenceveehikle
 
Introducing Elevate Capacity Management
Introducing Elevate Capacity ManagementIntroducing Elevate Capacity Management
Introducing Elevate Capacity ManagementPrecisely
 
NetWeaver Data Management process
NetWeaver Data Management processNetWeaver Data Management process
NetWeaver Data Management processTony de Thomasis
 
Callidus Software On-Premise To On-Demand Migration
Callidus Software On-Premise To On-Demand MigrationCallidus Software On-Premise To On-Demand Migration
Callidus Software On-Premise To On-Demand MigrationCallidus Software
 
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...CodeScience
 
On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?Callidus Software
 
OS_Process_Management_Chap4.pptx
OS_Process_Management_Chap4.pptxOS_Process_Management_Chap4.pptx
OS_Process_Management_Chap4.pptxDrAmarNathDhebla
 
Information Systems Life Cycle
Information Systems Life CycleInformation Systems Life Cycle
Information Systems Life Cycle4goggas
 
392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...
392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...
392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...SitiNurhaliza735951
 
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)Knoldus Inc.
 
Business Meets IT Presentatie
Business Meets IT PresentatieBusiness Meets IT Presentatie
Business Meets IT PresentatieRuud Stroet
 

Ähnlich wie ScaleFast Grid And Flow (20)

Accounting System Design and Development - System Planning and Development
Accounting System Design and Development - System Planning and Development Accounting System Design and Development - System Planning and Development
Accounting System Design and Development - System Planning and Development
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
Realtime search
Realtime searchRealtime search
Realtime search
 
informatica data replication (IDR)
informatica data replication (IDR)informatica data replication (IDR)
informatica data replication (IDR)
 
How to Migrate Without Downtime
How to Migrate Without DowntimeHow to Migrate Without Downtime
How to Migrate Without Downtime
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
Phase Two: What’s Next for Life Sciences and Enterprise Content Management
Phase Two: What’s Next for Life Sciences and Enterprise Content ManagementPhase Two: What’s Next for Life Sciences and Enterprise Content Management
Phase Two: What’s Next for Life Sciences and Enterprise Content Management
 
T3 Consortium's Performance Center of Excellence
T3 Consortium's Performance Center of ExcellenceT3 Consortium's Performance Center of Excellence
T3 Consortium's Performance Center of Excellence
 
Introducing Elevate Capacity Management
Introducing Elevate Capacity ManagementIntroducing Elevate Capacity Management
Introducing Elevate Capacity Management
 
NetWeaver Data Management process
NetWeaver Data Management processNetWeaver Data Management process
NetWeaver Data Management process
 
Callidus Software On-Premise To On-Demand Migration
Callidus Software On-Premise To On-Demand MigrationCallidus Software On-Premise To On-Demand Migration
Callidus Software On-Premise To On-Demand Migration
 
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
 
On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?
 
OS_Process_Management_Chap4.pptx
OS_Process_Management_Chap4.pptxOS_Process_Management_Chap4.pptx
OS_Process_Management_Chap4.pptx
 
Information Systems Life Cycle
Information Systems Life CycleInformation Systems Life Cycle
Information Systems Life Cycle
 
FreeFlow Process Manager
FreeFlow Process ManagerFreeFlow Process Manager
FreeFlow Process Manager
 
392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...
392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...
392976623-Accounting-Information-Systems-Essential-Concepts-and-Applications-...
 
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
 
Yongsan presentation 3
Yongsan presentation 3Yongsan presentation 3
Yongsan presentation 3
 
Business Meets IT Presentatie
Business Meets IT PresentatieBusiness Meets IT Presentatie
Business Meets IT Presentatie
 

Kürzlich hochgeladen

Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 

Kürzlich hochgeladen (20)

Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 

ScaleFast Grid And Flow

  • 1. Grid and Flow By Robert Betts robert.betts@scalefast.com
  • 2. The Offering A distributed, stable and well synchronised platform for grid computing. A methodology, toolset and library for deploying an efficient, co-ordinated and parallel platform.
  • 3. Current Operational Challenges Processing of large datasets open to high rates of failure Processing takes a long time to complete Processing is often executed sequentially Technology bottlenecks e.g. 32 bit software often only supports between 2 and 3 GB RAM 64 bit processes can hog available resources Most tools can’t exploit multi core configurations Dedicated hardware allocated to accommodate the maximum processing load Difficult to audit and trace processing problems
  • 4. With ScaleFast you can ... Centralise all business processes Define hierarchical processes with step inter-dependencies Parallelise the running of processes and steps Re-run failed processes from any point. Automatically split large process steps Make use of multi core/processor computers Distribute jobs across multiple computers Make use of user workstations and other idle computing resources
  • 5. ScaleFast Grid Distributed Computing Grid A distributor and worker nodes Implements map/reduce Workers can run on user workstations or dedicated infrastructure Can be easily deployed to a cloud platform Supports the native running of Python, Java and .Net
  • 6. ScaleFast Flow Process Workflow Engine Processes are made up of individual jobs which have inter-dependencies The output of one job can be the input of the next job Processes have notifications based on success or failure Flow has a built in scheduler which can be triggered by: Time with multiple time zone support User Interface API Processes can be restarted from any point of failure Processes can be made up of sub processes
  • 7. Common Use Cases Reporting and data processing Stabilising processes that fail due to resource constraints Speeding up processes that take a long time to run Improve and/or balance resource utilisation Process orchestration and scheduling Co-ordinating processes with event based synchronisation Parameter and data flow between process steps Centralisation and versioning of processes Reducing support administration with full process auditing General processing and application development Any application/process that would benefit from parallelism Risk Management and PL Processing Distributed Computations
  • 8.
  • 10. Reporting taking longer to run each day
  • 11. System support occupies a fulltime resource with additional assistance frequently required
  • 12. Overnight failures push EOD processing to t+2 (SLA at t+1 am)
  • 13.
  • 14. Fine grained audit trail of all processes
  • 15. EOD processing time reduced from 8 hours to 50 minutes
  • 17.
  • 18. Fine grained audit trail of all processes
  • 19. EOD processing time reduced from 4 hours to 30 minutes
  • 21. Risk Engine built on top of Grid and Flow.
  • 22.
  • 23. Tried shell scripts to split reports
  • 25. Reports still took 7 hours to run and taking longer to run each day
  • 26. A single failure required a complete restart
  • 27. A failure and restart would result in SLA failures to all downstream systems
  • 28.
  • 29. On failure, reporting process can now resume from any step
  • 30. Completed reports are now processed in 40 minutes
  • 31. Fine grained audit trail of process to aid support staff
  • 32.
  • 33. SCALEFAST Architecture Flow stores, versions and schedules workflows which are predefined and synchronised grid jobs. Grid Clients are any processes able to submit grid jobs. Grid Distributor receives job requests and maps the reduced jobs as tasks across workers. Grid Workers request and process job tasks Flow Grid Client 0 Grid Client ... Grid Client Q Grid Distributor Worker 0 Worker ... Worker X Server 0 Runner Runner Runner Server ... Server N Local disk Shared Storage
  • 34. Flow GUI A simple process example with 3 steps Processes can have multiple branch dependencies e.g. 1 to many and many to 1 Processes can be build up from sub processes Flow highlights the status of the individual steps By clicking on a step, you are redirected to the Grid for further details. Processes can be paused and restarted. On a processes failure, it can be restarted at any step in the process.
  • 35. Grid Job Details GUI Parameters, status and details of a grid job Individual tasks can be drilled down into Stderr and Stdout out can be accessed and queried across all tasks Input parameters, context and output visible at job or task level
  • 36. Grid Summary GUI High level view on the Grid status and activity View active worker nodes View job activity and history

Hinweis der Redaktion

  1. Scheduling Tools: Active Batch
  2. Issues in the top grouping can be addressed by tools like Active Batch
  3. Support staff can now see in very granular process details where process failed.It is no easier to determine the cause of a failure, was it: Resource issues Bad static data Bugs in code
  4. There is also an instance of flow that can be easily integrated into an Enterprise Workflow/Automation Application