SlideShare ist ein Scribd-Unternehmen logo
1 von 36
Processing Patterns for  PredictiveBusiness TM Event Processing Symposium  March 14, 2006  Tim Bass, CISSP  Principal Global Architect  TIBCO Software Inc.
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Introduction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
A Vocabulary of Confusion  (Work in Progress) Resource Management Data Fusion Sensor Fusion Information Fusion Tracking Data Mining Correlation Planning Complex Event Processing Processing Management Sensor Management Control Estimation Event Stream Processing Adapted from:  Steinberg, A., & Bowman, C.,  CRC Press,  2001
US Legislation - Monitoring Requirements The Predictive Enterprise
PredictiveBusiness TM Source:  Ranadiv é , V.,  The Power to Predict , 2006.
Example PredictiveBusiness TM  Scenarios ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
PredictiveBusiness TM  & Complex Event Processing (CEP) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]," Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 "  --- Gartner July 2003 Graphic Sources:  TIBCO Software Inc & IBM CEP Situation Manager Event Streams Historical Data Real-time Detection  and Prediction
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
A Traditional Event-Driven Architecture (Fraud) HTTP request / response  Structured messages Screen  Audit events Message Audit events Screen/ message  Audit events Fraud Detection Rules Fraud Detection Rules Queue Client/Server Channel Queue Fraud Detection Rules EMS Channel Queue Fraud Detection Rules Screen  Based Channel Fraud  Event Sensor Preprocessing Service API Queue Fraud Detection Rules HTTP Channel Queue Fraud Detection Rules API Channel … 1234Joe01021970….. Fraud  Event Fraud  Event Fraud  Event Fraud  Event Structured messages Queue Unix/ VT Channel Fraud Detection Rules Fraud Event Network TAP
Emerging Event-Decision Architecture Customer Profiles Purpose-Built Analytics Distributed Multisensor Infrastructure Internet/Extranet  Sensors Human  Sensors Edge/POC Sensors Operations Center  Other References Complex Event Processors  Sensors are Everywhere!
Capstone Constraints & Requirements ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],22
Event-Inference Hierarchy 22 Impact Assessment Situational Assessment Relationship of Events Identify Events Location, Times and Rates of Events of Interest Existence of Possible Event of Interest Data/Event Cloud Analysis of Situation & Plans Contextual and Causal  Analysis Causal Analysis, Bayesian Belief Networks, NNs, Correlation, State Estimation, Classification Use of Distributed Sensors for Estimations Raw Sensor Data (Passive and Active) Adapted from: Waltz, E. & Llinas, J., Multisensor Data Fusion, 1990 HIGH LOW MED
Event-Decision High Level Architecture 22 EVENT CLOUD (DISTRIBUTED DATA SET) KS KS KS KS KS KS KS KS KS KS KS KS KS KS Adapted from:  Engelmore, R. S., Morgan, A.J., & and Nii, H. P., Blackboard Systems, 1988 & Luckham, D., The Power of Events, 2002
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],HLA - Knowledge Sources KS KS KS
Event-Decision Architecture 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL .  .  .  LEVEL ONE EVENT REFINEMENT USER INTERFACE COMPLEX EVENT PROCESSING (CEP) DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION REFINEMENT LEVEL THREE IMPACT ASSESSMENT LEVEL FOUR PROCESS REFINEMENT Adapted from JDL:  Steinberg, A., & Bowman, C.,  Handbook of Multisensor Data Fusion, CRC Press,  2001
Structured Processing for Event-Decision ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Level of  Inference Low Med High
CEP Level 0 –  Event Preprocessing  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 1 – Event Refinement  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 2 – Situation Refinement ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 3 – Impact Assessment ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 4 – Process Refinement  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP - Database Management Examples ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CEP Level 5 – User Interface / Interaction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Our Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Processing Patterns Business Context Inference Processing Techniques Processing Patterns for PredictiveBusiness TM
Inference Algorithms for Event-Decision Processing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Map Business Context to Classical Methods Business Context Inference Processing Techniques ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Note: For Illustrative Purposes Only
Bayes Net:  Identity Theft Detection / Phishing  Source:  Bass, T., TIBCO Software Inc., January 2006 Uses Proxy Alert Service Account Lockout Profile Mismatch Brand Phishing Alert Security Alert Customer Known Fraud IP Identity Theft Login Success Phishing Alert Brand Misuse
Bayes Net:  Simple Web-Click Behavior  Click Pg Subtype Click Elapsed Associate Session ID Stores Visited Click Pg Type Click to Purchase Session Time # Items Purchased ID Browser Recognize Session ID OS Total Purchase Session ID Code Click Price Price Click Count Source:  Ambrosio, B., CleverSet Inc., December 2004
Recurring Pattern(s) for PredictiveBusiness TM ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Event-Decision Processing Characteristics Adapted (this and the next slide) from:  Steinberg, A., & Bowman, C.,  Handbook of Multisensor Data Fusion, CRC Press,  2001 Sensor Output Individual Event Aggregation  (situation) Effect  (situation, given plan) (Action) Entity  Estimate Sensor Processing Event  Processing Situation Assessment Impact  Assessment Decision  Making Activity Detection Assignment (L0)  Event Preprocessing Attribution Assignment (L1)  Event Refinement Relational Aggregation (L2)  Situation Refinement Plan  Interaction Aggregation (L3)  Impact  Assessment (Control) Planning (L4)  Process Refinement Estimation Process Association Process JDL Model  Levels
Comparison of Event-Decision Models Sense Detect Detect Analyze Analyze Decide Respond Sense & Respond Sensor Processing Collate Orient Sensor Processing (L0)  Event Preprocessing Sensor Acquisition Collect Observe Sensing --- Activity Intelligence  Cycle Boyd  Loop Waterfall  Model JDL Model  Levels Decision Execution Disseminate Act (L5)  Visualization Collate Evaluate Evaluate Disseminate Event  Processing Situation Assessment Impact  Assessment Decision  Making Orient Pattern Processing / Feature Extraction (L1)  Event Refinement Orient Situation Assessment (L2)  Situation Refinement Orient --- (L3)  Impact  Assessment Decide Decision  Making (L4)  Process Refinement
Key Takeaways ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Thank You! Tim Bass, CISSP Principal Global Architect [email_address] Complex Event Processing at TIBCO
JDL Example:  Inference ScoreCards Event Stream Raw Data Level 0 Pre-Processing Fraud Events Event Stream Level 1 Event Refinement ScoreCard Fraud Situations Fraud Events Level 2 Situation Assessment Business Impact Fraud Situations Level 3 Impact Assessment ScoreCard ScoreCard ScoreCard Event Source Task Level 4 Process Refinement ScoreCard Modified from:  Steinberg, A., & Bowman, C.,  CRC Press,  2001

Weitere ähnliche Inhalte

Was ist angesagt?

User and entity behavior analytics: building an effective solution
User and entity behavior analytics: building an effective solutionUser and entity behavior analytics: building an effective solution
User and entity behavior analytics: building an effective solutionYolanta Beresna
 
Why the DoD Uses Advanced Network-traffic Analytics to Secure its Network
Why the DoD Uses Advanced Network-traffic Analytics to Secure its NetworkWhy the DoD Uses Advanced Network-traffic Analytics to Secure its Network
Why the DoD Uses Advanced Network-traffic Analytics to Secure its NetworkNovetta
 
Novetta Entity Analytics
Novetta Entity AnalyticsNovetta Entity Analytics
Novetta Entity AnalyticsNovetta
 
Fluturas presentation @ Big Data Conclave
Fluturas presentation @ Big Data ConclaveFluturas presentation @ Big Data Conclave
Fluturas presentation @ Big Data Conclavefluturads
 
Cyber forensics intro & requirement engineering cit dec 21,2013
Cyber forensics intro & requirement engineering cit dec 21,2013Cyber forensics intro & requirement engineering cit dec 21,2013
Cyber forensics intro & requirement engineering cit dec 21,2013subramanian K
 
nullcon 2011 - Enterprise Paradigm for Controlling Data Leakage
nullcon 2011 - Enterprise Paradigm for Controlling Data Leakagenullcon 2011 - Enterprise Paradigm for Controlling Data Leakage
nullcon 2011 - Enterprise Paradigm for Controlling Data Leakagen|u - The Open Security Community
 
Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...
Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...
Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...SparkCognition
 
Network DDoS Incident Response Cheat Sheet (by SANS)
Network DDoS Incident Response Cheat Sheet (by SANS)Network DDoS Incident Response Cheat Sheet (by SANS)
Network DDoS Incident Response Cheat Sheet (by SANS)Martin Cabrera
 
"How to document your decisions", Dmytro Ovcharenko
"How to document your decisions", Dmytro Ovcharenko "How to document your decisions", Dmytro Ovcharenko
"How to document your decisions", Dmytro Ovcharenko Fwdays
 

Was ist angesagt? (9)

User and entity behavior analytics: building an effective solution
User and entity behavior analytics: building an effective solutionUser and entity behavior analytics: building an effective solution
User and entity behavior analytics: building an effective solution
 
Why the DoD Uses Advanced Network-traffic Analytics to Secure its Network
Why the DoD Uses Advanced Network-traffic Analytics to Secure its NetworkWhy the DoD Uses Advanced Network-traffic Analytics to Secure its Network
Why the DoD Uses Advanced Network-traffic Analytics to Secure its Network
 
Novetta Entity Analytics
Novetta Entity AnalyticsNovetta Entity Analytics
Novetta Entity Analytics
 
Fluturas presentation @ Big Data Conclave
Fluturas presentation @ Big Data ConclaveFluturas presentation @ Big Data Conclave
Fluturas presentation @ Big Data Conclave
 
Cyber forensics intro & requirement engineering cit dec 21,2013
Cyber forensics intro & requirement engineering cit dec 21,2013Cyber forensics intro & requirement engineering cit dec 21,2013
Cyber forensics intro & requirement engineering cit dec 21,2013
 
nullcon 2011 - Enterprise Paradigm for Controlling Data Leakage
nullcon 2011 - Enterprise Paradigm for Controlling Data Leakagenullcon 2011 - Enterprise Paradigm for Controlling Data Leakage
nullcon 2011 - Enterprise Paradigm for Controlling Data Leakage
 
Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...
Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...
Using a Cognitive Analytic Approach to Enhance Cybersecurity on Oil and Gas O...
 
Network DDoS Incident Response Cheat Sheet (by SANS)
Network DDoS Incident Response Cheat Sheet (by SANS)Network DDoS Incident Response Cheat Sheet (by SANS)
Network DDoS Incident Response Cheat Sheet (by SANS)
 
"How to document your decisions", Dmytro Ovcharenko
"How to document your decisions", Dmytro Ovcharenko "How to document your decisions", Dmytro Ovcharenko
"How to document your decisions", Dmytro Ovcharenko
 

Andere mochten auch

Using Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise SecurityUsing Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise SecurityTim Bass
 
Event Processing Reference Architecture, March 2006
Event Processing Reference Architecture, March 2006Event Processing Reference Architecture, March 2006
Event Processing Reference Architecture, March 2006Tim Bass
 
Proposed Event Processing Definitions ,September 20, 2006
Proposed Event Processing Definitions,September 20, 2006Proposed Event Processing Definitions,September 20, 2006
Proposed Event Processing Definitions ,September 20, 2006Tim Bass
 
Kennel volunteer program intro 2012
Kennel volunteer program intro 2012Kennel volunteer program intro 2012
Kennel volunteer program intro 2012guidedog
 
SCARF- Navigate Social Aspects Of The Workplace
SCARF- Navigate Social Aspects Of The WorkplaceSCARF- Navigate Social Aspects Of The Workplace
SCARF- Navigate Social Aspects Of The WorkplaceMark Coffey
 
Canine care operations overview
Canine care operations overviewCanine care operations overview
Canine care operations overviewguidedog
 
Intelligent Marketing
Intelligent MarketingIntelligent Marketing
Intelligent MarketingDavid Moore
 
Manifest Destiny
Manifest DestinyManifest Destiny
Manifest DestinyNick Hansen
 
Encore Tibco AMX BPM Architecture
Encore Tibco AMX BPM ArchitectureEncore Tibco AMX BPM Architecture
Encore Tibco AMX BPM ArchitectureDavid Moore
 
Business Intelligence Productionization
Business Intelligence ProductionizationBusiness Intelligence Productionization
Business Intelligence ProductionizationDavid Moore
 

Andere mochten auch (11)

Using Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise SecurityUsing Event Processing to Enable Enterprise Security
Using Event Processing to Enable Enterprise Security
 
Event Processing Reference Architecture, March 2006
Event Processing Reference Architecture, March 2006Event Processing Reference Architecture, March 2006
Event Processing Reference Architecture, March 2006
 
Proposed Event Processing Definitions ,September 20, 2006
Proposed Event Processing Definitions,September 20, 2006Proposed Event Processing Definitions,September 20, 2006
Proposed Event Processing Definitions ,September 20, 2006
 
Kennel volunteer program intro 2012
Kennel volunteer program intro 2012Kennel volunteer program intro 2012
Kennel volunteer program intro 2012
 
SCARF- Navigate Social Aspects Of The Workplace
SCARF- Navigate Social Aspects Of The WorkplaceSCARF- Navigate Social Aspects Of The Workplace
SCARF- Navigate Social Aspects Of The Workplace
 
Canine care operations overview
Canine care operations overviewCanine care operations overview
Canine care operations overview
 
Intelligent Marketing
Intelligent MarketingIntelligent Marketing
Intelligent Marketing
 
Manifest Destiny
Manifest DestinyManifest Destiny
Manifest Destiny
 
Encore Tibco AMX BPM Architecture
Encore Tibco AMX BPM ArchitectureEncore Tibco AMX BPM Architecture
Encore Tibco AMX BPM Architecture
 
Business Intelligence Productionization
Business Intelligence ProductionizationBusiness Intelligence Productionization
Business Intelligence Productionization
 
Tibco Amx Bpm
Tibco Amx BpmTibco Amx Bpm
Tibco Amx Bpm
 

Ähnlich wie Processing Patterns for Predictive Business

Complex Event Processing (CEP) for Next-Generation Security Event Management,...
Complex Event Processing (CEP) for Next-Generation Security Event Management,...Complex Event Processing (CEP) for Next-Generation Security Event Management,...
Complex Event Processing (CEP) for Next-Generation Security Event Management,...Tim Bass
 
Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006Tim Bass
 
Combating Fraud and Intrusion Threats with Event Processing
Combating Fraud and Intrusion Threats with Event ProcessingCombating Fraud and Intrusion Threats with Event Processing
Combating Fraud and Intrusion Threats with Event ProcessingTim Bass
 
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...Nathaniel Palmer
 
CEP and SOA: An Open Event-Driven Architecture for Risk Management
CEP and SOA: An Open Event-Driven Architecture for Risk ManagementCEP and SOA: An Open Event-Driven Architecture for Risk Management
CEP and SOA: An Open Event-Driven Architecture for Risk ManagementTim Bass
 
Detecting Opportunities and Threats with Complex Event Processing: Case St...
Detecting Opportunities and Threats with Complex Event Processing: Case St...Detecting Opportunities and Threats with Complex Event Processing: Case St...
Detecting Opportunities and Threats with Complex Event Processing: Case St...Tim Bass
 
Complex Event Processing
Complex Event ProcessingComplex Event Processing
Complex Event ProcessingJohn Plummer
 
Situation Awareness In A Complex World
Situation Awareness In A Complex WorldSituation Awareness In A Complex World
Situation Awareness In A Complex Worldvsorathia
 
Adding Rules to Improve Flexibility and Effectively Manage Complex Events
Adding Rules to Improve Flexibility and Effectively Manage Complex EventsAdding Rules to Improve Flexibility and Effectively Manage Complex Events
Adding Rules to Improve Flexibility and Effectively Manage Complex EventsTim Bass
 
SQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightSQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightEduardo Castro
 
Event Stream Processing SAP
Event Stream Processing SAPEvent Stream Processing SAP
Event Stream Processing SAPGaurav Ahluwalia
 
Open Source Event Processing for Sensor Fusion Applications
Open Source Event Processing for Sensor Fusion ApplicationsOpen Source Event Processing for Sensor Fusion Applications
Open Source Event Processing for Sensor Fusion Applicationsguestc4ce526
 
Complex Event Prosessing
Complex Event ProsessingComplex Event Prosessing
Complex Event Prosessingugur candan
 
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft Private Cloud
 
Stream Processing and Complex Event Processing together with Kafka, Flink and...
Stream Processing and Complex Event Processing together with Kafka, Flink and...Stream Processing and Complex Event Processing together with Kafka, Flink and...
Stream Processing and Complex Event Processing together with Kafka, Flink and...HostedbyConfluent
 
Perpetual Analytics - Health in Motion
Perpetual Analytics - Health in MotionPerpetual Analytics - Health in Motion
Perpetual Analytics - Health in Motionmrosenthal
 
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
5733   a deep dive into IBM Watson Foundation for CSP (WFC)5733   a deep dive into IBM Watson Foundation for CSP (WFC)
5733 a deep dive into IBM Watson Foundation for CSP (WFC)Arvind Sathi
 
Recom Banking Solution
Recom Banking  SolutionRecom Banking  Solution
Recom Banking Solutionjagishar
 

Ähnlich wie Processing Patterns for Predictive Business (20)

Complex Event Processing (CEP) for Next-Generation Security Event Management,...
Complex Event Processing (CEP) for Next-Generation Security Event Management,...Complex Event Processing (CEP) for Next-Generation Security Event Management,...
Complex Event Processing (CEP) for Next-Generation Security Event Management,...
 
Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006
 
Combating Fraud and Intrusion Threats with Event Processing
Combating Fraud and Intrusion Threats with Event ProcessingCombating Fraud and Intrusion Threats with Event Processing
Combating Fraud and Intrusion Threats with Event Processing
 
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...The “Predictive” Battlespace:  Leveraging the Power of Event-Driven Architect...
The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architect...
 
CEP and SOA: An Open Event-Driven Architecture for Risk Management
CEP and SOA: An Open Event-Driven Architecture for Risk ManagementCEP and SOA: An Open Event-Driven Architecture for Risk Management
CEP and SOA: An Open Event-Driven Architecture for Risk Management
 
Detecting Opportunities and Threats with Complex Event Processing: Case St...
Detecting Opportunities and Threats with Complex Event Processing: Case St...Detecting Opportunities and Threats with Complex Event Processing: Case St...
Detecting Opportunities and Threats with Complex Event Processing: Case St...
 
Complex Event Processing
Complex Event ProcessingComplex Event Processing
Complex Event Processing
 
Situation Awareness In A Complex World
Situation Awareness In A Complex WorldSituation Awareness In A Complex World
Situation Awareness In A Complex World
 
Adding Rules to Improve Flexibility and Effectively Manage Complex Events
Adding Rules to Improve Flexibility and Effectively Manage Complex EventsAdding Rules to Improve Flexibility and Effectively Manage Complex Events
Adding Rules to Improve Flexibility and Effectively Manage Complex Events
 
SQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightSQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsight
 
Event Stream Processing SAP
Event Stream Processing SAPEvent Stream Processing SAP
Event Stream Processing SAP
 
Open Source Event Processing for Sensor Fusion Applications
Open Source Event Processing for Sensor Fusion ApplicationsOpen Source Event Processing for Sensor Fusion Applications
Open Source Event Processing for Sensor Fusion Applications
 
Complex Event Prosessing
Complex Event ProsessingComplex Event Prosessing
Complex Event Prosessing
 
Big Data Application Architectures - Fraud Detection
Big Data Application Architectures - Fraud DetectionBig Data Application Architectures - Fraud Detection
Big Data Application Architectures - Fraud Detection
 
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview Presentation
 
Stream Processing and Complex Event Processing together with Kafka, Flink and...
Stream Processing and Complex Event Processing together with Kafka, Flink and...Stream Processing and Complex Event Processing together with Kafka, Flink and...
Stream Processing and Complex Event Processing together with Kafka, Flink and...
 
Perpetual Analytics - Health in Motion
Perpetual Analytics - Health in MotionPerpetual Analytics - Health in Motion
Perpetual Analytics - Health in Motion
 
Intellica evam summary
Intellica evam summaryIntellica evam summary
Intellica evam summary
 
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
5733   a deep dive into IBM Watson Foundation for CSP (WFC)5733   a deep dive into IBM Watson Foundation for CSP (WFC)
5733 a deep dive into IBM Watson Foundation for CSP (WFC)
 
Recom Banking Solution
Recom Banking  SolutionRecom Banking  Solution
Recom Banking Solution
 

Mehr von Tim Bass

A High Level Blackboard Architecture for Cyber SA
A High Level Blackboard Architecture for Cyber SAA High Level Blackboard Architecture for Cyber SA
A High Level Blackboard Architecture for Cyber SATim Bass
 
A Journey Into Cyberspace
A Journey Into CyberspaceA Journey Into Cyberspace
A Journey Into CyberspaceTim Bass
 
Event Processing Technical Society Event Processing Reference Architecture W...
Event Processing Technical SocietyEvent Processing Reference Architecture W...Event Processing Technical SocietyEvent Processing Reference Architecture W...
Event Processing Technical Society Event Processing Reference Architecture W...Tim Bass
 
Leveraging Business Rules in TIBCO BusinessEvents
Leveraging Business Rules in TIBCO BusinessEventsLeveraging Business Rules in TIBCO BusinessEvents
Leveraging Business Rules in TIBCO BusinessEventsTim Bass
 
Next-Generation IDS: A CEP Use Case in 10 Minutes
Next-Generation IDS: A CEP Use Case in 10 MinutesNext-Generation IDS: A CEP Use Case in 10 Minutes
Next-Generation IDS: A CEP Use Case in 10 MinutesTim Bass
 
A Survey of Event Processing Languages (EPLs), October 7, 2006
A Survey of Event Processing Languages (EPLs), October 7, 2006A Survey of Event Processing Languages (EPLs), October 7, 2006
A Survey of Event Processing Languages (EPLs), October 7, 2006Tim Bass
 

Mehr von Tim Bass (6)

A High Level Blackboard Architecture for Cyber SA
A High Level Blackboard Architecture for Cyber SAA High Level Blackboard Architecture for Cyber SA
A High Level Blackboard Architecture for Cyber SA
 
A Journey Into Cyberspace
A Journey Into CyberspaceA Journey Into Cyberspace
A Journey Into Cyberspace
 
Event Processing Technical Society Event Processing Reference Architecture W...
Event Processing Technical SocietyEvent Processing Reference Architecture W...Event Processing Technical SocietyEvent Processing Reference Architecture W...
Event Processing Technical Society Event Processing Reference Architecture W...
 
Leveraging Business Rules in TIBCO BusinessEvents
Leveraging Business Rules in TIBCO BusinessEventsLeveraging Business Rules in TIBCO BusinessEvents
Leveraging Business Rules in TIBCO BusinessEvents
 
Next-Generation IDS: A CEP Use Case in 10 Minutes
Next-Generation IDS: A CEP Use Case in 10 MinutesNext-Generation IDS: A CEP Use Case in 10 Minutes
Next-Generation IDS: A CEP Use Case in 10 Minutes
 
A Survey of Event Processing Languages (EPLs), October 7, 2006
A Survey of Event Processing Languages (EPLs), October 7, 2006A Survey of Event Processing Languages (EPLs), October 7, 2006
A Survey of Event Processing Languages (EPLs), October 7, 2006
 

Kürzlich hochgeladen

Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 

Kürzlich hochgeladen (20)

Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 

Processing Patterns for Predictive Business

  • 1. Processing Patterns for PredictiveBusiness TM Event Processing Symposium March 14, 2006 Tim Bass, CISSP Principal Global Architect TIBCO Software Inc.
  • 2.
  • 3.
  • 4. A Vocabulary of Confusion (Work in Progress) Resource Management Data Fusion Sensor Fusion Information Fusion Tracking Data Mining Correlation Planning Complex Event Processing Processing Management Sensor Management Control Estimation Event Stream Processing Adapted from: Steinberg, A., & Bowman, C., CRC Press, 2001
  • 5. US Legislation - Monitoring Requirements The Predictive Enterprise
  • 6. PredictiveBusiness TM Source: Ranadiv é , V., The Power to Predict , 2006.
  • 7.
  • 8.
  • 9.
  • 10. A Traditional Event-Driven Architecture (Fraud) HTTP request / response Structured messages Screen Audit events Message Audit events Screen/ message Audit events Fraud Detection Rules Fraud Detection Rules Queue Client/Server Channel Queue Fraud Detection Rules EMS Channel Queue Fraud Detection Rules Screen Based Channel Fraud Event Sensor Preprocessing Service API Queue Fraud Detection Rules HTTP Channel Queue Fraud Detection Rules API Channel … 1234Joe01021970….. Fraud Event Fraud Event Fraud Event Fraud Event Structured messages Queue Unix/ VT Channel Fraud Detection Rules Fraud Event Network TAP
  • 11. Emerging Event-Decision Architecture Customer Profiles Purpose-Built Analytics Distributed Multisensor Infrastructure Internet/Extranet Sensors Human Sensors Edge/POC Sensors Operations Center Other References Complex Event Processors Sensors are Everywhere!
  • 12.
  • 13. Event-Inference Hierarchy 22 Impact Assessment Situational Assessment Relationship of Events Identify Events Location, Times and Rates of Events of Interest Existence of Possible Event of Interest Data/Event Cloud Analysis of Situation & Plans Contextual and Causal Analysis Causal Analysis, Bayesian Belief Networks, NNs, Correlation, State Estimation, Classification Use of Distributed Sensors for Estimations Raw Sensor Data (Passive and Active) Adapted from: Waltz, E. & Llinas, J., Multisensor Data Fusion, 1990 HIGH LOW MED
  • 14. Event-Decision High Level Architecture 22 EVENT CLOUD (DISTRIBUTED DATA SET) KS KS KS KS KS KS KS KS KS KS KS KS KS KS Adapted from: Engelmore, R. S., Morgan, A.J., & and Nii, H. P., Blackboard Systems, 1988 & Luckham, D., The Power of Events, 2002
  • 15.
  • 16. Event-Decision Architecture 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT REFINEMENT USER INTERFACE COMPLEX EVENT PROCESSING (CEP) DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION REFINEMENT LEVEL THREE IMPACT ASSESSMENT LEVEL FOUR PROCESS REFINEMENT Adapted from JDL: Steinberg, A., & Bowman, C., Handbook of Multisensor Data Fusion, CRC Press, 2001
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26. Processing Patterns Business Context Inference Processing Techniques Processing Patterns for PredictiveBusiness TM
  • 27.
  • 28.
  • 29. Bayes Net: Identity Theft Detection / Phishing Source: Bass, T., TIBCO Software Inc., January 2006 Uses Proxy Alert Service Account Lockout Profile Mismatch Brand Phishing Alert Security Alert Customer Known Fraud IP Identity Theft Login Success Phishing Alert Brand Misuse
  • 30. Bayes Net: Simple Web-Click Behavior Click Pg Subtype Click Elapsed Associate Session ID Stores Visited Click Pg Type Click to Purchase Session Time # Items Purchased ID Browser Recognize Session ID OS Total Purchase Session ID Code Click Price Price Click Count Source: Ambrosio, B., CleverSet Inc., December 2004
  • 31.
  • 32. Event-Decision Processing Characteristics Adapted (this and the next slide) from: Steinberg, A., & Bowman, C., Handbook of Multisensor Data Fusion, CRC Press, 2001 Sensor Output Individual Event Aggregation (situation) Effect (situation, given plan) (Action) Entity Estimate Sensor Processing Event Processing Situation Assessment Impact Assessment Decision Making Activity Detection Assignment (L0) Event Preprocessing Attribution Assignment (L1) Event Refinement Relational Aggregation (L2) Situation Refinement Plan Interaction Aggregation (L3) Impact Assessment (Control) Planning (L4) Process Refinement Estimation Process Association Process JDL Model Levels
  • 33. Comparison of Event-Decision Models Sense Detect Detect Analyze Analyze Decide Respond Sense & Respond Sensor Processing Collate Orient Sensor Processing (L0) Event Preprocessing Sensor Acquisition Collect Observe Sensing --- Activity Intelligence Cycle Boyd Loop Waterfall Model JDL Model Levels Decision Execution Disseminate Act (L5) Visualization Collate Evaluate Evaluate Disseminate Event Processing Situation Assessment Impact Assessment Decision Making Orient Pattern Processing / Feature Extraction (L1) Event Refinement Orient Situation Assessment (L2) Situation Refinement Orient --- (L3) Impact Assessment Decide Decision Making (L4) Process Refinement
  • 34.
  • 35. Thank You! Tim Bass, CISSP Principal Global Architect [email_address] Complex Event Processing at TIBCO
  • 36. JDL Example: Inference ScoreCards Event Stream Raw Data Level 0 Pre-Processing Fraud Events Event Stream Level 1 Event Refinement ScoreCard Fraud Situations Fraud Events Level 2 Situation Assessment Business Impact Fraud Situations Level 3 Impact Assessment ScoreCard ScoreCard ScoreCard Event Source Task Level 4 Process Refinement ScoreCard Modified from: Steinberg, A., & Bowman, C., CRC Press, 2001