SlideShare ist ein Scribd-Unternehmen logo
1 von 16
Downloaden Sie, um offline zu lesen
Introduction to
Knowledge Engineering
Dr. Albert Fornells Herrera
http://www.linkedin.com/in/afornells
Introduction to Knowledge Engineering - Albert Fornells
1. Definition
2. Process of Knowledge Engineering
3. References
2
Outline
Introduction to Knowledge Engineering - Albert Fornells
 What is Knowledge Engineering?
 The process of building intelligent knowledge-based systems
 Build a KBS is still more art than engineering
 Knowledge is incomplete, uncertain and heuristic
 Complex to estimate the nature and the amount of knowledge needed
 An incremental and prototyped approach is the way to incrementally evolve
the system from its conception until the final version
 Six iterative basic phases (Waterman, 1986; Durkin, 1994):
1. Problem assessment
2. Data and knowledge acquisition
3. Development of a prototype system
4. Development of a complete system
5. Evaluation and revision of the system
6. Integration and maintenance of the system
3
Knowledge Engineering
Introduction to Knowledge Engineering - Albert Fornells
4
The process of Knowledge Engineering
Introduction to Knowledge Engineering - Albert Fornells
Problem assessment
Data and knowledge
acquisition
Development of a
prototype system
Development of a
complete system
Evaluation and revision of
the system
Integration and
maintenance of the system
Determine the characteristics of the problem
Identify the main participants in the project
Specify the project’s objective
Determine the resources needed for building the system
Collect and analyze data and knowledge
Make key concepts of the system design more explicit
Choose a tool for building an Intelligent system
Transform data and represent the knowledge
Design and implement a prototype system
Test the prototype with test cases
Prepare a detailed design for a full-scale system
Collect additional data and knowledge
Develop the user interface
Implement the complete system
Evaluate the system against the performance criteria
Revise the system as necessary
Make arrangements for technology transfer
Establish an effective maintenance program
Requirements
Concepts
Structure
KBS ‘beta’
Reformulation
Redesign
Tuning
KBS tested
Maintenance
Assessment
Redesign
Iterations
 Goal: (1) Determine the characteristics of the problem; (2) Identify the participants
of the project; (3) Specify the objectives of the project and (4) Determine what
resources are needed for building the system.
 The problem characterization implies:
 Determine the problem type:
 Diagnosis. Inferring malfunctions of an object from its behavior and recommending solutions
 Selection. Recommending the best option from a list of possible alternatives
 Prediction. Predicting the future behavior of an object from its behavior in the past
 Classification. Assigning an object to one of the defined classes
 Clustering. Dividing an heterogeneous group of objects into homogeneous subgroups
 Optimization. Improving the quality of solutions until an optimal one is found
 Control Governing the behavior of an object to meet specified requirements in real-time
 Input and output variables and their interactions
 The form and content of the solution.
5
KE process: Problem assessment
Introduction to Knowledge Engineering - Albert Fornells
 Identify the participants of the project. There are two critical participants:
 The knowledge engineer (a person capable of designing, building and testing an
intelligent system)
 The domain expert (a knowledgeable person capable of solving problems in a
specific area or domain)
 Specify the project’s objectives
 Gaining a competitive edge, improving the quality of decisions, reducing labor costs,
and improving the quality of products and services, etc
 Determine what resources are needed for building the system
 Computer facilities
 Development software
 Knowledge and data sources (human experts, textbooks, manuals, web sites,
databases and examples)
 Money
6
KE process: Problem assessment
Introduction to Knowledge Engineering - Albert Fornells
 Toy problem: Found X.
 Do you think that a KBS is needed?
 X=sqrt(3^2+4^2)=5
 Maybe it is not necessary
 But maybe experts do not think the same because they misunderstand the
questions.
 They apply computer vision techniques for text recognition.
 Who is wrong?
7
KE process: Problem assessment
Introduction to Knowledge Engineering - Albert Fornells
4cm
3cm
x
4cm
3cm
x
Here is
 Goal: To obtain further understanding of the problem domain by (1) collecting and
analyzing both data and knowledge and (2) making key concepts of the system’s
design more explicit
 The choice of the system building tool depends on the acquired data. Data is
often collected from different sources and thus can be of different types.
 Continuous variables vs variables divided into several ranges vs normalized (i.e. 0 - 1)
 Symbolic (textual) data vs numerical data vs heterogeneous data
 Imprecise and noisy data vs well-defined and clean data
 As a result, the data must be transformed, or massaged, into the form useful for
a particular tool. There are three important issues that must be resolved before
massaging the data (Berry and Linoff, 1997)
 Incompatible data
 Inconsistent data
 Missing data
8
KE process: Data and knowledge acquisition
Introduction to Knowledge Engineering - Albert Fornells
 Knowledge acquisition is an inherently iterative process based on:
 Start with reviewing documents and reading books, papers and manuals related to
the problem domain in order to become familiar with the problem
 Collect further knowledge through interviewing the domain expert. The expert is
asked to identify four or five typical cases, describe how each case is solved and
explain, or ‘think out loud’, the reasoning behind each solution (Russell, 2002)
 Study and analyze the acquired knowledge, and repeat the entire process again
 However, extracting knowledge from a human expert is a difficult process – it is
often called the ‘knowledge acquisition bottleneck’.
 Quite often experts are unaware of the amount of knowledge they have and the
problem-solving strategy they use, or are unable to verbalize it.
 Experts may also provide us with irrelevant, incomplete or inconsistent information
 Example of cheese factory (Donald Michie, 1982)
 The data and knowledge acquired should enable us to describe the problem-
solving strategy at the most abstract level, but any detailed analysis has to be
done before evaluating the prototype
9
KE process: Data and knowledge acquisition
Introduction to Knowledge Engineering - Albert Fornells
 Goal: Create an intelligent system – or, rather, a small version of it – and test it
with a number of test cases.
 What is a prototype?
 A prototype system can be defined as a small version of the final system
 It is designed to test how well we understand the problem, or in other words, to make
sure that the problem-solving strategy, the tool selected for building a system, and
techniques for representing acquired data and knowledge are adequate to the task
 It also provides us with an opportunity to actively engage the domain expert in the
system’s development
 What is a test case?
 Once it is built, the prototype’s performance needs to be tested (usually together with
the domain expert)
 The domain expert takes an active part in testing the system and, as a result,
becomes more involved in the system’s development
 What should we do if we have made a bad choice of the system-building tool?
 We should throw the prototype away and start the prototyping phase over again
10
KE process: Development of a prototype system
Introduction to Knowledge Engineering - Albert Fornells
 Goal: Once prototype is assessed, we develop a plan, schedule and budget for
the complete system, and also clearly define the system’s performance.
 Main milestones:
 Adding data and knowledge to the system (i.e. collect core rules, additional
historical examples, etc.)
 Develop the user interface for delivering information to a user. Some system
focuses on explaining its reasoning process and justify its advice. Others represent
data in graphical forms.
 The development is like an evolutionary process where, as the project needs,
new data and knowledge are collected and added to the system.
11
KE process: Development of a complete system
Introduction to Knowledge Engineering - Albert Fornells
 Goal: To evaluate an intelligent system is, in fact, to assure that the system
performs the intended task to the user’s satisfaction
 Intelligent systems are designed to solve problems that quite often do not have
clearly defined ‘right’ and ‘wrong’ solutions
 A formal evaluation of the system is normally accomplished with the test cases
selected by the user
 The system’s performance is compared against the performance criteria that were
agreed at the end of the prototyping phase
 The evaluation often reveals the system’s limitations and weaknesses, so it is
revised and relevant development phases are repeated
 There are specific methodological procedures to assess the performance
(Ian Witten, 2005)
 X Stratified cross-validation
 Leave-one out
 Statistical tests are a common way for comparing the performance of the
approach with respect to experts and also with respect to others approaches
12
KE process: Evaluation and revision
Introduction to Knowledge Engineering - Albert Fornells
 Goal: It involves integrating the system into the environment where it will
operate and establishing an effective maintenance program
 What means integration?
 Integration means interfacing the new intelligent system with existing systems within
an organization and arranging for technology transfer
 What means establish a maintenance program?
 Intelligent systems are knowledge-based systems, and because knowledge evolves
over time, the system has to be able to be upgraded
 The organization that uses the system should have in-house expertise to maintain
and modify the system
 Is maintenance forever?
 No, a big change in the knowledge domain could imply to throw away the intelligent
system and start from the beginning.
13
KE process: Integration and maintenance
Introduction to Knowledge Engineering - Albert Fornells
14
Summary: Roadmap for tackling a problem
Introduction to Knowledge Engineering - Albert Fornells
Knowledge
Problem
Analysis
Data
Analysis
Data
Processing
Modeling
Evaluation
Production
Problem: I want to develop an intelligent system that can help me to fix
malfunctions of my computer.
Describe, from a global point of view, how will you tackle the process of Knowledge
Engineering?
1. Problem assessment
2. Data and knowledge acquisition
3. Development of a prototype system
4. Development of a complete system
5. Evaluation and revision of the system
6. Integration and maintenance of the system
15
Exercise: Will a KBS work for my problem?
Introduction to Knowledge Engineering - Albert Fornells
 A. J. González y D. D. Dankel, The Engineering of Knowledge Based Systems.
Theory and Practice. Prentice Hall, 2000.
16
References
Introduction to Knowledge Engineering - Albert Fornells

Weitere ähnliche Inhalte

Was ist angesagt?

Unit 11-systems-analysis-and-design
Unit 11-systems-analysis-and-designUnit 11-systems-analysis-and-design
Unit 11-systems-analysis-and-designZile Mafantiri
 
A survey of controlled experiments in software engineering
A survey of controlled experiments in software engineeringA survey of controlled experiments in software engineering
A survey of controlled experiments in software engineeringJULIO GONZALEZ SANZ
 
SYSTEM ANALYSIS AND DESIGN Assignment help
SYSTEM ANALYSIS AND DESIGN Assignment helpSYSTEM ANALYSIS AND DESIGN Assignment help
SYSTEM ANALYSIS AND DESIGN Assignment helpjohn mayer
 
System Analysis and Design (SAD)
System Analysis and Design (SAD)System Analysis and Design (SAD)
System Analysis and Design (SAD)Sachith Perera
 
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...CS, NcState
 
Systems Analysis
Systems AnalysisSystems Analysis
Systems AnalysisBli Wilson
 
Artificial Intelligence: Case-based & Model-based Reasoning
Artificial Intelligence: Case-based & Model-based ReasoningArtificial Intelligence: Case-based & Model-based Reasoning
Artificial Intelligence: Case-based & Model-based ReasoningThe Integral Worm
 
system analysis and design chapter 1 Kendall & Kendall
system analysis and design chapter 1 Kendall & Kendallsystem analysis and design chapter 1 Kendall & Kendall
system analysis and design chapter 1 Kendall & KendallDana dia
 
Applying Systems Thinking to Solve Wicked Problems in Software Engineering
Applying Systems Thinking to Solve Wicked Problems in Software EngineeringApplying Systems Thinking to Solve Wicked Problems in Software Engineering
Applying Systems Thinking to Solve Wicked Problems in Software EngineeringMajed Ayyad
 
A survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithmsA survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithmsAhmed Magdy Ezzeldin, MSc.
 
A&D - Fact Finding Techniques
A&D - Fact Finding TechniquesA&D - Fact Finding Techniques
A&D - Fact Finding Techniquesvinay arora
 
Big shadow test
Big shadow testBig shadow test
Big shadow testmunsif123
 
7. evalution of interactive system
7. evalution of interactive system7. evalution of interactive system
7. evalution of interactive systemKh Ravy
 
Test equating using irt. final
Test equating using irt. finalTest equating using irt. final
Test equating using irt. finalmunsif123
 

Was ist angesagt? (20)

Unit 11-systems-analysis-and-design
Unit 11-systems-analysis-and-designUnit 11-systems-analysis-and-design
Unit 11-systems-analysis-and-design
 
Ced unit 1 notes-new
Ced unit 1 notes-newCed unit 1 notes-new
Ced unit 1 notes-new
 
A survey of controlled experiments in software engineering
A survey of controlled experiments in software engineeringA survey of controlled experiments in software engineering
A survey of controlled experiments in software engineering
 
SYSTEM ANALYSIS AND DESIGN Assignment help
SYSTEM ANALYSIS AND DESIGN Assignment helpSYSTEM ANALYSIS AND DESIGN Assignment help
SYSTEM ANALYSIS AND DESIGN Assignment help
 
System Analysis and Design (SAD)
System Analysis and Design (SAD)System Analysis and Design (SAD)
System Analysis and Design (SAD)
 
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
 
Systems Analysis
Systems AnalysisSystems Analysis
Systems Analysis
 
Artificial Intelligence: Case-based & Model-based Reasoning
Artificial Intelligence: Case-based & Model-based ReasoningArtificial Intelligence: Case-based & Model-based Reasoning
Artificial Intelligence: Case-based & Model-based Reasoning
 
System analysis
System analysisSystem analysis
System analysis
 
system analysis and design chapter 1 Kendall & Kendall
system analysis and design chapter 1 Kendall & Kendallsystem analysis and design chapter 1 Kendall & Kendall
system analysis and design chapter 1 Kendall & Kendall
 
Prvt file test
Prvt file testPrvt file test
Prvt file test
 
Applying Systems Thinking to Solve Wicked Problems in Software Engineering
Applying Systems Thinking to Solve Wicked Problems in Software EngineeringApplying Systems Thinking to Solve Wicked Problems in Software Engineering
Applying Systems Thinking to Solve Wicked Problems in Software Engineering
 
A survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithmsA survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithms
 
A&D - Fact Finding Techniques
A&D - Fact Finding TechniquesA&D - Fact Finding Techniques
A&D - Fact Finding Techniques
 
Big shadow test
Big shadow testBig shadow test
Big shadow test
 
7. evalution of interactive system
7. evalution of interactive system7. evalution of interactive system
7. evalution of interactive system
 
Wcre13b.ppt
Wcre13b.pptWcre13b.ppt
Wcre13b.ppt
 
Chap06
Chap06Chap06
Chap06
 
Check upload1
Check upload1Check upload1
Check upload1
 
Test equating using irt. final
Test equating using irt. finalTest equating using irt. final
Test equating using irt. final
 

Andere mochten auch

Spunti - Biblioteca como 18 febbraio 2014
Spunti - Biblioteca como 18 febbraio 2014Spunti - Biblioteca como 18 febbraio 2014
Spunti - Biblioteca como 18 febbraio 2014MIUR
 
CERT - NADIA - OFFICE ADMINISTRATION
CERT - NADIA - OFFICE ADMINISTRATIONCERT - NADIA - OFFICE ADMINISTRATION
CERT - NADIA - OFFICE ADMINISTRATIONMaria Raju
 
Y O G A A N D S T R E S S Dr
Y O G A  A N D  S T R E S S  DrY O G A  A N D  S T R E S S  Dr
Y O G A A N D S T R E S S Drdrsolapurkar
 
Neurosensory disturvances following surgucal removal of mandibular third molar
Neurosensory disturvances following surgucal removal of mandibular third molarNeurosensory disturvances following surgucal removal of mandibular third molar
Neurosensory disturvances following surgucal removal of mandibular third molarIndian dental academy
 
Cálculos e recomendações para correias de transmissão de potência em v good...
Cálculos e recomendações para correias de transmissão de potência em v   good...Cálculos e recomendações para correias de transmissão de potência em v   good...
Cálculos e recomendações para correias de transmissão de potência em v good...Diogo Bellé
 
Autonomous car
Autonomous carAutonomous car
Autonomous carJay Patel
 
PLANTAS VASCULARES SIN SEMILLA
PLANTAS VASCULARES SIN SEMILLAPLANTAS VASCULARES SIN SEMILLA
PLANTAS VASCULARES SIN SEMILLAAlondra Cervantes
 
Autonomous car
Autonomous carAutonomous car
Autonomous carAnil kale
 

Andere mochten auch (17)

Spunti - Biblioteca como 18 febbraio 2014
Spunti - Biblioteca como 18 febbraio 2014Spunti - Biblioteca como 18 febbraio 2014
Spunti - Biblioteca como 18 febbraio 2014
 
JOSE CASIMIRO File
JOSE CASIMIRO FileJOSE CASIMIRO File
JOSE CASIMIRO File
 
CERT - NADIA - OFFICE ADMINISTRATION
CERT - NADIA - OFFICE ADMINISTRATIONCERT - NADIA - OFFICE ADMINISTRATION
CERT - NADIA - OFFICE ADMINISTRATION
 
Iss
IssIss
Iss
 
2009 11 16 Landerd Bedrijfspresentatie Op Internet
2009 11 16 Landerd Bedrijfspresentatie Op Internet2009 11 16 Landerd Bedrijfspresentatie Op Internet
2009 11 16 Landerd Bedrijfspresentatie Op Internet
 
geoext_dwins
geoext_dwinsgeoext_dwins
geoext_dwins
 
Y O G A A N D S T R E S S Dr
Y O G A  A N D  S T R E S S  DrY O G A  A N D  S T R E S S  Dr
Y O G A A N D S T R E S S Dr
 
Balancing It All
Balancing It AllBalancing It All
Balancing It All
 
Resume - Muhammad Shahzad
Resume - Muhammad  ShahzadResume - Muhammad  Shahzad
Resume - Muhammad Shahzad
 
Neurosensory disturvances following surgucal removal of mandibular third molar
Neurosensory disturvances following surgucal removal of mandibular third molarNeurosensory disturvances following surgucal removal of mandibular third molar
Neurosensory disturvances following surgucal removal of mandibular third molar
 
Electricity
ElectricityElectricity
Electricity
 
Licopodios
LicopodiosLicopodios
Licopodios
 
Cícadas
CícadasCícadas
Cícadas
 
Cálculos e recomendações para correias de transmissão de potência em v good...
Cálculos e recomendações para correias de transmissão de potência em v   good...Cálculos e recomendações para correias de transmissão de potência em v   good...
Cálculos e recomendações para correias de transmissão de potência em v good...
 
Autonomous car
Autonomous carAutonomous car
Autonomous car
 
PLANTAS VASCULARES SIN SEMILLA
PLANTAS VASCULARES SIN SEMILLAPLANTAS VASCULARES SIN SEMILLA
PLANTAS VASCULARES SIN SEMILLA
 
Autonomous car
Autonomous carAutonomous car
Autonomous car
 

Ähnlich wie Introduction to knowledge discovery

Systems Analysis And Design 2
Systems Analysis And Design 2Systems Analysis And Design 2
Systems Analysis And Design 2MISY
 
MODULE 1_Introduction to Data analytics and life cycle..pptx
MODULE 1_Introduction to Data analytics and life cycle..pptxMODULE 1_Introduction to Data analytics and life cycle..pptx
MODULE 1_Introduction to Data analytics and life cycle..pptxnikshaikh786
 
Artificial Intelligence Notes Unit 5
Artificial Intelligence Notes Unit 5Artificial Intelligence Notes Unit 5
Artificial Intelligence Notes Unit 5DigiGurukul
 
Section b
Section bSection b
Section bPCTE
 
IS L02 - Development of Information Systems
IS L02 - Development of Information SystemsIS L02 - Development of Information Systems
IS L02 - Development of Information SystemsJan Wong
 
Expert system (unit 1 & 2)
Expert system (unit 1 & 2)Expert system (unit 1 & 2)
Expert system (unit 1 & 2)Lakshya Gupta
 
System analysis and design
System analysis and design System analysis and design
System analysis and design Razan Al Ryalat
 
Concepts of system analysis
Concepts of system analysisConcepts of system analysis
Concepts of system analysisALFIYA ALSALAM
 
Different Approaches To Sys Bldg
Different Approaches To Sys BldgDifferent Approaches To Sys Bldg
Different Approaches To Sys BldgUSeP
 
Senior ipt term4_casestudy
Senior ipt term4_casestudySenior ipt term4_casestudy
Senior ipt term4_casestudyhccit
 
06 styles and_greenfield_design
06 styles and_greenfield_design06 styles and_greenfield_design
06 styles and_greenfield_designMajong DevJfu
 
Hsc project management 2018pptx
Hsc project management 2018pptxHsc project management 2018pptx
Hsc project management 2018pptxgreg robertson
 
System_Analysis_and_Design_Assignment_New2.ppt
System_Analysis_and_Design_Assignment_New2.pptSystem_Analysis_and_Design_Assignment_New2.ppt
System_Analysis_and_Design_Assignment_New2.pptMarissaPedragosa
 
Unit 2 Classical Systems Development Methodology.pptx
Unit 2 Classical Systems Development Methodology.pptxUnit 2 Classical Systems Development Methodology.pptx
Unit 2 Classical Systems Development Methodology.pptxVrundaPatadia
 
Online auction system srs riport
Online auction system srs  riportOnline auction system srs  riport
Online auction system srs riportDilip Prajapati
 
Online auction system srs riport
Online auction system srs  riportOnline auction system srs  riport
Online auction system srs riportDilip Prajapati
 

Ähnlich wie Introduction to knowledge discovery (20)

Systems Analysis And Design 2
Systems Analysis And Design 2Systems Analysis And Design 2
Systems Analysis And Design 2
 
Mis
MisMis
Mis
 
Sdlc1
Sdlc1Sdlc1
Sdlc1
 
MODULE 1_Introduction to Data analytics and life cycle..pptx
MODULE 1_Introduction to Data analytics and life cycle..pptxMODULE 1_Introduction to Data analytics and life cycle..pptx
MODULE 1_Introduction to Data analytics and life cycle..pptx
 
Artificial Intelligence Notes Unit 5
Artificial Intelligence Notes Unit 5Artificial Intelligence Notes Unit 5
Artificial Intelligence Notes Unit 5
 
Section b
Section bSection b
Section b
 
IS L02 - Development of Information Systems
IS L02 - Development of Information SystemsIS L02 - Development of Information Systems
IS L02 - Development of Information Systems
 
Expert system (unit 1 & 2)
Expert system (unit 1 & 2)Expert system (unit 1 & 2)
Expert system (unit 1 & 2)
 
System analysis and design
System analysis and design System analysis and design
System analysis and design
 
Concepts of system analysis
Concepts of system analysisConcepts of system analysis
Concepts of system analysis
 
Different Approaches To Sys Bldg
Different Approaches To Sys BldgDifferent Approaches To Sys Bldg
Different Approaches To Sys Bldg
 
Senior ipt term4_casestudy
Senior ipt term4_casestudySenior ipt term4_casestudy
Senior ipt term4_casestudy
 
06 styles and_greenfield_design
06 styles and_greenfield_design06 styles and_greenfield_design
06 styles and_greenfield_design
 
Hsc project management 2018pptx
Hsc project management 2018pptxHsc project management 2018pptx
Hsc project management 2018pptx
 
SAD_SDLC.pptx
SAD_SDLC.pptxSAD_SDLC.pptx
SAD_SDLC.pptx
 
System_Analysis_and_Design_Assignment_New2.ppt
System_Analysis_and_Design_Assignment_New2.pptSystem_Analysis_and_Design_Assignment_New2.ppt
System_Analysis_and_Design_Assignment_New2.ppt
 
Unit 2 Classical Systems Development Methodology.pptx
Unit 2 Classical Systems Development Methodology.pptxUnit 2 Classical Systems Development Methodology.pptx
Unit 2 Classical Systems Development Methodology.pptx
 
Introduction
IntroductionIntroduction
Introduction
 
Online auction system srs riport
Online auction system srs  riportOnline auction system srs  riport
Online auction system srs riport
 
Online auction system srs riport
Online auction system srs  riportOnline auction system srs  riport
Online auction system srs riport
 

Kürzlich hochgeladen

AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 

Kürzlich hochgeladen (20)

AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 

Introduction to knowledge discovery

  • 1. Introduction to Knowledge Engineering Dr. Albert Fornells Herrera http://www.linkedin.com/in/afornells Introduction to Knowledge Engineering - Albert Fornells
  • 2. 1. Definition 2. Process of Knowledge Engineering 3. References 2 Outline Introduction to Knowledge Engineering - Albert Fornells
  • 3.  What is Knowledge Engineering?  The process of building intelligent knowledge-based systems  Build a KBS is still more art than engineering  Knowledge is incomplete, uncertain and heuristic  Complex to estimate the nature and the amount of knowledge needed  An incremental and prototyped approach is the way to incrementally evolve the system from its conception until the final version  Six iterative basic phases (Waterman, 1986; Durkin, 1994): 1. Problem assessment 2. Data and knowledge acquisition 3. Development of a prototype system 4. Development of a complete system 5. Evaluation and revision of the system 6. Integration and maintenance of the system 3 Knowledge Engineering Introduction to Knowledge Engineering - Albert Fornells
  • 4. 4 The process of Knowledge Engineering Introduction to Knowledge Engineering - Albert Fornells Problem assessment Data and knowledge acquisition Development of a prototype system Development of a complete system Evaluation and revision of the system Integration and maintenance of the system Determine the characteristics of the problem Identify the main participants in the project Specify the project’s objective Determine the resources needed for building the system Collect and analyze data and knowledge Make key concepts of the system design more explicit Choose a tool for building an Intelligent system Transform data and represent the knowledge Design and implement a prototype system Test the prototype with test cases Prepare a detailed design for a full-scale system Collect additional data and knowledge Develop the user interface Implement the complete system Evaluate the system against the performance criteria Revise the system as necessary Make arrangements for technology transfer Establish an effective maintenance program Requirements Concepts Structure KBS ‘beta’ Reformulation Redesign Tuning KBS tested Maintenance Assessment Redesign Iterations
  • 5.  Goal: (1) Determine the characteristics of the problem; (2) Identify the participants of the project; (3) Specify the objectives of the project and (4) Determine what resources are needed for building the system.  The problem characterization implies:  Determine the problem type:  Diagnosis. Inferring malfunctions of an object from its behavior and recommending solutions  Selection. Recommending the best option from a list of possible alternatives  Prediction. Predicting the future behavior of an object from its behavior in the past  Classification. Assigning an object to one of the defined classes  Clustering. Dividing an heterogeneous group of objects into homogeneous subgroups  Optimization. Improving the quality of solutions until an optimal one is found  Control Governing the behavior of an object to meet specified requirements in real-time  Input and output variables and their interactions  The form and content of the solution. 5 KE process: Problem assessment Introduction to Knowledge Engineering - Albert Fornells
  • 6.  Identify the participants of the project. There are two critical participants:  The knowledge engineer (a person capable of designing, building and testing an intelligent system)  The domain expert (a knowledgeable person capable of solving problems in a specific area or domain)  Specify the project’s objectives  Gaining a competitive edge, improving the quality of decisions, reducing labor costs, and improving the quality of products and services, etc  Determine what resources are needed for building the system  Computer facilities  Development software  Knowledge and data sources (human experts, textbooks, manuals, web sites, databases and examples)  Money 6 KE process: Problem assessment Introduction to Knowledge Engineering - Albert Fornells
  • 7.  Toy problem: Found X.  Do you think that a KBS is needed?  X=sqrt(3^2+4^2)=5  Maybe it is not necessary  But maybe experts do not think the same because they misunderstand the questions.  They apply computer vision techniques for text recognition.  Who is wrong? 7 KE process: Problem assessment Introduction to Knowledge Engineering - Albert Fornells 4cm 3cm x 4cm 3cm x Here is
  • 8.  Goal: To obtain further understanding of the problem domain by (1) collecting and analyzing both data and knowledge and (2) making key concepts of the system’s design more explicit  The choice of the system building tool depends on the acquired data. Data is often collected from different sources and thus can be of different types.  Continuous variables vs variables divided into several ranges vs normalized (i.e. 0 - 1)  Symbolic (textual) data vs numerical data vs heterogeneous data  Imprecise and noisy data vs well-defined and clean data  As a result, the data must be transformed, or massaged, into the form useful for a particular tool. There are three important issues that must be resolved before massaging the data (Berry and Linoff, 1997)  Incompatible data  Inconsistent data  Missing data 8 KE process: Data and knowledge acquisition Introduction to Knowledge Engineering - Albert Fornells
  • 9.  Knowledge acquisition is an inherently iterative process based on:  Start with reviewing documents and reading books, papers and manuals related to the problem domain in order to become familiar with the problem  Collect further knowledge through interviewing the domain expert. The expert is asked to identify four or five typical cases, describe how each case is solved and explain, or ‘think out loud’, the reasoning behind each solution (Russell, 2002)  Study and analyze the acquired knowledge, and repeat the entire process again  However, extracting knowledge from a human expert is a difficult process – it is often called the ‘knowledge acquisition bottleneck’.  Quite often experts are unaware of the amount of knowledge they have and the problem-solving strategy they use, or are unable to verbalize it.  Experts may also provide us with irrelevant, incomplete or inconsistent information  Example of cheese factory (Donald Michie, 1982)  The data and knowledge acquired should enable us to describe the problem- solving strategy at the most abstract level, but any detailed analysis has to be done before evaluating the prototype 9 KE process: Data and knowledge acquisition Introduction to Knowledge Engineering - Albert Fornells
  • 10.  Goal: Create an intelligent system – or, rather, a small version of it – and test it with a number of test cases.  What is a prototype?  A prototype system can be defined as a small version of the final system  It is designed to test how well we understand the problem, or in other words, to make sure that the problem-solving strategy, the tool selected for building a system, and techniques for representing acquired data and knowledge are adequate to the task  It also provides us with an opportunity to actively engage the domain expert in the system’s development  What is a test case?  Once it is built, the prototype’s performance needs to be tested (usually together with the domain expert)  The domain expert takes an active part in testing the system and, as a result, becomes more involved in the system’s development  What should we do if we have made a bad choice of the system-building tool?  We should throw the prototype away and start the prototyping phase over again 10 KE process: Development of a prototype system Introduction to Knowledge Engineering - Albert Fornells
  • 11.  Goal: Once prototype is assessed, we develop a plan, schedule and budget for the complete system, and also clearly define the system’s performance.  Main milestones:  Adding data and knowledge to the system (i.e. collect core rules, additional historical examples, etc.)  Develop the user interface for delivering information to a user. Some system focuses on explaining its reasoning process and justify its advice. Others represent data in graphical forms.  The development is like an evolutionary process where, as the project needs, new data and knowledge are collected and added to the system. 11 KE process: Development of a complete system Introduction to Knowledge Engineering - Albert Fornells
  • 12.  Goal: To evaluate an intelligent system is, in fact, to assure that the system performs the intended task to the user’s satisfaction  Intelligent systems are designed to solve problems that quite often do not have clearly defined ‘right’ and ‘wrong’ solutions  A formal evaluation of the system is normally accomplished with the test cases selected by the user  The system’s performance is compared against the performance criteria that were agreed at the end of the prototyping phase  The evaluation often reveals the system’s limitations and weaknesses, so it is revised and relevant development phases are repeated  There are specific methodological procedures to assess the performance (Ian Witten, 2005)  X Stratified cross-validation  Leave-one out  Statistical tests are a common way for comparing the performance of the approach with respect to experts and also with respect to others approaches 12 KE process: Evaluation and revision Introduction to Knowledge Engineering - Albert Fornells
  • 13.  Goal: It involves integrating the system into the environment where it will operate and establishing an effective maintenance program  What means integration?  Integration means interfacing the new intelligent system with existing systems within an organization and arranging for technology transfer  What means establish a maintenance program?  Intelligent systems are knowledge-based systems, and because knowledge evolves over time, the system has to be able to be upgraded  The organization that uses the system should have in-house expertise to maintain and modify the system  Is maintenance forever?  No, a big change in the knowledge domain could imply to throw away the intelligent system and start from the beginning. 13 KE process: Integration and maintenance Introduction to Knowledge Engineering - Albert Fornells
  • 14. 14 Summary: Roadmap for tackling a problem Introduction to Knowledge Engineering - Albert Fornells Knowledge Problem Analysis Data Analysis Data Processing Modeling Evaluation Production
  • 15. Problem: I want to develop an intelligent system that can help me to fix malfunctions of my computer. Describe, from a global point of view, how will you tackle the process of Knowledge Engineering? 1. Problem assessment 2. Data and knowledge acquisition 3. Development of a prototype system 4. Development of a complete system 5. Evaluation and revision of the system 6. Integration and maintenance of the system 15 Exercise: Will a KBS work for my problem? Introduction to Knowledge Engineering - Albert Fornells
  • 16.  A. J. González y D. D. Dankel, The Engineering of Knowledge Based Systems. Theory and Practice. Prentice Hall, 2000. 16 References Introduction to Knowledge Engineering - Albert Fornells