2. 1. Definition
2. Process of Knowledge Engineering
3. References
2
Outline
Introduction to Knowledge Engineering - Albert Fornells
3. What is Knowledge Engineering?
The process of building intelligent knowledge-based systems
Build a KBS is still more art than engineering
Knowledge is incomplete, uncertain and heuristic
Complex to estimate the nature and the amount of knowledge needed
An incremental and prototyped approach is the way to incrementally evolve
the system from its conception until the final version
Six iterative basic phases (Waterman, 1986; Durkin, 1994):
1. Problem assessment
2. Data and knowledge acquisition
3. Development of a prototype system
4. Development of a complete system
5. Evaluation and revision of the system
6. Integration and maintenance of the system
3
Knowledge Engineering
Introduction to Knowledge Engineering - Albert Fornells
4. 4
The process of Knowledge Engineering
Introduction to Knowledge Engineering - Albert Fornells
Problem assessment
Data and knowledge
acquisition
Development of a
prototype system
Development of a
complete system
Evaluation and revision of
the system
Integration and
maintenance of the system
Determine the characteristics of the problem
Identify the main participants in the project
Specify the project’s objective
Determine the resources needed for building the system
Collect and analyze data and knowledge
Make key concepts of the system design more explicit
Choose a tool for building an Intelligent system
Transform data and represent the knowledge
Design and implement a prototype system
Test the prototype with test cases
Prepare a detailed design for a full-scale system
Collect additional data and knowledge
Develop the user interface
Implement the complete system
Evaluate the system against the performance criteria
Revise the system as necessary
Make arrangements for technology transfer
Establish an effective maintenance program
Requirements
Concepts
Structure
KBS ‘beta’
Reformulation
Redesign
Tuning
KBS tested
Maintenance
Assessment
Redesign
Iterations
5. Goal: (1) Determine the characteristics of the problem; (2) Identify the participants
of the project; (3) Specify the objectives of the project and (4) Determine what
resources are needed for building the system.
The problem characterization implies:
Determine the problem type:
Diagnosis. Inferring malfunctions of an object from its behavior and recommending solutions
Selection. Recommending the best option from a list of possible alternatives
Prediction. Predicting the future behavior of an object from its behavior in the past
Classification. Assigning an object to one of the defined classes
Clustering. Dividing an heterogeneous group of objects into homogeneous subgroups
Optimization. Improving the quality of solutions until an optimal one is found
Control Governing the behavior of an object to meet specified requirements in real-time
Input and output variables and their interactions
The form and content of the solution.
5
KE process: Problem assessment
Introduction to Knowledge Engineering - Albert Fornells
6. Identify the participants of the project. There are two critical participants:
The knowledge engineer (a person capable of designing, building and testing an
intelligent system)
The domain expert (a knowledgeable person capable of solving problems in a
specific area or domain)
Specify the project’s objectives
Gaining a competitive edge, improving the quality of decisions, reducing labor costs,
and improving the quality of products and services, etc
Determine what resources are needed for building the system
Computer facilities
Development software
Knowledge and data sources (human experts, textbooks, manuals, web sites,
databases and examples)
Money
6
KE process: Problem assessment
Introduction to Knowledge Engineering - Albert Fornells
7. Toy problem: Found X.
Do you think that a KBS is needed?
X=sqrt(3^2+4^2)=5
Maybe it is not necessary
But maybe experts do not think the same because they misunderstand the
questions.
They apply computer vision techniques for text recognition.
Who is wrong?
7
KE process: Problem assessment
Introduction to Knowledge Engineering - Albert Fornells
4cm
3cm
x
4cm
3cm
x
Here is
8. Goal: To obtain further understanding of the problem domain by (1) collecting and
analyzing both data and knowledge and (2) making key concepts of the system’s
design more explicit
The choice of the system building tool depends on the acquired data. Data is
often collected from different sources and thus can be of different types.
Continuous variables vs variables divided into several ranges vs normalized (i.e. 0 - 1)
Symbolic (textual) data vs numerical data vs heterogeneous data
Imprecise and noisy data vs well-defined and clean data
As a result, the data must be transformed, or massaged, into the form useful for
a particular tool. There are three important issues that must be resolved before
massaging the data (Berry and Linoff, 1997)
Incompatible data
Inconsistent data
Missing data
8
KE process: Data and knowledge acquisition
Introduction to Knowledge Engineering - Albert Fornells
9. Knowledge acquisition is an inherently iterative process based on:
Start with reviewing documents and reading books, papers and manuals related to
the problem domain in order to become familiar with the problem
Collect further knowledge through interviewing the domain expert. The expert is
asked to identify four or five typical cases, describe how each case is solved and
explain, or ‘think out loud’, the reasoning behind each solution (Russell, 2002)
Study and analyze the acquired knowledge, and repeat the entire process again
However, extracting knowledge from a human expert is a difficult process – it is
often called the ‘knowledge acquisition bottleneck’.
Quite often experts are unaware of the amount of knowledge they have and the
problem-solving strategy they use, or are unable to verbalize it.
Experts may also provide us with irrelevant, incomplete or inconsistent information
Example of cheese factory (Donald Michie, 1982)
The data and knowledge acquired should enable us to describe the problem-
solving strategy at the most abstract level, but any detailed analysis has to be
done before evaluating the prototype
9
KE process: Data and knowledge acquisition
Introduction to Knowledge Engineering - Albert Fornells
10. Goal: Create an intelligent system – or, rather, a small version of it – and test it
with a number of test cases.
What is a prototype?
A prototype system can be defined as a small version of the final system
It is designed to test how well we understand the problem, or in other words, to make
sure that the problem-solving strategy, the tool selected for building a system, and
techniques for representing acquired data and knowledge are adequate to the task
It also provides us with an opportunity to actively engage the domain expert in the
system’s development
What is a test case?
Once it is built, the prototype’s performance needs to be tested (usually together with
the domain expert)
The domain expert takes an active part in testing the system and, as a result,
becomes more involved in the system’s development
What should we do if we have made a bad choice of the system-building tool?
We should throw the prototype away and start the prototyping phase over again
10
KE process: Development of a prototype system
Introduction to Knowledge Engineering - Albert Fornells
11. Goal: Once prototype is assessed, we develop a plan, schedule and budget for
the complete system, and also clearly define the system’s performance.
Main milestones:
Adding data and knowledge to the system (i.e. collect core rules, additional
historical examples, etc.)
Develop the user interface for delivering information to a user. Some system
focuses on explaining its reasoning process and justify its advice. Others represent
data in graphical forms.
The development is like an evolutionary process where, as the project needs,
new data and knowledge are collected and added to the system.
11
KE process: Development of a complete system
Introduction to Knowledge Engineering - Albert Fornells
12. Goal: To evaluate an intelligent system is, in fact, to assure that the system
performs the intended task to the user’s satisfaction
Intelligent systems are designed to solve problems that quite often do not have
clearly defined ‘right’ and ‘wrong’ solutions
A formal evaluation of the system is normally accomplished with the test cases
selected by the user
The system’s performance is compared against the performance criteria that were
agreed at the end of the prototyping phase
The evaluation often reveals the system’s limitations and weaknesses, so it is
revised and relevant development phases are repeated
There are specific methodological procedures to assess the performance
(Ian Witten, 2005)
X Stratified cross-validation
Leave-one out
Statistical tests are a common way for comparing the performance of the
approach with respect to experts and also with respect to others approaches
12
KE process: Evaluation and revision
Introduction to Knowledge Engineering - Albert Fornells
13. Goal: It involves integrating the system into the environment where it will
operate and establishing an effective maintenance program
What means integration?
Integration means interfacing the new intelligent system with existing systems within
an organization and arranging for technology transfer
What means establish a maintenance program?
Intelligent systems are knowledge-based systems, and because knowledge evolves
over time, the system has to be able to be upgraded
The organization that uses the system should have in-house expertise to maintain
and modify the system
Is maintenance forever?
No, a big change in the knowledge domain could imply to throw away the intelligent
system and start from the beginning.
13
KE process: Integration and maintenance
Introduction to Knowledge Engineering - Albert Fornells
14. 14
Summary: Roadmap for tackling a problem
Introduction to Knowledge Engineering - Albert Fornells
Knowledge
Problem
Analysis
Data
Analysis
Data
Processing
Modeling
Evaluation
Production
15. Problem: I want to develop an intelligent system that can help me to fix
malfunctions of my computer.
Describe, from a global point of view, how will you tackle the process of Knowledge
Engineering?
1. Problem assessment
2. Data and knowledge acquisition
3. Development of a prototype system
4. Development of a complete system
5. Evaluation and revision of the system
6. Integration and maintenance of the system
15
Exercise: Will a KBS work for my problem?
Introduction to Knowledge Engineering - Albert Fornells
16. A. J. González y D. D. Dankel, The Engineering of Knowledge Based Systems.
Theory and Practice. Prentice Hall, 2000.
16
References
Introduction to Knowledge Engineering - Albert Fornells