1. Competitive advantage from Data
Mining: some lessons learnt in the
Information Systems field
Mykola Pechenizkiy, Seppo Puuronen
Department of Computer Science
University of Jyväskylä
Finland
Alexey Tsymbal
Department of Computer Science
Trinity College Dublin
Ireland
PMKDâ05 Copenhagen, Denmark August 22-26, 2005
2. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
OutlineOutline
⢠Introduction and What is our message?
⢠Part I: Existing frameworks for DM
â Theory-oriented: Databases; Statistics; Machine learning; etc
â Process-oriented: Fayyadâs, CRISP, Reinartzâs
⢠Part II: Where we are? â rigor vs. relevance in DM
⢠Part III: Towards the new framework for DM
research
â DM System as adaptive Information System (IS)
â DM research as IS Development: DM system as artefact
â DM success model: success factors
â KM Challenges in KDD
â One possible reference for new DM research framework
⢠Further plans and Discussion
3. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
What isWhat is Data MiningData Mining
Data mining or Knowledge discovery is the process of
finding previously unknown and potentially interesting
patterns and relations in large databases (Fayyad, KDDâ96)
Data mining is the emerging science and industry of
applying modern statistical and computational
technologies to the problem of finding useful patterns
hidden within large databases (John 1997)
Intersection of many fields: statistics, AI, machine
learning, databases, neural networks, pattern recognition,
econometrics, etc.
4. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
H.H. Information SystemsInformation Systems
ďą H.0 GENERAL
ďą H.1 MODELS AND PRINCIPLES
ďą H.2 DATABASE MANAGEMENT
⢠H.2.0 General
â Security, integrity, and protection
⢠H.2.8 Database Applications
â Data mining
â Image databases
â Scientific databases
â Spatial databases and GIS
â Statistical databases
⢠H.2.m Miscellaneous
http://www.acm.org/class/1998/ valid in 2003
5. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
I. Computing MethodologiesI. Computing Methodologies
ďą I.5 PATTERN RECOGNITION
⢠I.5.0 General
⢠I.5.1 Models
â Deterministic
â Fuzzy set
â Geometric
â Neural nets
â Statistical
â Structural
⢠I.5.2 Design Methodology
â Classifier design &
evaluation
â Feature evaluation &
selection
â Pattern analysis
⢠I.5.3 Clustering
â Algorithms
â Similarity measures
⢠I.5.4 Applications
â Computer vision
â Signal processing
â Text processing
â Waveform analysis
ďą I.2 ARTIFICIAL INTELLIGENCE
⢠I.2.0 General
â Cognitive simulation
â Philosophical foundations
⢠I.2.1 Applications and Expert Systems
⢠I.2.2 Automatic Programming
⢠I.2.3 Deduction and Theorem Proving
⢠I.2.4 Knowledge Representation
Formalisms and Methods
⢠I.2.5 Programming Languages and
Software
⢠I.2.6 Learning
â Analogies
â Concept learning
â Connectionism and neural nets
â Induction
â Knowledge acquisition
â Language acquisition
â Parameter learning
⢠I.2.7 Natural Language Processing
⢠I.2.m Miscellaneous
6. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
G. Mathematics of ComputingG. Mathematics of Computing
ďą G.3 PROBABILITY AND STATISTICS
⢠Correlation and regression analysis
⢠Distribution functions
⢠Experimental design
⢠Markov processes
⢠Multivariate statistics
⢠Nonparametric statistics
⢠Probabilistic algorithms (including Monte Carlo)
⢠Statistical computing
7. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Our MessageOur Message
⢠DM is still a technology having great expectations
to enable organizations to take more benefit of
their huge databases.
⢠There exist some success stories where
organizations have managed to have competitive
advantage of DM.
⢠Still the strong focus of most DM-researchers in
technology-oriented topics does not support
expanding the scope in less rigorous but
practically very relevant sub-areas.
⢠Research in the IS discipline has strong traditions
to take into account human and organizational
aspects of systems beside the technical ones.
8. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Our MessageOur Message
⢠Currently the maturation of DM-supporting processes which would
take into account human and organizational aspects is still living its
childhood.
⢠DM community might benefit, at least from the practical point of
view, looking at some other older sub-areas of IT having traditions to
consider solution-driven concepts with a focus also on human and
organizational aspects.
⢠The DM community by becoming more amenable to research results
of the IS community might be able to increase its collective
understanding of
â how DM artifacts are developed â conceived, constructed, and
implemented,
â how DM artifacts are used, supported and evolved,
â how DM artifacts impact and are impacted by the contexts in
which they are embedded.
9. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Part IPart I
⢠Existing Frameworks for DM
â Theory-oriented
⢠Databases;
⢠Statistics;
⢠Machine learning;
⢠Data compression
â Process-oriented
⢠Fayyadâs
⢠CRISP-DM
⢠Reinartzâs
11. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Database PerspectiveDatabase Perspective
⢠DM as application to DBs
â âIn the same way business applications are currently supported
using SQL-based API, the KKD applications need to be provided
with application development support.â
â query KDD objects, support for finding NNs, clustering, or
discretization and aggregate operations.
⢠Inductive databases approach
â query concept should be applied also to data mining and
knowledge discovery tasks
⢠âthere is no such thing as discovery, it is all in the power of the
query languageâ
â contain not only the data but the theory of the data as well
Imielinski, T., and Mannila, H. 1996, A database perspective on knowledge discovery.
Communications of the ACM, 39(11), 58-64.
Boulicaut, J., Klemettinen, M., and Mannila, H. 1999, Modeling KDD processes within
the inductive database framework. In Proceedings of the First International
Conference on Data Warehousing and Knowledge Discovery, Springer-Verlag,
London, 293-302
12. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Reductionism ApproachReductionism Approach
⢠Two basic Statistical Paradigms
â âStatistical Experimentâ
⢠Fisherâs version, inductive principle of maximum likelihood
⢠Neyman and Pearson-Waldâs version, inductive behaviour
⢠Bayesian version, maximum posterior probability
⢠âStatistical learning from empirical processâ
â âStructural Data Analysisâ
⢠SVD
⢠Data mining â statistics - the issue of computational feasibility has a
much clearer role in data mining than in statistics
â data mining area approaches that emphasize on database integration,
simplicity of use, and the understandability of results
â theoretical framework of statistics does not concern much about data
analysis as a process that includes several steps
13. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Machine Learning ApproachMachine Learning Approach
⢠âlet the data suggest a modelâ can be seen as a
practical alternative to the statistical paradigm âfit a
model to the dataâ
⢠Constructive Induction â a learning process, two
intertwined phases: construction of the âbestâ
representation space and generating hypothesis in the
found space (Michalski & Wnek, 1993).
â Feature transformation (PCA, SVD, Random
Projection)
â Feature selection
â LSI
14. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Data Compression ApproachData Compression Approach
â Compress the data set by finding some structure or
knowledge for it, where knowledge is interpreted as a
representation that allows coding the data by using fewer
amount of bits.
â Theories should not be ad hoc that is they should not
overfit the examples used to build it.
â Occamâs razor principle,14th century.
⢠"when you have two competing models which
make exactly the same predictions, the one that
is simpler is the better".
Mehta, M., Rissanen, J., and Agrawal, R. 1995, MDL-based decision tree pruning.
In U.M. Fayyad, R. Uthurusamy (Eds.) Proceedings of the KDD 1995, AAAI Press,
Montreal, Canada, 216-221.
15. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Other Theoretical frameworks for DMOther Theoretical frameworks for DM
⢠Microeconomic view
â the key point is that data mining is about finding actionable
patterns: the only interest is in patterns that can somehow be used
to increase utility;
â a decision theoretic formulation of this principle: the goal can be
formulated in finding a decision x that tries to maximise utility
function f(x).
Kleinberg, J., Papadimitriou, C., and Raghavan, P. 1998, A microeconomic view of data
mining, Data Mining and Knowledge Discovery 2(4), 311-324
⢠Philosophy of Science
â logical empiricism, critical rationalism, systems theory
â formism, mechanism, contextualism
â dispersive vs. integrative, analytical vs. synthetic theories
â subjectivist vs. objectivist, nomothetic vs. ideographic,
nominalism vs. realism, voluntarism vs. determinism,
epistemological assumptions
â Explanation, prediction, understanding
17. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Knowledge discovery as a processKnowledge discovery as a process
Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R.,
Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press, 1997.
I
18. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
CRISP-DMCRISP-DM
http://www.crisp-dm.org/
19. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
KDD: âVertical SolutionsâKDD: âVertical Solutionsâ
Business
Understanding
Data
Understanding
Data
Preparation
Data
Exproration
Data
Mining
Evaluation &
Interpretation
Deployment
Experience accumulation
Reinartz, T. 1999, Focusing Solutions for Data Mining.
LNAI 1623, Berlin Heidelberg.
20. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Conclusion on different frameworksConclusion on different frameworks
â Reductionist approach of viewing data mining as statistics has
advantages of the strong background, and easy-formulated
problems.
â The data mining tasks concerning processed like clusterisation,
regression and classification fit easily into these approaches.
â More recent (process-oriented) frameworks address the issues
related to a view of data mining as a process, and its iterative
and interactive nature
21. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Part IIPart II
Where we are?
Rigor and Relevance in DM Reseach
22. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
So, where are we?So, where are we?
⢠Lin in Wu et al. notices that a new successful
industry (as DM) can follow consecutive phases:
1. discovering a new idea,
2. ensuring its applicability,
3. producing small-scale systems to test the market,
4. better understanding of new technology and
5. producing a fully scaled system.
⢠At the present moment there are several dozens
of DM systems, none of which can be compared
to the scale of a DBMS system.
â This fact indicates that we are still in the 3rd phase in
the DM area!
23. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Rigor vs Relevance in DM ResearchRigor vs Relevance in DM Research
Relevance
Rigor
Relevance Rigor
24. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Where is the focus?Where is the focus?
⢠Still! ⌠speeding-up, scaling-up, and increasing the accuracies of
DM techniques.
⢠Piatetsky-Shapiro : âwe see many papers proposing incremental
refinements in association rules algorithms, but very few papers
describing how the discovered association rules are usedâ
⢠Lin claims that the R&D goals of DM are quite different:
â since research is knowledge-oriented while development is
profit-oriented.
â Thus, DM research is concentrated on the development of new
algorithms or their enhancements,
â but the DM developers in domain areas are aware of cost
considerations: investment in research, product development,
marketing, and product support.
⢠However, we believe that the study of the DM development and DM
use processes is equally important as the technological aspects and
therefore such research activities are likely to emerge within the DM
field.
25. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Part IIIPart III
Towards the new framework for
DM research
26. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
DMS in the Kernel of an OrganizationDMS in the Kernel of an Organization
DM Task(s)
DMS (Artifact)
Organization
Environment
⢠DM is fundamentally application-oriented area motivated by business
and scientific needs to make sense of mountains of data.
⢠A DMS is generally used to support or do some task(s) by human
beings in an organizational environment both having their desires
related to DMS.
⢠Further, the organization has its own environment that has its own
interest related to DMS, e.g. that privacy of people is not violated.
27. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
The ISs-based paradigm for DMThe ISs-based paradigm for DM
Ives B., Hamilton S., Davis G. (1980). âA Framework for Research in Computer-based MISâ
Management Science, 26(9), 910-934.
âInformation systems are powerful instruments for organizational
problem solving through formal information processingâ
Lyytinen, K., 1987, âDifferent perspectives on ISs: problems and solutions.â ACM Computing Surveys, 19(1), 5-46.
User
Environment
IS
Development
Environment
IS
operations
environment
The
Use
Process
The
Development
Process
The
Operation
Process
The Organizational Environment
The External Environment
The
Information
Subsystem
(ISS)
28. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
DM Artifact DevelopmentDM Artifact Development
DM Artifact
Development
Experimentation
Theory Building
Observation
Adapted from: Nunamaker, W., Chen, M., and Purdin, T. 1990-91, Systems
development in information systems research, Journal of Management
Information Systems, 7(3), 89-106.
A multimethodological approach to the construction of an artefact for DM
29. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Research methods in a paper on DMResearch methods in a paper on DM
â Theoretical approach: theory
creating
⢠Hypothesis, new
algorithm, etc.
â Constructive approach
⢠Prototype of a DM tool
â Theoretical approach: theory
testing and evaluation
⢠Artificial, benchmark,
real-world data
⢠Evaluation techniques
â Conclusion on theory
30. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
The Action Research and Design ScienceThe Action Research and Design Science
Approach to Artifact CreationApproach to Artifact Creation
Design
Knowledge
Awareness of business
problem
Action planning
Action taking
Conclusion
Business
Knowledge
Artifact Development
Artifact Evaluation
Contextual
Knowledge
31. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
DM Artifact Use: Success Model 1 of 3DM Artifact Use: Success Model 1 of 3
System
Quality
Information
Quality
Use
User
Satisfaction
Individual
Impact
Organizational
Impact
Service
Quality
Adapted from D&M IS Success Models
32. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
DM Artifact Use: Success Model 2 ofDM Artifact Use: Success Model 2 of
33
⢠What are the key factors of successful use and impact of
DMS both at the individual and organizational levels.
1. how the system is used, and also supported and
evolved, and
2. how the system impacts and is impacted by the
contexts in which it is embedded.
Coppock: the failure factors of DM-related projects.
⢠have nothing to do with the skill of the modeler or the
quality of data.
⢠But those do include:
1. persons in charge of the project did not formulate
actionable insights,
2. the sponsors of the work did not communicate the
insights derived to key constituents,
3. the results don't agree with institutional truths
the leadership, communication skills and
understanding of the culture of the organization are
not less important than the traditionally emphasized
technological job of turning data into insights
33. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
DM Artifact Use: Success Model 3 ofDM Artifact Use: Success Model 3 of
33
⢠Hermiz communicated his beliefs that there are
the four critical success factors for DM projects:
⢠(1) having a clearly articulated business problem that needs
to be solved and for which DM is a proper tool;
⢠(2) insuring that the problem being pursued is supported by
the right type of data of sufficient quality and in sufficient
quantity for DM;
⢠(3) recognizing that DM is a process with many components
and dependencies â the entire project cannot be "managed"
in the traditional sense of the business word;
⢠(4) planning to learn from the DM process regardless of the
outcome, and clearly understanding, that there is no
guarantee that any given DM project will be successful.
34. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
KM PerspectiveKM Perspective
⢠A knowledge-driven approach to enhance the
dynamic integration of DM strategies in knowledge
discovery systems.
⢠Focus here is on knowledge management aimed to
organise a systematic process of (meta-)knowledge
capture and refinement over time.
â knowledge extracted from data
â the higher-level knowledge required for managing DM
techniquesâ selection, combination and application
⢠Basic knowledge management processes of
â knowledge creation and identification,
representation, collection and organization,
sharing, adaptation, and application
⢠DEXAâ05: TAKMA WS paper&presentation are available
Knowledge
Creation &
Acquisition
Knowledge
Organization &
Storage
Knowledge
Distribution&
Integration
Knowledge
Adaptation &
Application
Knowledge Evaluation, Validation and Refinement
35. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
New Research Framework for DMNew Research Framework for DM
ResearchResearch
People
Roles
Capabilities
Characteristics
Organizations
Strategy
Structure&Culture
Processes
Technology
Infrastructure
Applications
Communications
Architecture
Development
Capabilities
Environment Knowledge Base
Foundations
Base-level theories
Frameworks
Models
Instantiation
Validation Criteria
Design knowledge
Methodologies
Validation Criteria
(not instantiations
of models but KDD
processes, services,
systems)
Develop/Build
Theories
Artifacts
Justify/
Evaluate
Analytical
Case Study
Experimental
Field Study
Simulation
Assess Refine
(Un-)Successful Applications in
the appropriate environment
Contribution to Knowledge Base
DM Research
ApplicableKnowledge
BusinessNeeds
Relevance Rigor
36. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Further WorkFurther Work
⢠Definition of Relevance concept in DM research
⢠The revision of the book chapter
⢠Further work on the new framework for DM
research
⢠Organization of Workshop or Special Track or
Working conference on
â more social directions in DM research likely with one of
the focuses on IS as a sister discipline.
Few options:
â IRIS Scandinavian Conference on IS is one option
â Next PMKD
â Workshop in Jyväskylä
37. PMKDâ05 Copenhagen, Denmark August 22-26, 2005
Competitive advantage from DM: lessons learnt in the IS field by M.
Thank You!Thank You!
Book chapter draft is available on request from
Mykola Pechenizkiy
Department of Computer Science and Information Systems,
University of Jyväskylä, FINLAND
E-mail: mpechen@cs.jyu.fi
Tel.: +358 14 2602472 Fax: +358 14 260 3011
http://www.cs.jyu.fi/~mpechen
Feedback is very welcome:
⢠Questions
⢠Suggestions
⢠Collaboration
Hinweis der Redaktion
<number>
<number>
ACM classification system for the computing field:
DM is a subject of database applications (H.2.8), database management (H.2), and information systems field (H.)
SPSS whitepaper [4] states that âUnless thereâs a method, thereâs madnessâ. It is accepted that just by pushing a button someone should not expect useful results to appear. An industry standard to DM projects CRISP-DM is a good initiative and a starting point directed towards the development of DM meta-artifact (methodology to produce DM artifacts). However, in our opinion it is just one guideline, which is too general-level, that every DM developer follows with or without success.
In fact, the study of development and use processes was recognized to be of importance in the IS fields many years ago, and therefore it has been introduced into the different IS frameworks.
Nevertheless, so far in the DM community there exist too few research activities directed towards the study of a DM system as an artifact aimed to enable certain DM tasks in a certain context (Figure 1). In the IS discipline two research paradigms â the behavioral-science paradigm and design-science paradigm â have
The first efforts in that direction are the ones presented in the DM Review magazine [9, 21], referred below. We believe that such efforts should be encouraged in DM research and followed by research-based reports.
Lin in Wu et al. [43] notices that in fact there have been no major impacts of DM on the business world echoed. However, even reporting of existing success stories is important. Giraud-Carrier [18] reported 136 success stories of DM, covering 9 business areas with 30 DM tools or DM vendors referred. Unfortunately, there was no deep analysis provided that would summarize or discover the main success factors and the research should be continued.
In order to distinguish between the knowledge extracted from data and the higher-level knowledge
(from the KDS perspective) required for managing techniquesâ selection, combination and application
we will refer to the latter as meta-knowledge.