This has provided a broad overview of semantic Web mining. We examined the semantic Web concepts and described the impact of Web mining. We started with a discussion of semantic Web mining concepts and discussed a concept of operation. Then we discussed mining XML and RDF documents as well as the semantic interoperability of these documents. We also discussed the use of agents in semantic Web mining and described the notion of incorporating mining into the semantic Web when the semantic Web is considered to be a heterogeneous database manager. Finally, we discussed the evolution of the semantic Web and also examined the use of Web mining in E-commerce.

1. Chapter 14
Semantic Web Mining
INTRODUCTION
As mentioned in Chapter 8, although the ideas are still somewhat fuzzy,
the Web is evolving into the semantic Web. The technologies that are being
developed will eventually give us machine-understandable Web pages. Fur-
thermore, Web mining is now a reality. The challenge therefore is to carry
out semantic Web mining. That is, we need to mine the semantic Web. The
question is, what does it mean to mine the semantic Web? Note that the
semantic Web is about machine-understandable Web pages to make the
Web more intelligent and able to provide useful services to the user. This
means that the information on Web pages may have to be mined so that the
machine can understand the content. Essentially, we need to carry out
machine learning on the Web.
Another challenge is to mine the RDF and XML documents to develop
semantic Web services. One can also mine the ontologies and other data-
bases so that the Web is made more intelligent. While we briefly discussed
semantic Web mining in Chapter 8, we will explore this topic in some detail
in this chapter. In particular, we will examine each of the technologies dis-
cussed in Chapter 8 and discuss the impact of Web mining.
CONCEPTS IN SEMANTIC WEB MINING
Chapter 8 discussed the semantic Web and provided an example con-
cept of operation. In this section, we discuss semantic Web mining. It is
essentially mining the information pertaining to the semantic Web. This
means mining Web pages so that the machine can better understand the
information. It also means mining the data sources to develop an effective
semantic Web.
Exhibit 1 illustrates semantic Web mining. It is essentially mining vari-
ous XML and RDF documents as well as mining ontologies and metadata. It
also includes mining the data sources on the Web and mining the informa-
tion relating to the information management technologies. Note that in
Chapter 7, while we discussed semantic Web, it was difficult to give a defi-
nition of the semantic Web. That is, it is not possible to say that this is
where the Web ends and the semantic Web begins. The semantic Web is
essentially an evolution of the Web. We will not end with the semantic Web.
It will evolve as far as we can see. In the same way, semantic Web mining
231
Copyright © 2003 CRC Press LLC

2. Web mining
XML, RDF,
metadata,
ontologies and
data modeling
Information technologies Database
management technology:
technology: query,
collaboration, transactions,
multimedia, metadata,
knowledge storage
management
Semantic
Web mining
Exhibit 1. Semantic Web Mining Technologies
will evolve from Web mining. The ultimate goal is to make the Web easier
to use. In addition, we want to mine the large quantities of information on
the Web so that humans can better perform their tasks.
Exhibit 2 illustrates a semantic Web mining concept of operation. As we
have mentioned in Chapter 8, the brokering service matches subscribers
and publishers. With semantic Web mining, the brokering services use Web
mining to determine the best publishers for the subscribers and to advise
them. That is, it is not just a matching service; it mines the information and
data sources, finds the best match, and provides advice for future services
and enhancements.
Exhibit 3 shows how Web mining agents can be integrated with the loca-
tor agents and user agents. That is, the Web mining agents give advice to the
user agents as well as to the locator agents in terms of finding the resources.
XML, RDF, AND WEB DATA MINING
As we have mentioned in Part I, the Web has both structured and
unstructured documents. That is, while some of the data may be in rela-
tional databases, XML is becoming the de facto standard for Web docu-
ments. The W3C (World Wide Web Consortium) has developed standards
for RDF. Therefore, we can expect both XML and RDF documents to
become the standard document exchange languages for the Web. This
232
Copyright © 2003 CRC Press LLC

3. Publisher service: Publisher service:
finding the best route maintaining
schedules
Broker/matching service
uses Web mining
Subscription:
request the Subscription:
best route request schedules
Exhibit 2. Concept of Operation for Semantic Web Mining
Agents acting on behalf of users
User agent User agent User agent
Agents carrying out Web mining
Mining
Mining Mining agent
agent agent
Agents finding resources
Locator
Locator Locator agent
agent agent
Resource Resource
Resource
Exhibit 3. Agents for Semantic Web Mining
233
Copyright © 2003 CRC Press LLC

4. XML/RDF
database management XML/RDF
system + data database
mining tool
Exhibit 4. Tight Integration between Data Miner and XML/RDF Database
Mining tool
XML/RDF
XML/RDF
database management
database
system
Exhibit 5. Loose Integration between Data Miner and XML/RDF Database
means that there will be XML and, eventually, RDF databases on the Web.
One can think of the semantic Web in terms of different kinds of databases
such as XML, RDF, multimedia, relational, and object databases, which also
have to be mined.
Exhibits 4 and 5 illustrate the tight coupling and loose coupling
approaches for semantic Web mining. In the tight coupling approach, the
Web miner and XML data manager are tightly integrated. In the loose cou-
pling approach, the data miner is an interface to the XML data manager.
Note that, as illustrated in Exhibit 6, one can extract structure from the
XML and RDF documents and mine the structured database. Some of the
key points were discussed in the previous chapters on mining semistruc-
tured databases.
ONTOLOGIES AND WEB DATA MINING
In Chapter 8, we discussed ontologies for the semantic Web. Ontologies
were also discussed in Chapter 5 when we addressed metadata for Web
data management. The question is, what is the relationship between ontol-
ogies and Web mining? We feel that this is a two-way relationship. Web min-
ing can help to develop ontologies. Note that ontologies are somewhat
standard definitions about various entities and events such as vehicles,
wines, animals, etc. One could mine the data on the Web and develop ontol-
234
Copyright © 2003 CRC Press LLC

5. Data mining
tool
Extract
Structure from
XML/RDF
documents
XML/RDF
documents
Exhibit 6. Extracting Structure and then Mining
ogies. For example, by mining the data we may find that there is a special
kind of wine. This type of wine can then be included in the wine ontology.
One can also mine the ontologies on the Web so that it is made more
intelligent. This is illustrated in Exhibit 7. While ontologies are developed
from metadata and ontologies in turn generate, for example, RDF schemas,
one could mine the metadata, ontologies, and RDF schemas toward devel-
oping the semantic Web. That is, the information mined from ontologies
and schemas can be used perhaps to better understand Web pages. There
is still much to be done here.
AGENTS AND WEB DATA MINING
In Chapter 13, we discussed agents and Web mining. Agents are a key
technology for the semantic Web. Therefore, we will revisit agents for com-
pleteness. As we have mentioned earlier, agents are essentially processes
that have well-defined interfaces, and communicate with each other
through well-defined protocols. The question is, what is the relationship
between agents and Web mining? One is that agent technology can be used
for Web mining. That is, we could use agents to mine the data on the Web.
While we could use simple processes for data and Web mining, with agents
we can have protocol-based communication between the agents. A mining
agent could communicate with a resource manager agent and a resource
manage agent could communicate with a locator agent, all through well-
defined protocols.
235
Copyright © 2003 CRC Press LLC

6. Metadata
concepts
Web
mining
Semantic
+ Ontologies
Web
ontology
mining
XML + RDF
schemas
Exhibit 7. Ontologies and Web Data Mining
How can Web mining support agents? With Web mining, we can find var-
ious resources, patterns, and trends. This in turn could help toward con-
structing various types of agents. We can use Web mining to support a vari-
ety of agents to carry out their jobs. These will include locator agents,
resource management agents, and agents for knowledge management and
collaboration, among others.
Exhibit 8 illustrates agents mining data as well as using Web mining to
locate resources on the Web. The ideas discussed here also apply for the
semantic Web. That is, we need agents to mine the data on the Web as well
as Web pages so that the machine can understand the Web pages better.
Eventually, this means that the burden placed on the human could be
eased.
WEB MINING AND THE SEMANTIC WEB AS A DATABASE
In Chapter 8, we discussed an alternative way to view the semantic Web.
In this view, the semantic Web is considered a database management sys-
tem managing millions of objects. This means that the semantic Web can
query and manage the data in a large distributed database.
In Exhibit 7, Chapter 8, we illustrated how database functions can be
applied to the semantic Web. In Exhibit 9, we illustrate how data mining can
236
Copyright © 2003 CRC Press LLC

7. Mining agent A: Mining agent B:
does collaborative does collaborative
mining with mining mining with mining
agent B agent A
Web
Locator agent:
Web mining: locates resources
mines the on the Web based
data sources on information given
on the Web by the Web miner
Data source
on the Web
Exhibit 8. Revisiting Agents and Web Mining
Semantic Web + mining
Information Information
source A source B
Database system + Data mining:
integration, modeling,
querying, indexing,
data mining
Information Information
source C source D
Exhibit 9. Web Mining and Semantic Web as a Database
237
Copyright © 2003 CRC Press LLC

8. Ontologies
RDF RDF
Ontology
mining
XML XML
Interoperability
manager
Web mining
Exhibit 10. Web Mining and Semantic Interoperability
be included as part of the semantic Web function. That is, the database sys-
tem which is the semantic Web can carry out data mining to extract often
previously unknown patterns and trends. There is still a lot of research to
be done in considering the semantic Web as a large heterogeneous data-
base system. For example, we need to examine schema integration, feder-
ated database concepts, and many other issues. We also need to include
data mining as part of this research.
SEMANTIC INTEROPERABILITY AND WEB MINING
In Chapter 8, we discussed semantic interoperability. Here, the various
RDF and XML documents have to interoperate. The database researchers
have carried out extensive research on semantic heterogeneity in hetero-
geneous and federated databases. Many of the concepts are also applica-
ble to the semantic Web. Note that this is essentially what we discussed in
Chapter 8 when we considered the semantic Web as a large heterogeneous
databases manager.
Exhibit 10 illustrates an approach for the interoperability of RDF docu-
ments. Note that there may be several interpretations of the same entity
and the same interpretation for different entities. The interoperability man-
238
Copyright © 2003 CRC Press LLC

9. Integration manager
uses Web mining
XML XML XML
Data source C
Data source A
Data source B
Exhibit 11. Mining for Semantic Interoperability of XML Databases
ager has to handle such heterogeneity. This is where Web mining could
help. The various ontologies could be mined to support interoperability.
In addition, the data sources on the Web could be mined. The informa-
tion extracted from mining could be given as input to the interoperability
manager. Exhibit 11 illustrates an example of integrating XML documents.
Here again, the interoperability manager uses Web mining to handle
semantic heterogeneity.
WEB SERVICES AND WEB MINING
In Chapter 8, we provided an overview of Web services. This is one of
the rapidly growing areas in Web technology. The idea is essentially to pro-
vide a number of Web services, including publishing, subscription, direc-
tory, and many others, to Web users, who make use of the services by call-
ing appropriate service routines.
Exhibit 12 illustrates the concept of incorporating data mining not only
as a Web service but also using it to enhance the Web services. For exam-
ple, one could have a data mining service mine the data sources on the
Web. Data mining can also be used to broker the information. That is, the
broker service could use data mining to match publishers with subscrib-
ers. This is an area that is still largely unexplored. While we have the vari-
ous pieces such as Web services and Web mining, we need to integrate the
two to provide better quality services to the users.
239
Copyright © 2003 CRC Press LLC

10. Data/information Data
management mining
services service
Web services
use data mining for managing services
Publishing Directory
subscription service
service
Exhibit 12. Web Services and Web Mining
WEB MINING VS. SEMANTIC WEB MINING
In Chapter 8, we discussed the evolution of the Web to the semantic
Web. As we have mentioned, we cannot say at which point the Web ends
and the semantic Web begins. It will be a continuous process. The Web will
continually evolve to become the semantic Web and, some say, the knowl-
edge Web eventually. There is only one Web, and as we make progress with
technologies and applications, the Web will continue to be enhanced. We
also stress that applications as technologies alone will not be sufficient.
Applications will drive the way the Web evolves. This is why we provide a
brief overview of various applications in Chapter 16. While our focus is on
Web mining for these applications, we will discuss how these applications
are driving the Web.
Exhibit 13 illustrates semantic Web mining. We started with data mining
in the mid-1990s. At that time, many of the data mining tools mined only
relational databases. In 2000, we began to see multimedia data mining
tools, especially text mining tools, and Web mining tools appear. These
tools will eventually evolve into semantic Web mining tools. That is, seman-
tic Web mining is about data mining on the Web to make the Web more
intelligent. By 2005, we should begin to see such tools emerge. For exam-
ple, these tools will mine Web pages and give advice to both the system
and the users.
A NOTE ON E-COMMERCE, BUSINESS INTELLIGENCE, AND
SEMANTIC WEB MINING
E-commerce is the killer application for the Web. We have provided an
overview of E-commerce in Chapter 2, and we have discussed the semantic
240
Copyright © 2003 CRC Press LLC

11. Data mining in 2005:
semantic Web mining
i.e., mining the semantic Web
and intelligent Web mining
Data mining in 2000:
multimedia data mining
Web mining
Data mining in 1995:
mainly mining
relational databases
Exhibit 13. Evolution of Semantic Web Mining
Web and E-commerce in Chapter 8. We revisit E-commerce in this chapter
for completion. E-commerce is about carrying out transactions on the Web
between the consumer and the business or between businesses. In
[THUR00], we discussed how Web data management technologies could be
used for E-commerce. We also briefly illustrated how Web mining could
help business intelligence. We revisit this topic in this section.
Web mining is one of the key technologies to gather business intelli-
gence. It is illustrated in Exhibit 14. With Web mining, one can develop cus-
tomer profiles and buying trends, and carry out targeted marketing. We can
use many of the mining techniques such as classification, clustering, and
E-commerce Mine usage
site patterns
Strategy to
Mine public and
improve
private databases
Web site
Exhibit 14. Web Mining for E-Commerce
241
Copyright © 2003 CRC Press LLC

12. link analysis to develop profiles of customers. We can extract information,
such as “customers who bought X also bought Y,” and “customers who pur-
chased X and Y are likely to purchase Z.” This way, we could market Z prod-
ucts to customers who have purchased X and Y.
We will discuss the various applications of Web mining in Chapter 17. As
we have stated, applications drive the development of the Web. That is,
applications drive technologies, and technologies in turn generate new
applications. There are many similarities between gathering business intel-
ligence and gathering intelligence for counter-terrorism. This is one of the
reasons we have discussed business intelligence analysis throughout this
book. We will focus on counter-terrorism in Part III, and will revisit busi-
ness intelligence in Chapter 27.
SUMMARY
This chapter has provided a broad overview of semantic Web mining.
We examined the semantic Web concepts discussed in Chapter 8 and
described the impact of Web mining. We started with a discussion of
semantic Web mining concepts and discussed a concept of operation.
Then we discussed mining XML and RDF documents as well as the seman-
tic interoperability of these documents. We also discussed the use of
agents in semantic Web mining and described the notion of incorporating
mining into the semantic Web when the semantic Web is considered to be
a heterogeneous database manager. Finally, we discussed the evolution of
the semantic Web and also examined the use of Web mining in E-commerce.
The previous four chapters essentially examined data management,
information retrieval, information management, and semantic Web technol-
ogies and discussed the impact of Web mining. The next three chapters will
address some related topics. Chapter 15 will discuss Web usage and struc-
ture mining, which is another major component of Web mining. Web mining
prototypes and products will be the subject of Chapter 16. We provide a
broad overview of applications in Chapter 17. This will then set the stage
for discussion in Part III of counter-terrorism as one of the critical applica-
tions for data and Web mining. Web mining is used to gather business intel-
ligence so that organizations can get information about competitors and
improve their business practices. Similarly, business intelligence can be
used to learn about the strategies of the adversary to counter terrorism.
242
Copyright © 2003 CRC Press LLC