Whenever we face requirements that implies a search in documents, like microsoft word,pdf, XML files or even long text stored in an Oracle table. The solution is usually searched in the market, as an expensive tool, or developed as in-house complex solutions involving many people for a long time project. There is another solution full of resources at our reach. Oracle databases, with licenses for (EE, SE, PE) have an option for indexing text and documents, embedded or referenced in tables, with resources to deal with stop-lists, prefix search, accents, case, references, XLM, HTML and tags. This option is called Oracle Text and this presentation is an overview of its resources and features.

1. Carlos Oliveira / May 31, 2012

2. Agenda
 Oracle Text Overview
 Introduction
 Oracle Text Overview
 Types of Index
 Text Query Application
 Document Presentation and Highlighting
 Document Samples
 Oracle Text Indexing Process
 Indexing Classes
 Examples
 Contains Operators
 POC
 Training & Reference
 Questions

3. Introduction
I am a forward-looking Information Systems Architect with a
solid Oracle DBA background comprising the daily
infrastructure tasks of the DBA, several projects as a Data
Modeler, and performance management projects.
I Started on the mainframe business, and soon had a deep dive
in application development for Oracle databases. After
acquiring an Oracle certification, I worked on performance
enhancement for applications using Oracle databases, and later
worked several years as an infrastructure DBA, later I worked
on data modeling projects and more recently a performance
management project, on both application and database layers.

4. “The limits of my language
mean the limits of my world.”
Ludwig Wittgenstein

5. What is Oracle Text
•An option the database that extends the text indexes
•It is a free option for Oracle DB (EE, SE, and PE)
•Has cataloging, referencing and classification features
•Deals with tags, such as HTML or XML
•Extends indexing for:
•Documents stored in tables or referenced
•PDF, MS Word, XML, text, ...
•using data types as BLOB, BFILE, CLOB, long, ...
•even web pages, stored or referenced

6. Oracle Text Overview

7. Types of Index
Type of Query
Index Description Operator
CONTEXT Use this index to build a text retrieval application when your text consists of CONTAINS
large coherent documents. You can index documents of different formats such
as Microsoft Word, HTML, XML, or plain text.
You can customize your index in a variety of ways.
CTXCAT Use this index type to improve mixed query performance. Suitable for querying CATSEARCH
small text fragments with structured criteria like dates, item names, and prices
that are stored across columns.
CTXRULE Use to build a document classification application. You create this index on a MATCHES
table of queries, where each query has a classification.
Single documents (plain text, HTML, or XML) can be classified by using the
MATCHES operator.

8. Text Query Application

9. Document Presentation and
Highlighting
Output Procedure
Plain text version, no highlights CTX_DOC.FILTER
HTML version of document, no highlights CTX_DOC.FILTER
Highlighted document, plain text version CTX_DOC.MARKUP
Highlighted document, HTML version CTX_DOC.MARKUP
Highlight offset information for plain text version CTX_DOC.HIGHLIGHT
Highlight offset information for HTML version CTX_DOC.HIGHLIGHT
Theme summaries and gist of document. CTX_DOC.GIST
List of themes in document. CTX_DOC.THEMES

10. Document Samples

11. Oracle Text Indexing Process

12. Indexing Classes
Class Description
Datastore How are your documents stored?
Filter How can the documents be converted to plaintext?
Lexer What language is being indexed?
Wordlist How should stem and fuzzy queries be expanded?
Storage How should the index data be stored?
Stop List What words or themes are not to be indexed?
Section Group How are documents sections defined?

13. Example Parameters
EXEC ctx_ddl.drop_preference('address_lx'); begin
EXEC ctx_ddl.drop_preference('address_wl'); ctx_ddl.create_preference('address_lx','BASIC_LEXER');
EXEC ctx_ddl.drop_preference('address_st'); -- removes diacritics
EXEC ctx_ddl.drop_stoplist('address_sl'); ctx_ddl.set_attribute('address_lx','base_letter','YES');
ctx_ddl.create_preference('address_wl','BASIC_WORDLIST');
ctx_ddl.create_stoplist('address_sl', 'BASIC_STOPLIST');
ctx_ddl.add_stopclass('address_sl', 'NUMBERS');
ctx_ddl.add_stopword('address_sl', 'a');
...
ctx_ddl.add_stopword('address_sl', 'vocês');
ctx_ddl.create_preference('address_st', 'BASIC_STORAGE');
ctx_ddl.set_attribute('address_st','i_table_clause','TABLESPACE IDM2');
ctx_ddl.set_attribute('address_st','k_table_clause','TABLESPACE IDM2');
ctx_ddl.set_attribute('address_st','r_table_clause','TABLESPACE IDM2');
ctx_ddl.set_attribute('address_st','n_table_clause','TABLESPACE IDM2');
ctx_ddl.set_attribute('address_st',‘i_index_clause','TABLESPACE IDM2');
ctx_ddl.set_attribute('address_st','p_table_clause','TABLESPACE IDM2');
end;
/

14. Example DDL and Query
DROP INDEX dbapp.IDX_st_address_3; SET DEFINE OFF;
CREATE INDEX dbapp.IDX_st_address_3 ON SELECT NOM_st_address
dbapp.st_address(NOM_st_address) FROM dbapp.st_address
INDEXTYPE IS CTXSYS.CONTEXT WHERE CONTAINS (NOM_st_address, 'ST&MAJOR&OSCAR&STONE', 1) > 0;
PARAMETERS ('LEXER address_lx
WORDLIST address_wl Plan
STOPLIST address_sl
STORAGE address_st') SELECT STATEMENT CHOOSE Cost: 18 Bytes: 118 Cardinality: 1
PARALLEL 8; 2 TABLE ACCESS BY INDEX ROWID dbapp.st_address_3 Cost: 18 Bytes: 118 Cardinality: 1
COMMIT;
1 DOMAIN INDEX dbapp.st_address_3 Cost: 15
BEGIN
NOM_st_address
SYS.DBMS_STATS.GATHER_TABLE_STATS (
--------------------------------------------------
OwnName => 'dbapp'
OSCAR STONE MAJOR
,TabName => 'st_address'
,Estimate_Percent => NULL SET DEFINE OFF;
,Method_Opt => 'FOR ALL INDEXED COLUMNS SIZE SELECT NOM_st_address
AUTO ' FROM dbapp.st_address
,Degree => 8 WHERE CONTAINS (NOM_st_address, 'ST&OSCAR&STONE', 1) > 0;
,Cascade => TRUE
,No_Invalidate => FALSE); NOM_st_address
END; --------------------------------------------------
/ JOSE OSCAR STONE
........
OSCAR STONE MAJOR
OSCAR WEBBER STONE
28 rows selected.

15. CONTAINS Operators
•EQUIValence (=)
Query Expression Order of Evaluation
•NEAR (;)
•weight (*), threshold (>) w1 | w2 & w3 (w1) | (w2 & w3)
•MINUS (-)
•NOT (~) w1 & w2 | w3 (w1 & w2) | w3
•WITHIN
•AND (&) ?w1, w2 | w3 & w4 (?w1), (w2 | (w3 & w4))
•OR (|)
•ACCUMulate ( , )
abc = def ghi & jkl = mno ((abc = def) ghi) &
(jkl=mno)
•Wildcard Characters
•ABOUT dog and cat WITHIN body dog and (cat WITHIN body)
•stem ($)
•Fuzzy
•soundex (!)

16. Training
Resources at Oracle website
• Text Application Developer's Guide
http://docs.oracle.com/cd/B10501_01/text.920/a96517/toc.htm
• Text Reference
http://docs.oracle.com/cd/B10501_01/text.920/a96518/toc.htm

17. Thank you
Carlos Oliveira / May 31, 2012