SlideShare ist ein Scribd-Unternehmen logo
1 von 1
Ambit lick Solutions
Mail Id: Ambitlick@gmail.com , Ambitlicksolutions@gmail.Com
4. Information-Theoretic Outlier Detection for Large-Scale Categorical Data
Outlier detection can usually be considered as a pre-processing step for
locating, in a data set, those objects that do not conform to well-defined notions of
expected behavior. It is very important in data mining for discovering novel or rare
events, anomalies, vicious actions, exceptional phenomena, etc. We are investigating
outlier detection for categorical data sets. This problem is especially challenging
because of the difficulty of defining a meaningful similarity measure for categorical
data. In this paper, we propose a formal definition of outliers and an optimization
model of outlier detection, via a new concept of holoentropy that takes both entropy
and total correlation into consideration. Based on this model, we define a function for
the outlier factor of an object which is solely determined by the object itself and can
be updated efficiently. We propose two practical 1-parameter outlier detection
methods, named ITB-SS and ITB-SP, which require no user-defined parameters for
deciding whether an object is an outlier. Users need only provide the number of
outliers they want to detect. Experimental results show that ITB-SS and ITB-SP are
more effective and efficient than mainstream methods and can be used to deal with
both large and high-dimensional data sets where existing algorithms fail.

Weitere ähnliche Inhalte

Andere mochten auch

Tzarouk presentation
Tzarouk presentationTzarouk presentation
Tzarouk presentationtzarouk
 
Safe Screening Flow Chart
Safe Screening Flow ChartSafe Screening Flow Chart
Safe Screening Flow ChartDave Oakes
 
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...zillesubhan
 
English christmas assessment 2012
English christmas assessment 2012English christmas assessment 2012
English christmas assessment 2012AideenG
 
Tutorial de VODPOD
Tutorial de VODPODTutorial de VODPOD
Tutorial de VODPODFernanda Gr
 
Taller delgado-ullaguari_1D
 Taller delgado-ullaguari_1D Taller delgado-ullaguari_1D
Taller delgado-ullaguari_1DJorge Andres
 
Dispdir 131212 05787 evento american cup 2013
Dispdir 131212 05787  evento american cup 2013Dispdir 131212 05787  evento american cup 2013
Dispdir 131212 05787 evento american cup 2013Daniela Petrecca
 

Andere mochten auch (14)

Tzarouk presentation
Tzarouk presentationTzarouk presentation
Tzarouk presentation
 
iGuard
iGuard iGuard
iGuard
 
Safe Screening Flow Chart
Safe Screening Flow ChartSafe Screening Flow Chart
Safe Screening Flow Chart
 
Design Gerrit Yourself
Design Gerrit YourselfDesign Gerrit Yourself
Design Gerrit Yourself
 
Factsheet ITIL lite
Factsheet ITIL liteFactsheet ITIL lite
Factsheet ITIL lite
 
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
 
English christmas assessment 2012
English christmas assessment 2012English christmas assessment 2012
English christmas assessment 2012
 
Tutorial de VODPOD
Tutorial de VODPODTutorial de VODPOD
Tutorial de VODPOD
 
Web20 Bloglab
Web20 BloglabWeb20 Bloglab
Web20 Bloglab
 
Taller delgado-ullaguari_1D
 Taller delgado-ullaguari_1D Taller delgado-ullaguari_1D
Taller delgado-ullaguari_1D
 
Humor is The Health Of the Soul - Ratu Destian
Humor is The Health Of the Soul - Ratu DestianHumor is The Health Of the Soul - Ratu Destian
Humor is The Health Of the Soul - Ratu Destian
 
Dispdir 131212 05787 evento american cup 2013
Dispdir 131212 05787  evento american cup 2013Dispdir 131212 05787  evento american cup 2013
Dispdir 131212 05787 evento american cup 2013
 
Fitxa seguiment candidatura
Fitxa seguiment candidaturaFitxa seguiment candidatura
Fitxa seguiment candidatura
 
Diapositiva subir natali
Diapositiva  subir nataliDiapositiva  subir natali
Diapositiva subir natali
 

Mehr von ambitlick

DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...ambitlick
 
Low cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projectsLow cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projectsambitlick
 
Ambitlick ns2 2013
Ambitlick ns2 2013Ambitlick ns2 2013
Ambitlick ns2 2013ambitlick
 
Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013ambitlick
 
Handling selfishness in replica allocation
Handling selfishness in replica allocationHandling selfishness in replica allocation
Handling selfishness in replica allocationambitlick
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocolsambitlick
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsambitlick
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsambitlick
 
IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2  IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2 ambitlick
 
Adaptive weight factor estimation from user review 1
Adaptive weight factor estimation from user   review 1Adaptive weight factor estimation from user   review 1
Adaptive weight factor estimation from user review 1ambitlick
 
Integrated institutional portal
Integrated institutional portalIntegrated institutional portal
Integrated institutional portalambitlick
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocolsambitlick
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsambitlick
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsambitlick
 
Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”ambitlick
 
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...ambitlick
 
Energy efficient protocol for deterministic
Energy efficient protocol for deterministicEnergy efficient protocol for deterministic
Energy efficient protocol for deterministicambitlick
 
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...ambitlick
 

Mehr von ambitlick (20)

DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
 
Low cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projectsLow cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projects
 
Ambitlick ns2 2013
Ambitlick ns2 2013Ambitlick ns2 2013
Ambitlick ns2 2013
 
Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013
 
Handling selfishness in replica allocation
Handling selfishness in replica allocationHandling selfishness in replica allocation
Handling selfishness in replica allocation
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocols
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroups
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secrets
 
IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2  IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2
 
Adaptive weight factor estimation from user review 1
Adaptive weight factor estimation from user   review 1Adaptive weight factor estimation from user   review 1
Adaptive weight factor estimation from user review 1
 
Integrated institutional portal
Integrated institutional portalIntegrated institutional portal
Integrated institutional portal
 
Embassy
EmbassyEmbassy
Embassy
 
Crm
Crm Crm
Crm
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocols
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroups
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secrets
 
Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”
 
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
 
Energy efficient protocol for deterministic
Energy efficient protocol for deterministicEnergy efficient protocol for deterministic
Energy efficient protocol for deterministic
 
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
 

Kürzlich hochgeladen

Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Disha Kariya
 
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...Sapna Thakur
 
fourth grading exam for kindergarten in writing
fourth grading exam for kindergarten in writingfourth grading exam for kindergarten in writing
fourth grading exam for kindergarten in writingTeacherCyreneCayanan
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeThiyagu K
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3JemimahLaneBuaron
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfciinovamais
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpinRaunakKeshri1
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfagholdier
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxheathfieldcps1
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
Interactive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationInteractive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationnomboosow
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfAyushMahapatra5
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingTechSoup
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfJayanti Pande
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...christianmathematics
 
9548086042 for call girls in Indira Nagar with room service
9548086042  for call girls in Indira Nagar  with room service9548086042  for call girls in Indira Nagar  with room service
9548086042 for call girls in Indira Nagar with room servicediscovermytutordmt
 

Kürzlich hochgeladen (20)

Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..
 
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
BAG TECHNIQUE Bag technique-a tool making use of public health bag through wh...
 
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
 
fourth grading exam for kindergarten in writing
fourth grading exam for kindergarten in writingfourth grading exam for kindergarten in writing
fourth grading exam for kindergarten in writing
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and Mode
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpin
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
Interactive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationInteractive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communication
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdf
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdf
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
 
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptxINDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
 
9548086042 for call girls in Indira Nagar with room service
9548086042  for call girls in Indira Nagar  with room service9548086042  for call girls in Indira Nagar  with room service
9548086042 for call girls in Indira Nagar with room service
 

Information theoretic outlier detection for large-scale categorical data

  • 1. Ambit lick Solutions Mail Id: Ambitlick@gmail.com , Ambitlicksolutions@gmail.Com 4. Information-Theoretic Outlier Detection for Large-Scale Categorical Data Outlier detection can usually be considered as a pre-processing step for locating, in a data set, those objects that do not conform to well-defined notions of expected behavior. It is very important in data mining for discovering novel or rare events, anomalies, vicious actions, exceptional phenomena, etc. We are investigating outlier detection for categorical data sets. This problem is especially challenging because of the difficulty of defining a meaningful similarity measure for categorical data. In this paper, we propose a formal definition of outliers and an optimization model of outlier detection, via a new concept of holoentropy that takes both entropy and total correlation into consideration. Based on this model, we define a function for the outlier factor of an object which is solely determined by the object itself and can be updated efficiently. We propose two practical 1-parameter outlier detection methods, named ITB-SS and ITB-SP, which require no user-defined parameters for deciding whether an object is an outlier. Users need only provide the number of outliers they want to detect. Experimental results show that ITB-SS and ITB-SP are more effective and efficient than mainstream methods and can be used to deal with both large and high-dimensional data sets where existing algorithms fail.