SlideShare a Scribd company logo
1 of 1
Download to read offline
Automating CIRI Ratings of
Human Rights Reports Using GATE
Joshua Joiner and Karthikeyan Umapathy
School of Computing,
University of North Florida,
Jacksonville, FL USA 32224
R E S E A R C H C O N T E X T
This project involves parsing human rights reports
produced by the U.S Government and rating the human
practices for various countries. The U.S Human Rights
Reports are annual reports that cover internationally
recognized human rights practices in regards to individual,
civil, political, and worker rights.
T E X T M I N I N G T O O L
GATE is an open source text mining platform used for
developing custom text processing solutions.
G E N E R A T I N G C I R I R A T I N G U S I N G G A T E
C O N C L U S I O N S
In conclusion, I believe the automated process will not
provide a high accuracy when comparing to the CIRI
dataset because the dataset was compiled by humans. I do,
however, believe that processes involved in creating the
automated process can create a more objective standard
when analyzing country report text and producing ratings
for the human practices. There also needs to be more
patterns implements within the automated process to more
accurately match with the qualitative text from the
Women’s Rights and Independent Judiciary sections.
CIRI rating:
Text Mining of Human Rights Reports
Project Objective:
CIRI Sample Dataset
U.S. Department of State
CIRI coders rely on a manual process of reading through
the Human Rights Reports and then applying ratings to
each human rights practice for each country.
• The objective of this project is to automate the process
of scouring the human rights country reports.
CIRI (Cingranelli-Richards) Human Rights Data Project
rates the human rights practices of the U.S. Human Rights
country reports. Students, scholars, policymakers, and
analysts use the CIRI ratings for practical and research
purposes.
CIRI Rating of Human Rights Reports
Standard ANNIE process flow:
C I R I R A T I N G S C O M P A R I S O N
Rating Produced by Automation
GATE Architecture Overview:
E V A L U A T I O N P L A N C O N T R I B U T I O N S
• CIRI Coding Annotation Processing Resource
• Custom JAPE patterns for keywords and
phrases.
• Custom annotations for entity extraction.
• Custom implementation of sentiment
analysis.
• Ontology Storage
• CIRI Dataset Source Ratings.
• Automatically generated CIRI Ratings.
• For the Occurrence section which includes KILL,
DISAP, POLPRIS, and TORT the accuracy of the
rating is 60%.
• Women’s Right overall averaged 45% accuracy and
Independent Judiciary averaged 70%.

More Related Content

More from Karthikeyan Umapathy

Systematic Literature Review and Research Model to Examine Data Analytics Ado...
Systematic Literature Review and Research Model to Examine Data Analytics Ado...Systematic Literature Review and Research Model to Examine Data Analytics Ado...
Systematic Literature Review and Research Model to Examine Data Analytics Ado...Karthikeyan Umapathy
 
Finding Insights in Florida Voter Participation
Finding Insights in Florida Voter ParticipationFinding Insights in Florida Voter Participation
Finding Insights in Florida Voter ParticipationKarthikeyan Umapathy
 
A Systematic Review of Affordable Homeownership using Data Science Methods
A Systematic Review of Affordable Homeownership using Data Science MethodsA Systematic Review of Affordable Homeownership using Data Science Methods
A Systematic Review of Affordable Homeownership using Data Science MethodsKarthikeyan Umapathy
 
Identifying Communities with Opportunities for Positive Youth Development
Identifying Communities with Opportunities for Positive Youth DevelopmentIdentifying Communities with Opportunities for Positive Youth Development
Identifying Communities with Opportunities for Positive Youth DevelopmentKarthikeyan Umapathy
 
2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal Slides
2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal  Slides2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal  Slides
2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal SlidesKarthikeyan Umapathy
 
Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...
Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...
Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...Karthikeyan Umapathy
 
Profiling Florida Voter Participation
Profiling Florida Voter ParticipationProfiling Florida Voter Participation
Profiling Florida Voter ParticipationKarthikeyan Umapathy
 
2021 Florida Data Science for Social Good Big Reveal
2021 Florida Data Science for Social Good Big Reveal2021 Florida Data Science for Social Good Big Reveal
2021 Florida Data Science for Social Good Big RevealKarthikeyan Umapathy
 
2020 Florida Data Science for Social Good Big Reveal
2020 Florida Data Science for Social Good Big Reveal2020 Florida Data Science for Social Good Big Reveal
2020 Florida Data Science for Social Good Big RevealKarthikeyan Umapathy
 
Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...
Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...
Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...Karthikeyan Umapathy
 
Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...
Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...
Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...Karthikeyan Umapathy
 
Collaborative Community Engagement: Bringing Data Science to Societal Challen...
Collaborative Community Engagement: Bringing Data Science to Societal Challen...Collaborative Community Engagement: Bringing Data Science to Societal Challen...
Collaborative Community Engagement: Bringing Data Science to Societal Challen...Karthikeyan Umapathy
 
2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal
2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal
2019 Florida Data Science for Social Good (FL-DSSG) Big RevealKarthikeyan Umapathy
 
2018 Academy Health Annual Research Meeting Poster
2018 Academy Health Annual Research Meeting Poster2018 Academy Health Annual Research Meeting Poster
2018 Academy Health Annual Research Meeting PosterKarthikeyan Umapathy
 
2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation
2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation
2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal PresentationKarthikeyan Umapathy
 
Security and User Experience: A Holistic Model for CAPTCHA Usability Issues
Security and User Experience: A Holistic Model for CAPTCHA Usability IssuesSecurity and User Experience: A Holistic Model for CAPTCHA Usability Issues
Security and User Experience: A Holistic Model for CAPTCHA Usability IssuesKarthikeyan Umapathy
 
2017 Florida Data Science for Social Good Big Reveal
2017 Florida Data Science for Social Good Big Reveal2017 Florida Data Science for Social Good Big Reveal
2017 Florida Data Science for Social Good Big RevealKarthikeyan Umapathy
 
A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...
A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...
A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...Karthikeyan Umapathy
 
UNF Computing Senior Capstone Project
UNF Computing Senior Capstone ProjectUNF Computing Senior Capstone Project
UNF Computing Senior Capstone ProjectKarthikeyan Umapathy
 

More from Karthikeyan Umapathy (19)

Systematic Literature Review and Research Model to Examine Data Analytics Ado...
Systematic Literature Review and Research Model to Examine Data Analytics Ado...Systematic Literature Review and Research Model to Examine Data Analytics Ado...
Systematic Literature Review and Research Model to Examine Data Analytics Ado...
 
Finding Insights in Florida Voter Participation
Finding Insights in Florida Voter ParticipationFinding Insights in Florida Voter Participation
Finding Insights in Florida Voter Participation
 
A Systematic Review of Affordable Homeownership using Data Science Methods
A Systematic Review of Affordable Homeownership using Data Science MethodsA Systematic Review of Affordable Homeownership using Data Science Methods
A Systematic Review of Affordable Homeownership using Data Science Methods
 
Identifying Communities with Opportunities for Positive Youth Development
Identifying Communities with Opportunities for Positive Youth DevelopmentIdentifying Communities with Opportunities for Positive Youth Development
Identifying Communities with Opportunities for Positive Youth Development
 
2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal Slides
2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal  Slides2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal  Slides
2022 Florida Data Science for Social Good (FL-DSSG) Big Reveal Slides
 
Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...
Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...
Longitudinal Study on the Generational Impacts of Habitat for Humanity: A Res...
 
Profiling Florida Voter Participation
Profiling Florida Voter ParticipationProfiling Florida Voter Participation
Profiling Florida Voter Participation
 
2021 Florida Data Science for Social Good Big Reveal
2021 Florida Data Science for Social Good Big Reveal2021 Florida Data Science for Social Good Big Reveal
2021 Florida Data Science for Social Good Big Reveal
 
2020 Florida Data Science for Social Good Big Reveal
2020 Florida Data Science for Social Good Big Reveal2020 Florida Data Science for Social Good Big Reveal
2020 Florida Data Science for Social Good Big Reveal
 
Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...
Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...
Dashboard for Extracting Regional Insights and Ranking Food Deserts in Northe...
 
Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...
Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...
Developing a GIS Dashboard Tool to Inform Non-Profit Hospitals of Community H...
 
Collaborative Community Engagement: Bringing Data Science to Societal Challen...
Collaborative Community Engagement: Bringing Data Science to Societal Challen...Collaborative Community Engagement: Bringing Data Science to Societal Challen...
Collaborative Community Engagement: Bringing Data Science to Societal Challen...
 
2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal
2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal
2019 Florida Data Science for Social Good (FL-DSSG) Big Reveal
 
2018 Academy Health Annual Research Meeting Poster
2018 Academy Health Annual Research Meeting Poster2018 Academy Health Annual Research Meeting Poster
2018 Academy Health Annual Research Meeting Poster
 
2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation
2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation
2018 Florida Data Science for Social Good (FL-DSSG) Big Reveal Presentation
 
Security and User Experience: A Holistic Model for CAPTCHA Usability Issues
Security and User Experience: A Holistic Model for CAPTCHA Usability IssuesSecurity and User Experience: A Holistic Model for CAPTCHA Usability Issues
Security and User Experience: A Holistic Model for CAPTCHA Usability Issues
 
2017 Florida Data Science for Social Good Big Reveal
2017 Florida Data Science for Social Good Big Reveal2017 Florida Data Science for Social Good Big Reveal
2017 Florida Data Science for Social Good Big Reveal
 
A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...
A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...
A Research Plan to Study Impact of a Collaborative Web Search Tool on Novice'...
 
UNF Computing Senior Capstone Project
UNF Computing Senior Capstone ProjectUNF Computing Senior Capstone Project
UNF Computing Senior Capstone Project
 

Recently uploaded

Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
Rock Songs common codes and conventions.pptx
Rock Songs common codes and conventions.pptxRock Songs common codes and conventions.pptx
Rock Songs common codes and conventions.pptxFinatron037
 
CCS336-Cloud-Services-Management-Lecture-Notes-1.pptx
CCS336-Cloud-Services-Management-Lecture-Notes-1.pptxCCS336-Cloud-Services-Management-Lecture-Notes-1.pptx
CCS336-Cloud-Services-Management-Lecture-Notes-1.pptxdhiyaneswaranv1
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
Optimal Decision Making - Cost Reduction in Logistics
Optimal Decision Making - Cost Reduction in LogisticsOptimal Decision Making - Cost Reduction in Logistics
Optimal Decision Making - Cost Reduction in LogisticsThinkInnovation
 

Recently uploaded (16)

Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
Rock Songs common codes and conventions.pptx
Rock Songs common codes and conventions.pptxRock Songs common codes and conventions.pptx
Rock Songs common codes and conventions.pptx
 
CCS336-Cloud-Services-Management-Lecture-Notes-1.pptx
CCS336-Cloud-Services-Management-Lecture-Notes-1.pptxCCS336-Cloud-Services-Management-Lecture-Notes-1.pptx
CCS336-Cloud-Services-Management-Lecture-Notes-1.pptx
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
Optimal Decision Making - Cost Reduction in Logistics
Optimal Decision Making - Cost Reduction in LogisticsOptimal Decision Making - Cost Reduction in Logistics
Optimal Decision Making - Cost Reduction in Logistics
 

Automating CIRI Ratings of Human Rights Reports Using GATE

  • 1. Automating CIRI Ratings of Human Rights Reports Using GATE Joshua Joiner and Karthikeyan Umapathy School of Computing, University of North Florida, Jacksonville, FL USA 32224 R E S E A R C H C O N T E X T This project involves parsing human rights reports produced by the U.S Government and rating the human practices for various countries. The U.S Human Rights Reports are annual reports that cover internationally recognized human rights practices in regards to individual, civil, political, and worker rights. T E X T M I N I N G T O O L GATE is an open source text mining platform used for developing custom text processing solutions. G E N E R A T I N G C I R I R A T I N G U S I N G G A T E C O N C L U S I O N S In conclusion, I believe the automated process will not provide a high accuracy when comparing to the CIRI dataset because the dataset was compiled by humans. I do, however, believe that processes involved in creating the automated process can create a more objective standard when analyzing country report text and producing ratings for the human practices. There also needs to be more patterns implements within the automated process to more accurately match with the qualitative text from the Women’s Rights and Independent Judiciary sections. CIRI rating: Text Mining of Human Rights Reports Project Objective: CIRI Sample Dataset U.S. Department of State CIRI coders rely on a manual process of reading through the Human Rights Reports and then applying ratings to each human rights practice for each country. • The objective of this project is to automate the process of scouring the human rights country reports. CIRI (Cingranelli-Richards) Human Rights Data Project rates the human rights practices of the U.S. Human Rights country reports. Students, scholars, policymakers, and analysts use the CIRI ratings for practical and research purposes. CIRI Rating of Human Rights Reports Standard ANNIE process flow: C I R I R A T I N G S C O M P A R I S O N Rating Produced by Automation GATE Architecture Overview: E V A L U A T I O N P L A N C O N T R I B U T I O N S • CIRI Coding Annotation Processing Resource • Custom JAPE patterns for keywords and phrases. • Custom annotations for entity extraction. • Custom implementation of sentiment analysis. • Ontology Storage • CIRI Dataset Source Ratings. • Automatically generated CIRI Ratings. • For the Occurrence section which includes KILL, DISAP, POLPRIS, and TORT the accuracy of the rating is 60%. • Women’s Right overall averaged 45% accuracy and Independent Judiciary averaged 70%.