Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

NLTK in 20 minutes

83.784 Aufrufe

Veröffentlicht am

A sprint thru Python's Natural Language ToolKit, presented at SFPython on 9/14/2011. Covers tokenization, part of speech tagging, chunking & NER, text classification, and training text classifiers with nltk-trainer.

Veröffentlicht in: Technologie, Bildung
  • Dating for everyone is here: ♥♥♥ http://bit.ly/2Y8gKsI ♥♥♥
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Dating direct: ❶❶❶ http://bit.ly/2Y8gKsI ❶❶❶
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/qURD } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/qURD } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/qURD } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/qURD } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/qURD } ......................................................................................................................... Download doc Ebook here { https://soo.gd/qURD } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://shorturl.at/krvUW } ......................................................................................................................... Download Full EPUB Ebook here { http://shorturl.at/krvUW } ......................................................................................................................... Download Full doc Ebook here { http://shorturl.at/krvUW } ......................................................................................................................... Download PDF EBOOK here { http://shorturl.at/krvUW } ......................................................................................................................... Download EPUB Ebook here { http://shorturl.at/krvUW } ......................................................................................................................... Download doc Ebook here { http://shorturl.at/krvUW } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

NLTK in 20 minutes

  1. 1. NLTK in 20 minutesA sprint thru Pythons Natural Language ToolKit
  2. 2. Jacob PerkinsCo-founder/CTO @ Weotta (were hiring :)"Python Text Processing with NLTK 2.0 Cookbook"NLTK ContributorBlog: http://streamhacker.comNLTK Demos & APIs: http://text-processing.com@japerk
  3. 3. Why Text Processing?sentiment analysisspam filteringplagariasm detection / document similaritydocument categorization / topic detectionphrase extraction, summarizationsmarter searchsimple keyword frequency analysis
  4. 4. Some NLTK Featuressentence & word tokenizationpart-of-speech taggingchunking & named entity recognitiontext classificationmany included corpora
  5. 5. Sentence Tokenization>>> from nltk.tokenize import sent_tokenize>>> sent_tokenize("Hello SF Python. This is NLTK.")[Hello SF Python., This is NLTK.]>>> sent_tokenize("Hello, Mr. Anderson. We missed you!")[Hello, Mr. Anderson., We missed you!]
  6. 6. Word Tokenization>>> from nltk.tokenize import word_tokenize>>> word_tokenize(This is NLTK.)[This, is, NLTK, .]
  7. 7. Whats a Word?>>> word_tokenize("Whats up?")[What, "s", up, ?]>>> from nltk.tokenize import wordpunct_tokenize>>> wordpunct_tokenize("Whats up?")[What, "", s, up, ?]Learn More: http://text-processing.com/demo/tokenize/
  8. 8. Part-of-Speech Tagging>>> words = word_tokenize("And now for somethingcompletely different")>>> from nltk.tag import pos_tag>>> pos_tag(words)[(And, CC), (now, RB), (for, IN),(something, NN), (completely, RB), (different,JJ)]Tags List: http://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html
  9. 9. Why Part-of-Speech Tag?word definition lookup (WordNet, WordNik)fine-grained text analyticspart-of-speech specific keyword analysischunking & named entity recognition (NER)
  10. 10. Chunking & NER>>> from nltk.chunk import ne_chunk>>> ne_chunk(pos_tag(word_tokenize(My name is JacobPerkins.)))Tree(S, [(My, PRP$), (name, NN), (is, VBZ),Tree(PERSON, [(Jacob, NNP), (Perkins, NNP)]),(., .)])
  11. 11. NER not perfect>>> ne_chunk(pos_tag(word_tokenize(San Francisco isfoggy.)))Tree(S, [Tree(GPE, [(San, NNP)]), Tree(PERSON,[(Francisco, NNP)]), (is, VBZ), (foggy, NN),(., .)])
  12. 12. Text Classificationdef bag_of_words(words): return dict([(word, True) for word in words])>>> feats = bag_of_words(word_tokenize("great movie"))>>> import nltk.data>>> classifier = nltk.data.load(classifiers/movie_reviews_NaiveBayes.pickle)>>> classifier.classify(feats)pos
  13. 13. Classification Algos in NLTK Naive Bayes Maximum Entropy / Logistic Regression Decision Tree SVM (coming soon)
  14. 14. NLTK-Trainerhttps://github.com/japerk/nltk-trainercommand line scriptstrain custom modelsanalyze corporaanalyze models against corpora
  15. 15. Train a Sentiment Classifier$ ./train_classifier.py movie_reviews --instances parasloading movie_reviews2 labels: [neg, pos]2000 training feats, 2000 testing featstraining NaiveBayes classifieraccuracy: 0.967000neg precision: 1.000000neg recall: 0.934000neg f-measure: 0.965874pos precision: 0.938086pos recall: 1.000000pos f-measure: 0.968054dumping NaiveBayesClassifier to ~/nltk_data/classifiers/movie_reviews_NaiveBayes.pickle
  16. 16. Notable Included Corporamovie_reviews: pos & neg categorized IMDb reviewstreebank: tagged and parsed WSJ texttreebank_chunk: tagged and chunked WSJ textbrown: tagged & categorized english text60 other corpora in many languages
  17. 17. Other NLTK FeaturesclusteringmetricsparsingstemmingWordNet... and a lot more
  18. 18. Other Python NLP Librariespattern: http://www.clips.ua.ac.be/pages/patternscikits.learn: http://scikit-learn.sourceforge.net/stable/fuzzywuzzy: https://github.com/seatgeek/fuzzywuzzy
  19. 19. Learn Morehttp://www.nltk.org/http://streamhacker.comhttp://text-processing.comnltk-users mailing list
  20. 20. NLTK Tutorial @ PyConWhat would you want to learn in 3 hours?What kinds of NLP problems do you face at work?What do you want to do with text?

×