Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Entity Extraction from Natural Language Text using Apache NiFi and Idyl E3

896 Aufrufe

Veröffentlicht am

This presentation describes a dataflow pipeline created in Apache NiFi that uses Idyl E3 Entity Extraction Engine to extract entities from natural language text.

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Entity Extraction from Natural Language Text using Apache NiFi and Idyl E3

  1. 1. Entity Extraction from Natural Language Text in a Data Flow Pipeline Copyright 2017 Mountain Fog, Inc. All Rights Reserved.
  2. 2. Tools in Use ● Apache NiFi ● Facilitates data flow between disparate sources and services. ● https://nifi.apache.org/ ● Idyl E3 Entity Extraction Engine ● Extracts entities from natural language text via user- generated entity models through a REST API. ● http://www.mtnfog.com/
  3. 3. Goals ● Ingest text files from the file system. ● Extract entities from the text. ● Store entities in a MongoDB database. Text Entities Apache NiFi Dataflow S3
  4. 4. Idyl E3 Entity Extraction Engine Launch via the AWS Marketplace. Comes with an entity model for English-language person entities. REST API for entity extraction.
  5. 5. NiFi Processors A processor executes the dataflow work “of data routing, transformation, or mediation between systems.” [1] We are using: GetFile Processor IdylE3 Processor PutMongoDB Processor [1] https://nifi.apache.org/docs/nifi-docs/html/overview.html
  6. 6. Ingest Text Files Processor’s properties set to read all files from /ingest.
  7. 7. Send Text to Idyl E3 The Idyl E3 endpoint is set in the processor’s properties.
  8. 8. Store Entities in MongoDB The MongoDB URI set in the processor’s properties.
  9. 9. The NiFi Flow
  10. 10. The Result Start the NiFi flow. Files are removed from the ingest directory. Entities appear in the MongoDB collection. Take a well-deserved break.
  11. 11. Going Further Scale Idyl E3 behind a load balancer. Query entities via the Entity Query Language (EQL) processor. Extract other entity types through custom entity models.
  12. 12. support@mtnfog.com www.mtnfog.com

×