Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Machine Translation
Benchmark
by Intento
July 2017
• New MT providers: Global Tone Communications
(best for English-to-Chinese so far)
• Improved MT quality: Google, Microso...
Overview
• General Purpose Machine Translation Services with Public APIs
• IBM Watson Language Translator
• SDL Cloud Mach...
Performance vs. Price
Dataset WMT2016-17
Domain: News
Pairs: ru-en, en-ru, de-en, en-de, cs-
en, en-cs, tr-en, en-tr, fi-en...
Language coverage
Unique
language
pairs -
supported
exclusively by
one provider
1
100
10000
Google Yandex Microsoft Baidu ...
Language groups by
web popularity*:
P1 - ≥ 2.0% websites
P2 - 0.5%-2% websites
P3 - 0.1-0.3% websites
P4 - <0.1% websites
...
by service provider
Google Cloud
Translation API
Microsoft
Translator Text API
Yandex
Translate API
Systran
REST Translati...
Based on 100
1-segment
requests for
en-de pair
Service latency
Latency,seconds
0,1
1
10
100
Google Microsoft PROMT Yandex ...
API & Documentation changes in July 2017:
Change frequency
SDL MT 27
IBM 14
Google 13
Baidu 3
July 2017© Intento, Inc.
Discover the best service providers for
your AI task
Evaluate performance on your own data
at a fraction of the potential ...
Intento
https://inten.to
Konstantin Savenkov
CEO Intento, Inc.

<ks@inten.to>
Nächste SlideShare
Wird geladen in …5
×

Intento Machine Translation Benchmark, July 2017

8.220 Aufrufe

Veröffentlicht am

Evaluation of nine major Machine Translation providers: performance, quality, language coverage, latency, API update frequency.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

Intento Machine Translation Benchmark, July 2017

  1. 1. Machine Translation Benchmark by Intento July 2017
  2. 2. • New MT providers: Global Tone Communications (best for English-to-Chinese so far) • Improved MT quality: Google, Microsoft, Baidu • Language coverage: no changes • Added latency analytics (slide 8) • Added API changes analytics (slide 9) July 2017© Intento, Inc. Changes since May 2017
  3. 3. Overview • General Purpose Machine Translation Services with Public APIs • IBM Watson Language Translator • SDL Cloud Machine Translation • Google Cloud Translation API • Yandex Translate API • Baidu Translate API • Microsoft Translator Text API • Systran REST Translation API • PROMT Cloud API • GTCom YeeCloud MT • Benchmark Dimensions • Performance vs. Price • Language Coverage • API Latency • API Change frequency May 2017© Intento, Inc.
  4. 4. Performance vs. Price Dataset WMT2016-17 Domain: News Pairs: ru-en, en-ru, de-en, en-de, cs- en, en-cs, tr-en, en-tr, fi-en, en-fi, ro-en, en-ro, en-zh, zh-en Performance Normalised hLEPOR score aggregated across all language pairs in the dataset Affordability = 1/price Using public volume-based pricing tiers Legend • performance range • price range • average position © Intento, Inc. July 2017
  5. 5. Language coverage Unique language pairs - supported exclusively by one provider 1 100 10000 Google Yandex Microsoft Baidu Systran SDL MT PROMT IBM Watson GTCom 22 4 1 154 906 1 246 3 094 3 20 84 104109 756 3 306 8 556 10 506 Total Unique Supported language pairs © Intento, Inc. July 2017
  6. 6. Language groups by web popularity*: P1 - ≥ 2.0% websites P2 - 0.5%-2% websites P3 - 0.1-0.3% websites P4 - <0.1% websites 100% 94% 64% 88% 30% P1 P2 P3 P4 P1 P2 P3 P4 59% 100% 100% 94% 64% 100% 94% 94% 63% 63% 59% Supported language pairs by popularity * https://w3techs.com/technologies/overview/content_language/all© Intento, Inc. Language coverage 44% of possible language pairs July 2017
  7. 7. by service provider Google Cloud Translation API Microsoft Translator Text API Yandex Translate API Systran REST Translation API SDL Cloud Machine Translation PROMT Cloud API IBM Watson Language Translation July 2017© Intento, Inc. Language coverage Baidu Translate API
  8. 8. Based on 100 1-segment requests for en-de pair Service latency Latency,seconds 0,1 1 10 100 Google Microsoft PROMT Yandex SDL MT GTCom IBM Baidu Systran 75% 95% 99.9% July 2017© Intento, Inc.
  9. 9. API & Documentation changes in July 2017: Change frequency SDL MT 27 IBM 14 Google 13 Baidu 3 July 2017© Intento, Inc.
  10. 10. Discover the best service providers for your AI task Evaluate performance on your own data at a fraction of the potential cost saving Access any provider with no effort using to our Single API Intento Service Platform July 2017© Intento, Inc.
  11. 11. Intento https://inten.to Konstantin Savenkov CEO Intento, Inc. <ks@inten.to>

×