Presentation of the InVID tool for social media verification through contextual analysis, at the Media Informatics Lab meeting on detection and verification of socially shared videos.

1. Towards Automatic Detection of
Misinformation in Social Media
Symeon (Akis) Papadopoulos - @sympap
Information Technologies Institute (ITI) /
Centre for Research and Technology Hellas (CERTH)
Workshop on Tools for Video Discovery & Verification in Social Media
Dec 14, 2017 @ Thessaloniki, Greece

2. Real or fake?

3. Real or fake?

4. Real or fake?

5. Real or fake?

6. Types of misinformation

7. Towards computational verification

8. The Tweet Verification Assistant
A web-based service for marking an input
tweet as “real” or “fake”
2012
first ideas and experiments (SocialSensor)
2013-2016
main research, development and validation (REVEAL)
2016-now
incremental refinements and testing (InVID)

9. User interface
http://reveal-mklab.iti.gr/reveal/fake/

10. Credibility cues (aka features)

11. Building the classification model

12. Tweet Verification Corpus
• 53 events or hoaxes involving false and/or real
imagery and videos
• 257 cases of “fake” content, 261 of “real”
• 10,634 tweets sharing “fake” content, 7,223 tweets
sharing “real” content
• Examples events:
• Hurricane Sandy
• Boston Marathon bombing
• Sochi Olympics
• MA Flight 370
• Nepal Earthquake…
https://github.com/MKLab-ITI/image-verification-corpus

13. The “Verifying Multimedia Use” Task
•VMU: Organized in 2015 and 2016 as
part of the MediaEval benchmarking
initiative
•Goal: compare automated approaches
for fake tweet detection
•Outcomes: several methods from
different research groups across the
globe were tested and compared

14. Experimental validation
92.5% accuracy in identifying misleading posts
88-98% accuracy depending on language
(major languages tested: en, fr, es, nl)
New features, bagging and agreement-based
retraining led to significant improvements!
One of the top performing methods in the
VMU 2015 & 2016 tasks!

15. Context Analysis and
Aggregation
• Available at: http://caa.iti.gr
• YouTube, Facebook and Twitter videos
• metadata from APIs
• mentioned locations
• “verification”-related comments
• thumbnails for near-duplicate search
• weather at time and location of video
• video sharing on Twitter

16. Tip in comment led to debunking
A comment points to second 23 of the video
where suddenly the snake appears out of nowhere

17. # verification comments too high
1550 verification-related comments
out of 4219 total number of comments

18. Tweets sharing video are flagged
37 out of 43 tweets sharing
the video are classified as fake

19. Video verification experiments
• 117 fake videos and 110 real videos
• The dataset covers different types of manipulation:
• staged videos,
• videos misrepresenting the depicted event,
• videos of past events claimed to be captured now,
• digitally manipulated videos.
• A supervised learning approach using credibility
features extracted from video comments and video
metadata managed to achieve promising accuracy:
P=72%, R=86%, F=79%

20. Limitations
• Models are still based on aged training data
(could be affected by concept drift…)
• Results not always easy to justify or explain
to end users
• A well-informed adversary can easily fool the
model by emulating “credible-looking” posts
• Journalists are still expected to make the
final decision!

21. The future of misinformation

22. Acknowledgements
• Christina Boididou (Feature extraction, model
building, initial REST API development)
• Olga Papadopoulou (Validation, code refactoring,
REST API refinement and support)
• Lazaros Apostolidis (UI/UX)
• Markos Zampoglou (Evaluation)
• Yiannis Kompatsiaris (PI)

23. Thank you!
http://reveal-mklab.iti.gr/reveal/fake/
http://caa.iti.gr
Get in touch!
Akis Papadopoulos papadop@iti.gr / @sympap

24. References
• Boididou, C., Papadopoulos, S., Kompatsiaris, Y., Schifferes, S., & Newman, N. (2014,
April). Challenges of computational verification in social multimedia. In Proceedings
of the 23rd International Conference on World Wide Web (pp. 743-748). ACM
• Boididou, C., Middleton, S. E., Jin, Z., Papadopoulos, S., Dang-Nguyen, D. T., Boato, G.,
& Kompatsiaris, Y. (2017). Verifying information with multimedia content on twitter.
Multimedia Tools and Applications, 1-27
• Boididou, C., Papadopoulos, S., Apostolidis, L., & Kompatsiaris, Y. (2017, June).
Learning to Detect Misleading Content on Twitter. In Proceedings of the 2017 ACM
on International Conference on Multimedia Retrieval (pp. 278-286). ACM
• Castillo, C., Mendoza, M., & Poblete, B. (2011, March). Information credibility on
twitter. In Proceedings of the 20th international conference on World Wide Web (pp.
675-684). ACM
• Liu, M. Y., Breuel, T., & Kautz, J. (2017). Unsupervised Image-to-Image Translation
Networks. arXiv preprint arXiv:1703.00848
• Papadopoulou, O., Zampoglou, M., Papadopoulos, S., & Kompatsiaris, Y. (2017, June).
Web Video Verification using Contextual Cues. In Proceedings of the 2nd
International Workshop on Multimedia Forensics and Security (pp. 6-10). ACM
• Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation
using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593.