A summary of the workshop as presented at the 1st International Workshop on Benchmarking Linked Data (BLINK). (HOBBIT project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688227.)

1. Workshop Report
Benchmarking Linked Data
Axel-Cyrille Ngonga Ngomo
Irini Fundulaki
Anastasia Krithara
Horizon 2020
GA No 688227
01/12/2016–30/11/2018
ISWC 2016
Kobe, Japan
November 10, 2016
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 1 / 5

2. Core Questions
Core questions
1 Developers: How good is my tool?
2 Vendors: Who is my tool good for?
3 Users: Which tool(s) should I use for my
application?
Many questions
1 Tools
⇒ Bottlenecks, solved problems, distinctive
features
2 Benchmarks
⇒ Novel bechmarks, limitations, updates
3 Performance indicators
⇒ Limitations, correlations, innovations
4 Comparative studies
⇒ Across benchmarks, fields of application
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 2 / 5

3. Core Questions
Goal of BLINK
Provide a forum where the community can gather to address these challenges.
Core questions
1 Developers: How good is my tool?
2 Vendors: Who is my tool good for?
3 Users: Which tool(s) should I use for my
application?
Many questions
1 Tools
⇒ Bottlenecks, solved problems, distinctive
features
2 Benchmarks
⇒ Novel bechmarks, limitations, updates
3 Performance indicators
⇒ Limitations, correlations, innovations
4 Comparative studies
⇒ Across benchmarks, fields of application
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 2 / 5

4. Highlights
Insights
1 Scalable application stacks for Linked Data in demand (Keynote)
2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.)
3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL)
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5

5. Highlights
Insights
1 Scalable application stacks for Linked Data in demand (Keynote)
2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.)
3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL)
Positive
1 Emerging benchmarking platforms (GERBIL QA, HOBBIT)
2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL)
3 Uptake of methodology-based and data-driven benchmark design
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5

6. Highlights
Insights
1 Scalable application stacks for Linked Data in demand (Keynote)
2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.)
3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL)
Positive
1 Emerging benchmarking platforms (GERBIL QA, HOBBIT)
2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL)
3 Uptake of methodology-based and data-driven benchmark design
Todo
1 Need for benchmark repository (CKAN)
2 Community effort necessary for uptake of more benchmarks (HOBBIT)
3 Untackled real-world challenges (benchmarks coming up)
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5

7. That’s all Folks!
@hobbit_project
Axel Ngonga
AKSW Research Group
Augustusplatz 10, Room P905
04109 Leipzig, Germany
ngonga@informatik.uni-leipzig.de
@NgongaAxel
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 4 / 5

8. Acknowledgment
This work was supported by grants from the EU H2020 Framework Programme
provided for the project HOBBIT (GA no. 688227).
Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 5 / 5