Big data are precious, but only if we manage to extract information from them. To do this we need to write algorithms that analyse them. When dealing with big data we have to face a lot of challenges. Most of them are technological (storage, computational power), but there is also the aspect of how to efficiently write an analysis algorithm, taking into account all the technological limitations and also the business priorities. The best algorithm on paper often is not the most efficient “on code”. Taking inspiration from my personal journey from theoretical physics to data science, I will explain what it means to approach a problem theoretically and why big data are so tricky to approach also at the theoretical level.

1. When a perfect
algorithm meets real
data
the challenge of getting
insight from data
Alessandra Cagnazzo – Data Scientist
Big Data Oslo
25 June 2019

2. Data

3. Image
Recognition

4. Photos by CERN

6. Use the superpower
Of
Machine Learning
• It learns formulas partly by itself
• Give insight, indications
• One still has to engineer structures
and formulas but is a less
constrained way.

7. Threats are behind
the corner

8. Careful assuming the generality of a machine learning algorit
Deep is not always efficient.
Sometimes it is better to go shallow, at the cost of losing s
details.

9. 1 + 10 + 20 + 3 ≈ 5.83
1 + 10 + 20 + 3 ≈ 8.11
Parallelising, batching, and simplifying
ⅇlog
𝑎
2 𝑒log 2 +
𝑒 𝑎 log
𝑎
𝑏
ⅇlog
𝑎
2
+ 2
𝑎 𝑏
𝑐
+
𝑎 − 𝑒log 𝑎+log 𝑏
𝑐
−1
= 𝑎 + 𝑏 + 𝑐