Presentation about Jupyter at globo.com held at #JupyterCon 2017 in New York!. Authors: Diogo Munaro and Felipe Ferreira

1. Acceleratingdatadriven
cultureatthelargest
mediagroupofLatin
AmericawithJupyter

2. Diogo Munaro Felipe Ferreira

3. ABOUT GLOBO.COM

4. W E W E R E
BORN DIGITAL
TO TRANSFORM
BRAZIL’S INTERNET
by crafting digital products, services and operations for the
largest media group in Latin America

5. O U R M A I N O F F I C E I S I N
Rio de JaneiroOther offices: São Paulo and Porto Alegre

6. W E H A V E M O R E T H A N
87 MILLION USERS
C O M S C O R E M M X - M AY / 1 7

7. W E R E A C H
78%
OF BRAZIL’S INTERNET
C O M S C O R E M M X - M AY / 1 7

8. W E A R E
LEADERS IN
VIDEOS
SPORTS
NEWS
ENTERTAINMENT

9. 17 YEARSO F R E M A R K A B L E A C H I E V E M E N T S

10. breakingnews
localnews
contentverticals
tvshows
BRAZIL’S LARGEST
NEWS PORTAL

11. LARGER
THAN THE
RUNNER UP
2×
C O M S C O R E M M X - M AY / 1 7 & I N T E R N A L D ATA | G L O B O . C O M
1.000
54Mi UNIQUE
VISITORS
NEW ARTICLES
PER DAY
LAUNCHED LEADER
Designed and
developed
from scratch
G1 becomes
leader in Brazil
2006 2008 2017

12. 27 STATES
57 NEWSROOMS
5490 CITIES

13. THE LEADING
SPORTS PORTAL
IN BRAZIL
S I N C E 2 0 0 5
sportsnews
livecoverage
scouts
statistics

14. THE LEADING
SPORTS PORTAL
IN BRAZIL
S I N C E 2 0 0 5
sportsnews
livecoverage
scouts
statistics
61%
OF SPORTS
CATEGORY
C O M S C O R E M M X - M AY / 1 7

15. freemiummodel
hugeuserbase
highengagement
SINCE 2005
THE BIGGEST
FANTASY
GAME
IN BRAZIL
320.000SUBSCRIBERS
+11MiTEAMS
I N T E R N A L D ATA | G L O B O . C O M

16. ALL ABOUT
ENTERTAINMENT
tvshows,behindthescenes,
fashion,interiordesign,recipes,
music,BigBrotherBrasil…
S I N C E 2 0 1 4

17. B I G B R O T H E R B R A Z I L I S A N
INTERNET
PHENOMENON

18. B I G B R O T H E R B R A Z I L I S A N
INTERNET
PHENOMENON
NUMBER OF VOTES
IN A SINGLE POLL
149Mi PEAK VOTES
PER MINUTE
1Mi
I N T E R N A L D ATA | G L O B O . C O M

19. page visited
watching time video
sharing
comment
time reading news
scroll percentage
Horizon
Gateway
HORIZON - OUR EVENT TRACKER

20. O U R P R O D U C T A P P R O A C H I S
DATA
DRIVEN
DEVELOPMENTWe make our decisions based on
what really works for real people

21. HYPOTESIS EXPERIMENT ANALYSIS LEARN
Using Jupyter with our big data resources and looking for
KPIs, we can evaluate our design experience — like a
scientist
A SCIENTIFIC BIAS
FAST QUALITATIVE CYCLES

22. JUPYTER
is playing an
important role in
the data-driven
decisions
Jupyter notebook is the heart of our data
analysis process
In the jupyter notebooks are performed AB
test analysis, machine learning experiments,
peformance evaluation of the
recommendation systems and etc..

23. WHY DID WE
CHOOSE JUPYTER ?

24. • Scalability for Hundreds of
Data Scientists
BECAUSE OF GLOBO.COM NEEDS:
• Analysis of large, complex
data sets
• Works with R and Python
• Knowledge share and
reproducible studies
• Security: data access from
different sources

25. HOW TO MET THE
NEEDS?

26. SCALABILITY
FOR HUNDREDS
OF DATA
SCIENTISTS
+

27. LARGE
AMOUNT
OF DATA
+3billiondailyevents
+2millionconcurrentconnections
+50 million uniqueuserspermonth
+100thousandnewcontents permonth

28. WORKS
WITH R
AND PYTHON

29. - Python2.7
- Python3.6
- R
Different kernels

30. With this lib we accelerate data science
process on jupyter
WE CREATED OUR
INTERNAL UTILITY LIB
CALLED GLOBO DS

31. SPARK
MANAGEMENT
+ DATA SCIENCE
UTILITIES
G L O B O D S F U N C T I O N S

32. - Start(localorcluster)
- Stop
- Monitoring
TO SPARK
U T I L I T Y F U N C T I O N S

33. - Removeoutliers
- Plotgraphs
- Dateformat
- Devicetype
- Saveparquetand
csvinhdfs
TO DATA SCIENCE
PROCESS
U T I L I T Y F U N C T I O N S

34. …
SAFETY PACKAGE
INSTALL

35. KNOWLEDGE
SHARE AND
REPRODUCIBLE
STUDIES

36. KNOWLEDGE
SHARE AND
REPRODUCIBLE
STUDIES
…

37. SECURITY:
DATA ACCESS
FROM
DIFFERENT
SOURCES

38. WE HAVE SECURITY HOOKS
Load
Run Cell
Delete

39. ALL WITH OAUTH2
1. Access JupyterHub
2. Can Access ?
3. Authorized
User
Backstage
Accounts
4. Request User Data
5. User Data

40. JUPYTER ENVIRONMENT

41. Diogo Munaro Felipe Ferreira
www.linkedin.com/in/feliferr
felipe.felipeam@gmail.com
www.linkedin.com/in/dmvieira
diogo.mvieira@gmail.com