Nikolay Novozhilov of Wego.com introduces BigQuery, Google's service for interactive analysis of massive datasets, as Wego's main Big Data solution. Wego was founded in 2005 in Singapore as Asia Pacific and the Middle East's leading flight/hotel metasearch engine. BigQuery allows querying of billions of rows in seconds, uses a SQL-style query syntax, and pays only for resources used. Novozhilov discusses replacing MySQL with BigQuery and addresses concerns about data security, pricing, and open-source alternatives.

1. Nikolay Novozhilov
Wego.com
Using BigQuery as a main Big
Data solution

2. About Wego
Wego.com is Asia Pacific and the Middle East’s
leading flight/hotel metasearch engine used by
millions of travelers.
Wego was founded in 2005 in Singapore

3. Introducing BigQuery
Service for interactive analysis of massive datasets
(TBs)
Query billions of rows: seconds to write, seconds to
return
Uses a SQL-style query syntax
It's a service, accessed by a RESTful API
Pay only for what you use
Based on internal Google tool - Dremel
Column oriented, append only…

4. Data architecture in Wego
...

5. Why did we do it?
MySQL
“Zoo”
BigQuery

6. Why Hadoop is more popular?

7. My collection of concerns
Your data goes to cloud
Not open-source, Google can stop the service
“Strange” pricing model
Hadoop is trending, has bigger community
Append only database
???

8. Costs: storage + cost per query
Same fallacy again:
 “I want to launch a mom@pop – let’s buy a
building”
 “I want to build a site – let’s by servers”
 “I want big data – let’s build a data-warehouse”
Usual concerns:
 No realistic estimate upfront
 “Fear of running a query”

9. StackOverflow support
53
minutes!

10. Append only…
Slowly changing dimensions:
 daily re-load from MySQL
 daily upload from MySQL, keeping history
Absolutely necessary updates:
 do you really need it?
 BigQuery allows to save query to initial table:
Your
table Query

11. Actually useful - “Discovery mode”

12. Actually useful
Huge joins
REGEXT_MATCH(), …
Rich SQL - window functions
Nested data

13. My answer

14. What is Big Data revolution?
There is no difference
between big data and small
data anymore

15. Contacts
Blog: www.novozhilov.co
Email: nik@wego.com

16. “Yes, Sir, I tired to build an ROI case for
our BI project - but I couldn’t access any
reliable data!”
TimoElliott.com