Avancerad dataanalys och ”big data” har under de senaste åren klättrat på trendlistorna och är nu ett av de mest prioriterade områdena i utvecklingen av nya tjänster och produkter för ledarföretag i det digitala landskapet.
Informationen som byggs upp i systemen när kundmötena digitaliseras har visat sig vara guld värt. Här finns allt vi behöver veta för att göra våra affärer mer effektiva.
Sedan sommaren 2013 har Connecta tillsammans med Google ett etablerat samarbete för att hjälpa våra kunder med övergången till moln-tjänster för bland annat avancerad dataanalys. För att göra oss själva redo att hjälpa våra kunder har vi under ett antal år utvecklat såväl kunskaper som skaffat oss erfarenheter kring Googles olika moln-produkter, som exempelvis ”Big Query”.
Big Query är ett molnbaserat analysverktyg och en del av Google Cloud Platform. Big Query gör det möjligt att ställa snabba frågor mot enorma dataset på bara någon sekund. Big Query och Google Cloud Platform erbjuder färdiga lösningar för att sätta upp och underhålla en infrastruktur som med enkla medel gör allt detta möjligt.
På Connecta Digital Consultings tredje event för våren introducerade vi våra kunder och partners i koncepten dataanalys och Big Query.
Under eventet berördes följande punkter:
- Big Data och Business Intelligence (BI)
- “The Google Big Data tools” – framgångsfaktorer och hur man kommer igång
- Google Cloud Platform och hur man genomför en framgångsrik molnsatsning
Vi presenterade case och berättade om viktiga lärdomar vi dragit i samarbetet med Google och våra kunder.
29. For the past 15 years, Google
has been building out the world’s
fastest, most powerful, highest
quality cloud infrastructure on
the planet.
Images by Connie Zhou
30. Google has been running some of
the world’s largest distributed
systems with unique and stringent
requirements.
Images by Connie Zhou
35. May 2013
Google Compute Engine
(Preview)
PHP for App Engine
(Preview)
Big JOIN in BigQuery
The Last Year in the Cloud Platform
November 2013
Cloud Endpoints GA
Dedicated Memcache GA
August 2013
Layer 3 Load
Balancing
Encryption at
Rest for Cloud
Storage
December 2013
Compute Engine GA
Live Migration
Persistent Disks
July 2013
Dedicated
Memcache
Offline Disk
Import
February 2014
HIPAA Support
Cloud SQL GA
38. We can do better
Lower and simplify pricing
Make developers more productive
39. Prices are falling
• Public cloud prices
have dropped 6-8%
annually
Source: Google Internal Data
20142006
Public Cloud Prices
40. But prices are not falling fast enough
• Hardware costs have
dropped 20-30%
annually
Hardware Cost
Public Cloud Prices• Public cloud prices
have dropped 6-8%
annually
Source: Google Internal Data
20142006
41. Pricing Updates (Effective April 1st, 2014)
35% price drop on Compute Engine, across all sizes,
regions, and classes
37% price drop on App Engine frontend instance hours, 33%
on Datastore writes and 50% on Dedicated Memcache
68% price drop on Cloud Storage
On Demand pricing reduced by 85% - $5/TB
42. You should get the best price with...
No Upfront Payments
No Lock-in
No Complexity
43. 100%0% 20% 40% 60% 80%
Sustained Use
Previous
On Demand
New
On Demand
$0.11
$0.10
$0.09
$0.08
$0.07
$0.06
$0.05
$0.04
$0.03
Sustained-use discountsNetPricePerHour
45. • Managed VMs
• The Flexibility of Compute Engine
• The productivity of App Engine
• Provides best of both worlds
• IaaS + PaaS
Flexibility Managementand
Managed VMs
46. Developer Productivity
• Use the tools you know and love
• Fast, reliable deployments
• Isolate and fix issues in production
with Continuous Integration
Developer Productivity
Time to
Market
and
Robust
Design
47. 1000X BigQuery Streaming
• Near real-time analysis
• High fidelity, low latency
• Focus on results, not sharding
and transforming
$0.01 per 100,000 rows Real time availability of data100,000 rows per second
48. • Deployment Manager
• Replica Pools
• Cloud DNS
• Windows Server, SuSE, RHEL support
and so much more...
49. Agenda 25th, 2014
Google Cloud Platform Introduction, Gaining Momentum
Big Data on Google Cloud Platform
Discussion
2
3
1
52. • Applications at the heart
of business interactions
• Devices and sensors
• Lower cost of storage &
ingestion
• New programming
models
• New scale and
capabilities for SQL
• Easily available software
(Open Source)
• Easy on-ramp, cost
effective experimentation
• Unlimited scale, low TCO
• Combine Open Source
software and platform
services
Ability to process Cloud consumption modelData availability
Key drivers in the growth of Big Data
53. Google Cloud Storage
Mix and match storage and computation from OSS and Google Cloud Platform
BigQuery and Datastore Connectors
BigQueryDatastore
Hadoop
BigQuery
Connector
Datastore
Connector
Cloud
Storage
Connector
HBase HivePig
Hadoop Applications
Hadoop, Pig, HBase, and Hive are trademarks of the Apache Software Foundation.
56. Ease of use
• Simplified infrastructure for realtime use cases
• Stream events row-by-row via simple API
Use cases
• Server Logs, Mobile apps, Gaming, In-App real time
analytics
BigQuery Streaming
Low cost: $0.01 per 100,000 rows Real time availability of data100,000 rows per second
Customer example:
57. Google Analytics + BigQuery
Google Analytics Premium Platform Google BigQueryData Pipeline
Native Data Pipeline to Load Data into BigQuery Project
59. BigQuery in Action
" The interactive performance of Google BigQuery,
combined with Tableau’s intuitive visualization tools,
enabled our analysts to interactively explore huge
quantities of data – hundreds of millions of rows – with
incredible efficiency. Previously, analyses would
require hours or days to complete, if they would even
complete at all. With Google BigQuery it takes
minutes, if that, to process. This time-to-insight was
previously impossible"
– Giovanni DeMeo
Vice President
Global Marketing and Analytics
60. " The simulation cluster ran for nearly two months as
part of the ATLAS distributed compute grid, logging
over 5 million core-hours, completing 458,000
computationally intensive jobs and processing about
214 million events. The cluster achieved sustained
peak throughput of 15,000 jobs per day. “We had a
great experience with Google Compute Engine … and
think that it is modern cloud infrastructure that can
serve as a stable, high performance platform for
scientific computing”.
– Dr. Panitkin
CERN Atlas Project
CERN Atlas Compute Grid Extended on GCE
61. • 1.5TB in 60 seconds
• 8,412 cores
• Google Compute Engine
MapR Breaks Minute Record Sort
66. “[Google's] ability to build, organize, and operate a
huge network of servers and fiber-optic cables
with an efficiency and speed that rocks physics on
its heels.
This is what makes Google Google: its physical
network, its thousands of fiber miles, and those
many thousands of servers that, in aggregate, add
up to the mother of all clouds.”
- Wired
Images by Connie Zhou