2. Why am I here?
To share with you our 1st year journey & lesson learned.
Zillow <-> Splunk Profile
First year customer of Splunk
Aggressively learning and building Splunk for use by all product and
operations teams
My role
Seth Thomas (@theTommyRock)
Sr. Technical Program Manager, Ops
Splunk / Monitoring Advocate and Splunk Champion
3. Key stats
Zillow Traffic & Usage:
Nearly 77 million unique users visited Zillow, Inc.’s mobile applications and
websites in March 2014.
173 homes are viewed each second.
13 million rental shoppers come to Zillow rental sites each month.
Consumers submitted 21 million loan requests in ZMM over the past 12
months.
Zillow has data on more than 110 million U.S. homes, with Zestimates and
Rent Zestimates on approximately 100 million U.S. homes.
April 2014
4. First year with Splunk
Proof of concept: 1x end of life server hosting all Splunk
roles (indexer, search head, license server) for 100Gb term
license
Initial production deployment: Mixture of 4x physical
hosts (indexing) and 4x virtual machines
Indexed data per day: 100GB -> 350GB (and growing)
5. Speed
Why do my visualizations take so long!?
Perception
1) Timeliness of events
2) Time to display complete visualization
Stats: ~7,500,000,000 events
Time Events Second
s
1 hr 14,993,252 32.764
24 hours 309,386,436 683.882
7 days 2,051,733,993 4,831.735
6. Build to meet expectations
Understand the consumer
1. Fidelity
2. Freshness
3. Focus
Dashboard Panels or Reports
1. In-line Search -> Easiest, yet slowest
2. Accelerated Searches -> “turbo” button
3. Scheduled Searches -> Hurry and wait
4. Summary Index -> “Instant”
5. Scheduled Report -> Pro-active
Lessons Shared
1. Use “Fast Search”
2. Partition data used in real-time OR in-line searches
3. Evaluate expensive operations with “Inspect”
10. Zillow Sources
(April 2014 PR Digest)
Zillow Traffic & Usage:
Nearly 77 million unique users visited Zillow, Inc.’s mobile applications and websites in March 2014, representing year-over-year growth of 52%.
o Source: Internal tracking via Google Analytics
Real Estate Advertising Claims:
· The Yahoo!-Zillow Real Estate Network is the largest real estate network on the Web.
o Source: comScore Media Metrix Real Estate Category Ranking by Unique Visitors, February 2014, US Data.
· The Yahoo!- Zillow Real Estate Network is the #1 Real Estate brand in all of the top 20 local markets in the United States.
o Source: comScore Local Market Key Measures January 2014, Real Estate Category by Unique Visitors
· On desktop, Zillow is nearly twice the size of Trulia and Realtor.com.
o Source: comScore Media Metrix, March 2014
· On mobile web, Zillow is more than twice the size of Trulia and four times the size of Realtor.com.
o Source: Experian Marketing Services, January 2014
Rental Advertising Claims:
· Zillow.com is the largest rental site on the Web.
o Source: comScore Media Metrix Real Estate category ranking by Unique Visitors, February 2014, US Data
· The Zillow Rental Network is the largest rental network on the Web, comprised of the millions of rental shoppers on Zillow.com, Yahoo! and HotPads.
o Source: comScore Media Metrix Key Measures Real Estate Category Ranking by Unique Visitors, February 2014, US Data.
· 13 million rental shoppers come to Zillow rental sites each month.
o Source: Google Analytics, February 2014
Zillow Mobile Facts:
· Two-thirds of Zillow’s visits come from a mobile device; on weekends it’s more than 70%.
o Source: Google Analytics, March 2014
Zillow operates the most popular suite of mobile real estate apps with more than two dozen apps across the major platforms.
o Zillow Internal, March 2014
11. Zillow Sources
In March, more than 460 million homes were viewed on Zillow Mobile – that’s 173 homes per second.
Source: Google Analytics, March 2014
· Zillow operates two of the most popular and top-rated rental apps across iOS and Android.
o Source: App Store and Google Play ratings of 4+ and 4 stars, March 2014
Zillow Mortgage Marketplace:
Consumers submitted 21 million loan requests in ZMM over the past 12 months.
Source: Zillow Internal, March 2014
Consumers receive, on average, more than 30 customized loan quotes per loan request.
Source: Zillow Internal, March 2014
Zillow has published more than 70,000 reviews of mortgage professionals submitted by our users.
Source: Zillow Internal, March 2014
· One-third of home shoppers on Zillow have been approved for a home loan.
o Source: Zillow site survey, March 2014
Zillow Data & User Generated Content:
· Zillow has data on more than 110 million U.S. homes, with Zestimates and Rent Zestimates on approximately 100 million U.S. homes.
Source: Zillow Internal, March 2014
· Zillow has published more than half a million reviews of local real estate agents submitted by our users.
Source: Zillow Internal, March 2014
· 47 million homes on Zillow have been updated by our community of users.
o Source: Zillow Internal, March 2014
· Our users have submitted more than 840,000 questions and answers in our discussion forum, Zillow Advice.
o Source: Zillow Internal, March 2014
Who Uses Zillow?
Three-quarters of Zillow users are "in the market" – buying or selling within the next two years or looking to rent.
Source: Zillow site survey, March 2014
Hinweis der Redaktion
Where are we in our journey with Splunk?
1) Identify best of breed software (based on ~30 specific requirements)
2) POC - Limited term license
3) Infrastructure build out for scale
4) -- current -- Bring the data
5) -- this summer -- Splunk PS to kick-start
So what did all the buzz in the video mean in terms of usage?
April 2014 key highlights.
Each item driving more events into logs.
POC was EOL server and all roles on 1 box.
Prod deployment taking advantage of physical and virtual hosts.
Now that data is being indexed, I want to answer questions - quickly.
There is a lot of data available to answer simple questions.
Generation of NOW. Look at the time table to power a chart. – EACH TIME IT LOADS
What have I learned as a new customer of Splunk to reduce or remove "Loading..."
1) Understand who is consuming the dashboards & visualizations
- 3 key elements to determining what optimizations you can use.
2) Walk through the panel types
3) Other methods for answering the tuff questions faster
---
Dashboards, I don’t want to wait minutes or hours to view data that I review frequently.
1 Basic saved searches that are saved as dashboard panels are “in-line” searches. They are performed each time a dashboard/panel/chart is called. (super slow)
2 Saved searches using “accelerate this search”, faster but searches are still performed each time.
3 Saved searches using scheduled – faster, but the delta time from when the search was performed is slow. You do lose flexibility to wander outside your scheduled search terms.
4 Using a summary index – fast
5 Partition data that you need real-time / inline searches
6 Use “fast search”
7 Evaluate expensive operations using inspect
* real-time search – keep their timespan limited by # of events it must crawl – beyond 15min, what’s the point? (users of splunk typically don’t have rights to perform real-time.
Lastly – If you suspect there are inefficiencies with your search – use the inspector!
2 easy ways to launch.
Can tell you if you have a slow index server.
