SlideShare ist ein Scribd-Unternehmen logo
1 von 40
Downloaden Sie, um offline zu lesen
How to Tune Search Requests


                                                   Tom Hill / tomhill@amazon.com




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Agenda
!        Query Processing
!        Common Issues
!        Benchmarking
!        Analytics




    © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
…query processing
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Query Processing
                                                                                                         564                                              726




                                                                                                         726                                              564
Query


                                                                                                         123                                              123


         Matching                                 Filtering                                  Ranking                                                Sorting
  © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Query Processing
!   Matching
        •  Text Fields
        •  Literal Fields
!   Filtering
        •  Numeric terms, ranges
!   Ranking
        •  Score computation for each document
!   Sorting
        •  Based on rank computation, alphabetic, numeric
!   Faceting

 © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Matching
!   The starting point for results
!   Full text matching with “text” fields
!   Exact matching with “literal” fields
   	
  	
  




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Filtering
!   UINT fields
       •  Numbers
       •  Numeric ranges
!   After all of these you get results:
   	
  <hits	
  found="2432"	
  start="0">	
  




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Ranking
!   Score can be
       •  Text Relevance
       •  Rank expression
!   Done for every document that makes it past filtering




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Sorting
!   Last step
!   Again, cost a function of match set size
!   Sort by
       •  Text Relevance
       •  Rank Expression
       •  Field Value




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Performance: Match Set Size

<all documents>                                                       cat|dog                                    AND color:black                            AND age:0..6




Increasing Performance
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
…common issues
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Literals vs Uints
!   Literals will tend to improve performance
! Uints will tend to take less space




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Query Restriction: LiteralsNo Uints
                            Vs.
                                                                                                                                                                                    Restriction
	
  GeoMethod	
  	
  	
  	
  TextRel	
  	
  	
  	
  	
  Limits	
  	
  	
  	
  Queries	
  	
  Seconds	
  	
  QTimeMS	
  	
  	
  	
  Threads	
  CompletedQ	
  	
  	
  	
  	
  	
  AveHits	
  
	
  	
  	
  	
  	
  	
  NONE	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  6.2255	
  	
  	
  	
  	
  	
  622	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  8345450.00	
  
                                                                                                                                                                                      UINT
	
  CARTESIAN	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  15.6064	
  	
  	
  	
  	
  1560	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  8345450.00	
  
	
  	
  	
  	
  	
  	
  EQUI	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  19.7106	
  	
  	
  	
  	
  1971	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  8345450.00	
  
                                                                                                                                                                                    Restriction
	
  	
  	
  COSINES	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  27.4968	
  	
  	
  	
  	
  2749	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  8345450.00	
  
	
  HAVERSINE	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  31.2595	
  	
  	
  	
  	
  3125	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  8345450.00	
  
	
  
	
  	
  	
  	
  	
  	
  NONE	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  Numeric	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  9.1758	
  	
  	
  	
  	
  	
  917	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3807.00	
  
                                                                                                                                                                                      Literal
	
  CARTESIAN	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  Numeric	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  9.0255	
  	
  	
  	
  	
  	
  902	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3807.00	
  
	
  	
  	
  	
  	
  	
  EQUI	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  Numeric	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  9.1158	
  	
  	
  	
  	
  	
  911	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3807.00	
  
                                                                                                                                                                                     Restriction
	
  	
  	
  COSINES	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  Numeric	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  9.8321	
  	
  	
  	
  	
  	
  983	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3807.00	
  
	
  HAVERSINE	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  Numeric	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  9.1272	
  	
  	
  	
  	
  	
  912	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3807.00	
  
	
  
	
  	
  	
  	
  	
  	
  NONE	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  literal	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  0.8254	
  	
  	
  	
  	
  	
  	
  82	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3781.00	
  
	
  CARTESIAN	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  literal	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  0.5936	
  	
  	
  	
  	
  	
  	
  59	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3781.00	
  
	
  	
  	
  	
  	
  	
  EQUI	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  literal	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  0.6173	
  	
  	
  	
  	
  	
  	
  61	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3781.00	
  
	
  	
  	
  COSINES	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  literal	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  0.5916	
  	
  	
  	
  	
  	
  	
  59	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3781.00	
  
	
  HAVERSINE	
  	
  	
  	
  	
  	
  false	
  	
  	
  	
  literal	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  0.6289	
  	
  	
  	
  	
  	
  	
  62	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1	
  	
  	
  	
  	
  	
  	
  	
  	
  10	
  	
  	
  	
  	
  	
  3781.00	
  
	
  


    © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Negative Queries
!   CloudSearch supports negative queries
       •  &q=-amazon
!   Can match all documents
       •  if "doesntmatchanything" matches 0 docs, then
          -doesntmatchanything will match all docs.
!   Matching all docs means lots of computations
       •  Sorting, rank expressions, etc.



© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Implicit Limits
!   If the user doesn't give you restrictions,
    add some!
!   Top items for their category
!   Add implicit limits to broad queries
       •  &bq=important:1
       •  &bq=population:10000..
!   select the best


© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Wildcard Queries
!   Expands the query terms
       •  a* => aardvark, aaron, … azimuth
       •  Limited to first 2000 terms.
                 •  But that's still 2000 terms!
       •  Match set is all docs that contain any one of the terms
!   Match set gets big!
             a*                      a                       doc1                 doc9  doc17 doc80                                              doc85               doc90        do
                                  aaron                      doc3                doc50 doc87
                                   after                     doc99               doc110 doc117 doc111
                                  apple                      doc3                 doc5  doc18
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Wildcard Queries
!   Stemming
       •  cats* doesn't match cats
       •  cats is stemmed to "cat", but wildcards are not stemmed
!   Avoid negative wildcard queries
       •  -cat works fine
                 •  but may take a while to execute
       •  -cat123 may confuse you. It becomes:
                 •  -cat 123



© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Boolean Queries
!   Used to restrict your searches
       •  Faceting (e.g. category)
       •  Date
       •  Security
!   &bq=(and	
  title:'potter'	
  author:'rowling')	
  
!   Can improve performance
!   Can slow performance


© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Boolean Queries
!   Effects of additional AND
!   Effects of Additional OR

                                                                                                                             Color
                                                                                                                             Color

                                                                                                              Size Style
                                                                                                               Size Style
                                                                                                              Size  Style

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Rank Expressions Review
!   Used to enhance search results
!   Include non-text factors in scoring
       •  Popularity (likes/upvotes)
       •  Distance
       •  Price/Profit


&rank-­‐pop=((0.3*popularity)/10.0)+(0.7*text_relevance)&rank=pop	
  



© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Rank Expressions
  !   Execute once for each document in match set
          •  Reduce your match set with text matches, literals, uints
          •  Done after filters applied




                                                                                                                                     Faster!
Slower
   © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Rank Expressions
!   Avoid queries that match all (or many) documents
       •  -foo
       •  state:CA
! Precompute static parts of rank expression
       •  sqrt(rating)+200*log10(doc.votes) + 20*log10(doc.sales) =>
          "precomputed"
!   Add implicit limits to broad searches



© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Default Search Field
!   Searches all text fields by default
       •  Usually the right thing
       •  If not, remove unneeded fields from it
!   Changeable via the API
!   Alternative: explicitly select one field to search
       •  title:'harry potter'




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Target Slow Queries
!   Slow searches affect other searches
!   Optimizing the slowest search requests may speed up all
    of your searches.
!   Slow queries can produce timeouts (507s)




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
…benchmarking
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Testing Latency
!   Hard to estimate
       •  Depends on queries, usage patterns, data…
!   Build a domain & test
       •  It's the cloud, spin up & down as needed!
       •  Use your own data
       •  As close to real usage as you can
                 •  Log replay is good!
!   A/B Test your changes

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Two Approaches
!   Testing
       •  Just run some queries
!   Benchmarking
       •  Run enough to have some statistical validity




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Testing Approaches
!   Statistics provided

!   Browser tools
       •  Chrome
       •  Firebug for Firefox




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Benchmarking
!   Apache JMeter (or similar)
       •  Well documented
       •  Well tested
       •  General
                 •  This is good, and bad
!   Custom Code
       •  Usually looks a lot like your application
       •  So, not as much code as you might think
       •  Flexible


© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Custom Code
!   Multithread it for realistic results
       •  If it's simulating more than one client
!   Make sure the clients (tester) isn't the bottleneck
       •  Benchmark it with searches stubbed
       •  Avoid languages with global interpreter locks
!   Personally, I use:
       •  Java
       •  Apache Http Client


© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Custom Code Example
            	
  LinkedList<Thread>	
  threads	
  =	
  new	
  LinkedList<Thread>();	
  
            	
  int	
  nQ	
  =	
  queue.size();	
  
            	
  long	
  time	
  =	
  System.nanoTime();	
  
            	
  List<Consumer>	
  consumers	
  =	
  new	
  LinkedList<Consumer>();	
  
            	
  for	
  (int	
  i	
  =	
  0;	
  i	
  <	
  threadCount;	
  i++)	
  {	
  
            	
         	
  Consumer	
  c1	
  =	
  new	
  Consumer(queue);	
  
            	
         	
  consumers.add(c1);	
  
            	
         	
  Thread	
  t	
  =	
  new	
  Thread(c1);	
  
            	
         	
  threads.add(t);	
  
            	
         	
  t.start();	
  
            	
  }	
  
            	
  //	
  Wait	
  for	
  signal	
  that	
  we	
  have	
  processed	
  all	
  queries.	
  
            	
  for	
  (Thread	
  thread	
  :	
  threads)	
  {	
  
            	
         	
  thread.join();	
  
	
  


       © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Sample Data & Queries
!   Sample Data – must be realistic
                 •  BAD: a123 b456 xyzzyx
                 •  Better: use Wikipedia, project Gutenberg
                 •  Best: Your own data
!   Queries
       •      BAD: random words
       •      Better: read words from test data
       •      Best: Replay log files
       •      Always check your number of responses in benchmarks.
                 •  It's easy to get fast queries, if you get no hits.

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
…analytics
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Analytics & Metrics
!   You have to know what to tune
!   CloudSearch Metrics
!   Custom Logging




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
CloudSearch Metrics
!   Top Queries
!   Zero result queries
       •  Lack of data?
       •  Or query issues?




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Custom Logging
!   Log all requests on your end
!   Watch
       •  Longest running queries
       •  Failed Queries
       •  HTTP error codes
!   Track Changes over time




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Warning Signs
!   Know your http error codes
       •  500 series can be retried
       •  May indicate server is overloaded
       •  Long queries can tie up threads until timeout
                 •  More of an issue on small servers.




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
…wrap up
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Wrap Up
!   1) Limit match set size
!   2) Limit Match set size J
!   Be aware of the cost of features
       •  Test/Benchmark
!   Resources
       •      Slides will be on meetup group
       •      http://jmeter.apache.org/
       •      https://getfirebug.com/
       •      http://en.wikipedia.org/wiki/Category:Load_testing_tools

© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
Thanks!

                                                                   Tom Hill
                                                            tomhill@amazon.com




© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Weitere ähnliche Inhalte

Was ist angesagt?

OpenSearch
OpenSearchOpenSearch
OpenSearch
hchen1
 
Rest and Sling Resolution
Rest and Sling ResolutionRest and Sling Resolution
Rest and Sling Resolution
DEEPAK KHETAWAT
 
MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018
MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018
MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018
Amazon Web Services Korea
 
모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018
모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018
모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018
Amazon Web Services Korea
 
Sling models by Justin Edelson
Sling models by Justin Edelson Sling models by Justin Edelson
Sling models by Justin Edelson
AEM HUB
 

Was ist angesagt? (20)

마이크로서비스 아키텍처 기반의 의료정보시스템 고도화 전환사례.건국대학교병원.이제관
마이크로서비스 아키텍처 기반의 의료정보시스템 고도화 전환사례.건국대학교병원.이제관마이크로서비스 아키텍처 기반의 의료정보시스템 고도화 전환사례.건국대학교병원.이제관
마이크로서비스 아키텍처 기반의 의료정보시스템 고도화 전환사례.건국대학교병원.이제관
 
오픈소스 프레임워크 기반 웹 서비스 설계 (Example)
오픈소스 프레임워크 기반 웹 서비스 설계 (Example)오픈소스 프레임워크 기반 웹 서비스 설계 (Example)
오픈소스 프레임워크 기반 웹 서비스 설계 (Example)
 
AEM and Sling
AEM and SlingAEM and Sling
AEM and Sling
 
OpenSearch
OpenSearchOpenSearch
OpenSearch
 
Introduction to thymeleaf
Introduction to thymeleafIntroduction to thymeleaf
Introduction to thymeleaf
 
금융산업의 빅데이터 활용 및 이슈
금융산업의 빅데이터 활용 및 이슈금융산업의 빅데이터 활용 및 이슈
금융산업의 빅데이터 활용 및 이슈
 
AWS Fault Injection Simulator를 통한 실전 카오스 엔지니어링 - 윤석찬 AWS 수석 테크에반젤리스트 / 김신 SW엔...
AWS Fault Injection Simulator를 통한 실전 카오스 엔지니어링 - 윤석찬 AWS 수석 테크에반젤리스트 / 김신 SW엔...AWS Fault Injection Simulator를 통한 실전 카오스 엔지니어링 - 윤석찬 AWS 수석 테크에반젤리스트 / 김신 SW엔...
AWS Fault Injection Simulator를 통한 실전 카오스 엔지니어링 - 윤석찬 AWS 수석 테크에반젤리스트 / 김신 SW엔...
 
글로벌 기업들의 효과적인 데이터 분석을 위한 Data Lake 구축 및 분석 사례 - 김준형 (AWS 솔루션즈 아키텍트)
글로벌 기업들의 효과적인 데이터 분석을 위한 Data Lake 구축 및 분석 사례 - 김준형 (AWS 솔루션즈 아키텍트)글로벌 기업들의 효과적인 데이터 분석을 위한 Data Lake 구축 및 분석 사례 - 김준형 (AWS 솔루션즈 아키텍트)
글로벌 기업들의 효과적인 데이터 분석을 위한 Data Lake 구축 및 분석 사례 - 김준형 (AWS 솔루션즈 아키텍트)
 
Rest and Sling Resolution
Rest and Sling ResolutionRest and Sling Resolution
Rest and Sling Resolution
 
MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018
MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018
MSA를 넘어 Function의 로의 진화::주경호 수석::AWS Summit Seoul 2018
 
Spring Framework Petclinic sample application
Spring Framework Petclinic sample applicationSpring Framework Petclinic sample application
Spring Framework Petclinic sample application
 
모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018
모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018
모놀리스에서 마이크로서비스 아키텍처로의 전환 전략::박선용::AWS Summit Seoul 2018
 
Sling models by Justin Edelson
Sling models by Justin Edelson Sling models by Justin Edelson
Sling models by Justin Edelson
 
RESTful API Design, Second Edition
RESTful API Design, Second EditionRESTful API Design, Second Edition
RESTful API Design, Second Edition
 
NAVER TECH CONCERT_FE2019_오늘부터 나도 FE 성능분석가
NAVER TECH CONCERT_FE2019_오늘부터 나도 FE 성능분석가NAVER TECH CONCERT_FE2019_오늘부터 나도 FE 성능분석가
NAVER TECH CONCERT_FE2019_오늘부터 나도 FE 성능분석가
 
Force.com Canvas アプリケーション
Force.com Canvas アプリケーションForce.com Canvas アプリケーション
Force.com Canvas アプリケーション
 
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
 
[Partner TechShift 2017] AWS 마켓플레이스 등록을 위한 테크니컬 체크리스트
[Partner TechShift 2017] AWS 마켓플레이스 등록을 위한 테크니컬 체크리스트[Partner TechShift 2017] AWS 마켓플레이스 등록을 위한 테크니컬 체크리스트
[Partner TechShift 2017] AWS 마켓플레이스 등록을 위한 테크니컬 체크리스트
 
How to configure hikvision dvr cloud storage through web browser
How to configure hikvision dvr cloud storage through web browserHow to configure hikvision dvr cloud storage through web browser
How to configure hikvision dvr cloud storage through web browser
 
Evolve18 | Abhishek Dwevdi & Varun Mitra | Introduction to AEM Assets
Evolve18 | Abhishek Dwevdi & Varun Mitra | Introduction to AEM Assets Evolve18 | Abhishek Dwevdi & Varun Mitra | Introduction to AEM Assets
Evolve18 | Abhishek Dwevdi & Varun Mitra | Introduction to AEM Assets
 

Andere mochten auch

Meetup - Using CloudSearch with DynamoDB
Meetup - Using CloudSearch with DynamoDBMeetup - Using CloudSearch with DynamoDB
Meetup - Using CloudSearch with DynamoDB
Amazon Web Services
 
AWS Webcast - Location Based Search
AWS Webcast - Location Based SearchAWS Webcast - Location Based Search
AWS Webcast - Location Based Search
Amazon Web Services
 
AWS Enterprise Summit London 2013 - Stuart Lynn - Sage
AWS Enterprise Summit London 2013 - Stuart Lynn - SageAWS Enterprise Summit London 2013 - Stuart Lynn - Sage
AWS Enterprise Summit London 2013 - Stuart Lynn - Sage
Amazon Web Services
 
AWS Summit 2013 | Singapore - Public Sector Keynote, Teresa Carlson
AWS Summit 2013 | Singapore - Public Sector Keynote, Teresa CarlsonAWS Summit 2013 | Singapore - Public Sector Keynote, Teresa Carlson
AWS Summit 2013 | Singapore - Public Sector Keynote, Teresa Carlson
Amazon Web Services
 

Andere mochten auch (20)

Geospatial Search With Amazon CloudSearch
Geospatial Search With Amazon CloudSearch Geospatial Search With Amazon CloudSearch
Geospatial Search With Amazon CloudSearch
 
Amazon CloudSearch - Relevance, Ranking, Tuning and Analytics
Amazon CloudSearch - Relevance, Ranking, Tuning and AnalyticsAmazon CloudSearch - Relevance, Ranking, Tuning and Analytics
Amazon CloudSearch - Relevance, Ranking, Tuning and Analytics
 
Amazon cloud search comparison report
Amazon cloud search comparison reportAmazon cloud search comparison report
Amazon cloud search comparison report
 
Meetup - Using CloudSearch with DynamoDB
Meetup - Using CloudSearch with DynamoDBMeetup - Using CloudSearch with DynamoDB
Meetup - Using CloudSearch with DynamoDB
 
Enrich Search User Experience Using Amazon CloudSearch (SVC302) | AWS re:Inve...
Enrich Search User Experience Using Amazon CloudSearch (SVC302) | AWS re:Inve...Enrich Search User Experience Using Amazon CloudSearch (SVC302) | AWS re:Inve...
Enrich Search User Experience Using Amazon CloudSearch (SVC302) | AWS re:Inve...
 
AWS Webcast - Location Based Search
AWS Webcast - Location Based SearchAWS Webcast - Location Based Search
AWS Webcast - Location Based Search
 
Geospatial search with SOLR
Geospatial search with SOLRGeospatial search with SOLR
Geospatial search with SOLR
 
Japanese CloudSearch Use-Cases and Tech Deep Dive
Japanese CloudSearch Use-Cases and Tech Deep DiveJapanese CloudSearch Use-Cases and Tech Deep Dive
Japanese CloudSearch Use-Cases and Tech Deep Dive
 
I See NoSQL Document Stores in Geospatial Applications
I See NoSQL Document Stores in Geospatial ApplicationsI See NoSQL Document Stores in Geospatial Applications
I See NoSQL Document Stores in Geospatial Applications
 
AWS Canberra WWPS Summit 2013 - Extending your Datacentre with Amazon VPC
AWS Canberra WWPS Summit 2013 - Extending your Datacentre with Amazon VPCAWS Canberra WWPS Summit 2013 - Extending your Datacentre with Amazon VPC
AWS Canberra WWPS Summit 2013 - Extending your Datacentre with Amazon VPC
 
Advanced Topics - Session 2 - Introducing AWS OpsWorks
Advanced Topics - Session 2 - Introducing AWS OpsWorksAdvanced Topics - Session 2 - Introducing AWS OpsWorks
Advanced Topics - Session 2 - Introducing AWS OpsWorks
 
AWS Sydney Summit 2013 - Continuous Deployment Practices, with Production, Te...
AWS Sydney Summit 2013 - Continuous Deployment Practices, with Production, Te...AWS Sydney Summit 2013 - Continuous Deployment Practices, with Production, Te...
AWS Sydney Summit 2013 - Continuous Deployment Practices, with Production, Te...
 
AWS Enterprise Summit London 2013 - Stuart Lynn - Sage
AWS Enterprise Summit London 2013 - Stuart Lynn - SageAWS Enterprise Summit London 2013 - Stuart Lynn - Sage
AWS Enterprise Summit London 2013 - Stuart Lynn - Sage
 
Empowering Publishers Event - Intro - May-15-2013
Empowering Publishers Event - Intro - May-15-2013Empowering Publishers Event - Intro - May-15-2013
Empowering Publishers Event - Intro - May-15-2013
 
AWS Summit 2013 | Singapore - Public Sector Keynote, Teresa Carlson
AWS Summit 2013 | Singapore - Public Sector Keynote, Teresa CarlsonAWS Summit 2013 | Singapore - Public Sector Keynote, Teresa Carlson
AWS Summit 2013 | Singapore - Public Sector Keynote, Teresa Carlson
 
AWS Summit 2013 | Auckland - Extending your Datacentre with Amazon VPC
AWS Summit 2013 | Auckland - Extending your Datacentre with Amazon VPCAWS Summit 2013 | Auckland - Extending your Datacentre with Amazon VPC
AWS Summit 2013 | Auckland - Extending your Datacentre with Amazon VPC
 
Monetise your content with Amazon CloudFront
Monetise your content with Amazon CloudFrontMonetise your content with Amazon CloudFront
Monetise your content with Amazon CloudFront
 
AWS Summit 2013 | Singapore - Understanding AWS Storage Options
AWS Summit 2013 | Singapore - Understanding AWS Storage OptionsAWS Summit 2013 | Singapore - Understanding AWS Storage Options
AWS Summit 2013 | Singapore - Understanding AWS Storage Options
 
End Note - AWS India Summit 2012
End Note - AWS India Summit 2012End Note - AWS India Summit 2012
End Note - AWS India Summit 2012
 
AWS 101 Lunch & Learn March 2013
AWS 101 Lunch & Learn March 2013AWS 101 Lunch & Learn March 2013
AWS 101 Lunch & Learn March 2013
 

Mehr von Amazon Web Services

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 

Mehr von Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

How to Tune Search Requests using Amazon CloudSearch

  • 1. How to Tune Search Requests Tom Hill / tomhill@amazon.com © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 2. Agenda !   Query Processing !   Common Issues !   Benchmarking !   Analytics © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 3. …query processing © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 4. Query Processing 564 726 726 564 Query 123 123 Matching Filtering Ranking Sorting © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 5. Query Processing !   Matching •  Text Fields •  Literal Fields !   Filtering •  Numeric terms, ranges !   Ranking •  Score computation for each document !   Sorting •  Based on rank computation, alphabetic, numeric !   Faceting © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 6. Matching !   The starting point for results !   Full text matching with “text” fields !   Exact matching with “literal” fields     © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 7. Filtering !   UINT fields •  Numbers •  Numeric ranges !   After all of these you get results:  <hits  found="2432"  start="0">   © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 8. Ranking !   Score can be •  Text Relevance •  Rank expression !   Done for every document that makes it past filtering © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 9. Sorting !   Last step !   Again, cost a function of match set size !   Sort by •  Text Relevance •  Rank Expression •  Field Value © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 10. Performance: Match Set Size <all documents> cat|dog AND color:black AND age:0..6 Increasing Performance © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 11. …common issues © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 12. Literals vs Uints !   Literals will tend to improve performance ! Uints will tend to take less space © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 13. Query Restriction: LiteralsNo Uints Vs. Restriction  GeoMethod        TextRel          Limits        Queries    Seconds    QTimeMS        Threads  CompletedQ            AveHits              NONE            false                                        10      6.2255            622                    1                  10      8345450.00   UINT  CARTESIAN            false                                        10    15.6064          1560                    1                  10      8345450.00              EQUI            false                                        10    19.7106          1971                    1                  10      8345450.00   Restriction      COSINES            false                                        10    27.4968          2749                    1                  10      8345450.00    HAVERSINE            false                                        10    31.2595          3125                    1                  10      8345450.00                NONE            false        Numeric                  10      9.1758            917                    1                  10            3807.00   Literal  CARTESIAN            false        Numeric                  10      9.0255            902                    1                  10            3807.00              EQUI            false        Numeric                  10      9.1158            911                    1                  10            3807.00   Restriction      COSINES            false        Numeric                  10      9.8321            983                    1                  10            3807.00    HAVERSINE            false        Numeric                  10      9.1272            912                    1                  10            3807.00                NONE            false        literal                  10      0.8254              82                    1                  10            3781.00    CARTESIAN            false        literal                  10      0.5936              59                    1                  10            3781.00              EQUI            false        literal                  10      0.6173              61                    1                  10            3781.00        COSINES            false        literal                  10      0.5916              59                    1                  10            3781.00    HAVERSINE            false        literal                  10      0.6289              62                    1                  10            3781.00     © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 14. Negative Queries !   CloudSearch supports negative queries •  &q=-amazon !   Can match all documents •  if "doesntmatchanything" matches 0 docs, then -doesntmatchanything will match all docs. !   Matching all docs means lots of computations •  Sorting, rank expressions, etc. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 15. Implicit Limits !   If the user doesn't give you restrictions, add some! !   Top items for their category !   Add implicit limits to broad queries •  &bq=important:1 •  &bq=population:10000.. !   select the best © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 16. Wildcard Queries !   Expands the query terms •  a* => aardvark, aaron, … azimuth •  Limited to first 2000 terms. •  But that's still 2000 terms! •  Match set is all docs that contain any one of the terms !   Match set gets big! a* a doc1 doc9 doc17 doc80 doc85 doc90 do aaron doc3 doc50 doc87 after doc99 doc110 doc117 doc111 apple doc3 doc5 doc18 © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 17. Wildcard Queries !   Stemming •  cats* doesn't match cats •  cats is stemmed to "cat", but wildcards are not stemmed !   Avoid negative wildcard queries •  -cat works fine •  but may take a while to execute •  -cat123 may confuse you. It becomes: •  -cat 123 © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 18. Boolean Queries !   Used to restrict your searches •  Faceting (e.g. category) •  Date •  Security !   &bq=(and  title:'potter'  author:'rowling')   !   Can improve performance !   Can slow performance © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 19. Boolean Queries !   Effects of additional AND !   Effects of Additional OR Color Color Size Style Size Style Size Style © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 20. Rank Expressions Review !   Used to enhance search results !   Include non-text factors in scoring •  Popularity (likes/upvotes) •  Distance •  Price/Profit &rank-­‐pop=((0.3*popularity)/10.0)+(0.7*text_relevance)&rank=pop   © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 21. Rank Expressions !   Execute once for each document in match set •  Reduce your match set with text matches, literals, uints •  Done after filters applied Faster! Slower © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 22. Rank Expressions !   Avoid queries that match all (or many) documents •  -foo •  state:CA ! Precompute static parts of rank expression •  sqrt(rating)+200*log10(doc.votes) + 20*log10(doc.sales) => "precomputed" !   Add implicit limits to broad searches © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 23. Default Search Field !   Searches all text fields by default •  Usually the right thing •  If not, remove unneeded fields from it !   Changeable via the API !   Alternative: explicitly select one field to search •  title:'harry potter' © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 24. Target Slow Queries !   Slow searches affect other searches !   Optimizing the slowest search requests may speed up all of your searches. !   Slow queries can produce timeouts (507s) © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 25. …benchmarking © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 26. Testing Latency !   Hard to estimate •  Depends on queries, usage patterns, data… !   Build a domain & test •  It's the cloud, spin up & down as needed! •  Use your own data •  As close to real usage as you can •  Log replay is good! !   A/B Test your changes © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 27. Two Approaches !   Testing •  Just run some queries !   Benchmarking •  Run enough to have some statistical validity © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 28. Testing Approaches !   Statistics provided !   Browser tools •  Chrome •  Firebug for Firefox © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 29. Benchmarking !   Apache JMeter (or similar) •  Well documented •  Well tested •  General •  This is good, and bad !   Custom Code •  Usually looks a lot like your application •  So, not as much code as you might think •  Flexible © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 30. Custom Code !   Multithread it for realistic results •  If it's simulating more than one client !   Make sure the clients (tester) isn't the bottleneck •  Benchmark it with searches stubbed •  Avoid languages with global interpreter locks !   Personally, I use: •  Java •  Apache Http Client © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 31. Custom Code Example  LinkedList<Thread>  threads  =  new  LinkedList<Thread>();    int  nQ  =  queue.size();    long  time  =  System.nanoTime();    List<Consumer>  consumers  =  new  LinkedList<Consumer>();    for  (int  i  =  0;  i  <  threadCount;  i++)  {      Consumer  c1  =  new  Consumer(queue);      consumers.add(c1);      Thread  t  =  new  Thread(c1);      threads.add(t);      t.start();    }    //  Wait  for  signal  that  we  have  processed  all  queries.    for  (Thread  thread  :  threads)  {      thread.join();     © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 32. Sample Data & Queries !   Sample Data – must be realistic •  BAD: a123 b456 xyzzyx •  Better: use Wikipedia, project Gutenberg •  Best: Your own data !   Queries •  BAD: random words •  Better: read words from test data •  Best: Replay log files •  Always check your number of responses in benchmarks. •  It's easy to get fast queries, if you get no hits. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 33. …analytics © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 34. Analytics & Metrics !   You have to know what to tune !   CloudSearch Metrics !   Custom Logging © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 35. CloudSearch Metrics !   Top Queries !   Zero result queries •  Lack of data? •  Or query issues? © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 36. Custom Logging !   Log all requests on your end !   Watch •  Longest running queries •  Failed Queries •  HTTP error codes !   Track Changes over time © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 37. Warning Signs !   Know your http error codes •  500 series can be retried •  May indicate server is overloaded •  Long queries can tie up threads until timeout •  More of an issue on small servers. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 38. …wrap up © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 39. Wrap Up !   1) Limit match set size !   2) Limit Match set size J !   Be aware of the cost of features •  Test/Benchmark !   Resources •  Slides will be on meetup group •  http://jmeter.apache.org/ •  https://getfirebug.com/ •  http://en.wikipedia.org/wiki/Category:Load_testing_tools © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 40. Thanks! Tom Hill tomhill@amazon.com © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.