As a search consultant I need to understand how a search application is used with the end goal of providing a better search experience for the end user. That story can come from many places and part of that story can be found in the query logs.
Blog post about the same topic: http://nhhagen.wordpress.com/2013/11/28/query-log-analysis-using-logstash-elasticsearch-and-kibana/
This example is preformatted in JSON, but often the data is in plain text.
And more
GraphsQuery load over time Average query latency over timeTop Queries Top Queries with 0 hits Top search modes top refiners/facets usedChanges in query load pct. last hour, last day, last 30dRefiner/facet usage over time
GraphsQuery latency distribution count and pct Average query latency over timeQuery latency distribution count and pct over timeTable: queries over 200ms latency (really slow queries)
A “typical” search applicationThe user sends a query to the search applicationThat query finds its way to the search application APIThe logic in the application is set up to do 3 parallel queries against the search engines for different types of data
Query logs can be taken from the search engine, but are not the query log for the user, they are technical query logsThose logs can be analyzed to figure out when you need new servers, more RAM, CPU etc
When you create the query log in the search application API, just before returning to the user, you can put the context of the query and “sum” into the query log
For now only used to read the JSON log files and transport the log events to elasticsearch
Indexes all the log events in per day indices
Queries elasticsearch and builds graphs using facets. Provides an interactive application to generate graphs on the fly
Very simplified architecture
GraphsQuery load over time Average query latency over timeTop Queries Top Queries with 0 hits Top search modes top refiners/facets usedChanges in query load pct. last hour, last day, last 30dRefiner/facet usage over time
GraphsQuery latency distribution count and pct Average query latency over timeQuery latency distribution count and pct over timeTable: queries over 200ms latency (really slow queries)