This document proposes a framework to benchmark multiple public sector self-service solutions in Denmark. It involves tracking usage across 250+ solutions using a combination of server-side tracking, client-side tracking without JavaScript, and tying together sessions to create "master sessions." The goals are to prioritize spending, measure solution maturity and usage, and estimate cost savings compared to offline transactions. Key challenges include connecting discrete sessions and integrating data with the existing "FORM" business reference model to roll up benchmarks against public sector variables. The potential annual savings from moving transactions online for just one of hundreds of services is estimated at $1,000 per capita in Denmark.
1. A Self Service Solutions Benchmark
(Public Sector)
An Urchin-based framework for a benchmark of multiple,
non-HTML entities (in the Public Sector) hosted in a
distributed environment in order to prioritize Public
Spending against use and Value Created
Kristoffer Ewald
Director Analytics & Professional Services
The Milk / Guava
October 16th 2009
4. About
Summary
Fmr. Managing Director & Co-Founder of the first Scandinavian Google
Analytics Authorized Consultant: The Milk (2005-2007), Acquired by
Guava Online Marketing in 2007.
Now VP Analytics & Professional Services, in Guava
Guava Professional Services Team located in Denmark, Sweden and UK –
Provides all Analytics Professional Services
Fmr. Country Coordinator Web Analytics Association Denmark (2005-
2008)
Member Web Analytics Association Research Committee
Specialist, with background as an Economist, on Analytics and Online
Marketing Benchmarks – coined the term Online Space Management
with The Milk.
6. The Case
The public sector in the Kingdom of Denmark runs 250+ online-self-service solutions, build in a variety of
technologies, making it possible to do a lot of things online - i.e. Register that you’re moving to a new address, sign up
kids for kindergarten, Request unemployment compensation, Pension etc. (Denmark is probably one of the top 3
countries in regards to digital public sector).
These solutions have been build over time in a decentralized process, where the different departments have had a
large number of different suppliers deliver the solutions. Up until now no centralized tracking or benchmark of the use of
these digital self-service solutions have existed.
We’re developing a solution based on Urchin that will deliver a centralized data-warehouse for Analytics data
for all these digital-self-service solutions.
The trick we have to pull off is that a lot of things happen server-side, and not in the browser, and that we need to track
“inside” the self-service-solutions themselves, since they’re also used as widgets and made available in many different
locations under different domains. We also need to know when a success happens outside the self service solution and
even offline.
So we’re generating gif-hits server side and client side without the JavaScript.
After the data have been collected, the Urchin data will be pulled out with the API, stored in a data warehouse where it’ll
be matured with data from other sources (FORM), and presented via a custom build front end.
Lot’s of API use will happen. The Urchin reports themselves will only be used by the core team for analysis.
The challenge is to tie sessions in the self-service solutions to sessions in the browser with the same ID
contained, in order to be able to create “Mega Sessions”.
Current data quality that forms the basis of priorities is very aggregated – something like 16% of citizens in
OECD used a public sector self service solutions during the last 3 months.
7. Summary of key concepts
Server Side tracking
Non JS client side tracking
Creation of “Master Sessions”
Use of IDs to create key for integration via API
Unique ID-tagged links, generated on the fly, to bridge gap
between server side and client side tracking
Unique ID’s stamped in PDF’s + Tracked with Transaction
to measure offline success
Source: 8pt, medium gray
9. Why build a benchmarking system?
The public sector needs benchmarking in order to
prioritize what to develop and sustain
They need to know the maturity of the solutions
They need to know how much they’re used and on what
domains
They need to know the distribution across the public
portfolio of services
Measure the impact on the budget via integration with ”
FORM” – the public sector business reference model
Source: 8pt, medium gray
12. “FORM” and linking with our benchmark
The “FORM” (Public Sector Business Reference Model) - task
map is a meta data based tool that shows how each service is
placed within the collected portfolio of public services, it’s linked
for each service to the relevant section of the the Public Sector
budget, the relevant legislation, the relevant Department etc.
So, the end result will be a benchmark that will give the
combined Public Sector in Denmark insights into what Services
are effective, how they contribute to the Public Sector budget,
and where opportunities for improvement exist.
One example: when you move, the estimated cost to handle the
paperwork is USD 40, online it’s USD 1. (and we have several
hundred thousand people that move every year), and that’s just
one service…
Source: 8pt, medium gray
13. How we’ll build the Benchmark
There are three parts needed to do this:
1) Server side tracking of solution-use server side
2a) Client side calls to utm.gif from inside widget (i.e. Java)
2b) Client side calls to utm.gif from browser (via .js)
The real challenge is to tie these discrete utm.gif hits into “Master
Sessions”, and have a naming system for them including a key, to
be able to integrate with data in FORM, and measure conversions.
In other words – we need to create “Master-Sessions”,
containing several urchin-sessions (something also very
relevant in a number of other applications i.e. e-mailing
systems or CRM integration).
This is one of the reasons Urchin is necessary, since we can’t do
this with only 50.000 records, the current limit in GA.
Source: 8pt, medium gray
14. The Core Challenge in two parts
1. Find a way to connect discrete sessions into “Master
sessions” and Pass information from client side tracking to
server side, and Integrate with regular site tracking in order
to record conversions (outside the self service applet).
2. Extract the Master Sessions, mature it with information from
FORM, and create roll-up reports that’ll benchmark across all
the variables in FORM.
Source: 8pt, medium gray
15. Challenge 1 – Create “Master Sessions”
Solution: Create virtual pageviews of the following structure:
/FORM_ID/PropertyID/InstanceID/Folder/Action_label#?variable1=x&variable2=y etc.
To be supplemented with Transactions to get non-
aggregated actions with IDs, in order to integrate with offline
success – i.e. when a pdf is created (we’ll stamp the ID’s into
the pdf)
Label Value
FORM_ID Key
PropertyID Host where incident take place
InstanceID Random number+unix time code
Folder Folder name
Action_label# PV + VPV
17. API use
API will be used to pull data out, mature with “FORM_ID” as key
with the existing data matrix, and create roll-up benchmarks
against the Dimensions from Urchin vs FORM.
Source: 8pt, medium gray
18. Bridging
Create unique IDs in campaign data, to create a link btw. Server
Side Pageviews and Client Side Pageviews.
Source: 8pt, medium gray
19. Offline Integration
Create a unique ID, stamp it into PDF’s generated in the user
session, and set a Transaction, in order to tie forms printed and
send in to conversions
Source: 8pt, medium gray
20. Answer to the Quiz - potential saving?
Denmark Population = 5m
(OECD = 1.2 Bn)
Potential saving in Denmark:
$ (40 offline -1 online) = 39*
500,000 incidents *
250 systems =
4,875,000,000 $ = (approx.)
$1,000 p.a. pr. capita
