This is a guide for those with limited budget on how to engage end users in product development. For more information, see www.productdoctor.co.uk

1. 1
DIY USER ENGAGEMENT
A guide on how to engage users in product development
This White Paper is based on a Product Doctor Drop in Surgery that I ran with Katrina
Damianou during Over The Air 2011, 36 hours of Mobile Development. We offered 25
minute free Product Health Checks to the developers in attendance. Some of the examples
of products brought to us are referenced in this paper.
___________________________________________________________________________
Contents
Agree this premise first!
1. It is never too early (or too late) to engage end users
2. What do you show users?
3. How to find your end users
4. Can you have the conversation with end users?
5. How to begin the conversation
6. Write a test script
7. “I can’t explain what my product does”
8. Showing a prototype
9. Testing for usability
10. Keep checking back with users as you develop and improve each new feature
11. Build and test your product before developing your brand
12. Be honest with yourself as to why you are developing the App
Conclusion
Agree this premise first!
Before reading any further, we have to first accept that engaging users throughout the
development cycle and from the concept phase (i.e. before you build anything) is good
practice. I have proved both of these points over the past years, time and time again.
A. Engaging users WILL give your product a far greater chance at success. We can all think of
great examples of products that may work beautifully, but fail as they do not meet a user
need or desire.
B. Furthermore, products that are not tested properly with users often are too feature
heavy -there may be a handful of features that are used while the effort spent to build the
rest has been wasted.
When I asked patients whether they had engaged users at any stage of their development,
those with live Apps did say proudly that they certainly studied their App Store feedback.
October 2011 Product Doctor Insights from Over The Air 2011

2. 2
App Store feedback is not enough! Not only is it once the App is out there, but throughout
user tests I have carried out, I have found a bias on males giving written feedback on the
App Stores. Even then, it is only a certain type of personality that gives written feedback on
the App stores. They may give you suggestions on new features or how to improve their
user experience, but this will often be on one or two actions / features they are carrying out.
You are unlikely to get confirmation that the needs / desires you are trying to meet are valid
– again, this can throw the whole validity of the App in to question.
Remember that these people who give you feedback are likely to be early adopters. We
have to remember that Apps are still in the “introduction phase” of the product lifecycle.
This is the point where all the product and experience glitches/bugs will be highlighted and
ironed out. So by all means, you may well get good feedback on what is and isn’t working
but not on the bigger picture.
So here is my guide to DIY User Engagement
1. It is never too early (or too late) to engage end users
I am a bit of a broken record on this one, but seriously, it is something that recurs time and
time again! Before you put your time in to developing, at any stage, please test out what
you are intending that the product is going to do. It is crucial to check that the user needs
or desires exist and that the product and features you are going to build are a good solution
to those needs or desires.
2. What do you show users?
One of our patients was very surprised to hear that he did not need a prototype to start
consulting with users and he could in fact use a statement of the Product Concept at a very
early stage. They were part way through development of a home energy monitoring App.
They could see how this would save time and energy and focus them on spending their time
developing the right features.
So, you do not have to show people a prototype, you can just talk about your concept - try
to define it in one paragraph. It may help to write your concept up as a statement / user
story
"...As a user I need something that does "x" so that I can "y"..."
For example, "...As a user of the Boris Bikes scheme, I want to be able to easily find a free
bay to dock back my bike so that I don't have to waste my time trying to find one..."
You can also show functionality that you haven’t yet beautified on the front end (like the
energy App mentioned above). In fact, people are more comfortable to give you
constructive feedback if they see the product isn’t finished yet.
October 2011 Product Doctor Insights from Over The Air 2011

3. 3
3. How to find your end users
On questioning, every patient could identify what sort of people they thought would use
their product, but very few have checked their ideas out with end users that are not
relations or friends. Note that those close to you will want to be encouraging and
supportive and so their feedback may be biased.
Here are examples of how to find your users:
 ...for the Boris Bikes scheme App - go stand at the bike docking bays and chat to
users there;
 ...for an App to learn to Chinese - go to International House and talk to students and
teachers
 ...for navigating around a festival - go to a festival!
Once you talk to users you are likely to understand more about where your product sweet
spot is. You will find people who you thought would be interested in using your product that
are not; and you are highly likely to find scenarios and other target groups that you would
not have thought of. In my 10 years of talking to users on a large range of digital products I
have always found out something I had not imagined!
With many products, it makes sense to test in pairs of friends. Often the conversation
between the two respondents will be very open and you will gauge more open and honest
feedback from the discussion between them.
In some cases, it may be appropriate to ask your friends / family if they know anyone that
fits your criteria. Importantly, unless your product is specifically aimed at the tech
community, you really want someone outside of our industry. Sometimes, even tweeting or
facebooking who you are looking for can help.
You may also find groups around particular interests that may be happy to test out your
product – you can find them in Meet Up; Facebook groups; Ning and so on. They may be
happy to do this also without you have to pay them. Perhaps showing up with a tray of
donuts will do the job! If you have a product that is targeted to the youth, talk to me about
how to work with schools / colleges / universities as these sorts of exercises can double up
as learning experiences for young people. Many teachers really appreciate real life products
and people to use as case studies not only for enterprise skills development but also to
compliment particular courses, such as Product Design, Technology, Marketing, Business
Studies etc.
You may well need to reward the respondents. You can offer cash or a voucher; the amount
obviously depends on how long the interview is going to be. For the last set of user groups I
ran, I offered £40 for a 90 minute interview, so around £15 for 30 minutes. Sometimes a
free pint in the pub works – again – if that is where you can find your target group.
October 2011 Product Doctor Insights from Over The Air 2011

4. 4
4. Can you have the conversation with end users?
Some of our patients expressed concerns about their comfort levels with facilitating the
conversation with end users. If you are not comfortable, you could ask someone else - a
friend or family member who you think could help you. Of course, you can always ask
Product Doctor as we may be able to do it for you on an affordable hourly rate!).
Getting feedback from user is a practised skill; something I have been trained to do and
have practised over many years, but if you do not have any budget to hire someone in,
please try to do this yourself. H0nestly, it is better than not doing it all. Take a recording of
the interview – on your phone or on a dictaphone – that you can play back after the event.
You will pick up on comments that you missed at the time and also be able to improve on
your interview techniques, for example you will hear where you might have “lead the
witness”. If you want to be good at this, you can be and I am always happy to give you any
pointers you may want. You may even be able to provide a real life learning experience for
young people who are still studying – they could carry out the surveys for you – again, get in
touch if you would like to talk about this in more detail.
5. How to begin the conversation
Start by setting the scene; say (yes, lie if need be!!) that you did not develop this yourself,
you are just helping out a friend – people will be less likely to be honest if you seem too
invested in the product. Do not be passionate about the product. Do not appear to be
selling. Try to be as neutral as you can.
6. Write a test script
In addition to obviously being able to explain what the product is going to do, list out all the
assumptions you have made. This will include items like:
a. The purpose of the product / product concept
b. The user needs / desires that you think your product is meeting
c. Assumptions for financial modelling – e.g. how often the user is going to use your
product
d. The full features that you have / are intending to build
These are examples. Of course, you should also test out pricing, naming and so on.
So your questions will follow the above – example
a. Can you put in your own words, what this product does?
b. Which of the following needs can you relate to? Perhaps there are things we have
not thought of / how could the product help you? If it is not for you, do you know
anyone else that might use it? You can keep coming back to the user needs
throughout the interview as you will find they may well change throughout the
interview
c. How often could you see yourself using this product (based on scenarios you have
thought of above)
October 2011 Product Doctor Insights from Over The Air 2011

5. 5
d. Rate these features in order of importance to you – please do not rate any features
that you would not use at all. Are there any features you would like to see that are
not listed here? How would you prioritise those in this list?
Feature over-load is one of the most common mistakes I see being made.
Sometimes the killer benefit of the product is a feature that you are thinking of as a
side-product. This exercise will help you to find what is really going to make your
product stand out. One of our patients had an App relating to finding a taxi, which
had a number of different features. He had one feature that seemed particularly
unique and instinctively, it felt as if this could form the basis of the entire App
(patient confidentiality stops me from giving any detail).
7. “I can’t explain what my product does”
This was a wonderful quote from one of our patients. He had an idea that he felt would
help with person to person transactions. Since the rise of E-Bay we have certainly seen an
increase in person to person transactions. More traditionally, they have always existed –
direct purchases with no middle man - buying and selling cars and landlord/tenant
transactions. He talked us through his idea and then reflected, saying “I can’t really explain
what the product is” but he knew he was on to something. The prescription in this case,
was to go talk to people that have had experience of these person to person transactions.
Find out whether the pain he felt he could address really existed. From there, he could
develop a one pager for each of these user scenarios that he could pull out when presenting
what his product did. Stories are a great way to explain what your product does and how it
meets a user need. Products can be easily explained by exposing the solution to a user need
in a real life scenario.
8. Showing a prototype
As above, it is not always necessary to show the user something – testing out the concept
before build is crucial in product development. However, if you do have a prototype,
remember not to sell the product! Do not say things like “this bit is really cool”. Don’t look
excited, you are just trying to demo the functionality.
Once you have shared the prototype and worked through your questions, it is a good idea to
ask users how they would recommend you showed the product in the App Store. How they
would describe it and what screens they would show. You can also ask users how they
would expect to hear about the App. This will help you with your marketing plan.
9. Testing for usability
Once you have a prototype / something to show, you will also need to check usability and
design. The guide for usability testing is to list out a number of user tasks. Let’s take an
example: London Journey Planner App – so sample tasks might be
i) Find the route from here to Oxford Circus
ii) Find the current status of the Circle Line
October 2011 Product Doctor Insights from Over The Air 2011

6. 6
iii) Find the W7 Bus Route
These tasks should reflect all the different actions that users can take. Give them the task.
Then stop talking. Watch and listen. Try to not jump in and show a user how to do it. After
5 interviews (as per Jacob Neilson, the Godfather of Usability for his guidelines), you are
likely to see the same usability difficulties repeated. You will find the phrase “did that
behave as you expected?” to be a very useful question to ask. Try to replay for clarity “So
when you pressed that you expected….but instead it took you to….” Confirming what you
heard the user say is very important, you may find that you have made some interpretations
that are not correct.
Testing out design is part usability and part look and feel. Let your users know that they
should feel free to make suggestions and improvements as they go through. Again, try to
not feedback to their suggestions as to whether you like them or not or whether they are
technically possible. In some cases, user suggestions need further working through. So
hypothesise with them and walk through the change they suggest to make sure that it has a
logical conclusion.
In general, be very sparing with your words. State…Ask them what they think of that…Be
quiet and listen! If you find yourself doing any interpreting in your head, replay that back –
so “when I said that blah blah you said blee blee, does that mean bleugh bleugh?” Seek
clarity and try not to get side tracked in to a conversation off topic!
If users have difficulties with navigating through a particular feature, do ask them how they
would make it easier. Same goes for any icons that you are using. Just tips, and again, one of
my mantras, do not reinvent the wheel. If there is a good and well known user interface out
there that you can adopt for your product then use it. As much as users should be able to
navigate through an intuitive user interface, they love it when the interface is familiar and
will often suggest you should build it just like they do on x-another App.
10. Keep checking back with users as you develop and improve each new feature
Gathering user input is a necessary on-going exercise. If you are working with Scrum, you
can build it in to the sign off of the user stories for each iteration.
11. Build and test your product before developing your brand
I am a strong believer that you live and die by the strength of your product – product is king.
Brand and design are super important, but is nothing if your product is not meeting user
needs and desires and providing a great user experience. One of our patients was a
designer, rather than a developer. He had developed a brand before his product was up to
scratch. To me, this is the cart before the horse. I can empathise with developers that are
driving new product development as they have to fund developers to build their vision. So
many of our patients were developers hacking in their spare time – while they may need to
engage designers down the line, at least they are not having to pay out at this stage.
Recently I was asked to help a company come up with a name for their product, but they
could not explain in one paragraph what their product did. This illustrates the point well. I
do not see how you can get a product name or develop your branding without defining what
October 2011 Product Doctor Insights from Over The Air 2011

7. 7
your product does. Being product-centric, this view should not surprise you. Naming a
product and developing a brand is also an exercise where users need to be engaged.
12. Be honest with yourself as to why you are developing the App
I often meet young developers who are working on their pet App in their own time. They
may be doing this as their day jobs are not giving them the chance to work on particular
code or areas that they want to so they are supplementing their education and ability to
show their skill in their own time. Great. But in some cases, this is not about developing
profitable Apps. This is just about enhancing their CV – great to be able to say, “I developed
an App that has had x downloads and is rated as 4/5”. Leave it at that! Don’t drive yourself
crazy to find the revenues as that is not your core motivation. When I ask patients “what do
you see yourself doing in 5 years’ time?” very few can see themselves as running a company
that takes them away from their core love, which is developing! Many do not want to build
up the skills to market and PR their Apps for example. “Just leave me with the fun bit – the
coding”.
On the other hand, some developers absolutely are looking to make the killer App and want
to make their million. In these cases, it is really important to think through
a) The revenue model, examples
 This can include partnership conversations with revenue share
 Exit strategies where they may be able to sell their App
 App stands on its own two feet and revenue models work independently
b) The go to market strategy
In my opinion, you cannot rely on just putting your App in the App stores and hoping
for the best. You need to “DIY PR” to coin Lisa Devaney’s phrase – for example - how
to build up an online profile for the product and generate online noise so that for
example, industry bloggers write about your App.
Where are your target customers going to find out about your App – we had some
great examples of brainstorming with some of our patients of societies, online
groups centred around particular interests, magazines and known bloggers and so
on. There is no excuse; you can find groups on Meet Up, Facebook, Ning and so on
Additional Notes:
(i) Proving in the niches first
If you have a product, like Audio Books, that is going to have pretty much mass market
appeal, rather than try to tackle the market en mass, you can prove in the niches first
and work your way out from there. There may be some obvious places to start; or
perhaps places where the social media machine will work for you best; maybe you
already have contacts that can help you in those niches.
(ii) Nothing wrong with backing a few different horses
A few of our patients had products that could be own brand or white label. I am sure we
can all name many products that work in this way and we know the benefits of working
October 2011 Product Doctor Insights from Over The Air 2011

8. 8
with each tranche. Ultimately, the core functionality will be the same, with a different
wrap. If you have the capacity to manage both, then best to run with the two horses
and see what happens.
c) Your product positioning & understanding your competition
 How do you sit in relation to your competition – around half of the patients we saw
did not have an up to date grip on what was out there and how their product was
better / different.
 What are the 3 key product benefits to using your App
Again, no surprise here, but these entire elements can also be tested out with users. For
example for “go to market”, you can ask the users that say they would use your product
“How would we best reach you”. For Product Positioning “What do you think are the 3 key
benefits of using this product”?
I hope that you find this Guide useful, if you want to contact me for any clarification or
advice please do contact me:
T. 07956 376472
E. julia@productdoctor.co.uk
W. productdoctor.co.uk
October 2011 Product Doctor Insights from Over The Air 2011