Defining Experiments at OTA 2015

It was back to Over The Air last week and a new way to get people to think about reducing risks in the assumptions they make about people. I ran a very interactive workshop where people got to work on their own products. My intention is to make Product Doctor experiences INSIGHTFUL and ENTERTAINING so I was pleased to get a few laughs and the comments below:

“…a challenging but fun workshop on product development at #ota15…just a pity it wasn’t longer…I am going to use what I have learnt today at my workshop on Monday to design some research experiments”

OTA Session 2015

We were working on the honesty to admit assumptions we are making and then how to turn them in to SMART research experiments (Specific, Measurable, Attainable, Realistic and Timely). I didn’t use this acronym in the workshop itself so as not to expose my 1990s corporate experiences, but frankly, it does work well as a checklist. The participants worked in groups around great real-life case studies offered up by people working on them in the room using the process you will see in slide 4 of this powerpoint.

It’s great working with examples from the room – the range always fascinates me and we got to some really tightly defined experiments. The products ranged from testing out whether reminding people on a daily basis of their achievements improves their self esteem; whether children would still use phones if they had child-lock on them; whether an online whiteboard would improve willingness to study; whether different people in a business need different security levels on their internal systems; whether there was an appetite for business to buy “speed enhancing software for their e-commerce sites and finally how to improve take up of a co-working area “oystercard”.

Key areas of discussion were:

1). How important looking in the eyes is rather than using surveys when trying to assess pain levels and whether they are great enough to pay for solutions. How can you assess real pain without doing that?!

2). How easy it can be to get face to face in the natural environment rather than sending out a survey or getting on the phone. There seems to be a natural tendency to shy away from meeting people in the flesh, which is why we do a practice session later on in the workshop. e.g. if you are researching how people behave in co-working spaces, go there to observe and question people.

3). How easy it would be to do a smoke and mirrors mock up of a product to get it in to people’s hands and test out the reactions that you think it is going provoke. e.g. set up a manual text service sending positive messages to people rather than asking them whether they might like the idea.

4). Making sure that you are using evidence already available before designing your own experiments e.g. in the realm of education, there must be a lot of research showing the positive impacts of white boards used in the right way. Massive savings on time and money.

5). How important it is to research with customers who are buying the product, as well as the people who might just be using the product e.g. in the case of parents buying products for their children – research assumptions you have made with both groups.

6). How research output can be turned into the key proposition message. e.g. using quoted e-commerce site owners description of their pain as the proposition messages to others in their position.

So that was it for another year – always delighted to be a part of the grassroots hub-bub that is OTA. Big thanks to the participants of the workshop, all of whom got involved and to Dan, Margaret and Matthew & Team for having me back.

Product Doctor offers training, workshops, consultancy and qualitative research to help innovating product owners, marketers & entrepreneurs develop meaningful propositions, brands & products. Contact:

Scrum Masters running Experiments

I ran a session with the Agile Practitioners Meetup last week – it was an extract from workshops I run including those at The Mobile Academy.  It was a very effective demonstration of how to “co-curate” using examples and best practice from the room. With scrum masters from many different businesses including Read International, GiffGaff, Natwest, TFL, Squirrel and Pearson, there were lots of examples of assumptions being made about how customers feel, will behave and problems that they have.

Working in groups, each of these businesses was able to use the format in the presentation below to define experiments to reduce those risks. Critique flowed freely with good discussions took on topics like “Is Linkedin really a good way to recruit respondents?”, “How useful are surveys at enabling you to understand real pain felt by a customer?”, “How to use research sessions as a way to get Developers customer centricity”.

I urge Scrum Masters to share this approach with Product Owners as it will help them to truly represent their customers. It matters not if you deliver a product perfectly if it is not what the customer wants or needs.

Here is some feedback from a willing Scrum Master!

The Mobile Academy next runs from 1 Oct – 3 Dec at Idea London in Shoreditch. Get £100 off with code: Leanmob

Hoorah – the Industry responds to battery life problem…

At last, here is some response to the biggest issue that I find when talking to people about their mobile service: the battery life of their phone – but does it go far enough?

Wireless Charging

And hoorah for EE – here’s another great way to crack it (although no good for rural folk!)

 EE gives free portable battery chargers to all customers – The network is giving you a battery boost with a free charger you can swap for a fully charged replacement in any EE store

Full Article:

Start Up Advice

ffwd-logoYesterday I returned for an afternoon of mentoring on the first day of the new FFWD London Pre-Accelerator cohort.

Here are six themes that recurred:

1). Where is it you personally are trying to go?
I asked a number of founders where they actually wanted to go with their product – what was their personal vision? Some were after an exit and others wanted to be the CEOs of their business seeing their roadmap through to highly profitable businesses.

The responses were interesting in that not many had considered this question and once we played out the various answers, the relevance of asking the question early on became apparent. For example, take a business offering software packages to automate accounting processes: If they were building to sell out to one of the big four, they are more likely to focus their product set and proposition to client and personnel within those large organisations. If they were to be a self-managed independent business, they might be more likely to adopt a licence fee business model that caters for smaller sized firms and even to consider targeting the end clients / companies who employ the accountants, with offers that could reduce their accounting fees.

2. The Gift of a Physical Offering in a Digital Space
If your offering has a physical element to it there are a whole range of opportunities open for you to engage with and market to your end customer. Example: a “find a flatmate” business – why not set up a Meet Up group that invites those looking for flatmates to get together and meet each other? What a great way to find out what questions they ask each other – informing the questions you are going to build in to your online / mobile form. You will also be able to spot trends that help you to identify your market segments – as an example, perhaps this is very attractive to females of a particular age and situation?

It is interesting to draw a parallel with, who started fully online and have now integrated physical meetups and events to bolster their offering. As an ongoing strategy, having a physical element to your offering helps people to engage more deeply with your brand as they meet you, feel the atmosphere that you create and at the same time, help you to develop your offering. Win Win!

3. Understanding all the People in your Universe: People Centred Innovation
As we try to identify our target user and customer, we also come across different people that are in our scene / our “universe”. I tried out a quick exercise where some of the founders listed out the different groups of people that are in their universe. These are not necessarily customers or users, but potential collaborators, influencers and people in the value chain. Understanding all of these different people is incredibly useful. Profiling who is in your scene is an important piece of research – it can challenge what category you have put them in to – perhaps you discover new distribution channels or new end customers. This really can only be carried out in person. Look in to the eyes of people; feel their emotion; see their response; understand their motivation; then you can talk from a position of knowledge rather than assumption. I guarantee you will always find out something that you didn’t know.

Example: A discussion around the accounting landscape identified that a group initially thought of as potential customers could actually be a distribution channel.

4. Start on your Doorstep
The start up London scene is immense – with so many accelerators, courses, incubators and so on. If you are lucky enough to have a proposition where your target user is in this scene already, then it is somewhat of a gift – you are already on the inside! For example, the people that run the FFWD London programme are linked with a lot of the other players on the scene and no doubt have ways to get to the various co-ordinators of the many programmes and their databases. So, use it!

Another variation on this theme of doorstep starting with a fashion duo working on a new idea to offer affordable but high end dresses. They suspect their prime segment to be working women in their early 30s – many of whom you will find in this scene – a great place to start. An interesting follow on question came from this fashion duo – they knew that it was not a great idea to carry out early concept testing with their friends – although they are target market. The advice in this case is to ask friends and colleagues to introduce you to someone that fits their particular profile, so you are quite rightly more removed from the respondent.

5. Opening the Kimono
Creating an MVP is critical to getting your product out of the door. As we all know, it enables you to gather early feedback, create an early adopter base who will help co-design your future product and it starts your brand journey. I found myself encouraging some of the founders to be bolder in their brand vision. I think that it is good to sell a vision as long as you are not making promises to deliver particular functionality in specific timeframes.

Example: An App that enables restaurant go-ers to find out more about the menu is part of a broader vision to bring smaller restaurateurs the kind of analytics that are available to larger establishments, without the accompanying price tag. The business model is that the restaurant will be paying to be on the system. So why not sell them the broader vision; they are an early adopter of the first feature set and the intention is that they are in for the long haul journey.

Open the kimono – give them a flash of what you have – there is a strong marketing message associated with being the first in. Often benefits (usually price related) will follow for them in the longer term. Again, these early adopters will be the ones that inform your future product. Stay close to them. There is no harm with labelling your MVP – “BETA” is a known and accepted label for example – it means that there may well be clunks and a limited product set as this is an early stage of the product.

6. Don’t hide your Light under a Bushel
I met three founders who were building products based on their own insider-industry experience. With so much digital innovation out there, it is really important to show credibility and not be afraid to shout about it!

Example: One of the founders is 25 and is building an online training offering for entrepreneurs of “lifestyle” businesses. It sounded initially clichéd, with concepts like “gamification”. I boldly asked his age and what experience he had that qualified him to be coaching others. He went on to tell me (in a very modest way) how he had built up three successful offerings all profit-generating. I suggested that part of his elevator pitch would be to start with a bio of his top three successes (I don’t apologise for always thinking in threes!). Instant credibility. Not just another online training platform: a business built around sharing his personal experience.

Making the call on Usability fixes

Getting input from a wonderfully engaged audience!

A great question from the back!

We got in to one of my favourite discussions at one of my DIY User Research workshop at FFWD London.  I demonstrated how to run a number of different types of research and had the group practice for themselves.  One great question was around how diligent to be on fixing user experience glitches.

Years of research with the youth market has shown that they do not necessarily like to use products that are too “scripted” – at the extreme, experiences that have in-built tutorials. They tell me that half the fun of starting to use a new app or product is to work out how to use it. That is part of the “game” (their language). They enjoy learning how to use a product and are happy to find their own way around. So fixing every single little glitch might not be a good thing to do – crazy though it may sound.

Now having said all of that, I need to temper it

1) “Hygiene” processes need to be as simple and pain-free as possible. Examples of “hygiene” being registration, sign in, and most importantly, payment! Serious impact for your business if there are any issues here.

2) It does somewhat depend on the product and the segment. If you are a mobile banking app, people are using you for efficiency and convenience – and a glitchy experience is going to put people off and reduce their confidence – the last thing a bank needs. This is also a good example of where you need to think about the segment. My first point above is a younger user, but it would seem that older users, experiencing a banking app would not really find working it out “fun”. Annoying, more like.

3) The product could be SO worth it! My example here is Pinterest. I love Pinterest. It is really useful for me to discover and organise home decor images for my pending house move (fingers crossed!) and I love the convenience of being able to share my boards. There are lots of incidents where I feel that the product is not intuitive and where I am forced through tutorials for new features. But I love it – so useful – and honestly, my love is not diminished by these usability downers.

I love running these workshops – the conversations that flow, the light bulbs that go off and the questions that make me think. Please comment away on this post.


These questions came up in a recent round of Drop in Surgeries that I held and I thought they may be useful to share.

1. How do you test an idea where you are creating a desire, rather than providing a solution to a problem? 

Firstly, you need to work out the user’s context: where are they / what are they doing at the point that your product becomes relevant to them.  When you think of it this way, you may realise that you are giving them a more efficient / more entertaining way to do something that they already do.

Next, you need to work out how you simulate the experience so that they understand what your product/service does. What level of prototype do you think that you need? Less is more. If you can get away with a paper prototype or even a storyboard of the experience, all the better. Do you really need to go as far as to create a beta of the product? Then you can follow the advice that I give on how to set up and interact with people to get valid and useful feedback. 

2. How do I work out what features to include in my product?

Think about who your target customers are. You should have a number of different personas that you are using to help you focus. I suggest no more than 5. Who are your core target segment? Do you have 1 or 2 that you can “bullseye”? If so, you can run feature prioritisation exercises with them. This works well in a group session where you can also test brand, look and feel.

So you ask them to make a list: “What do you think that this product should do?” Then you show them what the product does. Next you ask them to prioritise all the features on both lists and come up with one list of prioritised features (and not to prioritise anything that they do not care for). Run this individually first, then try and get the group to come up with one agreed list. It is the conversation that is really important for you to listen to. You can also ask the group where the line is between the “must haves” and “nice to haves”. 

If your product is already live and you have an active engaged base, you can run this exercise online, asking respondents to put features in order of importance to them and then offering a free text box for anything that is missed out. It is a slightly different result, as you cannot easily get them them to prioritise anything that is not already in your list, but very useful for getting user input to your feature roadmap, especially if you are trying to develop the customers that you already have. (Short answer!)

3. Will I really be able to run my own research as you suggest in the DIY workshops?

Well this is the challenge. The Princeton professor that trained me said I would never be able to do it. She said that I was too passionate about my product. I knew that being able to carry out research with customers direct was going to improve my performance as the product owner so I was very determined. I proved her wrong.

There are a number of core skills that you need to carry out effective user interactions; that include being able to put people at ease; being totally impartial and being able to listen. Not everybody can do it. If you can, then great. If you can’t then find someone else to do it for you. Don’t opt out all together. If you are trying to keep costs down, you may still be able to write the brief, just find someone else to run it for you. I have plenty of other tips on how to do as much as you can on a budget, but still get valuable input. Please get in touch if you want to know more.

Product Doctor Diagnoses – OTA 2012

Alex Craxton visits the Product Doctor Surgery

Here’s the report from this year’s Product Doctor Drop in Surgery at OTA 2012.

Another interesting range of products; from making a good old phone call, through to tracking housekeeping budget, m-health to enhanced status posting and finishing with around the world travel.

From what I saw in the surgeries, a few trends were certainly coming through:

  • incorporation of scanning technology
  • the continued growth of products to support social networking status posting
  • m-health becoming a reality
  • increased adoption of value added mobile services by the corporate market
  • revenue models from businesses rather than individual spend

Diagnosis hinged around some familiar threads –

Tom Hume drops in to talk shop

1). End User Validation– making sure that user insights are gathered at concept phase and continued user testing continues. The point, as always, is that this is not just usability testing, but testing that the overall concept you have.  Identifying user need and desire, supporting revenue models and product feature set all need to be validated before you go and build your product.

2). Ensure it is a Genuine End User – friends, family, established business contacts and friendly existing customers do not count – they don’t want to upset you.  Remember also, that you are not representative of an entire segment – building something on your own needs is not validation.

Please see “DIY User Engagement” for more guidance.

Paul Moutray gets medical

3). Revenue Modelling – Really think hard about where the pots of money are; this year there was more talk about collecting and providing customer information to brands and generating sales leads for brands.  In this climate and market, a product really has to be amazing for an end user to want to pay for it.

4). Know your competition – make sure you understand who is vying for your customer money or attention.  Think hard about what you think you are selling and question whether it is already being provided today.

5). Just because you can, doesn’t mean you should. Technology brings many new opportunities and there are some very clever developers out there, but please check out the commercial bases before you give up your job and start building a new product.

There are a couple of other points that struck me this year. I thought about how useful it could be for my patients to hear each others session. Some have experience in areas that others have not and that “share” could have been helpful.  Tying this together with some feedback last year that this felt more like “product therapy”, I am wondering about running group surgeries next year…