How we conduct user research

# Introduction

This guide explains how we do user research at the NHS Business Services Authority, it applies to all user researchers both staff and contractors.

It builds on the guidance for user research in the GDS Service Manual, allowing us to build services that meet all 17 points of the NHS Service Standard.

The guidance builds on our Service Design Team Values.

# User research workflow

These are the things that user researchers usually do on projects.

1. Help setup the project and team
   Researchers get involved with project inception and planning. We review existing evidence. And we create research plans that show what we’ll do, why and how we’ll do it. We make sure the plan is achievable and allows us to do research at a sustainable pace.
   This helps teams to focus on understanding and meeting the needs of users, and to produce the best results with the people, time and resources we have.

2. Plan and do research activities
   This is the most obvious and visible part of our work. We choose research activities that provide strong evidence and reliable answers, for the least time, effort and cost. And we choose activities that help our teams and different kinds of users to participate.
   We record what we do and how we do it, for each other, and for those who come after us.
   This helps teams to embed regular research into their agile practices and get the best results from the effort they put in.

   • How we write and use discussion guides
   • How we take notes and record research sessions

3. Analyse research and produce findings
   Researchers carefully analyse the different kinds of data we collect. We analyse data in batches and record our findings from sprint to sprint. And we involve our colleagues in analysis to make sure our findings are clear and relevant.
   This helps teams understand, believe in and act quickly on what they learn.

4. Help teams make good decisions
   Researchers produce findings and other recommendations that help teams make good decisions about what to do next and how to approach it. We get actively involved in identifying opportunities, creating design ideas and setting priorities.
   This helps teams to create great services that work well for users, and to strike a balance between short-term wins and more complex, long term changes.

5. Create and share outputs
   Researchers create and share formal outputs like plans, presentations, reports and diagrams. These record and communicate the important things we learn. We create finished outputs from sprint to sprint, not just at the end of a project. And we think carefully about how we present and share these outputs to produce the most impact.
   This helps our teams and other stakeholders use and benefit from what we learned both during the project and after the researcher may have moved on to other things.

6. Take care of research data
   Researchers know it’s important to manage research data carefully. We follow our privacy policy and ISO27001 certification when collecting, processing and storing research data to make sure we comply with that General Data Protection Regulation.

   This helps teams to get the most from the research data they collect, while protecting the privacy of the people taking part.

# Recruiting research participants

Recruit people for research
Mainly the User Research Support Assistants or sometimes reserachers themselves, recruit appropriate participants for research activities. Finding the right participants at the right time is always challenging, so we start recruiting as soon as we can and use a variety of approaches to reach everyone we need. We keep track of who participates in our research, so we can be confident that we’re including the right people.
This helps teams to learn about all their users and produce services that work well for everyone.

• How we get informed consent for user research

• When and how we give incentives to research participants

# How we name and identify documents

It’s important that team members and stakeholders know they have the right research plan, discussion guide or other research document.

So we give our documents consistent file names like:

• YYYY MM DD - PROJECT - TYPE - RESEARCH ACTIVITY
• For example, 2020 12 01 - NHS Jobs - User research plan - GP practice onboarding Beta.

And a consistent front cover with:

• NHSBSA logo
• Project and phase
• Document type
• Date
• Main contact (who wrote it)

For example:

• NHS Jobs - Beta
• User research plan
• (Draft)
• 1 December 2020
• Sarah Stokes

# How we write and use discussion guides

We create discussion guides to help us plan and do research activities, like interviews, visits, workshops and usability tests.

They help us to:

• identify the purpose of the research activity and what we want to get from it
• review planned activities with team members and stakeholders so we stay aligned with project goals
• stay on track during research sessions
• make sure different researchers cover the same topics so participants have a consistent experience
• keep a record of what we did in each research activity

# The parts of a discussion guide

# Introduction

Recap the goals of the project and the research activity, describe the methods you’ll be using, and set out the overall structure of the research session.

# Preparation

Describe anything the researcher needs to have or do, to prepare for the session. Reference things like the information sheet to send to participants, describe how to set up the prototype you’ll be testing, and reference the notes template you want to use.

# Welcome

Set out the things you’ll say to the participant to let them them know what’s going on and make sure they feel comfortable. It can be helpful to write this out word for word, so you have something to fall back on when you’re tired, or get interrupted or distracted.

In your welcome you should:

• Say thank you
  Start off by letting the participant know you appreciate them offering their time (incentivised or not)
• Introduce yourself
  Let them know your name, role and who the NHSBSA is
• Summarise the research purpose
  Explain what you’re trying to find out, eg We’re interested in how you recruit teachers for your school
• Remind them it is voluntary
  They can choose to stop at any time
• Reassure tham that there are no right or wrong answers
  We’re interested in how the participant thinks and what influences that
• Ask them to be honest
  As researchers we won’t be offended by any comments, we just want to find out what people really think about a service and how it might be improved
• Describe how the session will go
  Let the participant know who is taking notes or observing, how you are recording the session, how long the session will take, and what you’ll be doing in that time, so the participant knows what to expect
• Check consent and non-disclosure
  Has this been done? Do they have any questions?

# Topics, tasks and activities

Have a section in your guide for each of the main interview topics, test tasks or workshop activities.

For each one write out:

• What you are trying to learn
• For an interview topic, the starter questions you’ll ask along with likely follow ups
• For a test task or workshop activity
• the steps to do
• explanations and instructions for participants, such asking them to think out loud during a test so you can understand what’s happening
• How long the topic, task or activity is likely to take
• The priority - is this essential, or could you leave it out if you’re short of time

# Wrap up

Describe how the researcher should conclude the session with the participant. This can include asking the participant for their final thoughts, asking them how the session went for them, letting them know what will happen next, and thanking them for their time.

Also include any steps for the researcher, like collecting and storing recordings, or deleting data from prototypes.

# Three discussion guide templates

We have template discussion guides for three different kinds of research activities:

• Interviews and contextual research
• Usability tests
• Workshops

# How we get informed consent for user research

You must have informed consent from participants before you start any research activity with them. And you must then act only within the consent the participant has given.

This guidance explains how we collect and keep evidence of informed consent at NHSBSA digital. It builds on the general guidance on getting informed consent in the Service Manual.

# Creating consent materials

NHSBSA consent materials must be used, unless we have been intructed to undertake research on behalf of another department eg NHSX.

• information sheet and
• consent form templates.
  NHSBSA should manage the research data and keep the associated records of consent, regardless if a we use a delivery parter to do th rserach activities.

Make sure your consent materials and process follow the good practice for informed consent described in the Service Manual.

# Creating an information sheet

For each round or batch of research you do, you should create a specific information sheet that describes the research you are doing, and gives the participant the information they need.

You can use the same information sheet for several rounds or batches of research, as long as you are:

• doing the research for the same purpose
• doing the same things during the sessions
• collecting the same research data
• working with the same third-party organisations and suppliers

We have template information sheets to use for the most common research situations.

# Creating a consent form

We have template consent forms for the most common research situations.

*• accessible consent forms **

# Sharing your consent materials and process

Some clients and stakeholders will not understand the ethical and legal reasons for getting informed consent, and will not be familiar with good consent practice.

To avoid problems later, share your consent materials and process as soon as you can with the client and with any organisations whose staff or members you are researching with.

# Collecting consent

# Providing the information sheet

Give each participant a copy of the information sheet when they are being recruited for a research session. Check which format and channel works best for them - email attachment, paper letter, easy read, etc.

For a face to face research session, make sure you can provide a paper copy of both the information sheet and the associated privacy statement/privacy policy, if a participant asks for one.

# Face to face research sessions

For user research sessions where you meet the participant in person, you can:

• give the participant a printed copy of the consent form
• send the consent form in an accessible digital format the participant can read using their assistive technology
• read the consent form aloudCheck with the participant which format works best for them.

Give the participant the consent form as soon as you can. Make sure they have time to read it carefully, and an opportunity to ask questions, before you ask the participant to sign .

# Remote or telephone research sessions

For user research sessions where you will not meet the participant face to face, you can:

• attach the consent form to an email in an accessible digital format
• send a paper consent form in the post

Ask the participant to send a reply, confirming their consent. They can reply to the email, or post back a signed paper consent form.

# During the session

At the start of the session, remind the participant that their participation is voluntary and they can stop or take a break at any time. Remind the participant about the different ways you are recording the session, and about anyone observing or listening in. Ask the participant to confirm their consent for the recording.
As the session continues, actively make sure you have the participant’s consent. For example, check in with the participant if they look concerned about an activity, or if you’re on a visit and want to take a photo in a new area.

# Keeping a record of consent

For paper consent forms, scan or photograph the signed consent form and keep the image file in the folder with the research data you collect in the session.

For email responses, keep a copy of the consent form and of the participant’s reply in the folder with the research data you collect in the session.

Make sure the consent form, email reply and any research data have the same participant identifier. Do not include participant names or other personal identifiers in the document names.

# When and how we give incentives to research participants

It‘s normal to pay members of the public an incentive when they take part in research activities with us.
This guidance explains how we agree incentives with clients and pay incentives to participants.

It builds on the basic guidance on giving incentives in the Service Manual.

# Who gets incentives

We will normally pay incentives to members of the public, including people who use a government service for their work. Professionals and members of groups may not be comfortable with getting an incentive themselves, and may want the incentive given to their organisation or donated to charity.

We do not expect to pay incentives to our internal NHSBSA colleagues or to other public servants who take part in research related to their work.

When we are paying an incentive, make sure the participant knows the amount of the incentive and how they will receive it. Include this in the [information sheet](link to consent guidance).

# Agreeing incentives

It’s best to include estimates of incentives and other research expenses in the statement of work.

You can then confirm the exact incentive with the client when you are planning a batch of research.

On previous project we have paid:

example from NHS digital below

• £30 of Love2Shop vouchers for 1.5 hours of in-person usability testing
• £50 cash for geographically-specific users to take part in 1 hour of remote mobile usability testing
• £100 cash for an hour-long discovery research session with people likely to need assisted digital support
• £100 cash for 45 minute research session with small businesses

# Paying incentives using shopping vouchers

Where we are paying the incentive, we use shopping vouchers. Normally from Love2shop.

Request the amount of vouchers you need through the … They should be in £5 and £10 denominations so they are easier to spend.

In a face-to-face research session, give the vouchers to the participant and have them sign a receipt.

For a remote research session, use the … Enclose the vouchers in a signed thank you sheet.

Handle the incentive at the start of the research session - either giving the incentive, or confirming that the incentive will be sent. This avoids any suggestion that participants have to ‘earn’ their incentive. It also means you don’t have to deal with the incentive if you want to stop a problematic session.

# Other ways to pay incentives

Other options for paying incentives might be a donation to a preferred charity, or a hamper that can be shared with a team.

# Withholding incentives

We do not pay the incentive (or any recruitment fee) if the participant doesn’t show up for their session. Although we might still pay the incentive if the participant tries to attend, but is stopped by an emergency or severe travel problem beyond their control.

If a recruitment agency provides a participant who does not match the brief, we will pay the participant their incentive, but we will not pay the agency their recruitment fee.

# How we take notes and record research sessions

To capture what happens in our research sessions we take notes and photos, make audio, video and screen recordings, and take copies of paperwork and other material that participants use or refer to.

We use these notes, photos and recordings during analysis to make sure we produce valid findings. And we use extracted quotes, images and clips to illustrate the findings we share.

There is good guidance on Taking notes and recording user research sessions in the Service Manual, and on Capturing User Research in UX Matters.

# Good practices to follow

1. Capture only what you need and have consent for
   Good user research practice, and good privacy practice, is to capture only the data you believe you need. For example, don’t make a screen or video recording, ‘just in case’, if you’ll only ever need a transcript of what the participant says.
   Make sure that anything you capture during a session is covered by the consent you’ve collected. If a participant changes their mind about what they consent to, be prepared to not make, or to delete recordings you no longer have consent for.
2. Do not use your personal devices
   Always use a NHSBSA provided device, rather than your personal device, when taking notes and photos, or making recordings.
   Using our own devices in research means that personal data about research participants can be mixed into our personal data stores, like cloud photo libraries. And also runs the risk of participants, client staff and colleagues seeing our personal messages and images.
3. Keep your notes and recordings secure
   As soon as possible after the session, transfer any notes and recordings from paper and other devices to your NHSBSA provided laptop. And from your laptop to the NHSBSA Google Drive or to the client’s research store.
   Keep only anonymised notes, quotes and images in tools like Miro and Trello.
4. Have a fall back
   The one certainty in user research is that things go wrong. So make sure you have other options to capture what you need.
   For example:
   • The participant may decide they don’t want to be recorded in a particular way, so be prepared to switch from video to an audio-only recording, or to just take notes
   • your preferred device, app or service may not be working properly, so have an alternative like a paper notebook
   • Your note taker, assistant or observer may not be able to join the session, so be prepared to take notes and recordings yourself
5. Note that if the participant refuses all recording, including note taking, you may want to abandon the session.

# Using your NHSBSA provided laptop

Whenever possible, use your NHSBSA provided laptop for taking notes and making recordings. Our laptops have encrypted drives and we follow good security policies.

With your laptop you can:

• capture notes to a local file or a Google Sheet
• record audio using a microphone and the Voice Memos or Quicktime apps
• record video using an external camera and Quicktime
• record your laptop screen using Quicktime or Screenshot

If research participants might use your laptop or see your screen during a research session, it’s a good idea to create a separate ‘research’ user account on your laptop with things like notifications and browser autofill turned off.

# Using a work device

Use a NHSBSA provided phone or tablet, rather than your personal device, to take photos and short videos during interviews, visits and workshops.

Remember to delete the photos and recordings from the device once you have transferred them to your laptop or to Google Drive.

# Using the notes template

We have a template for a Google Sheet you can use to take notes during a research session.

You can make a copy of the sheet for each research session, or make a copy for each batch of research and create tabs within the sheet for each research session.

# Using sticky notes and paper worksheets

We often use sticky notes and paper worksheets to capture information during research sessions, for example when researching users’ experiences.

Design paper worksheets, and give people advice on writing sticky notes, so you can easily use the information recorded.

Avoid capturing personally identifiable information about participants on sticky notes or paper. If you capture any sensitive informaton on sticky notes or paper, make sure to use a shredder to dispose of it.

Use a NHSBSA provided phone or tablet, rather than your personal device, to capture the contents of sticky notes and paper worksheets.

# Using a video conferencing service

Our preferred service for recording remote screen sharing sessions is MS Teams. Setup MS Teams meeting so that the recorded video is saved onto the User research library Stream channel or ensure that if using zoom or Google, make sure that the video is uploaded to the Sharepoint Research Library as soon as possible. Ensure that all videos are removed from your device.

# Using third-party facilities

If you are using a third party facility like a usability lab, transfer any recordings made by the lab to your NHSBSA provided laptop as soon as possible after the sessions are complete.

# User research principles

Our principles are not rules. They guide our work, keep us improving as a team.

1. Help teams understand people
   Researchers at NHSBSA help our teams build a deep understanding of the people who use our services. We know that‘s the best way to create public services that work well for the people who need them.
   We are facilitators, not gatekeepers. We actively involve our colleagues in research. And openly share what we do, how we do it and why it‘s important.
2. Find the truth. Tell the truth
   Researchers at NHSBSA create strong evidence and reliable answers so our teams can act with confidence. We are bold and focus on what‘s most important.
   We know we can learn things that are unexpected and challenging. So we communicate clearly and sensitively to help everyone make the best decision.
   (Credit to the great Dana Chisnell and the United States Digital Service for this one)
3. Take ethics seriously
   Researchers at NHSBSA know that the safety and trust of participants is our responsibility. We think about the ethics of our research at every step. From how we recruit participants and get their informed consent, through how we store and use the data we collect, to how we share our findings.
4. Be methodical, but not rigid
   Researchers at NHSBSA know that the quality of our findings depends on the quality of our methods. We use tried and tested methods, and take time to reflect and continually improve our practice.
   But we also understand that context is important. So we use the best approach for the question at hand and adapt our ways of working to fit the client and the project.
5. Learn, share and adapt
   Researchers at NHSBSA work in an agile way. We do research and analysis in small batches so we can continuously share and adapt to what we learn.
6. Make research inclusive
   Researchers at NHSBSA know how important it is that public services work for all the people who need to use them. We help teams understand the needs of all their users, and do research activities that everyone can participate in.
7. Build on existing evidence
   Researchers at NHSBSA help teams build on the knowledge and data they already have. We combine existing knowledge, poorly understood data and new research into a coherent picture.
8. Accept and admit constraints
   Researchers at NHSBSA do the best research we can within the constraints we have. We acknowledge and share the limits of our research and our findings. And we advocate for more research when it‘s needed to achieve the project outcomes.

# Programs you may encounter at the BSA

Here is a list of programs you may see in use at the BSA, your team will have its own preferences, but these are useful ones to be aware of.

What ‘should’ I be using?  
Ultimately, there are no set rules regarding what software or tools we should be using (with the exception of data storage). That said, there tends to be a steady rotation of software, websites and tools that URs use.

Debs Robinson has explained some of the tools she uses as a UR on the Medical Examiners project below:

• Trello – really useful to use as a way to organise your tasks.  I use it as a ‘to do list’ and it works great if multiple URs are on a project as you can both see what tasks need to be done.  I have it set up with different columns that I can move tasks between e.g. to do, in process, completed, queries, back burner

• Jira – as UR we never used to use this but were asked to join the board to ensure the whole team were represented.  We have our own research and design board (R&D) that UR, UX and CD use and its broken down into: to do, in progress, BA review and each of the disciplines has their own swim lane so it’s clear whose task is who’s.  We keep tickets simple and just include a small description about what we are wanting to do along with the aim, assumptions and any hypotheses.  We link each ticket to an epic and assign it to ourselves and the R&D board so it appears in our board (also shown in the overall team board). We add tickets for research we are doing e.g. recruit funeral directors, Prototype test X feature (these types of tickets might get added repeatedly as it’s always going to be something as UR that we need to be doing)

• Confluence – we have created a R&D section where we add information about key features that have required lots of design iterations/conversations.  This was done to ensure the whole team could see evidence as to why something was being done.  UR and UX update these pages.  We also have pages where we provide an update about findings from each sprint.  We can copy the aim etc from the Jira ticket and then we list the key findings and any recommendations.  We tag team members to these recommendations and ask them to update.  We also add a link to this page in the Jira tickets at the end of each sprint so the team can see full details

• Miro – we record most things in Miro as its really visual.  We create frames for everything we do so its easier to navigate.  We put the key findings from each UR session onto a board and theme them on the board.  We then hold team meetings where we take people through these findings and come up with recommendations and again record these on the board.  We conduct team workshops on Miro, create service blue prints etc.  Basically it’s a really useful tool if you like to see things visually and people are able to make comments so it’s a good way to keep a track on things

• Excel – We use an excel spreadsheets for our scripts and to record notes from UR sessions.  During prototype testing we used screen shots of the pages being tested and added the questions we would be asking in the cell next to the image and then we recorded feedback in the received in the cell next to this.  We asked any observers to record their own notes in the spreadsheet and the we use all this data and upload the key findings into the miro.  Having the screenshot and questions all together we found useful for observers and made script writing much quicker.  Our aim is to record any action/thought/observation the user does.  We also use excel for our UR calendar.  We add details of all UR sessions the dates, their participant number, what we are testing, testing method, how many observers are allowed, a space for people to add their name to this to attend and we then add the recording link afterwards so its easily accessible for the whole team

• Google Drive – this is no longer used as it is not seen as a secure place to store information (although I love it and prefer to use it as its great for collaborative working)

• Sharepoint – this is the place where you store all your information e.g. notes, scripts, personas, recordings, consent forms etc.  Literally everything you have you store here.

• Teams – we use Teams to conduct most UR sessions.  If participants don’t have access to Teams or would rather not use it they usually opt for a phone call but Teams seems to be the preference especially amongst professionals.  We use the recording facility within Teams which means your recording is automatically uploaded to Microsoft Teams.  You just need to go into this afterwards and update the name of the recording and amend who can access the recording e.g. allow everyone to view and add to UR interview group.  You can then download this file and upload into the relevant sharepoint folder”

You should have access to Jira and Confluence through using your @nhs.net accounts. For Teams and Sharepoint, you should have access through your Cipher account. For Miro you should use your @nhs.net account, but you will only be on the free version and may want to ask to get a license (see who should I go to for X). 
 
The list above is not exhaustive – there may be other things that you much prefer using. For example, you may ask your notetakers to take notes directly into Miro, or on a Word document. You may also like to print things out and highlight when doing analysis. It’s your call to use whatever you think will help you within your research, though of course there are tools that your team may prefer you use (e.g. in terms of visibility).

---

# Can you help us improve this guide?

This manual is constantly evolving based on your feedback.
If you’ve got a question or want to feedback, you can get in touch

• propose a change – read more about how to propose changes in GitHub