User testing script examples
Example user testing scripts to conduct user testing from.
Testing Plan
You always need a plan before you conduct a user test. A plan will define who you are testing, as well as the structure of each session (another word for this is a âresearch protocolâ).
Ushahidi typically structure a user testing session as follows:
Introduction and preamble - introduce the test, what you want to learn generally, and give the participant some guidelines. This also gives them a chance to ask you any burning questions before you begin.
Tasks - Observe the participant completing a set of tasks, usually about 3-5 of them.
General questions and closing - Any non-usability related things you want to know and a last chance to ask/answer questions.
The industry standard for task-based user testing with a clickable prototype is 5-8 people. Youâll know you have tested enough people when participants start to point out the same issues.
Example 1: Data export testing plan
Target User
Test ~5 users who are relatively expert users of the platform already, preferably also folks who match up with our âData workerâ persona e.g. a person who regularly works with data, reviewing, cleaning and/or analysing.
Ideally, we would test this with data workers from partner orgs in DREAMS.
Preamble
We are improving our data export features so that Ushahidi will be more effective for data work and M&E in your organization. Weâd like to get your feedback on some screens of a process where exporting data is important. We want to understand how easy or difficult it is to complete some tasks. Iâll ask you to complete 3 tasks and then ask some general questions at the end, if thatâs alright. The test should not take longer than 30 minutes.
Before we start there are a few things Iâd like you to keep in mind.
The prototype is clickable, but not fully-functional, so please donât worry if it doesnât function the way a normal website would.
You have no need to feel pressure here, weâre testing the designs, not you, so donât be afraid to explore and speak your mind.
You donât need to worry about hurting our feelings. We have very thick skin, so donât be afraid to be brutally honest.
It helps a lot for our analysis if you can think out loud and describe what youâre doing as much as possible while youâre using the prototype. Weâre not very good mind readers, and it will help us take notes and understand your thinking as youâre moving through the tasks.
We will be taking notes during the test, but there will be no recording of your voice or any video.
If at any point you choose to stop the session, thatâs absolutely fine. Just let me know.
Tasks
You could lead with a question where you ask the users to describe their experience of working with data and how much importance it has within their processes. May help to get them in the mindset to âwork with dataâ. Here we may extract some of the more general assumptions that we make as platform workers around âdata cleaningâ level of comfort of working the data in the platform before moving into the software they may feel more comfortable in (R studio/Excel/Spreadsheet programs).
Task 1: Locate data export items in the information architecture
Open a live Ushahidi deployment in production and ask the participant to point out/navigate to areas on the site where they would expect to find the ability to export all of their data. Allow them to explore for about a minute or so for each idea of where things are. They will think as they navigate.
âCan you tell us why this location came to mind?â
Ask for another place they might expect to find the ability to export data. Repeat step 2.
Task 2: Export data from the filter pane
Prototype opens on Data view with the filter pane open. Explain that they have filtered to include all posts in a particular order.
What would you expect to happen if you click on âExport filtered reportsâ?
Ask them to click âExport filtered reportsâ. How does this differ from what you expected? What do you think you have to do now?
Which of these data formats would you choose and why? Is there a combination of these they would download/export?
What do you expect will happen when you click âExport nowâ? Click button.
How did the result of clicking âexport nowâ differ from your expectations?
Task 3: Export data from settings
Navigate to settings
Which menu item would you choose in order to export all the data in your deployment? Why?
Click âExport dataâ
What do you think you need to do on this page? Have them complete the flow as they see fit and take notes on pain points.
What do you think âSelect fieldsâ means?
Click âselect fieldsâ and ask if this matches their expectations
How would you determine which fields to export?
Have participant complete export and ask for feedback, noting any pain points
Task 4: Assess exported file
Open an example file
Take a quick look at the exported file. How does this compare to your expectations?
If no, a quick description why and what would you do next? (re-export? change filters? export all data)
If yes, what would you do next?
General Feedback
What do you think overall?
If we released this feature tomorrow, how would it impact your work in your organization? If yes/no, ask for more information?
If you had to teach a colleague how to use these features, how long would it take you to teach them? Why?
Any final thoughts?
Example 2: HDX testing plan
Target User
Test ~5 users who are relatively expert users of platform already (if available), preferably also folks who match up with our âData workerâ persona. If not available the user testers must be tech savvy enough to understand the concept of âtagsâ they donât have to be experts in hxl or any kind of tagging language but they must understand the value of assigning a âlabelâ to a piece of data so that it can be âprocessedâ quicker. Users should know what spreadsheets are and be familiar with CSV as a term/file extension name. If we can test with someone self-identifying as partially-sighted or dyslexic this would be a good start to adding accessibility into our usertesting.
Ideally we would test this with data workers from partner orgs from COMRADES or organisations that have an interest in HXL and HDX already and have some level of familiarity. If they are too familiar we run the risk of the test being about what they specifically want from hxl/hdx and not about the UX of this system.
Preamble
We are improving the way that users can âtagâ and assign a label to specific kinds of data within an Ushahidi deployment so that the process of extracting meaningful insights from data will be easier. Weâre particularly interested in how clear the method of âtaggingâ is within our system. Weâd like to get your feedback on some pages of a process. Iâll ask you to complete some simple tasks and Iâll be asking questions as you complete these tasks. You have no need to feel pressure here, weâre testing the pages, not you, so donât be afraid to explore and speak your mind. In fact weâd encourage you to be an honest and talkative as possible. Weâll have some time for general feedback and questions at the end. Please wait until Iâve finished a question before clicking links or buttons if possible.
The test should not take longer than 30-40 minutes.
Before we start there are a few things Iâd like you to keep in mind:
The prototype is clickable, but not fully-functional, so please donât worry if it doesnât function the way a normal website would. If something doesnât work, try to explain what you were trying to do and expecting to happen. It helps a lot if you can think out loud and describe what youâre doing as much as possible while youâre using the prototype.
We will be taking notes during the test, but there will be no recording of your voice or any video. [My colleague name will also be on the call (introduce colleague) to take a duplicate set of notes just in case I miss anything while weâre talking]
We may ask you to slow down temporarily while we take accurate notes so donât worry if we ask you to go a little more slowly.
If at any point you choose to pause or stop the session, thatâs absolutely fine. Just let me know.
Task 1 - Previous experience with âtaggingâ (no screen required)
Q: Ask the users to describe what they think when they here the term âtaggingâ or âlabellingâ means in relation to websites, digital products, software etc.
Q: Ask users when they have previously had experience of âtaggingâ. What was the nature of the content? Was there any extraction of data afterwards? Whether there was anything they wish they could have done that they couldnât do.
Hypothesis, Do not share with user tester: Weâre looking for the users to describe either prescriptive tagging or freeform tagging. Do users expect to only be able to tag within a set of constraints or do they expect to write their own and those tags to be very intelligent.
Firstly, Imagine you work for a charitable organisation and are about download a spreadsheet of information that has been gathered about volunteers. These volunteers were tasked with handing out flyers about an event youâre organising. You want to download information from a system that collects feedback from these volunteers. The information that has been collected are things like where they handed out flyers, how many flyers they gave out and the date and time they were handing out flyers. I want you to remember that this scenario. If you need a reminder of this, just let me know.
Q: Can you show me where youâd look to download CSV spreadsheet files?
Q: (On Export page with 3 CTAs) Without clicking, can you explain to me what options you think are available from here?
Q: What would you need to do in order to feel comfortable choosing one of the three buttons?
Q: Can you tell me what you expect âhxl tags or attributesâ to be? Donât worry if you donât know, weâre looking for your best guess if youâve never heard of them before.
Q: If you havenât already, click a blue, underlined link.
Q: Were you expecting a page like this? If not how does it differ from your expectations?
Q: Spend a minute or so reading the information here. [Pause] Can you give me a summary of your understanding of hxl tags. (There are no wrong answers!)
Q: How would you get back to where you were?
Q: If you haven't already, please click the back button (you may need to click a few times on a ârealâ website.
Q: Imagine you want your volunteer data to include hxl tags.
(We can briefly explain here that hxl tags are a way of âassigningâ a âtermâ to a piece of information. Use Justinâs analogy of Instagram/Twitter hashtags!)
Q:Which option would you choose? (they can click at this point)
Q: As you choose this option, what are you thinking? What is your first impression of this page? (Should be on the first view of the export table)
Task 3 - Going through assigning hxl tags in the table
(You may want to repeat the scenario if tester needs reminding)
Q: Is it clear how you progress? What are the steps you would take to start this section?
Q: Can you show me how you would know if a certain section was going to be included in the eventual CSV download?
Q: Can you show me how you would add a HXL tag?
Q: (If they click on the âLeave emptyâ drop down) What do you think of the options presented in this drop-down list? What kinds of information does it give you?
Q: Can you add the hxl tag âactivityâ
Q: Can you show me (without clicking) how you might add a hxl attribute?
Q: (If they click on the âLeave emptyâ attribute drop down) What do you think of the options presented in this section? What kinds of information does it give you in relation to the tags? (Is the difference between tags and attributes clearer for the user at this point?)
Q: Can you now show me how you would add a custom hxl attribute? (Encourage clicking now)
Q: (on the âplease type an attributeâ field) What do you expect to be able to do here? (Go through the clicks in order to add the custom attribute)
Q: Has any information changed? Can you explain what you are seeing/understand?
Q: Can you now add a tag to âdescriptionâ. Can you point to a tag you might choose if you wanted to capture information relating to a âflyerâ (like a handout/leaflet/pamphlet) There are no wrong answers!
Q:(If they havenât already) Please click âitemâ and then add a custom attribute for âflyerâ.
Q: Can you show me how you would add a second attribute? (Once user find blue link) Can you now add the attribute âadultsâ
Q: Take a moment to pause, Can you describe your current thoughts on the process? Anything that has stood out to you?
Q: Please continue with the section âLocationâ and add a tag âgeoâ
Q: (Pause - see if they notice the list is less populated) Why do you think this list has less options? Q: (if they donât notice) Is there anything you notice about the drop down list? Why do you think this list has less options?
Q: Can you now add two hxl attributes âlongitudeâ and âlatitudeâ. (these are the last tags/attributes to add during the test the âexport to CSV and HDX buttons should be clickable)
Q: Take a moment to pause, Can you take a look at this screen and explain what you have just completed. Can you point out the HXL tags and attributes? Do you remember what they mean? If not can you give a good guess?
Q: Can you now show me how you would get a CSV file? What would you expect to see happen next?
Q: Can you tell me what you think âExport to HDXâ means? (Do people assume this is a file extension type? Do they understand itâs a whole system?) Go ahead and click âExport to HDXâ. Take me through your impressions of this message.
General wrap-up questions
Current Ushahidi user specific questions:
If we released this feature tomorrow, how would it impact your work in your organization? If yes/no, ask for more information?
All users can answer:
Any final thoughts?