6 Usability testing platforms rated best to worse
As a UX Designer, once you begin work on a certain project you always end up needing to perform usability testings on wireframes, prototypes, concepts, ideas or existing products. Doing so will uncover many interesting unknown facts about the product or validate already existing ones. In many cases, businesses do recognize that there is a need for usability testing in the product design workflow but almost every time they do not want to spend money on it.
The usability testing challenge
I was recently asked to find a remote usability testing platform that would fit the business needs and budget. But, what budget? There was no budget for testing planned at all. Because of that, the first step was to actually request some amount of money approved that I can use to spend on research tools and software.
After securing that, I started doing research on what platforms are out there, what features they offer, and how much they cost. I've put together a list that you might find useful and save you time in doing such research.
The information I have gathered below is relevant as of March 16, 2018
1. UserTesting.com
Subscription Cost: From $15,000 / year. Some of their other plans can go up to $25,000 per year.
Cost per participants: FREE. No additional cost is required to pay to add participants to your study. If you are unhappy with a participant it can be swapped free of charge.
Number of Studies: 12 / year. Unfortunately, this is quite a downside to have only 12 studies per year per organization
Account access: 3 people. You can have 3 people from the same organization running studies at the same time and having their own accounts
As part of your subscription, you also get participant screener questions. You can choose the device you would like to test on - desktop or mobile. Also available to filter by are options like age range, country, income, web expertise, and more
Personally, I believe this usability testing platform is great and they have over 1m testers across the world. Mainly in the United States but their panel size is quite decent in Australia, the United Kingdom, Canada & India. Although they are quite expensive, whenever you launch a test with them, the results come in within an hour which is quite handy if you are expected to deliver quick results. If you are unhappy with a video or a tester for some reason they would change it for you free of charge.
The experience while watching and analyzing a video is quite good too and offers features like adding notes, making little clips, and creating a highlight reel with the most important comments from all the videos in your study. That doesn't happen yet automatically yet but we can't have it all, can we!
If the business/project you are working on can afford this software, I would definitely recommend it! I was lucky to use it while I was working on a project to improve a design and the user experience for last minute deals listing on a travel agency website.
2. Validately.com
Subscription Cost: From $2,388 / year. Although this is the lowest plan it offers quite some features, it's also worth checking what their other plans are.[/toggle]
Cost per participants: $15 for unmoderated / $150 for moderated. Although you pay here for your participants separately, they offer you two types. Unmoderated is for remote usability testing where people think out loud and follow a script. Moderated is when you invite people on 1:1 video calls.
Number of Studies: 36 / year. If you run out of studies, you can top up your allowance for extra $588 and you will get 36 more studies until the end of your billing cycle.
Account access: 3 people. You can have 3 people from the same organization running studies at the same time.
As part of your subscription, you also get up to 2 participant screener questions. You get to choose the device you would like to test on - desktop or mobile. Choose the demographics of your participants - age range, country or if they have kids or not. You also get a very sweet option to create insight reports where you can add short clips with highlights and comment on them that are fully shareable.
Probably the best usability testing platform that you can find and will match expectations vs. costs and turnaround time of results. When you set up a test, results come up quite fast, probably in about 2 hours or so although they say it might take up to 5 business days. Their UI for previewing and analyzing videos can be improved but it is not bad or difficult to use. They also have good customer support if you have any questions that you would like to ask. They offer testers in Canada, the US, Australia & NZ, the UK, and the Rest of the world combined.
3. WhatUsersDo (Discontinued)
Subscription Cost: From $10,500 / year. They also offer a pay per credits plan. You can purchase 10 credits at a time for about $700 and exchange 1 credit for 1 participant in a study.
Cost per participants: FREE with the subscription"]If you have the subscription you don't pay anything for the participants in your studies but if you use the pay per credit option you would have to pay 1 credit for 1 participant. You can buy 10 credits for about $700.
Number of Studies: Unlimited. This is great!
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
As part of your subscription, you also get participant screener questions. Choose the device you would like to test on - desktop or mobile. Choose the demographics of your participants - age range, country and more.
Overall I think WhatUsersDo are great! Personally, for me, the downside is that they do not offer testers in Australia. If you are working on a product that has it's target customers based in Australia then this online software would not be suitable for your needs.
I think the best part of this usability testing platform is that when you sign up for an account, you can also sign up for their Slack Community and interact with other UX researchers. You can also talk directly to expert researchers from WhatUsersDo. I have personally used the community to discuss research approaches and methodologies. The community is very active and friendly and it's worth being a member of.
This platform was acquired by UserZoom Inc. so I am expecting more exciting features to come forward.
4. UserZoom.com
Subscription Cost: From $28,000 / year. These guys are not kidding! That is the lowest they offer and it allows you to do tests only on a desktop. If you want to do mobile testing as well be prepared to pay a sweet fee of $40,000 - $60,000 per year
Cost per participants: $30-$50. If you are using participants on a desktop would be around $30 but if you are using participants on mobile would be $50 per person.
Number of Studies: Unlimited. It better be for that annual subscription fee.
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
As part of your subscription, you also get participant screener questions. Choose the device you would like to test on - desktop or mobile. Choose the demographics of your participants - age range, country, and more. You can test with your own participants as well and run unlimited studies.
From what I have looked at, this usability testing platform seems to offer a great range of research tools all in one software which is great but if you are looking to make just usability testing I wouldn't recommend it. I never had the chance to try it but I have a feeling that for the price you are asked to pay it should be good! If your business can afford it then give it a try.
5. TryMyUI
Subscription Cost: From $3,588 / year. They also offer an enterprise subscription. If you are interested you can call them.
Cost per participants: 1 CR for desktop / 2 CR for mobile"]With your subscription you get 10 CR a month
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
As part of your subscription, you also get participant screener questions. Choose the device you would like to test on - desktop or mobile. Choose the demographics of your participants - age range, country and more.
I tried to set up a study with these guys but the UI and the UX of their usability testing platform made me give it all up. It's quite slow and super difficult to use. I mean, setting up a study is hard on its own and you do not need the complication of a badly designed product to add more stress to that.
6. UserTest.io
Subscription Cost: None. Probably the first platform as of yet that doesn't require an annual subscription cost to use it.
Cost per participants: $21. Pretty good price! No catch!
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
At the moment they do not offer anything but watching the videos. If you'd like to screen your participants you would have to contact their support to do it for you.
This platform was acquired by UserPeek.
Overall the usability testing platform has been working well but it has quite a slow turnaround time for videos. They have around 5k testers worldwide which reflects quite a lot on the timing of the tests. There is no option to screen your participants so for every test you set up you need to contact their customer support to do that for you, even though they are quite responsive sometimes your tests end up with people outside of your requested age range or screener requirements.
Read more about 10 Best Customer Experience Management Softwares of 2020
Redesign of a travel last minute deals and discounts page
Recently, I joined a business in the travel industry called Contiki. They offer guided tours and travel experience packages for travelers between 18-35 years old. As my first project as Senior UI/UX Designer at Contiki Marketing, I was assigned to redesign their existing last minute deals page. It is used to show trips that have discounted prices and are departing within the next month or so.
As every product evolves now and then it changes its overall design. Sometimes you can't transition all its features in one go, you have to do it step by step. The Last Minute Deals page's migration to the new design was one of these steps and it looked like this:
The Story Behind it
This page is a very important part of the product for selling to people who love discounts or would like to travel somewhere within the next weeks, the business has decided that we need to create an improved version of it.
Normally, if a person is browsing through the website the user journey of booking a trip would look like:
In the case of the last minute deals and discounts page, the individual trip page is meant to be secondary. Normally that page will contain all trip details including pricing and departure dates but in this case, a user is taken to the booking straight from the desired departure date.
Redesign Challenges
And here the first challenge of this redesign process arise:
What information should be shown about the product to motivate a customer to engage with the CTA and perform an action?
If you add all the information of a trip the card will become too overwhelming. It would have no greater value over the decision of the traveler compared to when the card would contain just the right information.
But how do you decide what is the right information?
- The answer is simple - just ask your customers.
Well, it's not that simple as just doing whatever your users might want, it's more about combining the customer's feedback with what the business wants to promote and then a good balance is found - you'd end up having a final functional design that satisfies the need of both sides - the business and it's customers:
The other challenge in this redesign piece was the layout of the card.
What should be the information hierarchy and the layout of the deals card? What would create that subconscious motivation in a traveler that will make him desire to book a trip?
Since it's a page about trips and discounts the users have already a "preset" in their minds that they would expect to see trips at discounted prices. Because of that, I have put the savings value first, with an attention-grabbing design.
An interesting thing here was that through user testing I found out that if the savings amount is too large people start to confuse it with the price of the trip, therefore that label had to have very balanced design between attention-grabbing properties and significance.
Designing Variations
Before the final design of the card was chosen, I did spend some time in making different variations that I was testing with customers to get feedback. I also observed how they interact with the information presented on the card. For my tests, I used the remote usability testing platform UserTesting which is one of the 6 usability testing platforms I use.
Variation A
This version focuses on a large promotional image while combining the already existing sliding cards from the left to the right where they show information of the itinerary on a map view, as well as the locations, visited. But after testing and consideration, this version was ruled out as it was creating too much noise on the listing page where you need to discover things fast.
The user testing results on this card showed that:
Only 40% of the users saw the savings label and understood that this is a deal
Only 20% of the users felt confident to book a trip. For the rest was unclear what's included in their trip
Only 20% of the users used the map
Now 20% of the users are confused that the highlighted "SAVE" price is the price of the trip
Variation B
This variation was featuring the trip image on the left which is conflicting with the visibility of the savings label. Based on the research performed with the previous card layout, I have made a decision to put the map as a secondary option therefor it was added in a tab format with the ability to switch between the ITINERARY and the MAP.
Now 60% of the users saw the savings label and understood that this is a deal
Only 40% of the users felt confident to book a trip as they understand where the trip will take them. For the rest was unclear what's included in their trip.
Only 20% of the users used the map
Now 20% of the users are confused that the highlighted "SAVE" price is the price of the trip
Variation C
This was the final design that was chosen for the layout of the card. The map and itinerary tabs were removed because that would clear the noise in the design.
Now 80% of the users saw the savings label and understood that this is a deal
Now 80% of the users felt confident to book a trip as they understand where the trip will take them and what's included in their trip. The rest felt uncomfortable clicking on a "BOOK NOW" button out of fear that the website will take their money before they can review their booking.
Now 80% of the users understand what is the trip's itinerary
The final design of the redesign
Through iteration and feedback, I managed to finalize the design for the page
I wonder in an A/B test which variation will perform better? Such was not performed in the design phase but I would be curious to find out with real numbers.