As a UX Designer, once you begin work on a certain project you always end up needing to perform usability testings on wireframes, prototypes, concepts, ideas or existing products. Doing so will uncover many interesting unknown facts about the product or validate already existing ones. In many cases, businesses do recognize that there is a need for usability testing in the product design workflow but almost every time they do not want to spend money on it.
The usability testing challenge
I was recently asked to find a remote usability testing platform that would fit the business needs and budget. But, what budget? There was no budget for testing planned at all. Because of that, the first step was to actually request some amount of money approved that I can use to spend on research tools and software.
After securing that, I started doing research on what platforms are out there, what features they offer, and how much they cost. I’ve put together a list that you might find useful and save you time in doing such research.
The information I have gathered below is relevant as of March 16, 2018
1. UserTesting.com
Subscription Cost: From $15,000 / year. Some of their other plans can go up to $25,000 per year.
Cost per participants: FREE. No additional cost is required to pay to add participants to your study. If you are unhappy with a participant it can be swapped free of charge.
Number of Studies: 12 / year. Unfortunately, this is quite a downside to have only 12 studies per year per organization
Account access: 3 people. You can have 3 people from the same organization running studies at the same time and having their own accounts
As part of your subscription, you also get participant screener questions. You can choose the device you would like to test on – desktop or mobile. Also available to filter by are options like age range, country, income, web expertise, and more
Personally, I believe this usability testing platform is great and they have over 1m testers across the world. Mainly in the United States but their panel size is quite decent in Australia, the United Kingdom, Canada & India. Although they are quite expensive, whenever you launch a test with them, the results come in within an hour which is quite handy if you are expected to deliver quick results. If you are unhappy with a video or a tester for some reason they would change it for you free of charge.
The experience while watching and analyzing a video is quite good too and offers features like adding notes, making little clips, and creating a highlight reel with the most important comments from all the videos in your study. That doesn’t happen yet automatically yet but we can’t have it all, can we!
If the business/project you are working on can afford this software, I would definitely recommend it! I was lucky to use it while I was working on a project to improve a design and the user experience for last minute deals listing on a travel agency website.
2. Validately.com
Subscription Cost: From $2,388 / year. Although this is the lowest plan it offers quite some features, it’s also worth checking what their other plans are.[/toggle]
Cost per participants: $15 for unmoderated / $150 for moderated. Although you pay here for your participants separately, they offer you two types. Unmoderated is for remote usability testing where people think out loud and follow a script. Moderated is when you invite people on 1:1 video calls.
Number of Studies: 36 / year. If you run out of studies, you can top up your allowance for extra $588 and you will get 36 more studies until the end of your billing cycle.
Account access: 3 people. You can have 3 people from the same organization running studies at the same time.
As part of your subscription, you also get up to 2 participant screener questions. You get to choose the device you would like to test on – desktop or mobile. Choose the demographics of your participants – age range, country or if they have kids or not. You also get a very sweet option to create insight reports where you can add short clips with highlights and comment on them that are fully shareable.
Probably the best usability testing platform that you can find and will match expectations vs. costs and turnaround time of results. When you set up a test, results come up quite fast, probably in about 2 hours or so although they say it might take up to 5 business days. Their UI for previewing and analyzing videos can be improved but it is not bad or difficult to use. They also have good customer support if you have any questions that you would like to ask. They offer testers in Canada, the US, Australia & NZ, the UK, and the Rest of the world combined.
3. WhatUsersDo (Discontinued)
Subscription Cost: From $10,500 / year. They also offer a pay per credits plan. You can purchase 10 credits at a time for about $700 and exchange 1 credit for 1 participant in a study.
Cost per participants: FREE with the subscription”]If you have the subscription you don’t pay anything for the participants in your studies but if you use the pay per credit option you would have to pay 1 credit for 1 participant. You can buy 10 credits for about $700.
Number of Studies: Unlimited. This is great!
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
As part of your subscription, you also get participant screener questions. Choose the device you would like to test on – desktop or mobile. Choose the demographics of your participants – age range, country and more.
Overall I think WhatUsersDo are great! Personally, for me, the downside is that they do not offer testers in Australia. If you are working on a product that has it’s target customers based in Australia then this online software would not be suitable for your needs.
I think the best part of this usability testing platform is that when you sign up for an account, you can also sign up for their Slack Community and interact with other UX researchers. You can also talk directly to expert researchers from WhatUsersDo. I have personally used the community to discuss research approaches and methodologies. The community is very active and friendly and it’s worth being a member of.
This platform was acquired by UserZoom Inc. so I am expecting more exciting features to come forward.
4. UserZoom.com
Subscription Cost: From $28,000 / year. These guys are not kidding! That is the lowest they offer and it allows you to do tests only on a desktop. If you want to do mobile testing as well be prepared to pay a sweet fee of $40,000 – $60,000 per year
Cost per participants: $30-$50. If you are using participants on a desktop would be around $30 but if you are using participants on mobile would be $50 per person.
Number of Studies: Unlimited. It better be for that annual subscription fee.
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
As part of your subscription, you also get participant screener questions. Choose the device you would like to test on – desktop or mobile. Choose the demographics of your participants – age range, country, and more. You can test with your own participants as well and run unlimited studies.
From what I have looked at, this usability testing platform seems to offer a great range of research tools all in one software which is great but if you are looking to make just usability testing I wouldn’t recommend it. I never had the chance to try it but I have a feeling that for the price you are asked to pay it should be good! If your business can afford it then give it a try.
5. TryMyUI
Subscription Cost: From $3,588 / year. They also offer an enterprise subscription. If you are interested you can call them.
Cost per participants: 1 CR for desktop / 2 CR for mobile”]With your subscription you get 10 CR a month
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
As part of your subscription, you also get participant screener questions. Choose the device you would like to test on – desktop or mobile. Choose the demographics of your participants – age range, country and more.
I tried to set up a study with these guys but the UI and the UX of their usability testing platform made me give it all up. It’s quite slow and super difficult to use. I mean, setting up a study is hard on its own and you do not need the complication of a badly designed product to add more stress to that.
6. UserTest.io
Subscription Cost: None. Probably the first platform as of yet that doesn’t require an annual subscription cost to use it.
Cost per participants: $21. Pretty good price! No catch!
Account access: 1 person. Only 1 person can access the account and run studies but you can run multiple studies at the same time.
At the moment they do not offer anything but watching the videos. If you’d like to screen your participants you would have to contact their support to do it for you.
This platform was acquired by UserPeek.
Overall the usability testing platform has been working well but it has quite a slow turnaround time for videos. They have around 5k testers worldwide which reflects quite a lot on the timing of the tests. There is no option to screen your participants so for every test you set up you need to contact their customer support to do that for you, even though they are quite responsive sometimes your tests end up with people outside of your requested age range or screener requirements.
Read more about 10 Best Customer Experience Management Softwares of 2020
