Posted by c4rl on November 23, 2011 at 6:51am
I'm considering using a service such as http://www.usertesting.com/ for usability testing on behalf of a client who doesn't necessarily have the time or budget for using a proper usability lab, but would probably benefit from some basic form of usability testing.
Has anyone tried a service such as this? If so, what pros/cons exist?
Comments
Haven't used that service
Haven't used that service personally, but have heard good things about it. Making sure you're asking them the right things is the critical part.
Have you considered doing it
Have you considered doing it yourself? http://drupal.org/node/1284300 Most of the learning comes from actually observing and feeling the pain as it happens.
I don't know of any other
I don't know of any other services that attempt to compete with usertesting.com
I've done a mix of "formal usability testing" (one-way glass, eye tracking, etc.), informal testing in an office environment, and usertesting.com testing. In each case I found I was getting most of the value with less of the cost (e.g. moving from formal to guerilla, and from guerilla to usertesting.com). You still get the pain of watching the user struggle on usertesting.com b/c you watch their video.
If you can do more than usertesting.com, great. If you can't? Go for usertesting.com
knaddison blog | Morris Animal Foundation
I have used usertesting.com a
I have used usertesting.com a few times now. I'd totally recommend it, depending on the study. The study you're doing doesn't require a certain type of participant. For example, I can't really use it for Acquia Cloud usability studies, because I need Drupal developers for that. yoroy is right, and in doing your own moderated study will be more informative. You can ask "why" questions or probe further when you're having a conversation. With usertesting.com you can ask a follow up question, but the only time I tried this, the response was "I don't remember, sorry." The moderated study budget doesn't have to be huge. $15 gift card for 2-30 mins of their time makes most people happy!
Here's my take on usertesting.com :
Pros
* Really quick turnaround. I get results within an hour, seriously.
* The tests get done. With moderated studies, we have a lot of no shows which means rescheduling or recruiting others.
* No recruiting
* Easy to repeat a test
Cons
* It's luck of the draw when it comes to informative results
* People can ramble or just read to you what they see on teh screen rather than actually analyse what they see
* They get really off track or stuck etc... and there's nothing you can do about it.
There are other online services that allow you get reactions to your site, but none quite like usertesting.com.
==================================
http://about.me/lisarex
My 2 cents
I agree with the comments here.
I have done many studies on usertesting.com and the pros and cons (besides what Lisa pointed out):
Pros:
- It is pretty affordable service. I think it is $39 for each participant including recruiting and compensation.
- Easier to get a refund if you are not happy with a participant. Just shoot them (to usertesting.com support) an email explain the things, and you will get credit.
Cons:
- Some participants are "professional testers". Professional testers try to say what they think you may want to hear. So, you have to be careful with analyzing those kind of participants.
- The post session questions are restricted to 4.
Also, there are tons of other services available in the market but usertesting.com does a decent job. I would suggest you to go with a particular method depending if you are testing a website or a web application. For websites, usertesting.com does a decent job.
Unfortunately, you cannot add attachments as comments, but if you shoot me a email message I can forward you a pdf (from one of my presentations) on conducting usability studies that has a section on usertesting.com and other services that are available in the market.
Hope this is helpful. Good luck with your project!
-Dharmesh
Dharmesh Mistry
UX Researcher | Acquia,Inc.