User Testing – A Tester’s Review

I’ve discovered the world of online user testing, and have so far really enjoyed it.  It is easy, fast, and fun – and I get to learn about new websites or changes that different companies are thinking of making to their existing websites.  There are a few companies that pay testers to test, and I have been trying them out to determine which ones I should be spending my time on.  User Testing is the first site I signed up with.

Overview

User Testing offers “videos of real people speaking their thoughts as they use websites, mobile apps, prototypes, and more.”  They provide clients with video and audio of real users within one hour.  Two account options are offered: the Basic account, for $49 or $99 per video, and the Pro account, which is based on the needs of the customer.  Past and current clients are listed, including large and recognizable companies.

Tester Compensation

A standard 10-20 minute test pays $10.  The rate is the same for computer, smartphone, or tablet tests.   Occasionally a test may take longer, but the tester will be notified upfront and the pay rate will be higher to compensate for the additional time.

After successfully completing 10 tests with acceptable ratings, users have the option to sign up for camera testing.  User Testing sends the tester an IPEVO camera to record smartphone navigation.  These tests pay $15.

Occasionally, testers are invited to participate in moderated sessions.  These are usually scheduled for a specific date, but at a time slot of the tester’s choosing.  If selected, the tester will receive a predefined amount (I’ve seen $40 much of the time).

All tests are paid out via PayPal exactly one week (to the minute) after the tester has taken them.  I have taken many, many tests and have never experienced an issue with payment.

Hiring Process

Easy.  Simply sign up for a tester account online, and you will receive an email asking you to take a sample (unpaid) test.  The first time feels a bit weird.  You are asked to navigate a site per the steps listed, and speak aloud all of your thoughts and expectations.  I wasn’t sure if I was doing it right but went ahead and submitted it anyway.

Once your test is submitted, you’ll receive an email indicating that it may take them up to a week to review it.  I was pleasantly surprised to receive a response the next day though, indicating that I could now fill out my tester profile to begin receiving payment for tests.  That’s all there is to it.

Taking Tests

Based on the information listed in your user profile, you will be invited to take tests via your online dashboard.  If your dashboard is up, you will hear a ping to alert you that there is a test available.  You can then answer a series of questions to determine if you are eligible to take the test or not.  If you are eligible, you can accept the test and then have no longer than 15 minutes to begin before it gets sent to another tester in the pool.

Tests go FAST, so you need to be on top of your game.  As soon as you hear the ping, you will need to click the “View Test” button on your dashboard to walk through the qualification questions.  Sometimes even then, you will be told after qualifying that all test spots have been filled.

Once you accept a test, you can plug in your headset/mic and then begin.  Your screen and voice will be recorded  as you walk through the test.  The key is to read the steps provided, follow the instructions, and constantly explain what you’re doing and why.  Honesty is important.  If you don’t like a site’s layout or had different expectations, companies want you to state that and to state why.  Once you complete the steps provided, the recording will end and you will be directed to answer 1-4 online questions (where you type in the answers).  Once that is done, your test will be uploaded and your Payment Pending amount will be updated.

Client Feedback

Clients have the opportunity to provide feedback on your tests (completeness, level of detail, clarity, helpfulness, etc.)  The Completed Tests tab should be checked periodically to see the feedback being provided so you can modify your approach as necessary.  Only about 25% of clients take the time to provide feedback – when they do it is based on a 1-5 star system, with comments.  An overall star rating is assigned to you based on the results of your last 12 tests.  Your star rating is extremely important as it will determine how quickly and in what priority you receive future tests.  Too many 1 or 2 star ratings and you will stop receiving tests completely.

What I Like

• Prompt weekly payment via PayPal
• $10 for 10-20 minute screen tests and $15 for IPEVO camera tests
• Fun and interesting work – I enjoy perusing sites and providing my feedback
• Opportunity to see and provide feedback on beta websites for major brands before they roll out
• Many tests available if you are quick to accept
• Friendly help when needed – I’ve emailed the Help Desk several times with questions and have always received quick and clear responses
• Opportunity to participate in additional studies and/or moderated sessions for increased compensation
• Clear and intuitive dashboard
• Pause capability – I can pause the test and recording if I have to step away

What I Don’t Like

• Tests are sporadic – this definitely is not going to provide a full time income
• At times it seems like tests fill up extremely quickly – even when responding within seconds, I often find that a test has already reached the required number of testers
• I have had issues with a lag between my iPhone and the dashboard notifications – I will receive a notification for a new smartphone test, yet my iPhone shows no tests available
• The quality of the test itself depends on the client – some questions may be confusing, navigation may be difficult to understand, etc.
• Star rating is completely contingent on a client’s perception of how you took the test – while I have not yet experienced any issues, you could conceivably have a client rate you poorly simply because they didn’t like what you said, instead of for a legitimate test-taking reason

Conclusion

Of all of the user testing companies I have tried, User Testing is my favorite.  There is a steady stream of available tests, prompt weekly payment, and friendly help.  On the flip side, tests get snapped up quickly, and consistency is sporadic – I may take four tests one day and then zero the next.  But overall, I feel it is a great legitimate company offering fair compensation for easy (and fun!) work.  A nice supplement to income for just a few minutes of work.  I highly recommend User Testing if you are interested in testing and providing feedback on various sites and applications.