Self-reported data give you information about users’ perception of the system and their interaction with it. At an emotional level, the data may tell you something about how the users feel about the system.
People are more likely to think of a product or site that they liked the last time they used it. They’re much less likely to remember how long the process was or that it took more mouse-clicks than it should have. That’s why users’ subjective reactions to a website, product, or store may be the best predictor of their likelihood to return or make a purchase in the future.
During a usability study, you might collect self-reported data in the form of verbatim comments from a think-aloud protocol while the participants are interacting with the product.
Two additional times when you might want to probe more explicitly for self-reported data:
immediately after each task (post-task ratings)
at the end of the entire session (post-study ratings).
Quick ratings immediately after each task can help pinpoint tasks and parts of the interface that are particularly problematic. More in-depth ratings and open-ended questions at the end of the session can provide an effective overall evaluation after the participant has had a chance to interact with the product more fully.
Post-Task Ratings
SEQ: Single Ease Question
NASA-TLX: NASA Task Load Index
Ease of Use
After-Scenario Questionaire (ASQ)
Expecation Measure
Postsession Ratings
SUPR-Q: Standardized User Experience Percentile Rank Questionaire
NPS: Net Promoter Score
UMUX: Usability Metric for User Experience
UMUX-LITE: Usability Metric for User Experience Lite
WAMMI: Website Analaysis and Measurement Inventory
PSSUQ: Post Study System Usability Questionaire
SUMI: Software Usability Metric
QUIS: Questionaire for User Satisfaction
PURE: Practical Usability Ratings by Experts
UEQ: User Experience Questionaire
How to Collect Ratings
Logistically, three techniques can be used to collect self-reported data in a usability test:
Answer questions or provide ratings orally. Having the participant provide responses orally is the easiest method from the participant’s perspective. It requires the observer to record the responses, and may introduce some bias as participants sometimes feel uncomfortable verbally stating poor ratings. This works best for a single, quick rating after each task.
Record responses on a paper form Paper forms are suitable both for quick ratings and for longer surveys. Paper forms may be easier to create than online, but they involve manual entry of data, including the potential for errors in interpreting handwriting.
Provide responses using some type of online tool Online forms are getting easier to create and participants are getting more accustomed to using them. One technique is to have a laptop or tablet with the online questionnaire next to the participant. The participant can then refer to the application or website easily while completing the online survey.