Self reported metrics

Importance of self-reported data

Self-reported data give you information about users’ perception of the system and their interaction with it. At an emotional level, the data may tell you something about how the users feel about the system.

People are more likely to think of a product or site that they liked the last time they used it. They’re much less likely to remember how long the process was or that it took more mouse-clicks than it should have. That’s why users’ subjective reactions to a website, product, or store may be the best predictor of their likelihood to return or make a purchase in the future.

When to collect self-reported data

During a usability study, you might collect self-reported data in the form of verbatim comments from a think-aloud protocol while the participants are interacting with the product.

Two additional times when you might want to probe more explicitly for self-reported data:

  • immediately after each task (post-task ratings)

  • at the end of the entire session (post-study ratings).

Quick ratings immediately after each task can help pinpoint tasks and parts of the interface that are particularly problematic. More in-depth ratings and open-ended questions at the end of the session can provide an effective overall evaluation after the participant has had a chance to interact with the product more fully.

Post-Task Ratings

  • SEQ: Single Ease Question

  • NASA-TLX: NASA Task Load Index

  • Ease of Use

  • After-Scenario Questionaire (ASQ)

  • Expecation Measure

Postsession Ratings

  • SUS: The System Usability Scale

  • SUPR-Q: Standardized User Experience Percentile Rank Questionaire

  • NPS: Net Promoter Score

  • UMUX: Usability Metric for User Experience

  • UMUX-LITE: Usability Metric for User Experience Lite

  • WAMMI: Website Analaysis and Measurement Inventory

  • PSSUQ: Post Study System Usability Questionaire

  • SUMI: Software Usability Metric

  • QUIS: Questionaire for User Satisfaction

  • PURE: Practical Usability Ratings by Experts

  • UEQ: User Experience Questionaire

How to Collect Ratings

Logistically, three techniques can be used to collect self-reported data in a usability test:

  1. Answer questions or provide ratings orally. Having the participant provide responses orally is the easiest method from the participant’s perspective. It requires the observer to record the responses, and may introduce some bias as participants sometimes feel uncomfortable verbally stating poor ratings. This works best for a single, quick rating after each task.

  2. Record responses on a paper form Paper forms are suitable both for quick ratings and for longer surveys. Paper forms may be easier to create than online, but they involve manual entry of data, including the potential for errors in interpreting handwriting.

  3. Provide responses using some type of online tool Online forms are getting easier to create and participants are getting more accustomed to using them. One technique is to have a laptop or tablet with the online questionnaire next to the participant. The participant can then refer to the application or website easily while completing the online survey.