Whether you work at a brand or an agency, everyone has experienced launching something new to disappointing results. Sometimes, as hard as we work, the best ideas simply go wrong. But that’s hardly a comfort when your market research and user testing indicated the design and execution were in alignment. And yet, somehow, the user flow or engagement isn’t where you expected.
At disappointing times like these, the problem wasn’t necessarily with your approach. But it may have been your approach to testing. Often, when we establish testing parameters, we give someone a clear task, such as asking an e-commerce user to find a yellow sweater and place it in their shopping basket. Users may then respond with the results you want, but they’re also paid testers sitting in a distraction-free environment. If they don’t do what you ask, they don’t get paid. The testing environment is a far different context from your audience's experience in the real world.
Taking into account every user's situation provides an effective means to evaluate how your work performs under less-than-ideal-circumstances. Often overlooked as brands review their marketing efforts, situational testing allows you to evaluate your designs through building a better understanding of your user. Given the stress of a pandemic and its impact on the way we work and how our kids attend school, everyone is operating in a changed environment. Consistent distraction is our new normal.
By accounting for the factors that may be affecting your audience, you gain a more honest picture of how well your design works.
The three types of situational research factored into the user experience
Digital experiences cannot be designed with the assumption that every user is focused and operating at some idealized physical capability. Research allows us to recognize our target audience and establish user personas. But after establishing that baseline, we need to provide accommodations for those having a different experience.
These varying situations deserving consideration fall under four headings that consider a user's abilities for touch, sight, hearing and speaking. Along with accounting for those experiencing a different physical situation, you have to also account for internal factors as well.
Permanent
When we think about designing for accessibility, this population is the most common consideration. These users could be functioning with a physical limitation, or have lost the use of their senses through blindness, deafness, or being non-verbal.
Temporary
Along with allowing for users experiencing long-term conditions, you have to consider those whose situation may have shifted when they encounter your product. An arm injury may impact one user’s physical capabilities, for example, or an ear infection may impact their hearing.
Fluid
Your users’ situations also may evolve depending on where and when they experience your product. A new parent’s physical capabilities may be affected by a need to hold their child, or another user may access your product in a crowded or noisy room. At any time, changing conditions may be compromising your user's abilities.
Social-Emotional
In some cases, your audience’s capacity to effectively use your product may be impacted by internal factors. Users may be hungry, in the middle of a bad day, or be forced to multitask in a way that leads to frustration, poor decision-making or difficulty using navigation tools. From the perspective of a researcher, all of these factors should be under consideration.
How building empathy for your user's situation creates stronger work
Taking into account the factors affecting how your campaign may be experienced provides far-reaching benefits. As with all accessibility considerations, work that has undergone situational testing will attract a wider audience because its not focused solely on users living in a perfect world. In design, perfect worlds simply don’t exist.
In addition, with accessibility front and center, you eliminate the guesswork around the ultimate goal for your design, which is to provide an intuitive, welcoming experience. Through situational testing, you understand the points where external factors impact the user experience. Think of a grandparent shopping over the Christmas holiday, which is a stressful time. If your ecommerce site considers this emotional state and how it impacts your user's capabilities, you’ll pursue a more streamlined design.
As a result, situational testing contributes to fewer abandoned shopping carts, harried support calls, and negative experiences with your brand.
How to establish effective situational testing strategies
Like conventional usability tests, situational testing can take place at any stage of the design process. However, testing for how well your product performs under specific, real-world circumstances requires more time to be effective.
Typically, situational tests have no moderator or facilitator to guide someone through key aspects of your design. In thinking about observing user behavior under varying conditions, it’s tempting to imagine anthropologists traveling to a remote region to observe an unfamiliar culture. However, modern tools allow users to test your designs from the comfort of their home and share their experience.
A remote qualitative testing platform like dScout allows users to keep a journal documenting their experience performing usability tasks. You can also set up contextual interviews, which allow users to narrate their actions as they navigate your site. As their experience progresses, they reveal points where frustrations appear.
Once your design nears completion, pressure testing evaluates the effectiveness of your work through pursuing as many negative outcomes as possible. This sounds fatalistic, but role-playing uncovers new problems as you discover new areas where the worst could happen. Ideally, you’ve found some evidence of these scenarios through dScout, but gaming out where a user experience may go awry prevents unexpected negative outcomes.
Granted, disappointing results could happen to any campaign as some situations simply can't be predicted. Who, for instance, could have imagined any number of the events of 2020 that contribute to our shared distracted mindset. But through understanding the context of your audience’s true experience, you increase your chances of finding a design approach that's as inclusive as it is effective. With that kind of foundation, your design has a much better shot at finding far-reaching success.
