How to Evaluate User Experience
Written by Janet Hwu, Lead User Experience Designer.
This past July I joined Shield AI, and so far it has been an experience designer’s dream. To work on a system that combines the physical and the digital, to create for a high-stakes use environment where design really matters, and to serve people who are committed to protecting others has been an opportunity that has proven both stimulating and challenging. As part of my responsibilities as UX (User Experience) Designer, I have been planning and conducting user evaluations for our work in development. User evaluation (also known as user testing or UX assessment) is the practice of seeking feedback from users throughout the design life-cycle, with the goal of improving their interactive experience with the final product. Here are a few key actions to take before beginning such a pursuit, inspired by the design thinking process taught at the Stanford d.school.
1. Get to know your user
One of my favorite parts of UX is developing empathy for people from other walks of life. There are so many facets of the human experience to discover, and each project brings fascinating learnings on how a life can be lived. Besides being fun, though, taking the time to get to know your users is a critical step of human-centered design that can easily get overlooked under the pressure of tight schedules and hard deliverables.
Stepping into the user’s shoes can involve a number of approaches — the creation of user personas, for example, is a conventional UX practice. Besides performing customary research online and through books and film, it’s also refreshing to get out of the office, track down people like your users, and get to know them as human beings. What motivates them? How do they think and why do they think that way? What line of reasoning do they follow when making decisions?
At Shield AI, our users are typically service members and first responders. I’m fortunate to work alongside a number of colleagues who have served in the military themselves. Learning from and bouncing ideas off of them has already led to a number of insights to be applied to the experience we are building.
2. Pick the appropriate method
Different phases of a project call for different methods of user evaluation or research. If you’re early on in the project and haven’t yet settled on a solution, consider taking a survey of your intended users to gather quantifiable data about them, or conduct one-on-one interviews to formalize the process of understanding them. You could organize ethnographic field studies wherein you observe and participate in the user’s natural environment to identify their needs, or pursue a line of contextual inquiry, which involves shadowing a user while asking them questions.
If your project is already well underway and you have functional prototypes to share, usability testing is the most useful and inexpensive approach to gaining awareness on how to improve the end result. This process typically requires observing one user at a time and evaluating how easily he/she can navigate your product while trying to perform a task. This has been among my primary methods of choice so far at Shield AI.
3. Simulate the final use context
This is important for usability testing in particular. Testing can be done in a lab for websites and apps that are purely digital experiences, but for products that include a physical component, more care is needed in creating the right context for evaluation. The more similar you can get to the proposed use case, the better your takeaways will be. Conditions of light and dark, the user’s physical and cognitive load, surrounding noise, and other environmental factors can drastically influence the way testers interact with your product.
Our use cases at Shield AI are challenging to replicate for obvious reasons, but we try to get as close as we can. We use a number of off-site facilities to test our robots for reliability and usability. During usability testing, our test users are asked to simulate anticipated conditions and to emulate our intended users as closely as possible.
These are some of the considerations that we take into account in how to best serve — and design for — our customers. At Shield AI we keep the human customer at the forefront of our product development, and we know the importance of user experience in fulfilling our company's mission: to protect service members and innocent civilians with artificially intelligent systems. My teammates are driven by a meaningful objective and shared values, which is what I appreciate most about working here.