Observation Rooms: Fantastic Value for Stakeholders

When most people think about in-person user research they focus on the activities that happen inside the testing room – the discussion, the tasks, the card sorting, the sketching, the ratings, the crying… (just kidding). However, what takes place behind the one-way glass can be just as meaningful. Here are some of the advantages to getting a variety of stakeholders into the observation room.

First-hand feedback

Stakeholders who view the research live get to hear the feedback first-hand. This can make a finding resonate much more powerfully than when it is just read in a report. Maybe team members will watch your concise, targeted, message-driven video clips later, or maybe they will get bogged down with the dozen other projects that they are working on and never make time for it. The best time to get attention to the project is when it is happening because it improves the odds that your stakeholders will hear the voice of the user. It can even be somewhat enticing because it pulls people out of their daily rut (I could just keep working on X, or I can change it up and see what our users are like in action, today!)

Collaboration

There is an opportunity for collaboration in the observation room, too. Designers, developers, and marketing can all pull together to see how the product is perceived, to learn where users are confused, to figure out where the workflow can be optimized… and then talk strategy. The discussion is evidence-based and pulls in a range of perspectives.

Those in the back room can also respond to feedback in real time and update whatever is being tested (easier for software, but not unheard of for hardware). Interface mock-ups, for example, can be developed in the observation room while the current iteration is being tested. Those in the observation room can discuss what they are seeing as it happens, identify when something is not working well, share ideas on how to address it, mock it up, and get immediate feedback on it in the very next session.

Answers now

Questions can be generated and answered on the fly. When people are observing, they can feed questions into the moderator to get them answered as they come up. This is an advantage over the alternative of only digesting the research after it is done. Once the study is over, your team either relies on the moderator’s inclination to inform a question or they wait for the next study to get it answered. The observation room can mean an increase in efficiency.

Consensus-building

Sometimes observers will hear something in a session, latch on to it, and immediately go off to design for it without fully considering if it is consistent across users and important enough to merit resources. A filled and effectively managed observation room can be used to reduce such instances. People will naturally discuss what they are seeing when they view it together. This is an advantage of having a dedicated room rather than team members just watching independently online.

Having everyone together in a room also allows for all of the players to get on the same page every day by having a daily debrief. Wrapping up the test day with 15-30 minutes of discussion can illuminate what observers saw, what they felt was important, and how that matches or mismatches with other observers. (Make sure to bring the moderator into this discussion!) This ensures that no one goes off and completely redesigns the app or product strategy because of one comment from one participant on one section of the product. Instead it allows the room to start building consensus.

Daily debriefs in the observation room also reduce the chances that a stakeholder will be blind-sided by a finding. No one will be surprised by the findings and recommendations in the report because they will have seen the story unfold as it progressed over the testing days. Most stakeholders will have had time to start digesting controversial findings and will have had an opportunity to get questions answered before the research project was wrapped up. This will reduce push-back on your recommendations.

Alternative to the “observation room”

The two main reasons that people may not utilize an observation room are: 1) They simply don’t have access to the appropriate lab setup, or 2) They can’t get that many people to show up to the facility because of time commitments or the team is distributed geographically.  Don’t let this stop you from getting your team together to observe sessions!

If you don’t have access to a traditional observation room, you can get creative. But don’t be tempted to just put observers in the testing room. I would consider this a major no-no. You can try to crowd some people into the testing room, but that can quickly become intimidating or distracting to the participant(s) after adding even 1 or 2 observers. Similarly, having a lot of remote observers call in to hear feedback on a call can make a respondent really self-conscious and negatively impact their feedback.

Instead, it is important to keep the observers as transparent to the participant as possible. It’s okay that they know the team wants to hear their feedback first-hand, but it shouldn’t be in-their-face.

An excellent solution is to set up remote “observation rooms” through video streaming and conferencing. This also solves the problem of distributed and busy teams; each local site can set up a small conference room with a video feed and observers can join when they are available. Just make sure they all pop in at the end of the day to debrief!