Back to Basics 3: Recruiting the Right Users for UX Research

Welcome back for another installment of what the team here at Human Interfaces is calling our “Back to Basics” blog series, looking at the in-s and out-s of conducting user research.  If you’ve been following along, you’ll recall that we tackled a big topic in last month’s post: how to go about selecting the right method to address your research questions.  Let’s assume now that you’ve carefully considered your research questions and you’ve selected the best methodology to provide the insights your organization needs.

While not all methods require study participants, it is entirely likely that you’re now facing another big challenge in the road to successful research:  How do we recruit the right users to participate in our research?  If you’ve settled on a research method such as focus groups, task-based usability testing, or in-depth interviews, there are several questions we need to address:

• Who can be a participant?
• Where can you find participants?
• How do you recruit participants?
• How do you get participants to show up?

Who can be a participant?

Generally, study participants are the users or potential users of the product or service to be tested. For example, if we are going to test tablet computers, suitable participants could be current tablet users. Equally important is that the participant sample includes participants of different backgrounds who represent the diversity of the target population. For example, some may use a tablet for work purposes, some for entertainment, and some for school, so each of these use cases should be accounted for across your participant sample.

If you are testing a new product prototype, you want to ensure that your participant sample includes users with varying interest in technology, and not just “early adopters” who are usually excited about new technology in general (regardless of how practical it may be in the “real world”).

As you can see, simply knowing if participants are users of your product or not is not enough to define your sample. There are many criteria that need to be considered when deciding who to recruit for your study. To ensure that they end up with a representative sample, researchers often define various “segments” of users that provide unique perspectives into the use of the product. For example, user demographics such as gender and experience with similar products in the past may provide insights specific to those different groups. Additionally, identifying “personas” related to characteristics within a population (e.g., remote workers versus desk-centric workers) will give you a more complete view of the scenarios in which the product or service is used, and specific features that may be required in each.

Once you have determined the type of participants you are looking for, you can begin the search.

Where can you find participants?

For some companies, a seemingly obvious place to find participants may be in their own customer databases. After all, customers are people who have already proven their intent to use a product by buying it. However, using a customer database for research recruitment purposes is not as straightforward as it may seem.

For starters, unless the organization already has approval to contact customers for research programs, doing so may be annoying to some people and raise privacy concerns for others. In some cases, internal company policies or politics may also be a roadblock to accessing these “cold call” lists from internal databases. We’ve found that the best way to address this issue is to have the client ask their own customers if they would be interested in participating in a study before passing these leads on to our recruiters.

For example, trade shows are a great place for clients to ask customers if they would be interested in participating in a study. Clients can then follow up with the interested customers via email to let them know that research recruiters from your firm will be contacting them shortly. By the time your recruiters receive the list, the customers have already expressed interest and have been notified that a recruiter will be reaching out. This not only increases incidence rates, but also helps insure that your company (and/or your client’s company) is not viewed in a negative light. Customers are already suspicious about companies who call them out of the blue, so the last thing you want to do is make customers think that your company sold their information to a research firm.

Another reason internal databases may be unsuitable is poorer diversity of participants, especially when the research can benefit from the perspectives of competitors’ customers and those who are less familiar with the company’s product. For example, if a company wants feedback on a new design of smartphones that they manufacture, it may be better to include people who use competitor products, rather than limiting the study to only their own customers. After all, the target market of a product typically includes competitors’ customers as potential users.

Similarly, recruiting only from sources like online user forums—a common approach—can be problematic. Not only is the participant selection potentially skewed towards certain brands, as with recruiting solely from customer databases, but there may be other biases as well. This includes aspects like technical experience, since it is reasonable to assume that regular participants of an online Q&A forum would be more knowledgeable than the average user. Research results for studies like usability studies may be impacted when new or inexperienced users are not tested.

In many cases, the best option for participant recruitment is a traditional, third-party research recruitment firm that maintains its own database of people who have opted-in for research studies. Not only is the selection of participants potentially much less biased, but such specialty agencies have trained personnel to perform recruitment and typically have an extensive database of candidates that can fill hard-to-find profiles. These databases are usually built from years of vetting suitable and motivated participants from various sources.

Although customer lists, forums, and other sources can still be used in conjunction with the services of a recruitment agency where appropriate, having the option to tap into an unbiased (and potentially novel) source of participants is always a good backup plan.

How do you recruit participants?

Whatever source you choose, it’s highly recommended that you use a formal screener as a guide for finding suitable participants for the study. A formal screener is a series of detailed questions that ensure that participants meet key eligibility criteria.

As previously discussed, a participant is usually a user or potential user of the product to be tested. However, that may not always be the case. If you are introducing a new product, or entering a new market, those potential users will not be familiar with your product (or maybe even with the type of product).

For instance, a basic study on tablet devices may include anyone who has an interest in using tablets, even those who do not currently own one. However, that leaves the door wide open (almost anyone today could be a potential tablet user). Of course, there are legitimate scenarios where we may want to leave things more open, but for this example let’s assume our tablet study involves the test of new, powerful video gaming features that make the tablet more expensive.

In this case, additional criteria that we may want to include could be household income as well as an estimate of how much they would be willing to spend on a powerful gaming tablet. These would help us decide if the candidate would be likely to even consider buying the device in the first place.

As our new tablet is aimed at video gamers, we may also want to ask candidates to provide details like the kinds of games they usually play, how much time they spend playing them in a week, and what devices they play them on. This will help us screen out less suitable participants who play only simple web-based games, or those who play very infrequently, from serious gamers who are more likely to buy and use the new device for its intended purpose.

As we discussed earlier, there could also be situations where we want to segment our group of suitable participants, even for the same study, to account for different target audiences or for specific research purposes. For example, in our gaming tablet study, in addition to recruiting a group of gamers who already use various mobile devices like laptops and handheld gaming systems, we may also want another group of gamers who do not currently use such devices but are open to the idea. In this way, we will not only be able to understand what expert tablet gamers may expect, we can also gain insights into how to make the product more appealing to potential users.

Developing a screener may seem complicated, but in many cases we can find help for getting started. Marketing departments in many organizations typically already have some definition of who the users or customers are for a specific product or market. This may come in the form of user “personas” or user profiles. These are great jumping-off points, but there is still work to be done to develop proper screeners for a research study.

Fortunately, this again is where a research recruitment agency will come in handy. In addition to understanding how to properly develop a screener and find suitable participants, professional recruiters will also perform the nitty-gritty task of actually conducting the screening and recruitment. This can involve the time-consuming work of conducting initial prescreening surveys, calling and interviewing shortlisted candidates, coordinating appointment schedules and plans, and acting as the central point of contact between participants and the researchers or research client. Additionally, recruitment firms usually either have their own facility to conduct studies, if one is required, or can make arrangements to find suitable places.

How do you get participants to show up?

After getting participants scheduled, the next step is to ensure that they actually attend the study sessions at the right time and place, since recruitment is usually done at least several days to a couple weeks in advance.

Besides being standard practice to compensate participants for their time in a study, this incentive also motivates them to remember their appointments and put in the effort to attend and provide good feedback for researchers. Compensation varies depending on many factors, like the type of participant you are seeking (people who are more difficult to find cost more to recruit), the amount of time each research session will take, and the region in which the research is being conducted, but can range from tens of dollars for 15-minute online surveys to anywhere from $100 to $250 for a one- to two-hour in-person study. Compensation details are set ahead of time between recruiter and client, and acknowledged by participants who agree to take part for the incentive offered.

A good recruitment agency also follows up with scheduled participants by providing directions to the venue, instructions for any tasks that need to be completed before their session (e.g. installation of specific software if the participant’s own device is used), as well as reminders to show up.

For many researchers and other stakeholders, the many hours required to properly prepare for, conduct, and handle logistics in a recruitment exercise would be better spent on study preparation, product development, data analysis, and other aspects of the research project. Unsurprisingly, these and other reasons make professional recruitment firms an attractive choice for most research projects that require participants.

Regardless of which route you take in recruiting users, getting the “right” participants is every bit as important as selecting the correct methodology for your research objectives.  Regardless of how sound your research design is, if you aren’t connecting with the right participants, your findings are unlikely to positively contribute to a world-class user experience with your product or service.

In our next installment of “Back to Basics”, we will be focusing on the various options and trade-offs for where to conduct user research, from online platforms to conference rooms to purpose-built user research facilities.  We hope you are enjoying our look at the fundamentals of user research, and we’d love to hear about your experiences in getting the most impact from your research activities.