Back to Basics 5: Testing Within Development Cycles

Welcome back for another installment of our “Back to Basics” blog series, focused on the ins and outs of conducting impactful user research. Our previous posts have tackled topics such as how to find and recruit the right users to participate in the research and selecting the right place & right tools to fit the study.

In this post, we are going to be looking at a critical and tricky aspect of user experience research: timing and executing research activities so that they both fit within, and are relevant to, the course of development cycles within organizations.

User experience research isn’t and shouldn’t be a separate endeavor managed in its own bubble. The fact is, testing can be more effective when handled as an integrated and continuous part of product development, with both product stakeholders and researchers working together to fit research type and pace within the preferred development model.

Get involved early and often

Regardless of how extensively and frequently study sessions are conducted, research works best when it is included early in the development lifecycle, even during the early planning stages. In fact, early involvement is preferable, as high-level ideas can be explored, defined, and changed (if necessary), more easily than much later in the game when money, time, and other resources have been spent towards a particular direction.

What are some of the important user testing areas to explore in the initial stages? It will be worth your while to identify overall user needs and interests in specific features, describe the context in which the user operates, brainstorm ideas that address critical problems, and determine practical solutions to those problems that don’t break the bank. Of course, this includes striking a balance between most users’ “we want everything” mentality, and selecting the functionality that will actually be usable.

Feedback from these efforts can then guide actual development in the right direction from the start, rather than having to make late-game adjustments (or being unable to make any changes at all) to accommodate crucial findings in what users want. Study methods used at this early stage can be as simple as short online surveys, or can dig deeper with in-depth interviews, or focus groups that may or may not involve simple wireframe prototypes or abstract mock-ups to help convey intent. Another key point to remember besides early involvement is that frequent testing, especially at important stages of development (e.g. completion of specific milestones or key features), is better than trying to test “everything and anything” at the very end. The main reason for conducting frequent testing is to confirm that development is on the right path and continually provide guidance on what are upcoming priorities, rather than discover too late that critical elements don’t meet user needs or design principles.

Small & focused vs. large & comprehensive

How extensive and how often should research be conducted? As with many things, it depends; there is no one model that fits all needs. As a rule of thumb, research should be conducted as often and as extensively as needed to answer pressing design questions that arise at various points in the development process.

For example, if the product team needs to know whether to go with version / widget / element A or B (or C, and so on), and the decision may be hard to reverse later, or may have great impact on later development (such as the implementation of a key feature), a study should be conducted to provide defendable evidence that supports whatever decision is made at that juncture.

Another factor to consider is the work process and timeline of the product team, as well as team norms for testing and quality-control cadences. For instance, some teams may favor regular weekly or monthly checks, whereas others may prefer a more milestone-based approach. Shorter intervals naturally lend themselves to smaller, quicker tests that may involve only a few participants (or none at all, if expert reviews are sufficient) and test only a few small changes. Conversely, longer intervals may call for longer test sessions with more participants that focus on multiple features and address more research questions, if only because more changes have been made since the last test.

The type or focus of the study itself also influences the size of the research and how frequently tests should be carried out. For example, competitive evaluations and other summative-type studies are necessarily larger simply because they cover many aspects of multiple systems.

One other aspect to consider when planning studies is whether or not to make changes during testing based on ongoing feedback. In some rapid, iterative testing models, proposed changes are tested while the study is still being carried out. While this provides rapid testing of new ideas, it also produces a number of challenges. In most cases, the option to make changes mid-test is not practical (e.g. as when hardware prototypes are used, or if there is a risk of messing up a delicate setup), if data consistency on a particular feature is a big concern, or if the results are still inconclusive. However, in other cases, especially in early prototypes or when it is clear that something needs to be tweaked, making changes on the fly could give you more bang for your buck by allowing you to test successive iterations of changes. Still, making these changes should be done carefully and in consultation with the researchers involved.

In summary, conducting research at multiple points throughout the design process, from product ideation to post-production, can help ensure a greater ROI, and shorten the development cycle in the long-run. In fact, designers should work closely with UX researchers at every phase, rather than in silos, to understand each others’ sticking points and frustrations. If you follow this approach, the end product will not only be new and innovative, but will be exciting and engaging for the end users.

In our next installment of our Back to Basics series, we will summarize all of our posts so far, and add a few final thoughts before we move onto the next set of articles for this series. Until then, drop us a note and tell us about experiences you’ve had with integrating and timing research to best fit your organization!