How to Conduct Message Testing

Message testing research is an important part of your communication and marketing development process. We’ve previously explored what message testing is, why it matters, and when to consider pursuing it. But once you have decided to embark on your message testing journey, you may be unsure exactly how to tackle it. There are a variety of methodologies, but which is the right method (or combination of methods)? Though your research objectives and ideal output will dictate which fits best, this article will explore your options and the factors to consider when choosing among them.

Market Research: Primary vs. Secondary

When it comes to market research, let’s start with the basics. There are two types of research: primary and secondary. Primary research, also called “field” research, focuses on new information, gathered first-hand. This type of research is helpful for developing a deeper understanding of customers. By contrast, secondary research, also called “desktop” research, taps into existing information and is helpful when building a baseline about a market, industry, or customer segment.

Message testing is used to evaluate how the message you’ve developed (or are developing) resonates with your target audience. This requires engaging directly with your potential customers or users to gather first-hand feedback. For this reason, primary research should be a critical focus of your message testing research.

Primary Research: Qualitative vs. Quantitative

There are two types of primary research: qualitative and quantitative. Qualitative research is usually more exploratory, open-ended, and conducted with a smaller, nonrepresentative sample. It helps you gain insights into people’s thoughts, feelings, motivations, and behaviors. Quantitative research is usually more tightly structured, closed-ended, and designed to reach a larger, more representative sample. It will help you test or confirm clearly defined hypotheses or concepts. Essentially, quantitative will help you define what people think and feel; qualitative will help you understand why they think and feel that way.

When you are choosing between qualitative and quantitative research, it’s important to consider where you are in your development process. In the initial stages of developing your communications, qualitative research can help you explore themes and come up with a general direction for your message. You can also pursue qualitative methods when you want to refine among a few message options before settling on the final direction. Then, once you are further in the message development process, you can leverage quantitative research to test and select the final message ahead of broader launch.

A Deeper Dive into Qualitative Research

Qualitative research is particularly helpful when you want to generate and hone ideas; test initial, rough concepts; uncover opinions and trends; and/or delve into unmet needs, attitudes, and behavior rationales. Because of the more open-ended, loosely structured format, you can probe and gain depth of insight across a smaller sample. It also allows you to iterate on your research and adjust along the way. However, given the smaller sample size, results are usually more directional rather than statistically valid.

When you are considering qualitative methods to support your message testing research, you have a few modalities at your disposal, including:

1. In-depth Interviews (IDIs): These are a one-on-one format of interaction between a skilled interviewer and an individual with first-hand experience, such as a customer or user. In the context of message testing, you can explore what they think about your message and hear them talk in their own words about the message(s) being tested. IDIs can be conducted virtually (audio and/or video) or in person. Face-to-face formats allow you to glean more insight from nonverbal cues such as facial reactions or body language. Because of the one-on-one format, you can dive deep and gain detailed insights; however, relative to other methodologies, it can also require more time and resources to conduct multiple interviews.
2. Focus Groups: This is a format that brings together a small group of participants for a real-time, professionally moderated discussion. While the exact structure of a focus group could vary based on your research objectives, they commonly entail four to six participants per group for 90 to 120 minutes in duration, conducted either virtually or in person (smaller groups may be referred to as dyads or triads). The participants themselves often share similar attributes or behaviors, and thus the discussion can help you gauge where there’s consensus around messaging or even spotlight areas of divergence. Given the group dynamic, it can allow you to gather many perspectives at once, which can save time and resources — and you can also uncover new insights as participants interact with and feed off one another.
3. Online Discussion Boards (ODBs): This format entails a private, online forum where a group of participants (about 15 to 30) asynchronously respond to moderator prompts and activities over the span of a few days. Conducted entirely online, the format allows for a variety of multimedia prompts and response modes, whether text, photo, and/or video. The asynchronous nature allows participants the flexibility to log in at their leisure, which can enable greater reach (e.g., across time zones) relative to a time-bound focus group. The format lends itself to candid and considered responses as opposed to real-time gut reactions. Since boards are typically conducted over the span of a few days, you can make tweaks to stimuli and iterate along the way. Boards can also be more time- and cost-effective than other methods.

Ultimately, qualitative methods of testing are ideal when you are looking to derive insight into how customers perceive your message on an emotional level and whether or not it is resonating.

A Deeper Dive into Quantitative Research

Compared with qualitative research, quantitative research is more structured and tightly controlled. The output of quantitative research is a more objective dataset that can be measured and analyzed. For this reason, it is useful in testing a well-defined communication concept that you want to validate before launch — which means it usually comes into play toward the end of your message development process.

One of the most popular ways to pursue quantitative research is to conduct a survey, which is a great method of collecting a higher volume of data in a short time period. It also has greater reach, which can help garner a wider range of respondents. This can result in a high-quality dataset that is more representative of the population.

When designing a survey, you can ask many types of questions ranging from rating, ranking, single- or multi-select, matrix tables, and other typically closed-ended questions. To message test, you can also consider a monadic or a sequential design to help refine and select optimal messaging. The former allows you to focus on a single message at a time to confirm whether it’s ready for launch, whereas the latter allows you to compare multiple messages with different angles to select the best concept.

You may want to message test via a survey if you want a measurable dataset with a more representative population — and if you are far enough along in your development that you have a well-defined message (or a set of a few messages) to test.

Factors to Consider When Choosing the Right Method(s)

There is no one-size-fits-all approach to market research. What’s more, you are not limited to pursuing a single methodology. In fact, most research would benefit from combining a few methods to gain deeper insights across your development journey. You can pursue IDIs to explore initial themes, then conduct focus groups to refine initial messaging concepts, and finally conduct a survey to select the best message ahead of launch. Even after launch, you may want to periodically revisit certain methods because customer attitudes can shift over time in response to changing market dynamics.

That said, depending on where you are in the development process, a few factors may dictate which method (or combination of methods) might be the right choice for you:

When designing your research plan, you need to consider many variables. While common considerations include your preferred mode of communication, desired output, timeline, and budget, each project is unique and other factors could influence your reasons for conducting one method over another. If you are not sure where to start, consider partnering with an experienced moderator or engaging a research partner to help you come up with the best research design based on your goals.

The Takeaway

Conducting primary research like message testing is an essential exercise to connect what your customers care about with the messaging you ultimately develop. While you can approach your research plan in many ways, the best method — or combination of methods — will help ensure your message is being interpreted correctly and producing the intended effect.

---

About Lauren Bledsoe and Meaghan Bradley

Lauren Bledsoe and Meaghan Bradley lead experienced teams supporting GLG’s Professional Services Firm clientele and Corporate clientele in the Americas, respectively. Their teams provide customized qualitative research offerings with either panel-only (B2B) or full-service options. Engagement formats include SME (subject matter expert) placements/staff augmentation, focus groups, panels/ad boards, online discussion boards, workshops, and in-person in-depth interviews.

Lauren has more than nine years of client service experience, and Meaghan has more than six years of client service experience in a qualitative research context spanning both the United States and EMEA. Lauren earned her bachelor’s degree after studying public policy and business at Duke University. Meaghan earned her bachelor’s degree in business administration at Manhattan College.

---

For more, read out articles What is Message Testing, and Why Does it Matter,
Improving Message Testing, or When to Conduct Message Testing.

You can also download our ebook, Message Testing: Best Practices.