Don’t Ignore Survey Screening Criteria

The screening section is the most important part of any survey, yet it is often undervalued or deemed unnecessary. Don’t fall into this trap. It is well worth the time it takes to design a tight screening section that helps identify relevant experts who are allowed to pass through and complete the survey.

How should you design a screening section to filter out unqualified respondents? It’s best practice to use a funnel approach. In the funnel approach, you start with a broad-category question and filter from there, adding progressively more granular questions until you know you have the respondent you’re seeking. It’s never a good idea to dive straight into the subject matter without first filtering out respondents who could potentially skew your data.

For example, if your target respondent is a health system marketing director, you would want to avoid starting with a “marketing decision making” question. Doing so would influence respondents’ perception of the survey, potentially impacting the quality of their responses.

Below, we’ve highlighted some categories to cover with the funnel approach:

• Industry: An industry question adds to the precision of discovering the ideal expert.
• Geography: A geography question can be at a country, regional, state, or metropolitan statistical area. This also allows you to analyze the geographic representation you’ve collected.
• Firmographics: Asking full-time employee size (FTE), annual revenue, etc. will ensure that you obtain the needed distributions across the firmographics (e.g., ensuring a 50%/50% split of <1,000 and >1,000 FTEs). This will also give you a sense of what type of establishment the perspective is coming from.
• Function/Role: When a job has a variety of titles for the same position, including a question that defines title variations is a great way to ensure you’re not losing good candidates.
• Responsibility: A title doesn’t always reveal a person’s day-to-day responsibilities. A question about responsibilities helps you ensure you’re getting insight from a decision maker.
• Familiarity (Vendor/Brand/Product/etc.): This question reveals if your respondents are aware of or have used the product in question. This should be at the end of your screening criteria as you’re getting into the granular details.

Sometimes it’s important to gauge a respondent’s level of knowledge within an area or sector. These questions establish trust that the respondent has the necessary knowledge and experience to speak to the topic at hand. These can be tied into the “familiarity” category.

What Is a Great Screener Question?

Great screener questions should trigger an innate response and contribute very little to the overall length of an interview. It’s easy for respondents to report on their industry or geography. You can actually improve the speed at which respondents are able to complete the survey by standardizing the format of your screening questions.

Screening questions also present an opportunity for quotas. Quotas are designed to enforce a preset distribution and monitor the counts per segment. For example, if your survey targets 50 marketing decision makers from health systems and 50 marketing decision makers from independent hospitals, quotas in the screening section will be critical to ensure you do not overindex in one area. Excluding this question from the screening section will create huge challenges when trying to monitor populations taking the survey.

Good screening questions can help you define how you’ll qualify a respondent to enter the main body of the survey (while filtering out the unqualified ones). Questions collecting demographic or firmographic information that will be used for data cuts should not be included in the screening criteria. Instead, use these segmentation questions at the end of the survey to ensure the experts who do not meet the criteria are not in the survey longer than necessary.

Sometimes there are key criteria that are difficult to target. In this instance, the screening section serves as an additional measure to ensure that only the most qualified experts pass through your survey.

Solving for Bias

Poorly constructed screener questions can lead to biased results. In order to create screener questions that are clear and unbiased, you first need to understand the types of bias that might arise:

• Leading questions can be yes/no questions where the respondent can determine which is the qualifying answer. It’s better to present the respondent with a list of options. Instead of, “Do you know about healthcare marketing?” ask, “Which of the areas below can you speak to knowledgeably?”
• Multibarreled questions ask more than one question at a time, which can lead to confusion and poor data quality. For example, asking, “Are you a current healthcare marketing director with oversight of digital platforms?” is asking position, recency, title, and platform knowledge all at once. Instead, split this into three questions, asking recency, title, and platform knowledge separately.
• Introduction blurbs can introduce the sponsor of a survey, their research, and often the information they’re seeking. It’s okay to use an introduction blurb, but its placement in the screening section is critical. If it is presented ahead of screening criteria, the respondent knows how they’re expected to answer. Use these blurbs after the respondent has been qualified by progressing through the screener questions.

A tight screening section leads to thoughtful responses in the body of the survey. When respondents are well versed in the topic, they are more likely to take their time answering each question. Avoid the unnecessary risk of two-word answers due to insufficient knowledge and use proper screening to gather comprehensive responses from well-qualified experts.

Ultimately, screening questions exist to increase the likelihood of gathering high-quality responses and data for your research. If you take the time to craft a detailed screener, you can rest easy when it comes time for analysis.

---

GLG Surveys are the best and fastest way to field-test your strategies. Our team of research specialists supports all phases of a project, from scoping, screening, and questionnaire design to panel selection, quality review, and data interpretation, so you can focus on the results.

What would you like to learn from your audience? Fill out the form below to get in touch and start surveying for insight now.

---

Check out the other articles in our Survey Series:

• When Statistical Significance Just Isn’t…Significant
• Top 8 Tenets of Survey Design
• What Type of Survey Do You Need?
• Are You Running the Right Survey for the Wrong Reason?
• Why the Screener Section of Your Survey Is Compromising Your Results
• Surveying Basics: The Right Way to Reach Respondents
• To Rate or to Rank? That Is the (Survey Design) Question
• Stop Letting Garbage Data Get in the Way of Good Survey Results