Ranking or Rating Survey Questions? The Ultimate Survey Design Question

Rating survey questions and ranking survey questions are two of the most common types of questions you’ll encounter in a survey. They are easy to understand, quick to answer, and provide great data. However, they each have optimal use cases, and using the wrong type can leave you with data that is frustratingly unusable for your research and analysis.

The terms rate and rank are often used interchangeably. However, they are distinct question types and collect different types of data. Here’s what you need to know to get the most out of both question types.

Rating Survey Questions

Rating survey questions use 3-point, 5-point, or even 11-point scales to gauge where you sit along a spectrum of possible outcomes. If you see question text that reads something like “on a scale of…,” then you’re looking at a rating question. These questions are great at measuring things like performance, quality, importance, or effectiveness.

However, therein lies the most notable limitation of rating questions. When you end up with data that has two competing products or brands that are rated the same, which one is preferred? Are they truly equal, or may slight nuances nudge the respondent to choose one over the other? With just a rating question, we cannot know the answer.

Notable Types of Rating Survey Questions:

• Numeric ratings use numbered scales, the most popular of which are 1-5 or 0-10. Net Promoter Score (NPS) ratings are a great example of numeric ratings.
• Frequency ratings ask how common an occurrence or behavior is and can be structured as a scale from “Never” to “Always.”
• Comparison ratings use qualitative answer choices like “Better” and “Worse” to evaluate two variables.
• Likert ratings ask respondents to identify how much they agree or disagree with a given statement.

There are many ways to rate – just choose the version that best gets at the question you need answered.

Ranking Survey Questions

Everyone loves a good ranking! What are your top three movies of 2019? What are the 10 worst bands of all time? We naturally gravitate toward framing things as “best” and “worst,” and ranking questions are designed to do exactly that. These questions are most effective when you need to know the relative positioning among many variables. Since ranking comes so naturally to us, this type of question is easy to interpret and answer for respondents.

However, as with all question types, there are limitations. What is the difference between first place and second place? How about third place and fourth place? Perhaps a respondent is equally passionate about their first- and second-place rankings and they have to flip the proverbial coin to decide. Then they get further down the list, and third place is still strong, but fourth place is barely worth a second look.

Ranking questions alone cannot capture the close tie between first and second place, and they also cannot capture the chasm between third and fourth place.

Tips for Using Rating and Ranking Survey Questions

A Better Way to Rank – Pick a Clear Winner (or Loser):

To get around this gap in data, have the respondent rank either their top three or bottom three choices. You still don’t capture the gap between ranks, but you also don’t get bogged down in ranking choices that are, in many cases, irrelevant.

If You Need to Know Both – Ask Both!

Both rating and ranking questions are easy for the respondent to answer, so asking both within the survey is not unduly burdensome to the respondent. If you need to know relative positioning but also understand the gaps between the ranks, simply ask both!

Bonus Pro Tip: Optimize for Mobile

While both rating and ranking questions are easy to answer, consider this: Most people take surveys on their phones. This is especially true for consumer surveys, but even for expert populations, between 30% and 40% of the participants take surveys on their phones as they travel from meeting to meeting or from location to location.

Obviously, phones are limited in their ability to display a large amount of data in a way that doesn’t require an inconvenient amount of scrolling back and forth. Now consider a numeric rating scale (0-10) that asks to rate the performance of 10 different vendors. While the matrix table only requires 10 answers, there are 100 options to display.

There are no hard-and-fast rules to follow here, but always think through the various ways a question displays on different screen sizes.

The Takeaway

To rate or to rank? Rating survey questions offer detail about a variable independent of other factors. Ranking survey questions answer the question of what’s the best and what’s the worst. And if you need both, ask both!

Check out the other articles in our Survey Series:

• Top 8 Tenets of Survey Design
• What Type of Survey Do You Need?
• Are You Running the Right Survey for the Wrong Reason?
• Why the Screener Section of Your Survey is Compromising Your Results
• Surveying Basics: The Right Way to Reach Respondents

---

About Will Mellor

Will Mellor leads a team of accomplished project managers who serve financial services firms across North America. His team manages end-to-end survey delivery from first draft to final deliverable. Will is an expert on GLG’s internal membership and consumer populations, as well as survey design and research. Before coming to GLG, he was the VP of an economic consulting group, where he was responsible for designing economic impact models for clients in both the public and the private sector. Will has bachelor’s degrees in international business and finance and a master’s degree in applied economics.