Are Professional Respondents Still An Issue?

By James Rogers, Managing Director APAC, PureSpectrum

Image by: Fabio Principe, PopTika – Shutterstock

What constitutes the nature of a Professional Respondent?  Is it the number of ‘panels’ that they have signed up for? Is it the number of surveys that they undertake per month? OR is it their overall motivation for participating in market research? Are these the real factors we need to be addressing?

While market research budgets are closely scrutinized and in a lot of cases reduced, year on year, this hasn’t necessarily translated into less demand in terms of the volume of interviews.  For the most part the requirement is to deliver the same OR more interviews for less! There is no question that the tightening of belts at the top has a direct impact on those lower in the market research supply chain. This practice has given rise to not only issues of quality, but also the emergence of new types of survey ‘fraud’ including bots, survey farms and AI-based challengers.

Whilst the dark forces at play are a major obstacle for the industry, the conversation around ‘professional respondents’ is still very much a valid topic, and one that deserves attention and further clarification.

In the single source panel world of days gone by, perhaps the idea of limiting the number of surveys per month and having lots of exclusion periods based on category, expecting panellists to be members of only 1-2 ‘panels’ was a novel expectation (maybe). But can the same be considered now? Professional respondents have been described in the past as those who participate frequently with the objective of directly benefiting from the incentives given. If price pressure continues to drive down CPIs, then by necessity so does the incentive itself, so at this point it’s not clear whether this still remains an accurate definition.

‘Traditional’ proprietary panels (single source as they are now referred to) were recruited via multiple channels but predominantly had one common ‘carrot’ that brought sign ups…money! The industry has indeed evolved; however, we should not forget that the trusted panels we now compare performance against were built by ‘list buying’, the promise of ‘instant’ rewards and wide scale SEO/Keywords such as ‘surveys for cash’ or ‘cash for opinions. Proprietary panels, where they exist now, operate in a mostly similar way, although now there is more diversity of channels, such as the rubber-stamping of social media influencers to entice recruits. The risk here of course is that the research buyer does not know where the respondents are coming from, nor how they got there. Worse still is a patchwork of ‘proprietary’ plus partners without the necessary safeguards in place.

Marketplaces, like that of PureSpectrum, are designed so clients can clearly see the respondent sources. The PureSpectrum Marketplace can detect and deduplicate where we see the same respondent coming from multiple sources. Given that all would agree we operate in a ‘multi-source’ ecosystem this should not come as a surprise; it may however be slightly uncomfortable when looking at the frequency in given markets. But should it be? There is not an unlimited percentage of the population that wants to undertake market research surveys. If the motivation of a respondent is to provide their opinions, then why would they not seek out multiple avenues for doing so? If the respondent is looking to ‘earn’ rewards, are they really of lesser value than those participating because they ‘enjoy’ it?

To illustrate the situation, we sampled N=1,000 respondents across five Asia Pacific countries. Respondents in Australia, China, Hong Kong, India, and Singapore were asked questions about their survey habits, primarily focusing on:

  1. How many market research ‘panels’ do you actively belong to?
  2. On average, how many surveys do you respond to/complete per month?

Now, not all these markets are equal; they vary wildly in terms of population size as well as the demand for research interviews. India and China dwarf Australia in terms of population and yet the latter is one of the most heavily researched audiences in the world with an abundance of MRAs and of course ‘Panel Suppliers’.

So, what did the data say? Some key takeaways

Panel membership for the most part demonstrates a very ‘loyal’ trend with over 43% maintaining a single panel affiliation. The idea of ‘professional respondents’ seemingly would apply to those on three panels or more, and under which contradicts the common expectation.

Is this data surprising? Given the markets in question and based on what is widely accepted—no, it is not. Is there a correlation between the frequency of memberships and the volume of surveys in each market? Yes, of course. Where this is a high demand correlating to population size, then those wanting to participate do indeed participate! Australia for example has the third largest population of markets reviewed and yet punches well above its weight per panellist in terms of survey sign ups.

When it comes to frequency of surveys per month, the data also delivered no surprises (for those that work within the fieldwork community at least).

  • The minority on average (<25%) participate four or less times per month.
  • Those participating in five to nine surveys per month average 28% across these markets.
  • The majority of the respondents (45%) undertake 10+ surveys per month, >2.5 per week. This varies again market by market and is again tied to the overall demand.
  • 2% provided the response of ‘prefer not to say’
  • Notably the smaller markets (Australia, Hong Kong and Singapore) increase the overall average because there is high demand and less addressable population.

When it comes to the ‘motivation’ for respondents this was also a key question and one that was weighted relatively equally (+/- 2%) across all markets. 60% of the respondents stated that their main reasons for participating were to ‘contribute their opinions’ and because ‘they enjoy taking surveys’. Again, not a large surprise here. Of course, for some the motivation is the incentive but this has always been the case, regardless of the medium methodology. We ultimately pay for people’s time and opinion, so rightfully they would be rewarded.

Multi-sourcing and multi-membership are a part of online surveys whether client’s source from ‘proprietary plus partners’ OR directly from marketplaces. The key to maintaining the best possible quality is to make sure you are aware of sources (and potential source bias) and the organization you are working with has capable guardrails and automation in place to make sure you are protected.

With the 3rd iteration of PureScore™ released this month and the full roll out of PureText™ (OE evaluation), PureSpectrum is at the forefront of time spent dealing with these challenges. Having more than five years and over 10 billion sessions in our armoury, we are comfortable having these conversations and demonstrating the tools we have to detect, manage and control the right respondents to join your surveys.

As you proceed to your next project, perhaps in addition to asking about professional respondents, perhaps you should start with:

  1. What proportion of your ‘panellists’ will come from a) Proprietary b) API and c) Partners?
  2. What processes and tools do you have in place to monitor and control biases
  3. What processes and tools do you have in place to assess behaviours?

About PureSpectrum

PureSpectrum offers a complete end-to-end market research and insights platform, helping insights professionals make decisions more efficiently, and faster than ever before. Awarded MR Supplier of the Year at the 2021 Marketing Research and Insight Excellence Awards, PureSpectrum is recognized for its industry-leading data quality. PureSpectrum developed the respondent-level scoring system, PureScore™, and believes its continued success stems from talent density and dedication to simplicity and quality. In the few years since its inception, PureSpectrum has been named one of the Fastest Growing Companies in North America on Deloitte’s Fast 500 since 2020, twice placed in the Top 50 of the GRIT Most Innovative List, and ranked twice on the Inc. 5000 list.

This article was first published in the Q2 2024 edition of Asia Research Media

Leave a Comment

Your email address will not be published. Required fields are marked *