Modern Data Companies

The Asia Research Annual Review of the Research & Survey Technology Business

The organisations that market research agencies have partnered with for so long, the ‘online panel providers’, are evolving into more sophisticated tech companies. So, we no longer use the term ‘panel companies’, but ‘modern data companies’. The panel or sample itself is being augmented and complemented by more holistic data collection and data management products that have an increasing range of applications for a wider client base.

As part of our annual review of the research and survey technology business in Asia, we interviewed the heads of the leading modern data companies (MDCs), including James Burge, managing director of Research Now SSI; Ludo Milet, managing director of Toluna; Sid Dutta, managing director of Lightspeed Research; Atsushi ‘Tomas’ Tamura, CEO of Rakuten Insight; and Fumiya Nakajima, managing director of GMO Research.

While research agencies still represent the core business of MDCs, in the last few years they have worked directly with more end clients, bypassing the agencies.

Ludo Milet from Toluna observes that traditional project cycles are becoming less common. The usual cycle of the agency designing a survey, fielding it, analysing, and reporting is increasingly being replaced by a series of shorter, lighter surveys, at a lower cost. While large strategic projects still go through the traditional cycle, clients can prefer to spend their research budget on a series of short, tactical research pieces. This is not so much a matter of budget – indeed, four tactical projects might cost the same as one strategic project – but it is about the need for speed.

Sid Dutta from Lightspeed comments that speed is also changing the way clients want their reporting. Lightspeed has developed end-to-end solutions, from templated questionnaires to reporting dashboards and data visualisation, e.g. through software such as Decipher and Power BI.

Clients are working ever more closely with MDCs, and consequently they are asking for a wider range of services, sometimes encroaching on full-service research. But all the MDCs concur that they draw a line between data collection and delivery and consultancy. Where the clients seek inputs to questionnaire coverage or interpretation of survey findings, MDCs will recommend the use of the full-service market research agency.

But the services the MDCs provide go far beyond the somewhat commoditised product of the ‘sample’, and now include a range of value-add services. Sid Dutta said that Lightspeed is organised around specific verticals to serve this need. This includes staff dedicated to subject matter expertise (e.g. for surveys that require purchase path analysis and specific data collection solutions), and a team dedicated to survey design. The latter is not so much about content (the area for agencies to advise on), but about enhancing survey usability (e.g. making them more engaging, fun, and easier to administer).

Sid points out that MDCs can be better at designing ‘modern surveys’. Agency researchers, who are used to the common 20–25-minute PC-based survey, might not be able to optimise their surveys for the speed and mobility that consumers really want.

James Burge from Research Now SSI comments that MDCs also have a better grasp of ‘in-the-moment’ surveys. Again, some of the more traditional researchers might think that consumers set aside time to do their surveys – think of the traditional 45-minute in-home interview with the interviewer, the clipboard, and the respondent patiently waiting for the next question. Sure, things have moved on a lot from in-home surveys, but researchers do still think of the respondent patiently filling in surveys on their PC, and thus might design their surveys accordingly.

James Burge comments that you need to look at the different stages of the consumer journey and you have to think of the right time, the right approach, and the right questions to ask respondents to ensure maximum response and engagement with the survey. In-the-moment customer experience research (e.g. during or immediately after a consumer has visited a store or a website) is progressively replacing the traditional tracking survey, which in some cases can be too backwards-looking and slow for today’s faster-paced marketing environment. The new generation of surveys is more about immediate interaction with the respondent, and is designed for the client to action the results within a matter of hours or days.

Tomas Tamura from Rakuten Insights says that AI itself will allow surveys to be shortened, for example by combining data from several sources so that we can ask fewer questions. However, ensuring that we have the consumer’s consent is key to achieving this. It is also important that these augmented data sources are trusted by the end clients.

All the MDCs are investeing in DIY services. Nothing new here, but among clients who need very fast turnaround, demand for these services is rising. Tomas Tamura continues that automation has improved efficiencies, but we should not forget that surveys still interface with people, and we still need people to ensure the quality of the survey design.


Despite the glamour behind the shiny new technology, for all MDCs sample provision is still their main business, and so too is the market research agency buying this sample. But for too long corporations have focused their campaigns on segments of the market with heavy internet usage and those living in the main conurbations. Online surveys alone miss huge parts of the population.

In the past, the concern has been about missing older age groups, something even more important given the aging populations in many Asian markets. But panels have been around for about 15 years, and panelists are getting older. James Burge points out that, while older consumers can be less ‘IT savvy’, they are usually ‘IT capable’ – the ‘silver surfers’ buy a lot online and keep in touch with family online, and hence are very capable of taking online surveys.

Across all the MDCs interviewed, the bigger challenge lies in accessing the lower socio-economic groups and those residents in lower-tier cities who are less represented on panels. In SE Asia in particular, a lot of research is still done offline. This is partly because online research in this region does not have such a large cost advantage over offline research, but markets like Indonesia, Vietnam, and the Philippines still have sample representation issues.

However, in some emerging markets, the move from offline to online also coincides with the adoption of alternative sample sources – for example, sampling is aligning to social media sources, reflecting a general trend in online behaviour.

The MDCs no longer promote themselves just based on their panel size, but on their reach, some of which is not even via panels.

Ludo Milet from Toluna says that cheaper smart phones from China can bring mobile surveys to more consumers, and data plans are getting more generous. Even web-enabled feature phones can be used for simple surveys, Ludo points out.

The MDCs are also working a lot with strategic partners with large databases. Telcos are great partners to work with because they have very large customer bases across most socio-economic groups at national levels, and they wish to monetise their databases. Respondents can be rewarded for taking part in surveys with increased data allowance, which helps to offset the consumption of data for the survey itself!

Ludo Milet from Toluna comments that mixed methods are being used more. It is no longer a case of the client asking for double opt-in panelists, but for a blend of panels, customer databases, river sampling, and even offline methods. Clients sometimes undertake a large pilot of mixed methodologies, analyse the findings, and then use these findings to determine the final sample blend for the main survey.

All the MDCs we interviewed use WeChat in China, whereby consumers can be accessed without being part of a panel. This ‘big river’ sampling opens up a huge population in the world’s most populous country, who can be rewarded instantly through WeChat itself.

Fumiya Nakajima from GMO Research comments that new big data companies in China are emerging, serving domestic clients who need a better understanding of their enormous market.


Automation and dashboarding are not new, but in the last year the biggest developments have been in DIY (e.g. clients self-serving for samples), often because they need very fast surveys and results (e.g. for tactical needs and pitches).

Again, not so new but growing in demand is passive data measurement for behavioural research. This can be supplemented by in-the-moment surveys, where the consumer is asked a couple of questions at a specified time so as to understand more about their behaviour. Ludo Milet says that Toluna is undertaking a lot of research that combines behavioural research via passive metering with ‘declared behaviour’ through surveys.

Panelists can also download applications that track their mobile and web behaviour. As panelists, they have ‘bought into’ the concept of research, and with sufficient incentives they are willing to be ‘stalked’ in this way. All the hype over privacy and GDPR is not hindering the research industry, at least not for now.

Fumiya Nakajima from GMO Research comments that new players (e.g. Happi) are creating a new survey ecosystem, for example through survey apps that provide higher rewards for shorter surveys and very fast turnaround. Furthermore, many clients run Survey Monkey, to “collect just a little bit of the ‘why’s behind the passive data they collect on customers,” but there can be limited insight on the analytical side of such surveys.

Other recent developments include data matching. This can combine survey data with the client’s CRM data and for programmatic, allowing clients to apply the survey findings more effectively – for example, to their media buying.

B2B research is still in its relative infancy in Asia. James Burge from Research Now SSI says they have quite good access to B2B through their specific partnerships, but B2B sampling mostly comes from profiling their consumer panel to understand more about their decision-making within companies. Sid Dutta from Lightspeed says that B2B is most advanced in the US, where Lightspeed is partnering with LinkedIn for B2B surveys, but Asia is still lagging quite far behind Western markets.


Intelligent questionnaires that adapt themselves to the responses to earlier questions have been talked about. This is not just about smart routing, but a sort of robot moderator similar to Chat Bot seen in some of the automated help desks of websites. Sid Dutta reports that Lightspeed is looking at this and has undertaken a lot of ‘research on research’ to assess how to optimise the open-ended question in surveys.

Tomas Tamura from Rakuten Insights says that we can do “really interesting things” with a combination of qualitative and quantitative, but he comments that one of the biggest challenges is the ability to roll this out across a very diverse region with all the different languages.

Fumiya Nakajima from GMO Research comments that data analysis, or any action plan backed by data, is still reliant on the human factor. But Qualtrics in the US (recently acquired by SAP) will see more of its solutions being automated through AI analytics.

Ludo Milet from Toluna comments that, for some research, the capacity to understand, adapt, and synthesise is still best served by human analysts, and changes in conversations are still best served by human moderators.

Well, let’s wait and see what this review says when we do it again in 2025!

By Piers Lee and Claudia Siregar, Asia Research Media