AI and the impact on Market Research. (Survey will evolve).

As I had 6 months to run down an NDA and a financial cushion, I was lucky enough to think about the MR industry and where it would go. I also had the chance to deep dive into machine learning and generative AI to really understand what it was and how it worked, rather than just looking at outputs in some sort of wonderment.

MR industry counts and gathers opinion

Within the MR industry, we essentially count and gather opinion.

• Counting things is fairly basic and the transition over the last decades has been to count using technology, whether that be server-based measurement (e.g. footfall, google analytics, etc) or panel-based measurement (e.g. UKOP, behaviour measurement). This is fine. It won’t last forever, but I would not be surprised if this is still happening at scale in 10 years time.
• Gathering opinions is somewhat harder. My playing with ML and AI did not find anything that would suggest that opinions on anything new (read – where training data did not already exist) is even in scope for current AI or ML methods. This means that survey in the broadest sense is likely to be the last bastion of the MR industry.

Opinions matter and this won't change for the foreseeable

People’s opinions will matter for as long as people make decisions. Without any evidence whatsoever, I estimate that ultimately human people will still make most decisions for the next half century, even if they are significantly biased by algorithms. 

The concerns about political manipulation right now are the most obvious examples of how people will lose the ability to make decisions, but none the less, even in an election, it is a human being that actually votes, not an algorithm.

So, if survey is here to stay, what will AI do to survey?

I started to think about what a survey is trying to achieve and recognised that a survey and a strategy consultant (or journalist, psychotherapist, or anyone else who 'interviews' someone) have the same goals, but they are approaching in different ways.

·    The life cycle of a survey currently is that a client comes to an MR company with a problem, and a survey consultant converts that into questions which are programmed and served as a script for people to answer. Statistics are then performed on the responses to identify where the script failed to achieve, and where it did achieve, to reach some insight.

·    A strategy consultant performing an interview is somewhat the same. Their client comes to the strategy consultant with a problem and the strategy consultant breaks that into some high-level key information objectives (KIOs). In the interview, likely only one or two initial questions are scripted. As the interview progresses, the strategy consultant creates questions based on the answers they have received and the information already known to them before the interview, before determining the next question.

Obviously, we run surveys because they can be done at scale and provide consistent results, for very little expenditure. But what if we could simulate the function of the strategy consultant, but in an online and scalable way, using AI.

The Online Strategy Interview

Let’s break down what an online strategy interview does.

Preparation

1. Understand the high-level problem being asked by the client/stakeholder.
2. Understand the KIOs that will be found during the interview.
3. Understand what information is available prior to the interview.

During the interview

The interviewer iterates a process that looks like this:

1. What information do I know already, and does this answer a key information objective?
2. If not, what will be the most efficient way to ask the next question to most likely get an answer?
3. Receive and process an answer.
4. Go back to step 1 until complete.

 After the interview

Process the information into answers to the key information objectives and report these as findings.

How it would work (technical requirements)?

In a future online survey world, AI could achieve these steps. It would need the following key capabilities:

Understand the high-level problem being asked and KIOs and Known Information

This would be required in the script as an input. It may be that an AI in the near future can deconstruct a high level problem into Key Information Objectives.

Evaluate whether there is enough information to answer a KIO

This step is very hard, but from various proof of concepts, I believe this is possible with current technology. At the moment, it will require some human guidance, but as time progresses, AI will be able to achieve this.

Determine what question to ask to most optimally get an answer that is closer to achieving a KIO

LLMs are already good at creating text on behalf of code. Initially, this will need some guidance from a human, but at some time in the near future, this guidance will reduce to zero.

Receive responses and understand them

We are already in a good position using LLMs. The respondent can provide anything they like so long as the solution is digital. This could be traditional survey responses (multi, single select; open text) or more novel (e.g. documents, webpages, datasets, video), but the abilities to read and interpret all of these methods is rapidly coming onto the market, and already AWS (Bedrock) and Microsoft (Azure Studio) and independents (…) are delivering one stop shops for deconstructing data.

 

The important thing to recognise is that each of these 4 modular capabilities is absolutely possible now, or at least is only a small technological evolution away. Once these 4 capabilities are created in the general case, then the strategy consultant / journalist / psychotherapist can be replaced with only a guidance script that states the high level objectives and KIOs required in any context.

Note: I have not stated here that all this will need a database schema that is fast and flexible enough to handle queries. This should be obvious, but semi-structured databases (e.g. Mongo) or structured databases containing json fields are obviously available now.

So what would the client experience look like?

The client connects to the system and enters in their problem that they are trying to solve. A “survey” would already be in place to ask the questions that would be asked by a sales consultant, such as relevant webpages, media, key information objectives.

The client would also need to ascertain (or maybe agree with recommendations) what level of accuracy or certainty is necessary.

They then let the system work. The system would keep interviewing from the panel until it had enough information to answer all the Key Information Objectives within the quality parameters set out (or the budget ran out).

And what would the respondent experience look like?

The respondent would get a series of questions. (These could be presented as chat or as video – that is somewhat by-the-by). They would answer in any way they liked, but guided by the interviewer.

So, how is this different?

The fundamental difference between how surveys are conducted now rather than on this system is that questions are dynamically invented in real-time rather than being scripted and programmed in advance.

How realistic is this?

Very. Each step is possible right now with some human guidance. At the moment, each use case needs a guidance script, but this won’t take forever to be made redundant.