The future of automation (Part 2)

Tribes Research 10th March 2017
Blog Latest News -

The future of automation (Part 1)

What does this mean for market research?

Automation, whether by digital assistants, robots, smart devices or A.I. means data, and lots of it. And it is data about how people are living their lives, their habits, lifestyles, choices and behaviours. What’s more, it is passive data – the individual is not having to answer a survey or talk to a market researcher, they don’t have to remember what they did and when, the data is being generated as the individual lives their life.

Market researchers love data, the more of it the better, the more accurate the better, the more up to date the better. However, this does not automatically mean that market research will benefit from the abundance, quality and immediacy of data being generated; there are difficulties to face. Two of the most significant difficulties are

  1. Data chaos and order. We said market researchers love data and the more of it the better but there is a caveat. They love data that they have specified, structured and validated. Researchers want ordered data, the concept of cards, columns and punches survives in MR decades after punch cards have been superseded. Passive data will be received with less structure, less order than survey data and a structure will have to be applied to it and it will be classified with other similar data. There will be masses of data but much of it will be superfluous, of no interest and it must be filtered out.
  2. Ownership. Market researchers are used to collecting data themselves on behalf of their clients. In some cases, such as syndicated studies, the market research agency owns the data and sells it to clients. In other cases, a standard as ad hoc for example, the client owns the data with the market research agency being responsible for collection, processing and analysis governed by a contract that explicitly allows such data processing. In a world where passive data is being collected from multiple sources, ownership and rights over data will become a little more complicated.

Data, data everywhere

How will market research organise, manage, and interrogate the masses of data produced from an increasingly automated world? The answer is by automating data processing and then automating much of the data interpretation and reporting.

Complex algorithms will automatically process, clean, tag and categorise each piece of raw data into manageable sets which a market researcher would recognise and sort for tabulation and analysis. However, given the potential volume and a continuous flow of data, it’s likely that tabulations, dashboarding and presentation will be automated and artificial intelligence will be trained to perform an initial evaluation of the data to identify trends, anomalies, and areas of interest.

Market researchers will still be needed. Firstly, for sense-checking and quality assurance, although much of this can also be automated, and secondly for insight.

Insight comes from understanding and interpreting data but it also comes from understanding context and the environment in which the subject is positioned and how it interacts with the world around it. This is a task for humans and as much as we might imagine a future where robots and artificial intelligence help us to live large parts of our lives, it is hard to imagine a time when a human’s thirst for knowledge and new experiences combined with a person’s unique quirks, irrationalities and inexplicable behaviour can be digitised and reduced to a level that a machine can understand, interpret, and explain, in an as meaningful and insightful a way as a fellow human being can.

Whose data is it anyway?

If a consumer buys a book from a bookseller a transaction is completed and both parties ‘own’ the information (data) from the transaction. There is some overlap in the information but also proprietary information known to only one party. Typically, both parties are free to sell the information to interested third parties (unless contractually forbidden).

The same principle applies with passive data. A supplier of a smart heating device will collect ‘transactional’ data when the heating in a household is switched on, off, temperature changed, etc. The information is stored on the supplier’s servers, combined with data from all other households and could be sold to third parties. The householder could also sell their passive data by connecting a metering device to their internet router that automatically uploads their heating transactions as well as their Smart TV and internet interactions.

For market researchers to access the data they must buy it from the smart device supplier or buy it from the producer of the router metering device. Buying from the smart device supplier would work in a similar way to retail audit data – a contract to acquire the data for a period of time in exchange for money or insight generated from the data, particularly if combined with data from other smart device suppliers. This is likely to favour the existing retail auditors and big MR companies that can do the deals to acquire the data and manage large volumes of disparate data, to slice, dice and distribute to appropriate departments in the organisation.

automation - data smart meter

For the other route to data acquisition, the market research company can acquire it from the metering device producer or develop their own metering device. In both cases, MR companies with large consumer research panels have an advantage because the placing of a metering device to collect ‘smart’ information passively about activity within the home will require consent from the householder. Consumer panel companies are ideally placed to find willing volunteers and will also have access to a whole host of complementary lifestyle, profiling and behavioural data on the household/individual to add depth and value to the passive data. The panel companies can then package this up and sell it on to other MR companies or end clients.

In the example above, there is a trade-off between the limited scope of data from independent smart devices on a large sample size and a wider range of data on a smaller sample size (requiring householder consent to collect) and market research companies have opportunities to exploit. However, there is an elephant in the room, or, to be more precise, a digital personal assistant in the room.

By connecting all the smart devices in the home the personal assistant will be controlling heating, lighting, TV viewing, music listening, food deliveries, monitoring commuting times, internet searching and shopping, managing diaries and more. And collecting a huge amount of data in the process. With data comes power and as the digital personal assistant market grows it will become more concentrated as the leaders emerge leaving a few winners to share the spoils.

The personal assistant winners will have huge power at their fingertips and it’s questionable how this will play out for market research. Maybe they will be able to buy the data and sell it on or maybe they will be cut out altogether as the data is processed and packaged automatically and sold directly to retailers and manufacturers. It is also conceivable the individuals could be prevented from selling their own data by terms and conditions that assign ownership to the P.A device producer.

We said earlier that human insight was needed to interpret human behaviour and choices but if the digital personal assistants take hold, maybe they will be making the choices and managing behaviour, in which case another artificial intelligence could interpret, evaluate and report it, and maybe market researchers will be surplus to requirements.