Data, a massive amount of data, seems to be the holy grail in building more sophisticated AI’s, creating human-like chatbots and selling more products. But is more data actually better? With GDPR significantly limiting the way we generate intelligence through collecting personally identifiable data, what is next? How can we create a specific understanding of our customers to exceed their expectations and needs with less data?
Many of us collect anything we can get our hands on from personal information, behavioral data, to “soft” data that one might run through a natural language processing (NLP) program to examine interactions between devices and humans to pull meaning from conversations to drive sales. We believe the more we collect, the more we know. But is that belief true or merely a belief?
Today, there are an estimated 2.7 Zettabytes of data that exists in the digital world, but only 0.5% of this data is analyzed. With all of this data being collected and only a fraction of it being analyzed, it is no wonder that 85% of data science projects fail, according to a recent analysis by Gartner.
Why?
Data science is utterly complex. If you are familiar with the data science hierarchy of needs, AI and machine learning, you know that for a machine to process data and learn on its own, without our constant supervision, it needs massive amounts of quality data. And within that lies a primary question — even if you have an enormous amount of quality data, is it the right data to drive the insights we need to know our customers?
A client of ours shared their dilemma, which might sound familiar to you. They collected all possible data points about their customers, data that was bias-free and clean.
“We know where the user came from, what they bought, how often they used our product, how much was paid, when they returned. Etc. We created clusters within our database, segmented them, overlaid with additional identifiable data about the user, and built our own ML algorithm so we could push the right marketing message and price point to existing and new customers to increase sales volume.”
All this effort and all this data did not lead to increased sales, they still struggled to convert customers. Was their data actionable? Our client assumed that the data available was directly correlated to driving the purchases. And here lies the dilemma with our assumptions and datasets — we believe the data we collect has something to do with the purchase when in reality the answer is, not always.
Solution?
Humans are “wired up” to use multiple factors to form a buying decision. Those decision are not made in a vacuum. They don’t just happen online, or offline. Each product, service or brand is surrounded by a set of factors or needs by the customer that play a vital role in their decision-making process, which in most cases has nothing or little to do with one’s demographics, lifestyle, price of the product, etc.
This idea is based on the principle of behavioral economics, which explains that multiple factors - cognitive, social, environmental and economic — play a role in one’s decision process and directly correlates to how we decide what to spend our money on and how we choose between competing products/experiences.
All these factors combined allow customers to individually determine if a brand can fulfill their expectations or why they may prefer one brand over another.
Relevance?
Product preference is directly linked to market share in sales volume. An analysis by MASB found a direct linkage between brand preference and market share across 120 brands and 12 categories. So, if the data that is collected does not directly link to preference how can any data model, ML algorithm, or AI be useful in stimulating sales?
Stephen Diorio, from Forbes, argues, “if more executives understood the economics of customer behavior and the financial power of brand preference they would be better armed to work with CMOs to generate better financial performance.” To remain competitive and stop the rat race for more data, many urge that “companies must apply the latest advances in decision science and behavioral economics to ensure that investment in market research and measurement will yield metrics that isolate the most critical drivers of brand preference.”
In the future, collecting data will become more complicated and in some senses, limited by GDPR, CCPA, and other privacy laws worldwide. To get ahead of the curve companies should quickly shift their perspective from gathering data about who, where, when, what and how, to a broader understanding as to why customers prefer what they prefer and what drives those preferences. Understanding your customers from a behavioral economics point of view will deliver superiority over the competition, how to exceed your shoppers’ expectations and how lean their preference in your favor.
