Are Your Social Listening Insights Actually Valid?

In my work with the Conversation Research Institute, I recently came across some social listening insights that I questioned.

There were over 800,000 posts on various social networks in the last year about planning a trip to some part of Asia. I was doing the conversation research for the Travel Blog Expo (TBEX) Asia-Pacific with the intent of telling travel brands and writers what North American consumers talked about online when they indicated they were planning vacations to Asia.

As we dove into the conversations, the phrase “won’t recommended” kept popping up, so we dove in to analyze why this negative term would be so prominent in posts about traveling to that part of the world. Granted, from a high level, it looked like a good portion of the people who talked about traveling to Asia were mentioning that it wasn’t recommended. When you drilled down to just the negative conversations, it was by far the most prominent sub-category.

For those selling travel to Asia, or for Asian destination marketers, this was a very bad thing. But that’s why you should always validate your social listening tools, charts and graphs with good, old-fashioned, human analysis.

Things Are Not Always What They Seem…

We furthered the research to categorize and organize any mention of the phrase “not recommended” and discovered some interesting commonalities. Paralleling the frequency of this phrase in negative conversations were the phrases “#GreatWall” and “#Terracotta warriors.” But those phrases were positive in sentiment.

Our experience tells us this means a single post was driving these results. It was one that was likely repeated over and over again – perhaps re-blogged or re-tweeted. And sure enough, we found it:

One influential blogger’s tweet was the culprit. Morgan Magazine has over 100,000 very engaged followers on Twitter. The post garnered hundreds of likes, shares and re-tweets. It was also echoed on other platforms beyond Twitter.

How Did This Change Our Insight Reporting?

First of all, any high volume negative topic is something to be concerned about in online conversations. It’s high volume for a reason, but you need to know a lot more before you actually know the reason.

In this case, one popular influencer made a flat statement about China. His network of fans and followers amplified it to the point of making its finer points emerge in a year-long look at the topic. So if you sell vacation packages to China, perhaps this is someone you want to reach out to, get to know and perhaps partner with to expose the good about traveling to China.

But this insight in and of itself doesn’t mean you have an image problem. It means you have one person who doesn’t think the world of you as a destination. The insight would be different if 1,500 different people said the same thing. But they didn’t. They simply echoed what one person said.

Validation Is As Crucial As Collection

Too often, brands who subscribe to social listening platforms forget that validating the data is just as important as finding them in the first place. But in order to do this, you have to have the time and analysis expertise to put your data through the rigors of proofing, testing, and validation.

This is why human analysis is required before you can act on a given insight that emerges from social listening platforms. It’s not enough to assume the software is right. You have to validate it is, then move forward with actions as a result.

To better understand your social listening data or see how conversation research can help your brand, drop me a line!