As Margaret described in her earlier post about Sentiment, our sentiment scores agree with humans about 75% of the time, right out of the gate.
“Is that good enough?” people ask us. “It depends,” is the answer.
It is extremely powerful to have the system score hundreds of thousands of posts in real-time so that you can be alerted to potentially volatile issues and situations without doing any special scoring work yourself. For example, when Netflix logged in earlier this month, (or maybe when they opened their daily email alert from Scout Labs) they saw this data:
Pretty clear that customers were not happy about something on Monday March 30th. Drilling in, Netflix could quickly see that the decision to increase the price of its Blu-Ray DVD rentals was not a popular one! We correctly scored lots of unhappy posts.
We didn’t score every post correctly. I found one post that we scored as negative, based on the following phrase: “Netflix blamed the company for…” We thought it meant that Netflix the company was being blamed, but really it was Netflix doing the blaming. But in aggregate, Netflix gets a very good sense in real-time what customers are thinking, thanks to Scout Labs’ automated sentiment scoring.
And when you care enough to make sentiment the very best, you can always correct or override the sentiment values in the application, which updates the value only for you and your team. If you are an agency (especially PR) this is part of the value that you can provide to your client: getting the sentiment values JUST right. There’s an added benefit to scoring / improving sentiment, as the team at Scout Labs uses those overrides as labeled data sets that we can use to improve our sentiment accuracy rate over time.
So is sentiment scoring at Scout Labs perfect? No. Is it still incredibly powerful and useful in identifying problems? Yes. It acts as an early-warning system and brings very important problems to your attention.
Post a comment