Your gaze drifts to the upper right corner of your MacBook Air. It’s four o’clock in the afternoon. Only one more hour of work to get through and then you’re free to go home and catch up on the latest episode of Game of Thrones. You’d think most people would feel great about this. But that’s not true for a share of the American population on Twitter. For these workers four o’clock in the afternoon is the unhappiest part of the workday.
This conclusion was drawn by a team of five researchers at Northeastern and Harvard universities, who studied 300 million tweets from septembre 2006 to août 2009. What seems like an awful amount of work for five individuals — going through hundreds of millions of 140-character texts and determining the mood of the person who wrote them — was made possible with the help of artificial intelligence.
Each tweet’s tone was categorized by an algorithm that matched its content with an ANEW (Affective Norms for English Word) word list and generated a score that determined if it was positive or negative. From there, the researchers were able to establish the feelings of the people behind the tweets and how they evolved from morning until evening.
The technology behind this is called sentiment analysis, and its applications go beyond identifying how people’s moods change throughout the day.
Emotionally intelligent machines
Sentiment analysis, or data mining, is a field within artificial intelligence that uses algorithms to determine if the tone of a written message is positive, negative or neutral. It goes through sentences, MicroBlogs (short messages of 140 or less characters, like tweets of Facebook posts) or even entire documents and establishes the feeling, attitude, or opinion of the person who wrote them towards a specific topic, product, or brand.
This is the basic version of sentiment analysis. Other, more complex algorithms try to determine how strong that opinion is. They attribute a weight to the positive or negative words and come up with a score that ranges from -1 to +1 and use the result to represent the sentiment of the sentence.
But how exactly do machines read our emotions?
There are three main approaches to sentiment analysis: rule-based, automatic, and hybrid.
Rule-based approaches work by first defining a set of rules in a scripting language that identify the tone of the written message at hand. These rules use a variety of inputs that range from classic Natural Language Processing techniques to other resources like dictionaries or lists of specific words. In practical terms, researchers would, for example, create a list of positive and a list of negative words. The algorithm would then analyze the written messages and match their content with the one on the lists. If a message has more positive than negative words, then its tone is deemed positive and vice versa. Otherwise, they’re considered neutral.
Automatic approaches, on the other hand, don’t rely on hand-crafted rules but on machine learning techniques that are far more complex. Simply put, automatic approaches rely on training a computer by showing it enough examples, or training instances, that it will begin to recognize patterns on its own and learn how to transform the input into the desired output. In this case, one could use a set of tweets and their corresponding sentiment as the input and measure the algorithm’s performance by looking at the percentage of following tweets it would correctly classify into each possible category.
Hybrid approaches combine the best of both worlds to achieve more accurate results.
Sentiment analysis was initially developed for market research. Bing Liu, Professor of Computer Science at the University of Illinois and author of several texts on the subject, explains that it emerged in the mid-2000s, when online reviews gained popularity, and companies wanted to look into them to understand what their customers were saying. Before sentiment analysis, the only ways businesses had to find out how people were reacting to a particular product or service was to gather focus groups or send out surveys, from which they had no guarantee of collecting a statistically significant number of replies.
Then algorithms came and made everything easier. They can analyze bigger amounts of data in shorter periods of time and don’t require companies to reach out to customers to collect their opinions; they simply take the ones they naturally share online.
A data gold mine
As consumers, we produce an immeasurable amount of online written data. We send out e-mails, exchange ideas on message boards, post on our friends’ walls and comment on their Instagram photos. We increasingly spend more time on social media. In 2017, the daily social media usage of global internet users rose to 135 minutes per day, up from 90 minutes just 5 years before. We take to Facebook, Instagram and other channels not only to share our likes and dislikes, but also voice our opinions about specific brands or products.
The data collected from online reviews, social media posts and other sources holds a lot of value for businesses, because the results of a sentiment analysis can be used to improve several different areas.
For marketing efforts, for example, it allows companies to track whether the reactions to different events or campaigns on social media were positive, negative or neutral, and rethink their strategy to better cater to their audience. It is also a useful tool to tap into people’s feelings towards the competition, to try to gain a competitive advantage in the market.
Sentiment analysis has also recently proven to be helpful in human resources and people operations. Companies like Frrole have developed a DeepSense AI that takes publicly available social data from job applicants and, based on the results, allows recruiters to get a sense of their behavioral traits and personality to assess if they’re a good fit for the team.
Other larger companies have started using this technology to understand how their current employees feel. IBM, for example, has an internal social networking platform called Connections, through which all of IBM’s 380,000 employees can get in touch with each other, post opinions, and comment on other people’s shared content. IBM pairs this platform with an internally developed sentiment analysis tool called Social Pulse, that scans what people are saying and identifies trends and possible red flags in employee satisfaction.
Reading customers’ minds
Sentiment analysis is equally useful, and important, for monitoring and improving customer experience.
Customers’ feelings towards a brand can be influenced by a number of factors. Product launches or changes, price increases, viral campaigns and other marketing actions, and customer service quality.
Companies can resort to sentiment analysis to go through product or service reviews, for example, and attribute a score to each of them, allowing customer service agents to reach out to the customers with the most negative opinions first and try to defuse the bad situation as soon as possible. As for the reviews with more positive scores, these allow for companies to understand what actions trigger positive emotions on customers as a benchmark going forward.
The same principle can be applied to inbound tickets. Sentiment analysis algorithms are able to sort tickets by urgency based on the email’s tone of voice. This way, agents can turn their attention to the most frustrated or dissatisfied customers without having to go through each of their tickets first to manually assess their priority.
For managers, this is a useful tool to measure customers’ overall satisfaction with the support team. The results are based not on requested feedback after a specific interaction with an agent, but on less intrusive and more honest opinions that customers type out somewhere online because they feel like it. It can complement internal performance reviews as an added layer of customer feedback, from which managers can get a clearer sense of the interactions that worked better and use those insights to improve processes.
But the technology isn’t completely accurate yet. Is it based on language, which isn’t as black as white as a list of positive and negative words. Both can be used to imply the exact opposite tone one would usually associate with them. For example, to comment on a delayed delivery, I can say: “Amazon hasn’t delivered my order in time. Brilliant!” The word “brilliant” commonly has a positive connotation, but, in this case, it’s being used to comment on an unsatisfactory service.
Although some AI companies are already training algorithms to recognize sarcasm, there are a lot more variables, such as context or even the complexities of certain languages that make it harder for machines to correctly make sense of the implied tone of the conversation.
Nonetheless, machines are continuously getting better at sensing human emotion. There is still a lot of room for improvement in sentiment analysis, but researches are investing in other sentiment reading tool like facial scans. Who knows, maybe in a few years we’ll have a report on the least productive hours of the workday based on how many time our computers scanned our faces for a yawn.
The post All the feels: using sentiment analysis to improve customer experience appeared first on Unbabel.
