A Look At Sentiment Analysis For User Experience Professionals
Dear Machine Learning, help me to know thy users and detect how they feel.
The advancement of artificial intelligence has opened new doors into the ways that we use computers to help us detect and categorize online opinions. The power of big data and intelligent machine systems has made room for new tools that can help user experience professionals and marketing professionals alike. Now that we are equipped with such valuable tools we need to better understand their design so that we can make use of them. One particular use of machine intelligence that is quickly gaining traction is the implementation of sentiment analysis, also known as opinion mining. Imagine the ability to use a simple algorithm to find out what people think about your product, why they think that about your product, and how strongly they feel about that opinion. This article aims to explain how sentiment analysis works and the use cases for UX professionals.
What is Sentiment Analysis?
Sentiment analysis is a tool that aims to explain…sentiment! What that means is that you can use a computer to detect the polarity (positive or negative) of an opinion with a certain degree of confidence. In the simplest application a sentiment analyzer will tell you if the opinion is positive, neutral or negative. How does this help? On a larger scale, using big data you can uncover themes of sentiment to detect how people feel about your product. On a smaller scale, you can tailor a chatbot conversation to help the chatbot respond to a user’s sentiment. This technique delivers a smarter and more human-like artificial intelligence, which can respond in a unique way based on the emotions you show in a written chat conversation.
How does it work you ask?
Lexicon-based vs. machine learning.
There are many ways to analyze sentiment, but the most common way being used today is the lexicon-based model. This model uses a dictionary of words that are annotated (by humans) with their polarity (good or bad) and strength (how good or how bad). The lexicon-based sentiment analyzer combs through text and picks out specific words or phrases, called tokens, and classifies their polarity and strength to capture the text’s opinion towards its main subject matter. You can read all about this model from a long academic article written by some really smart people here.
The end result of this technical algorithm work is essentially a score or some other rating for your sentence, phrase or paragraph that tells you if that analyzed text is positive, neutral or negative.
There are too many ways to show polarity to explain here, but if you want to see this analysis in action check out liwc.wpengine.com and scroll to the bottom of the page to try it out with your own sentence. I must admit the tool is crude and not particularly useful unless you plan on using it to analyze collective opinions on a larger scale; think thousands of Twitter posts or product reviews on Amazon.
Detecting negative and positive emotions from your users is one part. The next step might be to then pick out some of the more blatantly negative posts to uncover more about why those users hold such opinions. The problem with lexicon-based models is that they are bad at detecting sarcasm and language nuances because it is based on individual words rather than a more holistic assessment. While it is easy to implement, the value of a lexicon-based approach is really in how it scales up and not necessarily in the accuracy of the analysis.
The other method of sentiment analysis is done through machine learning. This model uses a corpus, a set of data, which are labeled as positive or negative by humans. The labeled corpus is then used to train a classifier, which will classify new text it sees based on the data it was given from the corpus.
Essentially, the machine learning is much smarter because the deep learning approach uses abstract representations of what it learns rather than assessing text at face value as the lexicon-based approach does.
It can classify data in more general ways and progressively learn so it becomes smarter over time. So why isn’t everyone using this approach? Well, teaching computers to learn requires a good corpus, or set of classified data, to learn from and this seems to be a stopping point for many practitioners. As you might assume, the construction of the artificial intelligence tools that analyze the data is not particularly easy either. If you’re curious how to make it work and you got some Python programming knowledge, watch this video to see how opinion mining can be programmed to help detect sentiment in IMDB movie reviews.