By: Jay Clegg
Intro – NLP & Sentiment Analysis
Although research in Natural Language Processing (NLP) dates back many decades, recent advancements in both computing hardware and NLP architectures have produced incredibly useful results that have attracted intense interest and investment in the field. In 2023, one merely had to ask “so who’s using ChatGPT?” in a crowded room to find plenty of people interested in talking about it.
There is a small subset of the NLP field called “sentiment analysis” (aka “opinion mining”) that I’d like to dive into today. Sentiment analysis is a method for detecting whether a natural language text is emotionally positive, negative, or neutral. “Sentiment” can be valuable feedback, but it is often obscured from traditional data processing due to it being in the form of unstructured, free-form text.
Some example use-cases for sentiment analysis are:
- Measuring sentiment in product or service reviews written by customers
- Tracking brand sentiment in social media posts
- Prioritizing customer service issues
Consider product reviews – product reviews can make or break sales for an e-commerce product or service. Sentiment analysis can help identify and extract positive or negative sentiment in an automated fashion, which can empower customer support/outreach teams to identify and help unhappy customers faster. Sentiment analysis can also enable these teams to serve a larger audience than they otherwise could.
I can’t cover the entire field in one blog post, but I’d like share some impressions of Microsoft Azure’s Sentiment Analysis service, which is one of their Azure AI Language services.
Sentiment analysis is part of a larger suite of read-to-use Azure AI Language tools that include:
- Language detection and language translation
- Question answering (ask conversational questions that are answered from a company knowledge base)
- Key phrase extraction (distill the main concepts in an unstructured text or user input)
- Text summarization (document & conversational)
- Named entity recognition (e.g. extract people, organizations, events, places, dates, products and more)
- PII recognition (identify, classify, and redact personal information)
- Conversational language understanding (chatbots that can identify human intent and entities)
Although the terms sentiment analysis and opinion mining are often used interchangeably in the field of NLP, Microsoft uses these terms to distinguish slightly different features in Azure AI Language service.
- Sentiment analysis will apply 3 labels to an overall text (“negative”, “neutral” and “positive”), each with a confidence score of 0 to 1 (i.e. 0%-100%). It will also return individual scores for each sentence. In both cases, all three confidence scores will sum to 1.
- Opinion mining returns more granular information about opinions related to words in the text. This information can be be optionally added to sentiment analysis requests to sometimes capture certain customer “assessments”. For example, “I thought the service was too slow” would return as a word pair: “Service”: “Slow”
Integration
AI Language service exposes REST API endpoints (collectively called “Prediction API”) that can be called to submit text for analysis and return results in JSON. To ease implementation, client libraries and quickstarts are provided by Microsoft in C#, Python, Javascript and Java.
Implementing custom clients in most other languages should not be difficult. Azure Synapse provides a machine learning wizard for sentiment analysis that makes getting started with Synapse integration almost trivial.
For highly secured environments, where it is not permissible to analyze data in the cloud, Azure provides a docker container that can run on-premise. These containers do all of the processing locally, but they do require an internet connection in order to relay billing information to Microsoft.
How Well Does Azure Sentiment Analysis Work?
My answer to this question will be subjective. I tried to find information on how Microsoft’s language model works internally, but it doesn’t appear that Microsoft has publicly published any details on how the language model works. I also couldn’t find any recent large scale objective metrics.
Based on my small experiments, I’d say it works quite well for some types of texts and not so well for others. It seems to recognize most negation phrases correctly (e.g. “I did not like this product” is obviously a negative sentiment), but it does not do well with oblique references and sarcasm. For example, phrases like, “If you like sitting on the phone with tech support for hours, then this product is for you” tend to produce false positives. It lacks the deep understanding of a large language model, such as ChatGPT.
The trade-off is ease of implementation and lower cost. With any implementation of sentiment analysis, proper evaluation against a representative sample of the intended target data is necessary. Also, given Microsoft’s recent shift to heavily leverage AI in various domains (including the newly released Microsoft Fabric platform), I would expect them to keep evolving their pre-built models over time.
For advanced use-cases, Microsoft provides all the tooling you need to train custom sentiment analysis models in Azure.
It’s worth noting that a very wide range of languages are supported, for both the pre-built and custom trained models. When training a custom sentiment model, it can be trained predominantly in one language, and then used against texts from multiple other languages.
Pricing
Azure sentiment analysis pricing has fairly simple pricing tiers, which it shares with several of its other AI Language features. The pricing is per 1,000 records, and records are defined as 1,000 character units. I’ve found that what I consider a short product review is usually less than 1,000 characters, with medium length review taking <3,000. The per-record cost starts at 1/10th of a cent, with substantial price breaks once you get into millions of records per month
Tutorial/Walkthrough
Experimenting with Azure sentiment analysis is free for the first 5,000 text records per month (at least when this post was written), so why don’t we dive in?
This walkthrough will show you the entire process of setting up the Azure AI Language service manually. We will end up with an instance that we will call using the Language Studio interface that can be used for ad hoc experimentation.
I’ll assume you have an Azure account already, so log into it, and let’s create a new resource group as a container for our AI Language service.
For the region, you can use whatever is appropriate for you, but for AI-related services, it’s definitely a good idea to check availability and compare pricing on Azure’s pricing page. The Free tier may not be available in all regions.
Click through the “next” prompts to create the resource group. Then locate and open the page for the resource group and click “+ Create”.
Next, when the marketplace page comes up, check “Azure services only” and enter “Language” or “Sentiment” in the search box. You should see “Language Service” as an option, so at the bottom, choose to create one.
You should see a page that lists several service features. Let’s just hit “Continue to create your resource”.
From there, things get just a little more complicated. Minimally, you need to make sure you have the correct subscription created. Choose the resource group you created earlier (“rg-JayDemoAI” for me). Again, choose the same region as before and give this new Language service a name (I chose “lang-JayDemoAI”).Make sure you have the “Free-F0” pricing tier selected.
Down at the bottom you’ll see a “Responsible AI Notice” and a check box to indicate you have read the documents. You will have to agree to that to proceed, so I recommend you read over those, and when you are ready to agree, check the box and click “Review and create”.
NOTE: there are extra tabs for Network/Identity/Tags. I’m going with the defaults, assuming we’ll just run some experiments and then clean up the whole resource group when we’re done. If you have special security or enterprise needs for this service, you may want to change some options there.
Once you’ve made it entirely through the process, you may see something like this below. It could take a couple minutes to deploy.
Eventually your new service will be up and running. You should see “Deployment Is Complete” and a button to go back to the resource group that contains this service. When you click that, it should take you back to your resource group overview and show the new service.
Then, let’s click on the newly created language service link (again, I named mine “lang-JayDemoAI”). You should see something like this.
Finally, since we’re just trying things out, scroll down to “Get Started with Language Studio”. Click that and it should launch a new tab for Language Studio, which should look something like the screenshot below. Locate the “Analyze sentiments and mine opinions” item and click “Try it out”.
And here we are, ready to do some sentiment analysis.
Ok, take a quick break and refill that coffee, now for the fun part. There’s not a ton of controls here, primarily a toggle for whether to additionally do “opinion mining” and a text box with a button to submit the text to be analyzed. So let’s try it. To save you a minute, Microsoft has some buttons further down to preload texts for you. Try “Service Review (long)” to repopulate the text box.
Down below that there’s another acknowledgement check box and a run button. When you click run you should see something like this.
(It’s possible you may get slightly different results, depending on whether something has changed since I wrote this article). At the top of the result pane, you should see the overall sentiment deleted for the entirety of the input text. I got 91% positive, 8% neutral, 1% negative for an overall score of “positive” with 91% confidence.
Scrolling down you’ll see sections for each sentence. You can expand these sections to see the scoring for each individual sentence that was detected.
If you left “opinion mining” turned on, below the individual sentence sentiments are the “opinions” from the text. These consists of a target and some number of assessments, each one having its own confidence value. You can see here I’ve hovered the cursor over a target and a pop-up shows the opinion in-sentence.
In Closing
Feel free to play around with their other samples and try some of your own. Existing e-commerce reviews or social media posts are a good source for sample text. I’ve found that short, single sentence texts tend to push the confidence toward 100%, even when it evaluates the sentiment incorrectly (because of sarcasm, etc). Longer, multi-sentence texts will break down the sentiment for each sentence and tend to give more reliable overall sentiment scores.
I hope this article and walkthrough gave you a good introduction to Azure’s sentiment analysis and opinion mining feature as part of their AI Language service. Let us know if you’d be interested in more posts about Azure’s AI Language offerings.
Also, if you are you looking for help in maximizing your Azure, you can learn more about our Azure consulting services here.
Keep Your Data Analytics Knowledge Sharp
Get the latest Key2 content and more delivered right to your inbox!
