⏳️ BETA: This feature is not yet available in all Vanilla communities. It is currently in a beta state and limited to beta testers. This note will be removed and the title updated when the feature is globally available. In addition, you must contact Vanilla Support and request this feature be enabled for your community.
Higher Logic Vanilla (Vanilla) communities include a Sentiment Analysis feature that leverages AI to help you track and understand the language being used across your community.
Sentiment Analysis offers two types of analysis:
- Site-wide sentiment
- Keyword sentiment
These two features gives you powerful insights into both the general tone of your community and the more granular sentiment attached to specific keywords.
Each type of analysis is explained in more detail in their associated sections below.
⚠️ IMPORTANT: In the Calibrate your sentiment levels section of this article, you'll learn how to calibrate the range between the four sentiment levels (extremely negative to extremely positive). While this calibration is configured on the Sentiment Keywords page, it affects your entire site's representation of sentiment (site-wide, Keyword, and even Automation Rules). If using this feature, this is generally the first thing you'll want to do.
How sentiment analysis benefits your community
Before we dive into each of these sentiment analysis features, let's first review how sentiment analysis can benefit your community and its managers. Here are a few key benefits:
- Understanding community mood: By analyzing the sentiment of posts, comments, and messages, community managers can gauge the overall mood of the community. This helps in identifying whether the community is generally positive, negative, or neutral at any given time.
- Identifying issues early: Sentiment analysis can highlight negative trends or spikes in negative sentiment, allowing managers to address issues before they escalate. This proactive approach can prevent potential crises and improve community satisfaction.
- Improving engagement: By understanding what topics or types of content generate positive sentiment, managers can tailor their content strategy to better engage the community. This can lead to higher participation and a more vibrant community.
- Measuring impact of changes: When new features or policies are introduced, sentiment analysis can help measure the community's reaction. This feedback is crucial for making informed decisions and adjustments.
- Enhancing customer support: For communities centered around products or services, sentiment analysis can help identify common pain points and areas for improvement. This can lead to better customer support and product development.
- Personalizing interactions: By understanding individual sentiment, managers can personalize their interactions with community members, making them feel valued and heard.
Site-wide sentiment analysis
This analysis provides an overall sentiment score for your community based on user posts. It tracks sentiment across discussions, categories, tags, and even users, allowing you to quickly spot trends in community sentiment. You'll see who the top advocates and top detractors are, and how the overall tone is shifting over time.
How to view this data
This analysis is provided to you in an out-of-the-box Dashboard called Sentiment, available in Vanilla Analytics:
By default, this Dashboard provides the following site-wide sentiment data:
- Post and Comment sentiment
- Average sentiment by Categories and Tags
- Top 10 positive and negative members
- Questions with low sentiment
- Top contributing member sentiment
📝 NOTE: Use the calendar dropdown at the top right to define a custom date range for more precise data.
Customize the Sentiment Dashboard
You cannot edit the out-of-the-box Sentiment Dashboard, but you can, while viewing it, click the Copy button at the top right to make your own version. This enables you to:
- Edit and/or delete any of the default charts
- Add your own custom charts
Once your copy is saved, it's stored in the Dashboards section of the Analytics page.
Keyword Sentiment
This feature enables you to analyze the sentiment around specific keywords. By tracking keywords in posts, you can monitor sentiment related to specific products, topics, or issues. For instance, you can track the sentiment around a new product launch, helping you measure community reactions in real-time.
How it works
- Each time a user creates a post, it's sent to Azure AI Services to be evaluated. The text is scanned for the presence of up to 10 of the most salient keywords, which are then analyzed and given a sentiment score.
- The AI does this by analyzing the context of the words surrounding a keyword. For example, if "Green" is the keyword and the sentence is, "I love the color green," Green would have a positive score. On the other hand, the sentence, "I think the color green is ugly" would result in a negative score.
- Each of these notable words/terms come back to Vanilla and are listed on the Sentiment Keywords page. From this page, you can perform several actions that we'll discuss below.
Access the Sentiment Keywords page
- Access the Dashboard.
- Navigate to Settings > Discussions > Sentiment Keywords.
Calibrate your sentiment levels
Click the settings icon (to the left of the Add Keyword button) to display the Calibrate Sentiment Levels dialog. Here, you can drag and drop the four sliders to calibrate the range between the four sentiment levels:
- Extremely Negative to Negative
- Negative to Balanced
- Balanced to Positive
- Positive to Extremely Positive
This calibration enables you to customize the ranges between each sentiment score, giving your organization the flexibility to determine what range is considered negative, what range is considered positive, etc.
As you move these sliders, the preview chart below will automatically update to preview how the content in your community would be distributed based on your adjustments.
✔️ TIP: This preview uses your community's actual content, so you can adjust these sliders to fit its "mental modal." For example, if you have a support-based community, it's expected for most of the content to be negative in nature because users are looking for help. We recommend skewing the scale so that most of your content is in the Balanced range, so the truly "negative" and " positive" content is more discernable.
Track a Keyword
One of the primary uses of sentiment analysis is tracking specific keywords to be further analyzed in Vanilla Analytics.
- Only keywords that are tracked can be leveraged in analytics.
- You can choose to track any keywords on the Sentiment Keywords page, including those that are automatically extracted from users' posts or those you have manually added yourself.
⚠️ IMPORTANT: You can only track a maximum of 100 keywords at a time. Keep this in mind when deciding the most important keywords your organization wants to analyze in Vanilla Analytics. With this tracking cap in mind, we recommend occasionally untracking keywords that are no longer needed or important.
Track/untrack keywords
For any keyword, click the Tracking toggle to control whether it's tracked or not.
Choosing to untrack a keyword results in the popup below, informing you that the keyword will no longer be tracked in analytics but that any historical tracking data will remain in tact.
Manually add (and track) a keyword
You're free to add your own keywords at any time, which organizations often do to prepare for an upcoming product release, event, conference, etc. This way, you can keep an eye on the sentiment regarding keywords important to your organization as time goes on.
⭐️ EXAMPLE: If your organization is launching Product XYZ next month, you could preemptively add "Product XYZ" as a keyword so you could monitor sentiment about it after its release.
To manually add a keyword:
- Click the Add Keyword button at the top right of the page.
- In the resulting popup, type your keyword in the text field.
- Click Save. Your new keyword will be added to the list, and automatically toggled to be tracked (you can untrack it, if desired).
Filter and sort keywords
Let's learn how to use the filter, sort, and page navigation options to more easily locate specific keywords.
Keyword filtering
To help pinpoint specific keywords, you can use the following filters:
✔️ TIP: After selecting one or more filters, click the Filter button to apply them. Click Clear All to remove them.
✔️ TIP: Applied filters and sorting are added to the page's URL, making it easy to save URLs tied to specific filters/sorting you want to easily view in the future or share with a peer.
- Date last used: Set a From and/or To date to limit keywords by when they were last used.
- Category: Filter keywords by a specific Category.
- Sentiment: Filter keywords by one of the five sentiment levels: Strongly Negative, Negative, Balanced, Positive, Extremely Positive.
- Filter by: Select Subcommunity or Locale and then use their associated dropdown below to choose one.
- Tracked Status: Filter keywords based on whether they're tracked or not. You can choose, All, Tracked Only, or Untracked Only.
Keyword sorting
By default, the Sentiment Keywords page is sorted by usage, from most used to least.
✔️ TIP: Applied filters and sorting are added to the page's URL, making it easy to save URLs tied to specific filters/sorting you want to easily view in the future or share with a peer.
Click the column headers to sort keywords by:
- All Time Sentiment (Extremely Positive to Extremely Negative / Extremely Negative to Extremely Positive)
- Keyword (alphabetical A/Z or Z/A)
- Number of Uses (least number of uses / greatest number of uses)
- Last Use (most recent / earliest)
✔️ TIP: An up or down arrow is displayed on the currently sorted column to indicate the sort direction.
Page navigation
Over time, your community will analyze a lot of keywords, and the list will grow to an expansive size. In addition to filtering and sorting the list to more easily find what you're looking for, you'll likely want to navigate through multiple pages to browse your community's keywords.
To do so:
- Click the →| icon to display the Jump to page fields, allowing you to type a specific page number to view.
- After entering the page number, click Go.
Analyze sentiment data in Vanilla Analytics
Vanilla Analytics includes several data sources that enable you to analyze the sentiment data across your community:
- Keyword
- Posts
- Post Modifications
- Tags
📝 NOTE: If you're not familiar with creating custom dashboards, adding charts, and using data sources to analyze analytics data, check out Create Custom Dashboards & Charts.
Posts, Post Modifications, and Tags data sources
While the Posts, Post Modifications, and Tags data sources are not specifically designed to analyze sentiment data, they include several options that are incredibly useful in doing so.
In these data sources, you can use the Group By (shown below) and Filter options to target the following sentiment data:
- Sentiment
- Positive Sentiment Score
- Neutral Sentiment Score
- Negative Sentiment Score
- Average Sentiment Score
📝 NOTE: The available options differ depending on the data source.
Sentiment analysis examples
These data sources and their associated Group By, Filter, and Metric options allow you to analyze sentiment data in many ways. Here are a few examples to give you some ideas:
- Average sentiment by Category (data source = Posts; Group By = Category Name; Function metric = Mean and Field metric = Sentiment Score)
- Average sentiment by Tag (data source = Tags; Group By = Tag Name; Function metric = Mean and Field metric = Sentiment Score)
- Average sentiment by Role (data source = Posts; Group By = User Role Name; Function metric = Mean and Field metric = Sentiment Score)
- Average sentiment by discussion (data source = Posts; Group By = Discussion Name; Function metric = Mean and Field metric = Sentiment Score)
