Adding a LLM AI bot to your community to answer (product) questions

gxjansen · 2023-04-16T18:24:54+00:00

There was an error rendering this rich post.

Many first timers (especially in product oriented communities) that go through your community (or even ask a question) aren't looking to join a new community at all, they're not there for your amazing community plan and experience (sorry): they just want an answer to their question, ideally as quickly as possible.

To facilitate this, we're already looking into using Vanilla's federated search which hopefully will allow our community members to simultaneously search through the Forum, our Documentation, Github, Academy (courses). Maybe even add Youtube and StackOverflow.

But I'd want to go beyond federated search.

I'm interested in enhancing the core community function of (quickly) answering questions, inspired by the latest LLM/AI developments.The LLM would need to be trained specifically on our data (Forum, Documentation, Github, Academy, maybe even support tickets) and when a user either searches on forum or posts a question in a specific subforum, the AI could post a near immediate response.

Just like with GPT-4, I wouldn't expect the answers to be perfect (but neither are many real forum responses). It could even state this as a default disclaimer. But it would be great if the bot could provide quick solution directions that help the user solve their problem faster/better. Maybe we can even build in a feedback model where community members rate the bots response and then feed that back into the training data.

A quick mockup/example of what this could look like in Vanilla:

Pffff-How-do-I-solve-this-It-s-so-complicated-—-Spryker-Community.png

I know I could connect GPT to our forum through Zapier, and that you can fine-tune GPT-3 (no 4 so far AFAIK). But would that be my best approach? Since I also want this training not to be one-off, but want to keep it trained on ongoing basis on new community messages. Are there already services around this (connected to GPT-4 or other LLMs)? Freelancers that could build this?

Find more posts tagged with

Comments

AndrewLapidus

Love the idea, but unless I've understood it wrong, it sounds to me more like a way to optimize Support rather than build a community. I totally agree that users are not interested in our community plans and experiences — they almost always arrive to solve a selfish problem. But I've always operated under the assumption that the key to building a community is "activating" these selfish users and turning them into collaborative and engaged community members.

I'm also very interested in integrating better AI-copilots in our user support journey, but I imagine this would need to be severed from community. Would be happy to hear other perspectives on this intriguing topic that's taking every tech company by storm!

BrendanP

Really interesting ideas shared here so thank you for the post. I've always subscribed to the notion that when you boil it down - community is about people speaking with other people. I think AI will play a role in the future of these platforms in different ways but similar to Andrew, it seems to me like community content could be something consumed by AI to help provide answers as part of a support function, rather than the main function of community.

Support is certainly a critical part of what communities provide. It's the ability for there to be more nuanced and complicated questions that make communities useful because it's a fellow human understanding your question and attempting to provide an answer.

I've seen the Zapier example you referenced before and it's certainly cool - I am just not quite sure if these platforms know enough about platforms like ours yet to provide good answers. I imagine there will be products developed based on the ChatGPT API to review all the content a company provides (blogs, KB content, docs, community content. etc) to provide answers. I think that could certainly be cool but I am personally a little bit uneasy about what that means for the future.

Customers expect self-service options and we all love getting answers to our support questions as quickly as possible. However, when I think about the longer-term goals of building a community, I think the truly special moments are when customers engage to provide insights and answers to one another. It's difficult to measure but I do believe there's something very impactful about the authenticity and connection that's created when your community peer provides you with an answer rather than someone from the staff or a bot.

gxjansen

@AndrewLapidus

the key to building a community is "activating" these selfish users and turning them into collaborative and engaged community members.

Why would adding a bot block or inhibit that? In my mind, providing a fast answer to a product question to solve an issue that I have right now is a great way to get people to return to the forum more often. And that in itself is a great start to start expanding that relation.

To be clear: I want this bot to be active in specific categories that are aimed at answering (product) questions/howto's. I don't want it to involve itself into discussions.

So for example: the bot can (try to) answer "Which extension should I use for [Feature X] and how do I configure it" in our Product Q&A category, but I wouldn't want it involved in our generic discussion forum involving itself in "How do you see e-commerce evolve in the next 5 years and how should our product adapt to that.

gxjansen

I actually was able to integrate ChatGPT on our Vanilla staging environment, it looks like this:

B2B-What-are-my-options-—-Spryker-Community (2).png

Despite this being the generic ChatGPT product that is not trained on our data, it's kinda neat that even with the input question here being generic (as in: it doesn't mention our brand name), you can still make it answer about your product/brand.

(Gave the bot its own user role so I could give it a different background to make it stand out from human answers.)

gxjansen

@BrendanP

I am just not quite sure if these platforms know enough about platforms like ours yet to provide good answers. I imagine there will be products developed based on the ChatGPT API to review all the content a company provides (blogs, KB content, docs, community content. etc) to provide answers.

FYI: There already are programs that use the ChatGPT "Engine" but not it's training data and then let you enter your own training data. Which is good for both GDPR/legal reasons as for being able to feed it up-to-date info about your own company/product and to let it train of forum/chat data itself, making the answers much more relevant to your users.

That training on your own data is something that is definitely needed imho before it makes sense to add this to our forum, but might depend on the kind of forum and topics you have.

gxjansen

@BrendanP

I think the truly special moments are when customers engage to provide insights and answers to one another. It's difficult to measure but I do believe there's something very impactful about the authenticity and connection that's created when your community peer provides you with an answer rather than someone from the staff or a bot.

I'm not disagreeing. But do you think an answer by a bot (and let's assume for a moment here that content-wise it's actually a decent answer) inhibits this? I mean: it's not that a bot answering prevents any human from answering? It might even spark the discussion when humans try to out-perform the bot if you have some competitive community members :).

In my mockup in the first post you can also see the line

Maybe we can ask @user1 and @user2 chime in here as well, as they seem knowledgeable 💡 about this topic.

IMHO That could be a nice addition to invite humans to this post: acknowledging that these users are experts on the topic of the question (based on the data of your forum) and have them correct/supplement the post of the bot.

Shauna

Love this discussion, I've been hearing a lot of folks talk about this lately.

Cool article was just posted on the Vanilla Blog that might be relevant here:

https://vanilla.higherlogic.com/blog/how-tools-like-chatgpt-and-generative-ai-are-being-evaluated-in-customer-success-programs/

Quick Links

Upcoming Events

How the Rules of SEO Are Being Rewritten in 2025
Generative Engine Optimization? AIOs? The end of search as we know it? 2025 promises to be a long, strange year for SEO. If you’re wondering how to survive it, you’re not alone. Learn more about the key trends driving change in search, which ones teams will need to adapt to, and which ones are just scary acronyms you’re…
HLV Connect: Open Session
⭐️ Got something cool to share? Have a victory in your community and want to show off your accomplishments with fellow community builders? 🙋 Got questions? Bring your community, tactical and random questions and get answers from peers and Vanilla experts! 🗓 Join me and @paddykelly, and a group of your peers for our next…