Get excited! We are doing a 5-part series on how to best leverage natural language processing (NLP) within chatbots, for the non-technical person. Feel free to share these posts with your friends and followers in business, marketing, and sales. No need to have a computer science degree to begin experimenting with NLP!
Natural Language Processing, What Gives?
People in the chatbot world talk a big game about natural language processing and artificial intelligence platforms. However, anyone who has received an, “I don’t know” message from a bot--insert eye-roll here.
The fact of the matter is that NLP separates the men from the boys; it and can add significant power to a chatbot, and there are many companies that create chatbots equipped with NLP and AI. The issue is often the time and cost to create, train, and deploy such a bot is on average between 3-6 months—and that’s just your first version…if you’re lucky.
Common Framework for Creating NLP Bots
The cost for some companies to train their first bot is in the six figures. Moreover, the main problem is that because chatbots are so new, the first attempt at building chatbots is rarely your best bot, and so you spend a lot of time and money training a bot that is likely not optimal upon deployment. This often leads to the common large #botfails with which we are so familiar (e.g. Facebook Assistant M). And because so much money and time is spent on them in the first place, the impact of the fail has a much greater magnitude.
Hence, for many companies, I propose a solution we are calling “Hybrid Bots”, on a rapid-release model designed to quickly deploy bots with a limited scope and adding in NLP slowly and effectively based off of users’ responses.
The rapid-release model is good for bot creators who want to leverage NLP and:
· have limited technical knowledge or resources
· want a better understanding of what their users need from a bot
· don’t have large amounts of time or money for bot deployment
· are planning a large and expensive bot deployment and want to do research first
If you might fit into any of these categories, read on to learn more.
Hybrid Bots for Rapid Release
Hybrid bots are just chatbots that begin as a decision-trees, or structured conversations, and layer in natural language processing (NLP) little by little over short time-frames, as you gain more insights from users. The reason that this is an effective model, is that it can be deployed and edited quickly on the fly and the NLP training will be based off of how users actually interact with the bot as opposed to data like customer service e-mails, which although accurate for the way users interact with your company, are not effective for capturing bot-based responses.
Why is this data different? People will boundary test bots over and over again, asking questions of a banking app such as, What’s the capital of Italy? or Are you a human being?, etc. This type of data will never be captured in your customer service e-mails.
How Do Hybrid Bots Work?
Create a simple decision tree bot with a limited scope of questions covering topics such as: a.) what your company does, b.) how do your products and services work c.) responding to your most commonly asked-sales questions. (You can usually pull this from sales operations teams at large companies or from a helpful, friendly sales person at a smaller company).
Your bot should have a simple introduction noting that it is a bot, and then letting the user know what help it can offer, such as: “Welcome to YS Savings Bank. What can I tell you more about?” And list between 4-6 options (for the banking example, maybe it’s “Opening an Account” or “Customer Service”, etc.)—any more than that and you may be reaching the paradox of choice. Then for 1-2 of those options, have a free text response. For example, the last option in your decision tree might say, “Ask Me a Question”, and then allow for users to freely-type whatever they want.
Make sure your bot makes sense and gives valuable answers or routes users to information or people that can provide correct information. Thereafter, you should deploy your bot on your site. Depending on the volume of users, you can start to evaluate the free-text responses to see common questions, themes, or needs from a bot.
Rapid-Release-Framework for Creating NLP Bots
You can then use this information to train your bot to answer these questions. For example, if 250 people ask the question. “Are you a human?”, than you can add this to the skillset of answers your bot can provide, such as: “Afraid not. I’m just a banking bot here to help you.” Easily layer in AI as you have enough data to do so, and let the decision tree become the default conversation. The idea is, over time, you can make your bot smart enough to handle most conversations on its own.
Now comes the interesting part. You might be thinking, Nice idea lady, but how do I use NLP in this bot? The beauty of this is, there are resources out there that allow non-technical people to leverage NLP developed by some of the most sophisticated companies in the world, namely: Google, Facebook, and Amazon. Sure, you can create your own proprietary machine learning systems, but ultimately, you’ll never have the data sets for training that are available to the goliaths of tech. While I’m sure niche NLP and machine learning systems from new players will pop-up, for beginners you can start learning by using the free services available. Beyond this, these companies are incentivized to help you, because the more you train your bot on their platforms, the better their platforms become. It’s a quid pro quo, friend.
To explain the efficacy of this, we will be attempting to build a bot and leverage the following free NLP platforms: a.) Dialogflow (formerly API.AI) from Google; b.) Wit.AI-owned by Facebook; and c.) Amazon’s Lex, respectively. Moreover, we’ll take you step-by-step on the process of leveraging their toolsets and show some of wins and challenges we encountered. We’ll do all this ensuring that someone who does not have a technical background will be able to follow along and build their own NLP bots—pretty nifty, right?
Interested in knowing how it all works out? Sign-up for our newsletter here!