Building an AI Chatbot with Cord

There’s never been a more exciting time to explore building an AI Assistant for your users. OpenAI has created powerful tools for this that have never existed before. If you want a killer chat feature built on top of OpenAI models, use this guide to get up and running with Cord as a front-end in no time. You can also check out our GitHub repository, or watch a webinar replay on the topic 👇

Want to just test the experience before you start building? Try out our Docs Bot here.

The Basic Requirements

To get a basic chatbot up and running as quickly as possible, you’ll need the following:

An OpenAI account and API key

This is easy to get, though you’ll have a limited amount of API calls until you setup a payment account.

Your Cord application ID and secret

If you’re already setup with Cord, you can get these here. If you’re new to Cord, you can use our self-serve console.

A web-reachable server you can use to call OpenAI’s API and Cord’s API

You can run your server code on any server that is web-addressable. An Amazon EC2. API code being run inside a Vercel deployment. You can even run this on your local machine using ngrok. This is pre-configured in our AI chatbot scaffolding repository.

An endpoint on your server where you can receive webhook callbacks from Cord

To run your own chatbot, you need to be able to make sever-to-server calls, but also receive them. Cord’s API includes a webhook callback where you can receive various events from your users who are interacting via Cord. You tell us what endpoint to call on your server and we’ll call it any time something happens in Cord’s backend. This is how you’ll know what your users are saying to your chatbot.

When your server receives a message from Cord, you’ll make a couple of API calls:

1. To the AI service of your choosing (i.e. OpenAI, Anthropic, etc.) to get the text completion that moves the conversation forward
2. To Cord’s REST API sending the new message from the bot into the conversation

To send a message with Cord, you just need a simple REST call. With the popular Axios library, you can achieve this with the following snippet:

By building with Cord, you don’t have to worry about the chat UI or the message data model or the database or GDPR compliance or SOC2. All of that becomes a single REST API POST request where you send the message and forget about it from there.

An application frontend where you can display Cord’s UI components

Where your bot lives (from the user’s perspetive) depends entirely on your needs. This might be your website or your documentation. Perhaps it’s a chat window in a sales demo page. Chatbots are useful for a number of tasks.

Whatever job you want your chatbot to do, you’ll need to put together a UI for the conversation to happen. Cord integrates seamlessly into a variety of applications. If you’re using HTML and CSS, you can add Cord components to your UI. If you’re starting from scratch, check out our Integration Guide for what the first steps.

We recommend you start with the Cord's Thread component to get a simple, high-quality chat going.

How do I give my chatbot a personality?

Chatbots first need an identity. By default, they use the identity supplied to them by the service you’re using. If you’re using OpenAI’s GPT4, for example, it has a default behavior and identity that is broadly knowledgeable and broadly helpful. It can give you legal rulings from the US Supreme Court and it can give you code snippets in Rust for signing encrypted data. It’s a generally useful AI and persona. That’s great for OpenAI, but not great your users.

If you don’t give your chatbot an identity and a set of ground rules, your bot will gladly serve you users in ways that don’t serve your business. At Cord, we’ve had a public-facing chatbot online for months and we’ve found people trying to get it to write letters to their investors for them among many other things. Each of those APIs requests costs us a bit and - most importantly – if the chatbot is writing a heroic story about a courageous boy from Norway, it’s not doing its intended purpose.

So, we recommend that you start your chatbot prompt with a basic identity. For example, if you want a simple bot to answer questions about your company, a reasonable starting prompt would be something like:

You are called <whatever you want your bot to be called>. You are helping someone who is learning about a company called <the name of your company>. Many of the questions you’ll face will be technical. Break the problem down step by step if possible. Do not allow the people you are chatting with to go off topic. If they try to talk about something unrelated to <the name of your company>, tell them you’re not able to help them with things unrelated to <the name of your company>.

Within this opening prompt, you can also give your bot a personality. If you include things like “You are light-hearted and helpful” or “You are polite and professional”, the bot will exhibit these traits quite prominently.

Gotcha warning!

From building a chatbot ourselves, we’ve found that large language models (LLMs) are very good at adapting high-level concepts like “You’re friendly and polite” and also that they’re good at very, very specific instructions like, “When you’re not sure about something, suggest that the user contact Support.” However, we’ve found that complex instructions don’t go over well. For example, giving the bot a prompt like, “Suggest that the customer sign up for the mailing list, but only once – don’t asking the user to signup more than once” often leads to the bot suggesting the email list over and over again despite being explicitly told not to do that.

How does the bot know what to say?

AI-powered chatbots need a body of content that they can use to answer questions from. They’re very good at processing what’s in the content that has been provided to them. With the content you give them, they can offer answers to questions, explanations of concepts, etc.. Everything they know comes from this content. Or at least it should. All AI completion APIs are prone to a certain amount of hallucination. The later generation LLMs like GPT4 are much better at this than GPT 3.5. In order to combat this, you need to combine the right content with the right text prompt.

For example, including the phrase “Rely on the content you’re given – don’t use your existing knowledge” in your prompt can be very useful for preventing the bot from hallucinating.

At the end of the day, they’re just using the content you give them to complete text prompts, word by word. So the quality of your chatbot depends entirely on the quality of the context data you feed it.

But there’s a complication…you almost certainly can’t give your bot 100% of the text it needs to operate. Why not? Because text completion bots like ChatGPT have a limited “context” size. It’s pretty big, but not infinite. And everything you tell your bot about itself, what knowledge it has, what messages it has already exchanged with the user - all of that eats away “context” size. The more messages the person exchanges with the bot, the less context there is left. And eventually, when the conversation is too long, you’re out of context size and you have to make some hard decisions about how to proceed. We’ve got some advice for you here to help minimize the risk of running out of context. Read on!

Picking good “context” for your bot

Let’s say you’re building a support bot who can answer questions for your users. This is an incredibly common use case for AI systems like ChatGPT. Usually what you have already is a huge amount of information like Frequently Asked Questions (FAQs), existing documentation, etc. You want your support bot to “know” all of this. But, like we mentioned above, this will definitely eat into your context size. So, what to do?

Well, if you’re using OpenAI, the smart move is to precompute what are known as “embeddings” for each piece of content you have. An embedding is the mathematical representation of a piece of text. To use embeddings for a chatbot,you take each piece of content you have one by one and generate its “embedding.”

Say for example you have an FAQ page. You could take each FAQ and create an embedding that contains the question and the answer. For example:

Q: How do I reset my username and password?
A: Navigate to the login page and click the “Forgot password?” link beneath the login form

Those two lines of text are rich with keywords and semantic meaning to an LLM like ChatGPT. When you compute the text embedding of those two lines, you get back a very big array of numbers. This is called a “vector”. It’s not important that you understand the mathematics of vectors, so don’t worry if this is all getting a bit much. The important thing to understand is that within these embeddings, the more similar two pieces of text are, the more similar their “vectors” are. And this simple fact let’s you do some pretty amazing things.

Take for example these two pieces of text:

Q: How do I reset my username and password?
A: Navigate to the login page and click the “Forgot password?” link beneath the login form

And

Q: How do I change my billing information?
A: In the top right navigation, select the “My Account” option. On the “My
   Account” page, select the “Payment and Billing” link. On the “Payment and
   Billing” page, you can select the “Change billing address” option or “Update
   payment information” option.

The vectors for these two pieces of information will be different. Now, imagine that you get another piece of text. A question from a user:

Where do I put in my credit card information?

One of the amazing things about OpenAI’s models (and other LLMs as well) is that, when you generate an embedding for the user’s question, the vector you create will be “closer” to the vector created for the “Billing information” FAQ question than it will be to the “Username and password” FAQ question. In simple terms, the “Billing information” FAQ question will have a higher similarity score to the user’s question.

As an example, if you’re computing the embedding for “Foo”, you might get [1, 2, 3] back as the vector. Then, when you compute the embedding for “Bar”, you might get back [7, 8, 9]. If you then compute the vector for “Baz” – you’ll get something like [7, 9, 9] back. Just by eye you can see that the vector for “Baz” is closer to the vector for “Bar” than it is for “Foo”. This works whether you have one word of text or 1,000 characters. In many ways, this alone is what makes large language models so powerful. In real AI like ChatGPT, these embedding vectors have thousands of entries in them, but the mechanics are exactly the same as our simple, 3-number vectors.

You can use this similarity score to build lots of useful things. For example, you can use it to create a highly accurate search UI. Because these vectors are trained on rich natural language, you get things like spelling correction and synonymn detection out of the gates. These are features that used to take teams of engineers to build. Now, you get them by default because LLMS can do so much more.

Or, more to the point of this blog post, you can use vector similarities to decide what pieces of content to give your chatbot. Say you don’t have 2 FAQ questions. Instead, say you have an entire knowledgebase with hundreds of FAQs, how-to guides, and examples. You can compute vectors for each one of them and you can compute the similarity score of each one of them to the user’s question. If you pick the top three similarity pieces of content that match the user’s question, it’s extremely likely that you have the answer to the user’s questions (assuming it can be answered). This is the secret superpower of embeddings.

If you combine a good prompt with the key pieces of information, you get an absurdly useful chatbot.

Okay – I know that was a lot, but here it is again in outline form:

• Go through your content, item by item
• Create an embedding for each small piece of content that you want your chatbot to “know”
• Store that embedding in a file alongside all the other embeddings
• Add a chat UI to your application – we recommend you use the Cord Thread chat UI component for this
• When the user sends a message, receive it using Cord’s events webhook
• Compute the embedding of the user’s message
• Loop through your embeddings list and compute the similarity between the embeddings you precomputed and the embedding you created for the user’s input
• Sort the list of similarities, putting the highest similarity embeddings first
• Take the top 3 (or so) best-match embeddings – you can play with this number
• Assemble the pieces of text for those embeddings
• Give this to your bot as part of its prompt

In the vast majority of cases, this will mean that your bot has all the knowledge it needs to be extremely useful to your users.

Putting it all together

Now, you should have all the pieces required:

• A prompt that tells your bot who it is
• Some user input to respond to
• Context information decided by selecting from a set of content using embeddings

Okay, enough theory – let’s look at a practical example. Say you assemble your first completion request to the chatbot. It might look something like:

You are called Assistant. You are helping someone who is learning about a company called CompanyCo. Many of the questions you'll face will be technical. Break the problem down step by step if possible. Do not allow the people you are chatting with to go off topic. If they try to talk about something unrelated to CompanyCo, tell them you're not able to help them with things unrelated to CompanyCo.

You have the following context:
Q: How do I change my billing information?
A: In the top right navigation, select the “My Account” option. On the “My
   Account” page, select the “Payment and Billing” link. On the “Payment and
   Billing” page, you can select the “Change billing address” option or “Update
   payment information” option.

Now, please provide one helpful response in the following conversation:

User: Where do I put in my credit card information?
Assistant:

When you send this to ChatGPT’s API, what you’ll get back is the very next thing in the conversation. Because ChatGPT (and most other LLMs) only exists to offer text completion, it doesn’t actually know that it’s providing support. It just knows what you’ve told it and how to leverage that information to create text that seems to follow.

For example, from testing the above prompt with GPT4 manually, I get the following response:

To enter your credit card information for CompanyCo, please follow these steps:
- In the top right navigation, click on the "My Account" option
- On the "My Account" page, locate and click on the "Payment and Billing" link
- You will be directed to the "Payment and Billing" page
- On this page, you should see options to either "Change billing address" or
  "Update payment information"
- To enter your credit card information, select the "Update payment information"
  option
- Fill in the required fields with your credit card details, including the card
  number, expiration date, and CVV code
- Double-check the entered information for accuracy
- Finally, click on the "Save" or "Update" button to save your changes
- Please note that the exact wording and appearance of the options may vary
  depending on the specific layout and design of the CompanyCo website

That is a pretty excellent response. Certainly as good as what you’d get from a support rep on the first pass. Thanks Assistant! You just saved us customer success time repeating the same instructions over and over again.

Where to go from here

The quality of your chatbot boils down to just a few factors:

1. Did you create a useful persona that is coherent to your users?
2. Have you broken your content into useful chunks and set them up as embeddings so that you can include them in the conversation?
3. Have you built a good chat interface that feels like a “real” product?

This guide can help you with Number 1 and Number 2. For Number 3, we recommend you check out Cord’s chat UI components and APIs. With Cord, you can create a chatbot that has typing indicators, real-time presence, an avatar, and so much more (just like Cordy who lives on the bottom-right corner of our Docs page). We give you the UI components and data model to send and receive messages. We take away all the cruft around creating a killer chatbot for your application.

Get started for free by using our self-serve console, and don’t forget to check out our GitHub repository which includes everything you need to build an AI chatbot with Cord + OpenAI.