OpenAI token usage
We offer a variety of AI tools at Bird, including:
These tools are powered by an integration called OpenAI, and use tokens to work.
Open AI tokens
Tokens are a way for OpenAI's large language models (LLMs) to understand and create sentences.
When OpenAI's LLM needs to process a sentence, such as a message sent to you by a customer, or create a sentence, such as formulating a reply to your customer's message, it breaks the words down into smaller pieces called tokens.
In short, LLMs make sense of sentences written by humans by converting them into a format that the LLM can understand.
The more tokens it uses, the more it understands, and the more accurate its replies will be.
Tokenization isn't an exact science. For example, when given the word 'cat', an LLM may decide that a token is the whole word, or it might decide that 'ca' is one token, and 't' is a second token.
Token usage in AI Agents
Understanding messages from customers and formulating responses aren't the only ways that OpenAI's LLMs uses tokens. They also use tokens to understand how they're supposed to work, and to perform specific tasks.
In AI Agents, the following processes use tokens:
To understand the instructions you've given the AI Agent about how it should work, including its purpose, tasks you've asked it to perform, the audience it's talking to, the personality you've given it, the guardrails it needs to follow, when it should hand over to an human, and any custom instructions or advanced configurations you've set up.
To understand messages that it needs to process.
To formulate responses to messages.
To perform actions like searching a knowledge base, sending messages, resolving conversations, or handing over to a human agent.
Diagram: Token usage by an LLM for an AI Agent
Every process that OpenAI's LLMs perform to make your AI Agent function will add to the total number of tokens used.
These process are what make our AI Agents so powerful in comparison to a chatbot that hasn't been given these instructions, doesn't understand any context, and hasn't been asked to perform any additional tasks.
The more complex the instructions you've given your AI Agent, the more tokens it will use.
See how many tokens your AI Agents are using
You can see exactly how many tokens your AI Agents are using.
When testing an AI agent, send a message to the agent, then click Inspect.
When examining a live AI Agent, go to the Agent, click Chats, then Inspect each chat.
How we charge for token usage
When you use the OpenAI integration with Bird, your OpenAI account will be billed for token usage, not Bird.
Tokens are billed by OpenAI based on the usage by your AI Agents, and Bird does not impose additional charges for these tokens.
Because our AI Agents use tokens to perform a variety of complex process, your token usage may be higher than expected, particularly if you're only used to working with basic OpenAI functions, such as the OpenAI step in Flows, or OpenAI assistant.
Good to know
Bird will never add any non-essential tokens to your processes.
Reducing token usage
The only way to reduce token usage by AI Agents is to limit its functionality. This can be done by removing some of the instructions you've given it during the set up.
However, we don't recommend this approach as it will limit the capabilities of the AI Agent, and will still use tokens.
Last updated