ㅤ
Make your chatbot faster and more comprehensive with new AI model from OpenAI
This is going to be follow-up post to “Self hosted ChatGPT with Cloudflare tunnel and Mattermost” so please first read following blog post.
Today, I’m going to test OpenAI’s latest iteration in the GPT series: GPT-4o. This model represents a continuation of OpenAI’s efforts to refine and improve their language models. I will also show how to switch between GPT-4o and GPT4-Turbo in chatbot using predefined commands / flags, and set desired Model temperature (temperature parameter in GPT models controls the randomness of the model’s output. It is a floating-point value that typically ranges from 0 to 1)
This is how OpenAI describes their newest model:
“GPT-4o (“o” for “omni”) is a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, image, and video and generates any combination of text, audio, and image outputs. It can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time(opens in a new window) in a conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models.”
ㅤ
Is it really that good?
First let’s have a look at my billing and see the costs for both models, I’m using similar number of promts every day of course number of tokens might differ but it should be at least similar, from the chart you can see that GPT-4o is much cheaper than GPT4-Turbo:
cost comparison: green bars represents GPT-4 Turbo, while blue bars represent GPT-4o, the difference is significant!
OpenAiModel dataclass:
Mattermost Bot initialize with GPT-4o model as a default one, user can change it back and forth by using command parameters described below:
command parameters:
model temperature parameter:
How Does Temperature Work?
- Low Temperature (e.g., 0.1): When the temperature is set to a low value, the model’s output becomes more deterministic and focused. The model is more likely to choose the highest probability tokens, leading to more predictable and conservative responses. This setting is useful when you need accurate and specific answers.
- High Temperature (e.g., 0.9): A higher temperature setting increases the randomness of the output. The model is more likely to choose less probable tokens, which can result in more creative and diverse responses. This setting is beneficial for tasks that require creative writing or brainstorming, but it can also lead to less coherent and more unpredictable results.
Optional parameters
Here I have added optional parameter for removing old chat context from Redis storage. Basically each conversation has its own id and it is stored in Redis under the same key name. So if I want to remove old conversations I can easily delete it from Redis using ‘–clean-db’ parameter This makes my redis instance free of orphan keys.
bot logs with info about cleaned keys:
All Python code is available in my previous post, I have updated it with new functionality so please have a look here: Self hosted ChatGPT with Cloudflare tunnel and Mattermost
Btw at the time of writing this post, these are the latest GPT Models:
- gpt-4o-2024-05-13
- gpt-4-turbo-2024-04-09
bot parameters usage:
That’s it!
it was quick post I hope you like it, I will be tinkering more around OpenAI and chatbots in the future so stay tuned :)