Make your chatbot faster and more comprehensive with new AI model from OpenAI

This is going to be follow-up post to “Self hosted ChatGPT with Cloudflare tunnel and Mattermost” so please first read following blog post.

Today, I’m going to test OpenAI’s latest iteration in the GPT series: GPT-4o. This model represents a continuation of OpenAI’s efforts to refine and improve their language models. I will also show how to switch between GPT-4o and GPT4-Turbo in chatbot using predefined commands / flags, and set desired Model temperature (temperature parameter in GPT models controls the randomness of the model’s output. It is a floating-point value that typically ranges from 0 to 1)

This is how OpenAI describes their newest model:

“GPT-4o (“o” for “omni”) is a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, image, and video and generates any combination of text, audio, and image outputs. It can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time(opens in a new window) in a conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models.”

Is it really that good?

First let’s have a look at my billing and see the costs for both models, I’m using similar number of promts every day of course number of tokens might differ but it should be at least similar, from the chart you can see that GPT-4o is much cheaper than GPT4-Turbo:

cost comparison: image alt text green bars represents GPT-4 Turbo, while blue bars represent GPT-4o, the difference is significant!

OpenAiModel dataclass:

@dataclass
class OpenAiModel():
    gpt4o: str = 'gpt-4o-2024-05-13'          # 128,000 tokens, Up to Oct 2023:
    gpt4turbo: str = 'gpt-4-turbo-2024-04-09' # 128,000 tokens, Up to Dec 2023:

Mattermost Bot initialize with GPT-4o model as a default one, user can change it back and forth by using command parameters described below:

command parameters:

case '--use-gpt4o':
    self.chat = ChatGPT(model=OpenAiModel.gpt4o)
    self.driver.reply_to(message, f'you are now using: {self.chat.model}')
case '--use-gpt4-turbo':
    self.chat = ChatGPT(model=OpenAiModel.gpt4turbo)
    self.driver.reply_to(message, f'you are now using: {self.chat.model}')

model temperature parameter:

case x if x.startswith('--set-temp='):
    self.chat.temperature = float(x.split('=')[-1])
    self.driver.reply_to(
        message,
        f'temperature set to: {self.chat.temperature}, current model: {self.chat.model}'
    )

How Does Temperature Work?

  • Low Temperature (e.g., 0.1): When the temperature is set to a low value, the model’s output becomes more deterministic and focused. The model is more likely to choose the highest probability tokens, leading to more predictable and conservative responses. This setting is useful when you need accurate and specific answers.
  • High Temperature (e.g., 0.9): A higher temperature setting increases the randomness of the output. The model is more likely to choose less probable tokens, which can result in more creative and diverse responses. This setting is beneficial for tasks that require creative writing or brainstorming, but it can also lead to less coherent and more unpredictable results.

Optional parameters

case '--clean-db':
    keys_removed = init_db_cleanup()
    if keys_removed:
        r.delete(*keys_removed)
    self.driver.reply_to(message, f'number of keys cleaned: {len(keys_removed)}')

Here I have added optional parameter for removing old chat context from Redis storage. Basically each conversation has its own id and it is stored in Redis under the same key name. So if I want to remove old conversations I can easily delete it from Redis using ‘–clean-db’ parameter This makes my redis instance free of orphan keys.

image alt text

bot logs with info about cleaned keys:

2024-07-10 00:11:30,361 :: INFO :: removing: chatgpt:thread:ii5xmrpp778i7m7xm6u4x6qn3e
2024-07-10 00:11:30,361 :: INFO :: removing: chatgpt:thread:qdqokabdgbbzxy93ic1zj38cec
2024-07-10 00:11:30,362 :: INFO :: removing: chatgpt:thread:pnkkit5hgfdofncnnyiquycdkc
2024-07-10 00:11:30,362 :: INFO :: removing: chatgpt:thread:q39begoboigdfp3xjypyknfstr
2024-07-10 00:11:30,362 :: INFO :: removing: chatgpt:thread:ehgin3wc1fnjimmagaykm818ne
2024-07-10 00:11:30,362 :: INFO :: removing: chatgpt:thread:tiojycwxytdatms34xh3z5eoer
2024-07-10 00:11:30,362 :: INFO :: removing: chatgpt:thread:rwg41sw7ijyxudeonp374mbrrw
2024-07-10 00:11:30,362 :: INFO :: removing: chatgpt:thread:eotbdqjddtf5mnaokbxmiz9rpo
2024-07-10 00:11:30,363 :: INFO :: removing: chatgpt:thread:n56b7oduwt84tmo5qy159b1z1h
2024-07-10 00:11:30,363 :: INFO :: removing: chatgpt:thread:kuyn81q3ytrkzphfojgumkparo
2024-07-10 00:11:30,363 :: INFO :: removing: chatgpt:thread:iu8xe4datpyf5ynstg7o66sp8r
2024-07-10 00:11:30,363 :: INFO :: removing: chatgpt:thread:7g7984x3mt863xu84chksujg6h
2024-07-10 00:11:30,363 :: INFO :: removing: chatgpt:thread:iajqr89753nxfg9zi47jc4tcco

All Python code is available in my previous post, I have updated it with new functionality so please have a look here: Self hosted ChatGPT with Cloudflare tunnel and Mattermost

Btw at the time of writing this post, these are the latest GPT Models:

  • gpt-4o-2024-05-13
  • gpt-4-turbo-2024-04-09

bot parameters usage:

image alt text

That’s it!

it was quick post I hope you like it, I will be tinkering more around OpenAI and chatbots in the future so stay tuned :)