GPT-4o mini: The current fastest and cost-effective LLM Model

OpenAI's latest LLM is 4o mini, a lightweight model designed to be cost-efficient while high-performing against other lightweight models and even some larger models. It's arrival saw the replacement of GPT-3.5 Turbo, the benchmark and beginning of the 'AI boom' in terms of LLM models - GPT-4o mini is 60% cheaper than 3.5 Turbo ($0.70 per 1M tokens to $0.24 per 1M tokens) while far surpassing it's predecessor in various benchmarks including textual intelligence and multimodal reasoning. GPT-4o mini has left quite an impact in the LLM realm, ranking well within the top 3 on LMSYS' leaderboard - exceeding models like Anthropic's Claude 3.5 Sonnet, and Google's Gemini 1.5 Pro.

Superior Textual Intelligence and Multimodal Reasoning

As previously mentioned, GPT-4o Mini excels in both textual intelligence and multimodal reasoning, outperforming now-legacy models like GPT-3.5 Turbo. It has shown higher performance on academic benchmarks, scoring 82% on the MMLU (Massive Multitask Language Understanding) benchmark. Additionally, performing well in multimodal reasoning tasks, scoring 59.4% on the MMMU (Multimodal Machine Understanding) evaluation, compared to 56.1% for Gemini Flash and 50.2% for Claude Haiku.

Pricing & Latency Changes

GPT-4o Mini introduces a highly competitive pricing model:

- Input Tokens: $0.15 per million tokens
- Output Tokens: $0.60 per million tokens

This pricing structure is approximately 60% cheaper than GPT-3.5 Turbo and much more affordable than previous frontier models including GPT-4 Turbo, GPT-4o. Additionally, GPT-4o Mini offers reduced latency, making it ideal for real-time applications such as our AI Call Agents and other conversational-based applications.

How It Changes the AI Call Agent Game

With the drastic improvements in cost and being able to perform at a high level we have been able to make immense changes to the pricing structure of our own AI Call Agents making it more affordable and accessible to our clients, decreasing price by 40% and maintaining high quality responses.

However it doesn't stop at pricing, the latency of the model is a key differentiator between OpenAI's past model - GPT-4o. Lowering the latency by ~35%. These small upgrades in latency get us one step closer to simulating human to human conversation as best as possible resulting in the AI Call Agents performing much better in the real world.

Overall, GPT-4o Mini represents a significant step forward in making advanced AI both accessible and cost-effective, paving the way for broader adoption and innovation in the AI field.