OpenAI launches GPT-4o Long Output with 16X token capacity

OpenAI is reportedly eyeing a cash crunch, however that isn’t stopping the preeminent generative AI firm from persevering with to launch a gradual stream of recent fashions and updates.

Yesterday, the corporate quietly posted a webpage asserting a brand new giant language mannequin (LLM): GPT-4o Lengthy Output, which is a variation on its signature GPT-4o mannequin from Might, however with a massively prolonged output dimension: as much as 64,000 tokens of output as a substitute of GPT-4o’s preliminary 4,000 — a 16-fold enhance.

Tokens, as you could recall, consult with the numerical representations of concepts, grammatical constructions, and mixtures of letters and numbers organized based mostly on their semantic that means behind-the-scenes of an LLM.

The phrase “Whats up” is one token, for instance, however so too is “hello.” You possibly can see an interactive demo of tokens in motion through OpenAI’s Tokenizer here. Machine studying researcher Simon Willison also has a great interactive token encoder/decoder.

By providing a 16X enhance in token outputs with the brand new GPT-4o Lengthy Output variant, OpenAI is now giving customers — and extra particularly, third-party builders constructing atop its utility programming interface (API) — the chance to have the chatbot return far longer responses, as much as a couple of 200-page novel in size.

Why is OpenAI launching an extended output mannequin?

OpenAI’s choice to introduce this prolonged output functionality stems from buyer suggestions indicating a necessity for longer output contexts.

An OpenAI spokesperson defined to VentureBeat: “We heard suggestions from our clients that they’d like an extended output context. We’re all the time testing new methods we will finest serve our clients’ wants.”

The alpha testing part is predicted to final for just a few weeks, permitting OpenAI to assemble information on how successfully the prolonged output meets person wants.

This enhanced functionality is especially advantageous for purposes requiring detailed and intensive output, resembling code enhancing and writing enchancment.

By providing extra prolonged outputs, the GPT-4o mannequin can present extra complete and nuanced responses, which might considerably profit these use instances.

Distinction between context and output

Already, since launch, GPT-4o provided a most 128,000 context window — the quantity of tokens the mannequin can deal with in anyone interplay, together with both input and output tokens.

For GPT-4o Lengthy Output, this most context window stays at 128,000.

So how is OpenAI in a position to enhance the variety of output tokens 16-fold from 4,000 to 64,000 tokens whereas retaining the general context window at 128,000?

It name comes all the way down to some basic math: regardless that the unique GPT-4o from Might had a complete context window of 128,000 tokens, its single output message was restricted to 4,000.

Equally, for the brand new GPT-4o mini window, the overall context is 128,000 however the most output has been raised to 16,000 tokens.

Which means for GPT-4o, the person can present as much as 124,000 tokens as an enter and obtain as much as 4,000 most output from the mannequin in a single interplay. They will additionally present extra tokens as enter however obtain fewer as output, whereas nonetheless including as much as 128,000 complete tokens.

For GPT-4o mini, the person can present as much as 112,000 tokens as an enter as a way to get a most output of 16,000 tokens again.

For GPT-4o Lengthy Output, the overall context window remains to be capped at 128,000. But, now, the person can present as much as 64,000 tokens price of enter in trade for a most of 64,000 tokens again out — that’s, if the person or developer of an utility constructed atop it desires to prioritize longer LLM responses whereas limiting the inputs.

In all instances, the person or developer should make a alternative or trade-off: do they wish to sacrifice some enter tokens in favor of longer outputs whereas nonetheless remaining at 128,000 tokens complete? For customers who need longer solutions, the GPT-4o Lengthy Output now gives this as an choice.

Priced aggressively and affordably

The brand new GPT-4o Lengthy Output mannequin is priced as follows:

• $6 USD per 1 million enter tokens
• $18 per 1 million output tokens

Examine that to the common GPT-4o pricing which is $5 per million enter tokens and $15 per million output, and even the brand new GPT-4o mini at $0.15 per million enter and $0.60 per million output, and you’ll see it’s priced relatively aggressively, persevering with OpenAI’s current chorus that it desires to make highly effective AI inexpensive and accessible to huge swaths of the developer userbase.

At the moment, entry to this experimental mannequin is restricted to a small group of trusted companions. The spokesperson added, “We’re conducting alpha testing for just a few weeks with a small variety of trusted companions to see if longer outputs assist their use instances.”

Relying on the outcomes of this testing part, OpenAI might take into account increasing entry to a broader buyer base.

Future prospects

The continuing alpha check will present precious insights into the sensible purposes and potential advantages of the prolonged output mannequin.

If the suggestions from the preliminary group of companions is optimistic, OpenAI might take into account making this functionality extra extensively accessible, enabling a broader vary of customers to learn from the improved output capabilities.

Clearly, with the GPT-4o Lengthy Output mannequin, OpenAI hopes to deal with a fair wider vary of buyer requests and energy purposes requiring detailed responses.