It has been a turbulent week for OpenAI, marked by executive departures and significant fundraising developments. Despite this, the startup is forging ahead, aiming to persuade developers to utilize its AI models to create tools at its 2024 DevDay. The company unveiled several new tools on Tuesday, including a public beta of its "Realtime API" for building apps with low-latency, AI-generated voice responses.
Also Read :
In a briefing with reporters prior to the event, OpenAI's chief product officer, Kevin Weil, assured that the recent departures of chief technology officer Mira Murati and chief research officer Bob McGrew would not impede the company's progress. He expressed gratitude for their leadership and emphasized that OpenAI would not slow down.
As OpenAI experiences another C-suite overhaul, reflecting the turmoil following last year's DevDay, the company is striving to demonstrate to developers that it remains the top platform for building AI apps. Despite boasting over 3 million developers building with its AI models, OpenAI faces growing competition in the market.
OpenAI highlighted that it had reduced costs for developers to access its API by 99% over the past two years, likely in response to competitors such as Meta and Google continuously undercutting their prices.
One of OpenAI's new features, the Realtime API, allows developers to create nearly real-time, speech-to-speech experiences in their apps, offering a choice of six voices provided by OpenAI. These voices are distinct from those offered for ChatGPT, and developers are not permitted to use third-party voices to avoid copyright issues. Additionally, OpenAI has not made available a voice reminiscent of Scarlett Johansson's.
During the briefing, OpenAI's head of developer experience, Romain Huet, presented a demo of a trip planning app built with the Realtime API. The application enabled users to verbally interact with an AI assistant about an upcoming trip to London and receive low-latency responses. The real-time API also provides access to various tools, allowing the app to annotate a map with restaurant locations as it responds.
Huet also demonstrated how the Realtime API could engage in a phone conversation with a human to inquire about ordering food for an event. Unlike Google's Duo, OpenAI's API cannot directly call restaurants or shops; however, it can integrate with calling APIs like Twilio to do so. Notably, OpenAI is not adding disclosures for its AI models to automatically identify themselves on calls, although their AI-generated voices sound quite realistic. As of now, it appears to be the developers' responsibility to include this disclosure, which could be mandated by a new California law.
During its DevDay announcements, OpenAI introduced vision fine-tuning in its API. This will allow developers to use images, in addition to text, to fine-tune their applications of GPT-4o. The goal is to help developers improve the performance of GPT-4o for tasks involving visual understanding. However, developers will not be able to upload copyrighted imagery (such as a picture of Donald Duck), images depicting violence, or other imagery that violates OpenAI’s safety policies.
OpenAI is working to keep up with its competitors in the AI model licensing space. Its prompt caching feature is similar to the feature Anthropic launched several months ago, allowing developers to cache frequently used context between API calls, reducing costs and improving latency. OpenAI says developers can save 50% using this feature, whereas Anthropic promises a 90% discount for it.
Additionally, OpenAI is offering a model distillation feature to allow developers to use larger AI models, such as o1-preview and GPT-4o, to fine-tune smaller models such as GPT-4o mini. Running smaller models generally provides cost savings compared to running larger ones, but this feature should let developers improve the performance of those small AI models. As part of model distillation, OpenAI is also launching a beta evaluation tool so developers can measure their fine-tune’s performance within OpenAI’s API.
However, there was no news about the GPT Store announced during last year’s DevDay. OpenAI has been piloting a revenue share program with some of the most popular creators of GPTs, but the company hasn’t shared much about this program since then.
OpenAI also announced that it’s not releasing any new AI models during DevDay this year. Developers waiting for OpenAI o1 (not the preview or mini version) or the startup’s video generation model, Sora, will have to wait a little longer.
.png)
.jpg)
0 Comments