ChatGPT is poorly designed. So I fixed it

ChatGPT is one of the fastest-growing, fastest-growing apps in history. But with hundreds of millions of people using this every day, has no one stopped to ask, "Why is this app so confusing now?"

Let's demonstrate that with a conversation. I have ChatGPT pulled up, and we actually see two buttons to interact with voice. This is the voice-to-text option, and this is voice-to-voice. Hey, OpenAI, how's it going?

Hey there, I'm doing great. Thanks for asking. How about you, what's new on your end?

Sounds pretty good, with a little bit of vocal fry. But let's try something a little bit more descriptive. I want you to write a group email just saying, "I'm super excited for the AI Engineer World's Fair, and I want to meet up for lunch sometime."

Sure thing. How about something like this? "Excited for the AI Engineer World's Fair? Let's meet for lunch." "I'm super excited about the upcoming AI Engineer World's Fair."

It does respond with a nicely worded email, but it can only respond through voice. If you want to collaborate on the written email, the best you can do is end the call and look for a voice transcript with the nice formatting at the end.

Ideally, this would be something multi-modal with text and voice together, but it really feels like these two apps were built by two different companies. Scott Hanselman called this "shipping the org chart," which he explained really well talking about sitting in an EV.

I'm looking at the map, and then I turn, and then I'm looking at the climate thing, and then I'm looking at the speedometer, and they're all different fonts. And then I realized it was three Android tablets chained together, and I could suddenly see the organizational chart of this large international auto company.

And OpenAI is guilty of this as well. You have some wiz-kid who ships a technical improvement, it turns out it's exactly what consumers are craving, marketing's never consulted, and you end up with a science fair full of potential options.

Like O3, 4.0, Xbox 361 Series S Pro, you get the picture. So let's see how to fix it. There's two things I want to change about ChatGPT today: allowing voice and text at the same time, and smartly choosing theright model depending on your ask.

And we can do this with off-the-shelf tools. 4.0 real-time gives you a live audio chat, and tool calls can handle the rest. You can ask it to send a text for longer details, like links and drafts. And maybe a research tool could hand off to a smarter model and come back with an answer.

Let's bring this concept to life. We have our voice button that takes us from our dormant state to voice mode, which works exactly the same, floating or binol. You have your mute, your end call, and also a new button for chat.

This pulls up a panel that looks like iMessage. It kind of feels like texting your friend, where you're on a FaceTime call, with your call controls at the top, a little reminder of what you might have asked if you're looking through history, and then a text response in case it needs more detail for an email draft.

Now, what if we want to ask something that requires more detail? This is something we explore at Warp Terminal, which is a developer tool to write code in any environment. I have a project pulled up here, and I can ask it for simple things like "undo my last commit."

This hands off to a coding agent and decides to help me out by running commands in my terminal. It can also handle more complex questions, though, like "refactor this entire codebase to use Flutter instead." It detects that it's complex and decides to write a plan with a reasoning model to make sure the code actually works.

This is a really effective pattern, and you can do with some heuristics. If you asked for details and pros and cons, for example, we could hand off to reasoning, tell you how long it's thinking, and hand back a more detailed response.

Now let's go build it using off-the-shelf APIs. Hey there, how's it going?

Hey there. I'm doing well, thank you. How about you? How's your day going?

Pretty good. I just landed in California. Can you send me a link to a park that I should go visit?

Of course. I'll send you a link to a popular park in California.

Yeah, I've heard about Yosemite. Can you tell me more about its history? Go deep.

Yosemite National Park's history is rich, beginning with its ancient Native American heritage and leading to its establishment as a national park. Check the chat for more details.

And I'm handling text input just using tool calls. So we have this "send chat message" tool where you send details that are easier to explain via text. Didn't even need a system prompt, just added this description, and it was smartly able to send theright stuff with text.

It's honestly amazing how far you can go with very simple prompts these days. And for reasoning models, we have another tool call. This is sending off whenever a user wants to go deeper on a topic. You can send some details and then respond back to the model with whatever it needs to say, or just dump it straight in the client if you're building an app like this.

If you want to see the source code, it's available on my GitHub under /fixgpt. With that, subscribe and enjoy the rest of the conference.