# ChatGPT is poorly designed. So I fixed it AI Engineer ยท 2025-06-03 Ben Holmes critiques ChatGPT's confusing design, calling it a case of 'shipping the org chart' where voice and text features feel like separate apps. He demonstrates a fix by building FixGPT, an app that allows simultaneous voice and text interaction and smartly routes requests to the right model via tool calls. Using off-the-shelf APIs like 4.0 real-time and simple prompts, Holmes shows how a 'send chat message' tool enables detailed text responses alongside voice, while a reasoning model tool handles complex queries. The source code is available on GitHub, showcasing how to avoid OpenAI's science fair of options. ## Questions this episode answers ### What is 'shipping the org chart' and how does it apply to ChatGPT's design? Ben Holmes says ChatGPT feels like two separate apps built by different teams, with redundant voice buttons and no integration between voice and text. He invokes Scott Hanselman's term 'shipping the org chart,' which Hanselman used to describe a car's dashboard made of three mismatched Android tablets. This reflects how OpenAI's technical advancements ship without unified user experience, causing confusion. [1:13](https://aie.addtry.com/55ba5122-f09a-43ff-858f-95e73ba6d77c?t=73000) ### How did Ben Holmes enable voice and text interaction at the same time in his ChatGPT fix? He built a demo app using OpenAI's 4.0 real-time API and tool calls. A 'send chat message' tool lets the model send text responses during a voice call, so users see links, drafts, or details in an iMessage-like panel while talking. The interface merges call controls with a text chat, making it feel like a natural, multi-modal conversation. [2:53](https://aie.addtry.com/55ba5122-f09a-43ff-858f-95e73ba6d77c?t=173000) ### How does Ben Holmes' app decide when to use a more powerful reasoning model? The app uses heuristics based on the user's request. If someone asks for deeper analysis, like the history of Yosemite, a tool call sends the question to a reasoning model, which responds via text after a brief thinking period. This pattern, also used in Warp Terminal for complex coding tasks, ensures simple queries stay fast while harder ones get detailed answers. [3:09](https://aie.addtry.com/55ba5122-f09a-43ff-858f-95e73ba6d77c?t=189000) ## Key moments - **[0:00] Intro** - [0:00] ChatGPT presents two separate voice interaction buttons, causing confusion for users. - **[0:16] Confusing UI** - **[1:13] Multimodal Gap** - [1:28] Scott Hanselman: "I could suddenly see the organizational chart of this large international auto company." - **[2:02] The Fix** - **[2:39] App Design** - [2:39] The redesigned app adds a chat button during voice calls, enabling simultaneous text interaction. - **[3:09] Smart Routing** - **[3:56] Live Demo** - [4:18] When asked to 'go deep' on Yosemite history, the app hands off to a reasoning model for a richer response. - **[4:37] Tool Calls** - **[5:12] Outro** ## Speakers - **Ben Holmes** (guest) ## Topics AI Product Design ## Mentioned OpenAI (company), 4.0 real-time (product), ChatGPT (product), Warp Terminal (product) ## Transcript ### Intro **Ben Holmes** [0:00] ChatGPT is one of the fastest-growing, fastest-growing apps in history. But with hundreds of millions of people using this every day, has no one stopped to ask, "Why is this app so confusing now?" ### Confusing UI **Ben Holmes** [0:16] Let's demonstrate that with a conversation. I have ChatGPT pulled up, and we actually see two buttons to interact with voice. This is the voice-to-text option, and this is voice-to-voice. Hey, OpenAI, how's it going? **Speaker 2** [0:30] Hey there, I'm doing great. Thanks for asking. How about you, what's new on your end? **Ben Holmes** [0:35] Sounds pretty good, with a little bit of vocal fry. But let's try something a little bit more descriptive. I want you to write a group email just saying, "I'm super excited for the AI Engineer World's Fair, and I want to meet up for lunch sometime." **Speaker 2** [0:51] Sure thing. How about something like this? "Excited for the AI Engineer World's Fair? Let's meet for lunch." "I'm super excited about the upcoming AI Engineer World's Fair." **Ben Holmes** [0:59] It does respond with a nicely worded email, but it can only respond through voice. If you want to collaborate on the written email, the best you can do is end the call and look for a voice transcript with the nice formatting at the end. ### Multimodal Gap **Ben Holmes** [1:13] Ideally, this would be something multi-modal with text and voice together, but it really feels like these two apps were built by two different companies. Scott Hanselman called this "shipping the org chart," which he explained really well talking about sitting in an EV. **Speaker 3** [1:28] I'm looking at the map, and then I turn, and then I'm looking at the climate thing, and then I'm looking at the speedometer, and they're all different fonts. And then I realized it was three Android tablets chained together, and I could suddenly see the organizational chart of this large international auto company. **Ben Holmes** [1:47] And OpenAI is guilty of this as well. You have some wiz-kid who ships a technical improvement, it turns out it's exactly what consumers are craving, marketing's never consulted, and you end up with a science fair full of potential options. ### The Fix **Ben Holmes** [2:02] Like O3, 4.0, Xbox 361 Series S Pro, you get the picture. So let's see how to fix it. There's two things I want to change about ChatGPT today: allowing voice and text at the same time, and smartly choosing theright model depending on your ask. And we can do this with off-the-shelf tools. 4.0 real-time gives you a live audio chat, and tool calls can handle the rest. You can ask it to send a text for longer details, like links and drafts. And maybe a research tool could hand off to a smarter model and come back with an answer. ### App Design **Ben Holmes** [2:39] Let's bring this concept to life. We have our voice button that takes us from our dormant state to voice mode, which works exactly the same, floating or binol. You have your mute, your end call, and also a new button for chat. This pulls up a panel that looks like iMessage. It kind of feels like texting your friend, where you're on a FaceTime call, with your call controls at the top, a little reminder of what you might have asked if you're looking through history, and then a text response in case it needs more detail for an email draft. ### Smart Routing **Ben Holmes** [3:09] Now, what if we want to ask something that requires more detail? This is something we explore at Warp Terminal, which is a developer tool to write code in any environment. I have a project pulled up here, and I can ask it for simple things like "undo my last commit." This hands off to a coding agent and decides to help me out by running commands in my terminal. It can also handle more complex questions, though, like "refactor this entire codebase to use Flutter instead." It detects that it's complex and decides to write a plan with a reasoning model to make sure the code actually works. This is a really effective pattern, and you can do with some heuristics. If you asked for details and pros and cons, for example, we could hand off to reasoning, tell you how long it's thinking, and hand back a more detailed response. ### Live Demo **Ben Holmes** [3:56] Now let's go build it using off-the-shelf APIs. Hey there, how's it going? **Speaker 2** [4:03] Hey there. I'm doing well, thank you. How about you? How's your day going? **Ben Holmes** [4:08] Pretty good. I just landed in California. Can you send me a link to a park that I should go visit? **Speaker 2** [4:14] Of course. I'll send you a link to a popular park in California. **Ben Holmes** [4:18] Yeah, I've heard about Yosemite. Can you tell me more about its history? Go deep. **Speaker 2** [4:26] Yosemite National Park's history is rich, beginning with its ancient Native American heritage and leading to its establishment as a national park. Check the chat for more details. ### Tool Calls **Ben Holmes** [4:37] And I'm handling text input just using tool calls. So we have this "send chat message" tool where you send details that are easier to explain via text. Didn't even need a system prompt, just added this description, and it was smartly able to send theright stuff with text. It's honestly amazing how far you can go with very simple prompts these days. And for reasoning models, we have another tool call. This is sending off whenever a user wants to go deeper on a topic. You can send some details and then respond back to the model with whatever it needs to say, or just dump it straight in the client if you're building an app like this. ### Outro **Ben Holmes** [5:12] If you want to see the source code, it's available on my GitHub under /fixgpt. With that, subscribe and enjoy the rest of the conference. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.