The UX of AI: Making AI-Powered Apps Your Users Don't Hate - Kathryn Grayson Nanz, Progress Software

Hi, my name is Kat. I'm a Senior Design and Developer Advocate at Progress Software, and I have spent most of my career focused on the front end. After taking a somewhat meandering path through design and into development, I eventually found my niche in the space where those two worlds overlap.

My design background has given me a lot of insight into the UX side of software engineering, and the user research that I get to do as part of my work as Developer Advocate is one of my favorite parts of the role.

I have spent a lot of my career teaching developers about design, about UX, and about how to build more user-friendly software—something that's always been valuable, but is more crucial than ever now that AI has entered the picture. I don't need to tell you that AI is a pretty unique new technology; it is rapidly developing and has so much potential to improve systems and software in ways that we couldn't have even dreamed of just a few years ago.

However, along with all of that potential is also quite a bit of risk. AI's non-deterministic nature means that the output can wildly range in quality, format, style, and accuracy, even when we're trying our best to prevent that. On top of that, it requires entirely new types of interaction patterns to use.

In fact, we've seen the emergence of new vocabulary, techniques, and sometimes entire roles meant to address that need. For those of us that are living and breathing this technology every day, concepts like prompt engineers, hallucinations, retrieval-augmented generation, and so much more have all become regular parts of our lives.

But for the vast majority of our users, that simply isn't the case yet. In any kind of technology, there's a gap between the developer, who has by nature kind of become the subject matter expert, and the user. The technical literacy of the average software engineer is always going to be higher than the average user of said software.

That's where UX comes in; it's how we bridge the gap between what we intended when we built an application and what the user actually, well, experiences. After all, it doesn't actually matter what our software can do if our users hate using it so much that they will avoid it at all costs.

Andright now, AI has a kind of serious UX problem. Our users are seeing AI features get added to basically everythingright now, from their work software to their social media apps,right down to their computer and phone operating systems. But as a wise technologist once said, "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should."

We're really pushing our users to adopt these new tools and features, but we're not always making it easy to do so. It's common to see situations where a user is presented with a poorly designed AI feature that they really only have the barest idea how to use.

Then, when they try it and it doesn't do what they wanted or expected it to do, they feel annoyed and off-put and kind of negative about AI generally. The more times they try an AI feature and get subpar results, the less likely they are to engage with it again in the future.

Which means that, as the creators of these features and software, we really only have so many chances to get thisright before our users just disengage and stop trying at all. In my personal opinion, this knowledge gap between developer and user when it comes to AI is one of the widest that I've seen with any technology.

That makes it extremely hard for us to build AI-powered features that our users are actually going to get the full benefit from. When Macintosh introduced their graphical user interface, it was, like AI is now, something that was very different and unfamiliar to their users.

But a large part of their success and reputation was built on their ability to create a user experience that met users where they were. They borrowed vocabulary and user flows from real life, they included lots of icons and visual representations, and they really made sure to walk users through how to get the most out of their new personal computers.

And over time, as the average user became more comfortable and more literate in the space, the user experience of the Macintosh evolved with them. We start to see more technical terms and fewer abstractions. Look at the difference here between the control panels in Macintosh System/1 and System/6.

If you had presented a System/1 user with the words "rate of insertion point blinking" or "RAM cache," it would have been pretty much meaningless to them. Similarly, we see that those low and high volume icons are gone in System/6, along with the little turtle and rabbit speed icons,right?

That kind of more literal depiction wasn't needed anymore. Right now, we are probably somewhere around System/3 in this metaphor of introducing AI to users. We may not need the turtle and rabbit icons anymore, but we also can't yet assume that they will sit down with our AI software as experts.

The good news here is that, unlike the Macintosh UI, we're not totally starting from scratch here. Our users have existing mental models about software usage that's going to carry over to what we're building. They're just not always going to map over exactly one-to-one.

But that's where we come in. As developers, it's our role to leverage these patterns that our users are already familiar with, start to combine them in new ways, and then introduce them to our users gradually in a way that's not going to feel overwhelming.

A good user experience will not only provide a user with the tools but also guide them how to use them. For instance, let's look at an AI chat interface. From a purely UI perspective, chatting with an LLM is almost, but not quite, like chatting with another user.

So we'll be able to borrow some familiar patterns here, like having the user's chats show up on one side and the AI chats on the other, messages that appear above an input field with a send button, a scrollable message history, and more.

That gives our users a really good jumping-off point and a lot of visual clues about how to start interacting with an AI chat. However, the experience isn't quite close enough that we can just lift an existing interface from direct messages and repurpose it wholesale.

A chat interface that was originally built for two people is not going to have the accommodations that we need for things like adding reference sources, pausing or stopping an in-progress reply from the LLM, leveraging agentic tools and integrations, or more.

So those are going to be the places where we have to adapt existing UX and UI patterns into something new. We have to be the bridge and help our users transition into familiarity with these new AI experiences. We can see the difference here,right?

Compare this Copilot research agent new chat with a new Teams chat. Both of them have familiar aspects,right? We see text boxes, we see the plus to attach files, and so on. But the Teams chat assumes a much higher level of familiarity.

It's not spelling out any of the iconography it uses, and we don't see some of the things intended to lower the barrier of entry, such as the examples, large text buttons, or voice interaction options that show up on the agent chat.

Now, before we really dive into this, I do think we have to take a moment to address kind of the elephant in the room,right? Why not just have AI create these solutions for us? Why do we have to be the ones who design and build new AI patterns when we have technology that can generate interfaces for us now?

The problem with that is that AI can only really remix things that already exist. It's fantastic for looking at those existing common patterns and replicating them, but the patterns for AI interfaces don't fully exist yet. Or at the very least, they are still in the process of being rapidly developed, and they are changing all the time as that technology advances.

There's just not a long history of standardized and familiar AI-related interfaces that we can reference to solve this problem. So, at least for now, we're kind of on our own here, and we can't yet defer it to AI.

Maybe that will change down the road, but for the time being, this is still a very human problem. Additionally, as many folks have begun to notice, the design that an AI creates is going to be a lot like every other design out there.

AI can reference and remix, but you're just not going to get brand-new concepts from it. An AI-generated UI tends to look pretty darn average. And while that can be a really great starting point, it's not usually a great ending point.

If you just need a quick landing page, it's probably going to do the trick. But since we're dealing with whole new interaction modes that need more attention paid to the UX rather than less, it's just not going to be enough to get us where we need to goright now at the quality level that our users need and deserve.

So if we're going to build new patterns and new interfaces to help our users get the most value out of this technology, where do we start? Well, like most things in design, it's best to start with the user, specifically the user's problems.

That means that when we're thinking about building our AI features, we need to be thinking about what challenges our users have with AI and the user flows that will mitigate those issues as much as possible. From the user research I've been doing and the users I've had the chance to speak to, I've started to group the main challenges here into five categories: trust, clarity, control, transparency, and meaningful benefit.

In this talk, we're going to discuss each of these in depth, understanding the problems our users have when they're using AI, and looking at some examples of new patterns and techniques we can employ to address them. By making sure that we are addressing each of these, we can start to create AI experiences that support our users through the introduction of this new technology and ensure that they are in a place to use it to its fullest potential.

So let's get going. Trust. The number one, by far biggest hurdle that we need to overcome in order for our users to engage with the AI features we build is trust. Right now, to the average user, AI is a black box.

Many simply do not understand, from a technical perspective, how it works. And when you don't understand how something works, it makes it very, very hard to trust the output. On top of this, just about everyone who has interacted with an LLM before has, at some point, seen them hallucinate.

Even as the models keep getting better, we're still not at the point where we can claim any tool will be 100% guaranteed hallucination-free. There's always a chance, even if it's a small one, that the AI is going to provide an incorrect response.

So how many times can a user see output that's wrong and still trust it? It can be easy to think that the answer to this problem is to try and position the feature that we're launching as some kind of exception to this rule.

We say things like, "Other AI tools might not be trustworthy, but ours is different. Ours is higher quality, it's safer, it's more reliable," so on and so forth. But not only is this questionably true,right? I mean, after all, not many of us are really training our own models here, so a lot of this is going to be simply outside of our control.

But it's also a very difficult thing to try and sell your users on. In situations where we cannot promise the truth, we have to go above and beyond to earn it. Our honesty about current capabilities and limitations of our AI tools will go a lot further with our users than the denial of any potential problems.

So rather than trying to convince users that our solution is inherently trustworthy, we can instead build in patterns that empower them to see when incorrect information is returned and give them the tools to correct or mitigate it. You know the saying, "Trust but verify"?

That's kind of the goal here. By citing the references in an AI-generated response and linking users back directly to the source material, we give users the information they need to validate AI output themselves. The more a user is able to click through and see where an answer came from, the more they'll be able to trust the content, even if they don't choose to check every source every time.

In addition to allowing our users to vet the answers, citations also have a second important purpose. They allow users to repurpose the material in their own work while maintaining a trail of accuracy. This is really important for their own credibility and reputation.

After all, how often would you share content from an unverifiable source if you knew that any errors would ultimately be attached back to your own name? Generating output for the user might be the last step in our process, but it's really just the beginning for them.

Users want to take that content and turn it into a report, an email, a campaign, a presentation. And if the output can't be cited and trusted, then it's just not going to be used. There are a handful of different ways we can implement this, and which one you pick is going to depend on what you're building and the context in which it's being used.

But a few popular approaches include things like tooltips, inline links, and side panel reference windows. Tooltips allow you to provide a small snippet of a relevant quote, which can be really great for increasing credibility. Links, of course, are ideal if the content is coming from an external source, like another web page, as opposed to an internal source, like a document in a shared drive.

And if the feature you're building is meant to support more research-oriented work, then you might consider adding a side panel where the user can explore the source material in more detail, which kind of positions the AI assistant as more of a librarian than inherently a subject matter expert.

The other place where trust factors heavily into AI work is agentic workflows, where the AI is thinking and executing work on its own. This, understandably, can be kind of concerning for users, depending on the stakes of the project and how easy or hard it would be to roll back any incorrect actions, which is when keeping that human in the loop really becomes crucial.

One of the best ways to help users trust these systems enough to use them is to show an action plan to the user and allow them to approve it before the agent begins work. This has become a pretty common flow in a lot of foundation models,right?

If you're using agentic features in Claude or ChatGPT, you will have seen it create a list of the steps it will take to accomplish a given task and then ask for your confirmation before beginning. Of course, there are also settings you can add to toggle this off or always allow it, all of which is important if you're going to allow users to repeat a flow over and over and don't want them to have to babysit it.

Spoiler alert, we will talk a little bit more about permissions in the transparency section later. But if you're creating some kind of an agentic tool and no plan is ever shown to the user and an action happens without them understanding why or how, it can be very, very hard for them to trust both the result of that action and the agentic tool itself.

There is also, undeniably, and I would maybe even argue correctly, quite a bit of skepticism around whether or not content is AI-generatedright now. You've probably seen exchanges online where someone shares a photo or video, and another person is immediately commenting to tell them that's not actually real.

When we spend so much of our time and energy second-guessing and investigating content that's shared with us, it doesn't exactly foster an environment of trust. For now, AI-generated content is highly polarizing with users. While some have heavily leaned into using it in their daily lives, others will react to it negatively and dismiss it as slop,right?

The ability for AI to generate text, images, and video that is nearly indistinguishable from human-created work is very new, and for many users, that still feels uncanny and unsettling. I don't say this to start any kind of debate,right?

I mean, it seems harsh to say it, but our own personal opinions on AI-generated content don't really matter too much in this context. What's more important is the awareness that our users will react to AI-generated content in a wide variety of ways, and not all of those ways are going to be positive.

So if we want to include AI-generated content in our applications and we're not exactly sure how our users will feel about it, what can we do to retain their trust? One of the easiest things is to just mark any AI-generated content as AI-generated.

This could be as simple as a disclaimer in the app store description, a watermark on a photo, whether that's digital or visible, or maybe even just an asterisk at the end of a sentence that denotes something as AI-created.

When we do this, we remove that potential user perception of them feeling like we tried to sneak AI-generated content into our apps or somehow pull one over on them. If we're using AI and we think it's theright fit for our work, or we feel strongly that our users should be able to share AI-generated content, then there's really no shame at all in designating where it's been used.

Depending on where that AI is used, it might also be helpful to clue our users into which aspects of output they might need to manually verify, rather than having to review everything as though it was AI-generated, if that's not truly necessary.

Hand in hand with trust comes clarity. Clarity is one of the most important aspects of creating a good user experience, kind of generally. And the stakes only get higher when we start incorporating AI. Because, as we said, so much of AI still feels like an unknown to our users, we need to be as descriptive and straightforward about what's happening at any given time.

The more insights we can give them about what our software is doing and what they can expect to happen next, the more comfortable they will feel engaging with AI. I've noticed that there can be a kind of impulse to frame AI to our users as magic,right?

We don't even have to look further than the prevalence of the sparkle icon to designate AI as an example of that. However, while that might feel kind of mysterious and cool, it's really not accurate,right? AI is just another technology, and our users deserve clarity over dramatics.

They want to know what's going on, how a given output was created, and there's really no reason for us not to pull back that curtain whenever we can. One of the most obvious ways for us to do this is with streaming text or similar live generation of content.

Not only is this a really great way to disguise the latency issue of AI generation,right? Nobody likes to watch a loading spinner, but it's also a fantastic way to show the user what's happening in real time. If we make the user wait until an entire response is generated, there's a non-zero chance that it won't actually be what they wanted.

However, if we start showing a partial response as it's being generated, we give users the chance to start assessing it immediately. That means they can be coming up with follow-up questions or adjustments or maybe just stopping the process entirely if it's not returning what they expected.

This also keeps the user as an active participant in the process, preventing that XKCD, my code's compiling experience of pressing a button and walking away. You've probably experienced yourself how much harder it is to complete a task if you are constantly disengaging from the workflow.

Even if you know a process will only take a minute, if there's not something to keep you there, you're going to start checking your emails or replying to Slack messages or scrolling on your phone. And just like that, a one-minute wait has become a five to ten-minute side quest, making it much harder to pick up where you left off.

The more we can prevent this for our users, the more productive they can be when they're using our tools. Another thing that can help remove the wall between the AI and the user is having it think out loud as much as possible as it processes a request, especially in chat-based interactions.

Similarly to live streaming, this improves a user's confidence level in the output because they can kind of follow the breadcrumbs and have a deeper understanding of how a given conclusion was reached. After all, we do this all the time when we talk to other people,right?

If we're presenting an idea, we know it's going to be helpful for us to kind of talk through the rationale behind how we got there and why we think it's theright choice. Explaining ourselves is such a natural part of human interaction that we've really come to inspect it.

And when we don't see it mirrored in the systems that we use, it can feel kind of frustrating, like something is being unintentionally kept from us. Additionally, seeing that chain of thought process allows for better, more accurate revisions when the output wasn't what they expected.

If the user can understand all the steps that happened between point A and F, then they can really pinpoint where things went off the rails. And that allows them to give more specific instructions or clarify ambiguous parts of their request.

Some existing patterns are a little bit easier to adjust from their previous non-AI usage into this context. And highlighting new content is one of those. It's not groundbreaking, but when it's combined with everything else, it can be immensely helpful in helping users keep track of AI actions, especially in situations where an agent is acting autonomously.

The basic idea is that when new content appears or something gets changed or updated, it gets specially marked for the user in a way that draws their attention to it. Sometimes that might mean literally moving the focus up or down the page to see what's changed,right?

Like when new messages appear in an ongoing chat and the message history scrolls down to display it. Other times, though, it could just mean drawing the user's eye back to a space they might not have been actively watching, even if that doesn't include literal movement on the page.

Things like changing the color of revised text or putting a box or highlight around new material or even marking lines of code that are different.

AI tools are often framed as being a kind of personal assistant for the user, an intelligent junior coworker that they can delegate tasks to. However, as anyone who's ever delegated a task can already tell you, sometimes that means that things don't get done in the way that you would have done them.

When users are working with our AI tools, they need to know that they are still ultimately the ones in the driver's seat. After all, there's very little value in being able to assign the AI tasks if you can't then step in as you like to make adjustments, corrections, and fixes.

In that vein, it is also important for the user to be able to stop or override an AI action at any point, whether that's during content generation, partway through an agentic workflow, when it's running code or scripts, or something else entirely.

If the user is unable to abort the process, then they're not actually the ones in control, and frankly, that's not really acceptable. One of the most reassuring things we can offer nervous users is a clear and prominent emergency break that they can slam to bring everything to a halt for any reason.

That means it can't be tucked away in a menu somewhere or involving some specific command they have to remember, which they won't in a high-stress moment. We want to give them the equivalent of a big red button they can slam to make everything stop whenever they need.

Of course, once a user has stopped a process, we can probably guess what they want to do next,right? They want to undo the incorrect things that changed. Even way before AI, Nielsen's heuristics list safe exploration as one of the core principles required for a good user experience.

That means the user should be able to navigate back and forth, click and unclick things, and generally just kind of mess around in a piece of software without getting stuck or unintentionally making big permanent changes. Sometimes a user will try something and then change their mind, decide they don't like it, or it didn't do what they thought it would.

In those cases, we want to make it as easy as possible for them to roll things back to an earlier state. This kind of safety net was always valuable, but now that we're dealing with non-deterministic output, having some kind of a version history is going to be pretty much a non-negotiable.

Of course, the level of version history that's required will differ based on the task. The Git style of version control that we are used to as developers is probably more than you will need in a user-facing application. If this is a truly simple conversational interface, you may not need it at all.

The user could just rephrase their question and try again if they don't get the answer they wanted. However, if your tool is doing something more complex, like creating and editing a document, simple undo and redo actions that allow users to move back and forth within the last 10 or so steps would be really helpful.

And if you're wanting them to do truly advanced work with your tools, then it's worth building in the mechanisms for checkpoints or safe states. Consider what kind of tasks your user is going to be working on with your AI features, and think about how significantly each step forward is likely to change the existing content.

Ideally, this will also allow users the specificity to make targeted adjustments. You don't necessarily want them to have to wipe out and replace everything that happened in a given step if they weren't happy with it. It's common for AI-generated results to have a mix of quality in the output.

Users may want to keep some aspects while reverting others. And the more control we can give them over that creative process, the more enjoyable it will be for them to collaborate with our AI tools. This is especially true now, as the cost of tokens is starting to tick up.

Having granular control over what gets reworked allows for more specific and productive iteration without them just having to say, "Try again," and start over from scratch each time.

AI tools are strongest when they seamlessly integrate into the rest of a user's workflow, referencing internal documents, sending emails, making calendar appointments, drafting and sharing content, so on and so forth. However, most users won't integrate what they can't see and don't understand.

If the AI applications we build are going to become part of their daily lives, then users need transparency into exactly what they do, what they have access to, and what it will cost them in both time and money.

The more transparent we can make these features, the less hesitation our users will have in adopting them. The concept of asking for user permissions is certainly not AI-specific, but the stakes can feel a lot higher when you're asking users for permission to allow AI to act on their behalf.

That means we need to make it easy for users to see which applications we've allowed our tool access to and in what ways, keeping in mind that permissions are often more than just a binary yes or no. So can your agent just reference data from this database?

Or is it also allowed to delete tables? Can it only read your user's emails, or can it send from their address as well? Can it run scripts, search their local files? Each one of these actions requires direct user sign-off.

Permissions are also not a one-and-done situation. A user might feel comfortable allowing an action to happen once under their direct supervision, but they might not want to allow it permanently. They could give access to a specific folder, but not every folder.

They might approve an action, but then want to be notified every time it's taken. Consider which of these gray zones might exist in your permission structure and try to accommodate as many different options as you can. Another common sticking point with AI is data collection.

Often, it can be helpful to save information from past interactions, but storing this kind of data requires real attention to user permissions and data management. If you want to create a system that can remember things, then you also need to make sure that it can forget and that the user can not only see, but has the final say in what exactly gets remembered.

An ideal permissions flow will not only ask for a user's approval in the moment, but also create a space where they can see the history of what they've given access to and when, as well as allowing them to revoke that access or permission at any time.

Now, this one's pretty simple, so we won't spend too much time on it. But another crucial aspect of transparency is making sure users know exactly what they are committing to when they approve an action request. In addition to approving those actual steps and plan, as we already discussed in the trust section, they also need to know how much time something is going to take and what it's going to cost them in money or tokens or credits, however you are counting this.

Even if we can't specify an exact amount, if we can provide a rough estimate, that's generally going to give users enough information to work with and to potentially revise their request if it's going to exceed what they're comfortable with.

Similarly to the marking that denotes which content was AI generated, it's also a really good example of a chance to have a visual signal for when an AI tool is acting independently. If, for example, you're going to allow an agent to take control of the user's browser, then there needs to be some kind of banner or sidebar or outline, an indication that the user is no longer driving that interaction.

Not only is this a good thing to do just for transparency so the user understands the current state of the system, but it's also helpful to make sure they don't unintentionally interrupt or confuse an ongoing process. It's an especially important consideration for processes that you know are going to take an extended period of time where the user might have stepped away and then come back without necessarily keeping track of everything going on while they were gone.

Finally, all of the interesting technology and cool functionality in the world is meaningless if what we build isn't solving the problems that our users need solved. To do that, we need to create experiences that make it as simple as possible for our users to get the output they need from our AI features and start to leverage it in their work or daily lives.

One of the biggest mistakes we can make when developing AI features is assuming that our users know how to use them. As mentioned at the beginning of this talk,right? We know how to do things like provide context or specify formatting requirements or break large tasks into smaller ones and then iterate.

But our users often don't. So when we place a blank text box in front of a user and just tell them to ask AI, we're actually kind of asking them to do a lot of work in figuring out how to really use it.

They need to understand what kinds of problems the tool can solve, how specific they need to be, what information the AI tool needs to succeed if they need to add sources or references, how to recognize when a response needs refinement.

Rather than expecting this level of AI literacy from our users on day one, we can help them by providing examples, templates, suggested prompts, and guided workflows that demonstrate what success looks like.

Now, what will your users be doing with the content that your AI feature generates? If they're going to run a search or analyze a spreadsheet, what happens next? What do they do with the results? If they create an image, who are they showing it to?

How are they sharing it? Is it getting printed or posted online or sent to a friend? By introducing next step action buttons, we can make it as easy as possible for them to leverage the output of our tools in their own work.

If they're not able to action on the content that was created, it's never going to be more than a novelty to them. It's not going to be of realistic use. But we can help guide them to suggested next actions that will help them make the most of what they created.

If we want to take this one step further, we can start to build in direct integrations with their other most used tools. Maybe they want to generate a new document with the content in their word processing software from a draft.

Maybe they want to push new code to a linked repo. Maybe they want to open a new ticket in a tracking system with their findings from a test. The simpler we make it for them to get the information from our tool into the rest of their workflow, the more useful it's going to be to them.

Now, AI can generate content, write code, analyze data, automate tasks, empower our users to do all kinds of amazing work, but only if they're willing to try it and only if we make it easy enough for them to do.

When everything our AI tools are doing is hidden behind a curtain, it makes users feel like things are happening without their input, which is hard, especially when many of them are already kind of feeling some level of AI skepticism or hesitation.

If we unintentionally create AI experiences that take away our users' power, understanding, and autonomy, then they'll never be interested in using what we build because they'll never feel truly comfortable engaging with it. By focusing on patterns that reinforce those pillars of trust, clarity, control, transparency, and meaningful benefit, we can set guardrails around our AI features that will help make our users feel safe and confident.

Because the technology is here,right? The models are already really good, and they just keep getting better and better, faster and more efficient. That means that the differentiator for the AI-powered software we build isn't performance anymore. It's the quality of the experiences that we can build around them.

The line between design and development is blurring a little bit more every day. How users interact with the system, how much information they see, how much control they have, and how we earn their trust are now questions that developers have to consider when building AI-powered software, whether your title includes designer or not.

The technology may be incredible, but for it to be truly successful, users still need to be at the center of everything we built. Thank you guys very much for listening to this talk. I really hope this is helpful to you as you start integrating AI features into your own applications and software.

You can find these slides as well as a full transcript of this talk at the following links. And of course, feel free to reach out to me online if you have any questions at all. I'm always happy to chat.