Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation - Alex Liss

Hi, I'm Alex Liss, VP of Data Science and AI at Huge, a design and technology company, and today I'm going to talk about invisible users, invisible interfaces, which is how teams can use AI simulation to accelerate design. We're going to talk about three things today when it comes to the state of UX and AI: one, where we are now; two, where we could be; and three, an idea for how to get there.

So, in terms of where we are today, unfortunately, the reality is that AI has a trust gap. Some recent research by Edelman, which came out in December 2024, highlighted that only 32% of U.S. adults say they trust AI, and only 44% of adults globally say they feel comfortable with how businesses are using AI.

And the reason for that, unfortunately, is AI slop. And as these screen grabs show, AI slop is when you go to use a website, a product, an interface, and there's a genAI fail where it gives you something that you know is not correct.

Whether it's a search, a web search that tells you you should eat rocks every day, or a website that tells you you can buy a car for $1, we know that neither of those things can be true.

Website creators are stuffing AI chatbots and everything and telling users it's magic. But the real magic of genAI, as Cassie Kozikov says, you know, it is the fact that it's a UX revolution. It's that users can actually talk to a machine learning model in natural language, which has never been possible before.

So, with that in mind, we really want to think about the capability of that and go back to some UX-first principles for us to figure out how we want to use genAI to help design. So, going back to how it started, you know, some writings by Don Norman of the Nielsen Norman Group talk about the design of everyday things, talking about simplicity and efficiency.

And he put forward another great principle in his book, The Invisible Computer, which talks about invisible interfaces, which is software that feels so seamless and intuitive the users practically forget that they're using it. And I think the opportunity here is to use AI to design interfaces that actually feel like magic, not by stuffing chatbots into the website, but by accelerating needfinding.

And this part about needfinding is in asterisks because I just want to make it clear I'm not a designer. You're not going to see designs I produce today; I'm a data scientist. But you are going to see a proposal for a process for how we could deliver this AI-accelerated needfinding.

So, let's talk about what a new approach to the design lifecycle looks like, because in the world today, all of us are bombarded with so many screens, so much information, so much complexity, it's a little bit akin to a pilot operating in a cockpit.

And the way that pilots learn to operate in this environment is actually through simulation, through a lot of practice. And I think we can take this idea of simulation and evolve it and reinterpret it a bit. Our current design process, which comes from the pre-ChatGPT era, is data-driven, as designers collect, you know, qualitative observations, quantitative data, ethnographic observations, and they use that to guide the prototyping process.

But going forward, what if we could empower designers to work with invisible users in the form of AI simulation to turn the data artifacts that they've always collected as part of needfinding to active participants in the design process, and give designers their own mini feedback cycle to work with AI simulation as part of the broader needfinding process to help deliver better design?

So let's take a look at what an example of an accelerated needfinding process could look like, through an example project. So the new process for needfinding in the era of AI simulation has a lot of similarities with the existing process.

We're going to start by defining our audience, then mapping their intentions, identifying specific tasks they will perform to accomplish those intentions, conducting analysis of that process with finding insights, and ultimately developing design alternatives. However, in this era of AI simulation, there's going to be a few key differences in some of the components and some of the workflow.

So when we start thinking about audiences in the era of AI simulation, we want to start with data that represents these audiences. And at my company, Huge, we have a data platform, which we call Live, that has a mixture of datasets, demographic, psychographic, contextual, that allow us to simulate audience behaviors in the real world.

And this is a critical foundation to build on. Once we have this foundation, the next step is to actually apply intent mapping. And here's where we turn the data into something active: a simulation, which we call an Intelligent Twin, which represents a set of user behaviors and a set of desired outcomes, their needs and motivation.

Intelligent Twins become an active participant in the design simulation process. And from there, we can brief Intelligent Twins to evaluate interfaces by focusing on specific tasks, much as we might brief a human designer to conduct a heuristic analysis.

And in this way, we're able to evaluate a specific interface, or in the case of this sample project, a specific website. Now, for the sample project I looked at, I wanted to think about the advantages of this simulated methodology, the scale and speed.

Intelligent Twins can operate with different levels of velocity than a human team, so I wanted to apply that to do a global audit of sports websites. And we can imagine this happening from the perspective of a business that wanted to partner with sports leagues globally and learn how to cut across different countries, different cultures, to engage audiences there.

And this is where the Intelligent Twin methodology could provide some unique advantages. So, in terms of the actual data collection, with Intelligent Twins here in this sample project, I actually broke them out into two audiences, two personas: a casual fan, newer to sport, versus a super fan who's lifelong and very savvy.

And across these three different websites, we briefed a series of tasks across different categories like navigation, information architecture, and fan engagement, with four tasks per category. And that allows us to simulate 72 AI-simulated actions to give us a test drive for how this AI-accelerated design audit will perform.

When we get into the actual findings, so with the speed and the scale of Intelligent Twins, we can roll these insights up at a very high level or dive more granular, as we'll see in the next couple of slides.

But one of the benefits of this kind of global audited speed is it gives us high-level understandings across an entire category. And we can see from this audit that in the area of navigation, just kind of the starting place of a fan coming to the website, all the different leagues we looked at, you know, basketball, the Olympics, and the English Premier League, all those sites performed pretty well in terms of task completion when it comes to navigation.

But as we get deeper into the process, as fans started to browse content and information architecture and engagement pathways, that's where those initial successes started to drop off. And this goes back to the challenge I mentioned at the start of the presentation.

There's an AI trust gapright now because AI is being shoved in people's faces and websites, and it doesn't work very well. But hopefully, this methodology could help us understand what user pain points are and actually solve those pain points, and in that way, start to repair that trust gap.

We can also go deeper with this methodology, so we can scale the simulation to surface friction across specific areas, specific nuances of the experience. The different levels of altitude this methodology can operate at allows us to create design briefs which are really focused, which are really broad, and really deep as well.

And that's what we set up on this slide here, sort of closing out the audit process through AI acceleration, computer use models, computer vision models, human-in-the-loop observation. We can use this process to generate a design brief that allows the human team to really focus in the pain points that are distracting users in a specific category and try to solve them.

And in that way, we can use AI simulation to hopefully accelerate and improve the design process. As we look to the future, in the first half of 2025, there's been, you know, tremendous progress. The MCP protocol from Anthropic is already showing potential integrations to turn prototype components in Figma through to actual code components in React or Node.js or other frameworks.

And this is going to make it easier than ever to create new designs. And I think as the acceleration of these tools just makes it easier to create design that really emphasizes the importance of focusing on the why, the strategy, and the problem we want to solve for users in the design process, because it'll be easier than ever to solve for the how.

In terms of this methodology, obviously this is experimental, early stages, so there's some limitations and improvements to consider going forward. I think reproducibility is key. So as we evolve this methodology, my company and I will be standardizing things in a code repository that lays out parameters like the briefing instructions, the simulated audience dimensions, the number of audit runs, and the parameters from task completion and failure.

And also, once we standardize this, applying it through a test-and-control methodology will also be useful to help us isolate what the strengths are of Intelligent Twins for design needfinding and how that can be used alongside or in a complementary manner to human teams, especially applying to different industries, different geographies, and different domains.

In conclusion, just a few final thoughts here. When it comes to repairing the AI trust gap that we're struggling with at the moment, I hope we can all recognize that users don't need more websites that have genAI chatbots in them that don't work, but users could always benefit from better websites, better mobile apps, better surfaces that provide clarity and simplicity.

And hopefully, I've been able to show in this talk today how AI simulation can be a tool to help us empower design teams to gather insights in a smarter, faster, and better process, and ultimately create websites and interfaces that can restore and repair the trust gap.

Thank you.