Cooking with fire without burning down the kitchen: Dominik Kundel

Thanks, everyone, for joining. Let's start with a quick raise of hands. How many of you in the last 2 years got into AI? Alright, that's a pretty good amount. And how many of you were in the room when your boss was like, "Okay, so what is our plan about AI?

What do we do about AI?" Alright, that was pretty much me last year. So, in the next 20 minutes—time permitting—should be exactly 20 minutes. We're going to talk about our team at Twilio, our history, lessons we've learned around, you know, our team and how we operate within the company, the role we play, and to ensure, sort of, that Twilio can keep up with the rapid pace of AI.

Or in other words, how do we, you know, cook with fire without burning down the kitchen and upsetting our customers.

As Peter said, my name is Dominik Kundel. I lead product and design within a small team at Twilio called Emerging Tech & Innovation. We're a team of 16 people across engineering, product, design, and go-to-market, and we try to operate as self-contained as possible outside of our two more traditional Twilio business units around communications.

That is more focused on, sort of, the APIs that Twilio is known for around SMS, voice, email, etc. And then our customer data platform that is more focused on providing a holistic overview of your customer so you can appropriately engage with them.

And our team itself is focused on exploring what the future of customer engagement is going to look like, with an eye toward emerging technologies like AI. But we're not the AI team within Twilio. At Twilio, we sort of see AI as a feature that spreads across all of our products to help engage customers to better—uh, it helps our customers to better engage with their customers.

And you might have seen, sort of, our, like, customer AI billboards around that. That is really, sort of, the vision around that is: how do we use AI to better empower people to engage with their customers. So we have "innovation" in the name, though, so are we the only ones innovating at Twilio, then?

The answer is also no. Generally, it's everyone's job to innovate. But I think there's two types of innovation. If you've read The Innovator's Dilemma, this seems familiar to you. But basically, there's, sort of, the sustaining innovation and the disruptive innovation.

Sustaining innovation is really teams trying to innovate on top of their existing products and their existing customer needs. That is, sort of, often more the iterative one. It's like Sony trying to make a mirrorless camera better and, like, really innovating in that space.

While disruptive innovation is often, initially in its infant stage, much worse in quality. It's actually a characteristic of disruptive innovation: it's the poor quality that often does meet the need of a niche audience, though. So think about cell phone cameras when they initially came out.

Nobody would leave their camera at home, go on vacation, and be like, "I'm going to take pictures on this, like, Motorola." But these days, like, most people don't even have a camera anymore. They—like, I just went on vacation and only took my phone.

The exciting thing about AI is that innovation is really happening on both ends of the spectrum. We see both, sort of, augmentation of existing workflows like, you know, Photoshop's generative fill. A lot of what Apple announced was baked into existing products,right?

It wasn't a, "Hey, here's a new AI thing," but it was more built into the features. So that's more on the sustaining innovation side. While I would say agents, especially, and GPTs in general, are more on the disruptive side.

Because the reality is agents are not ready for enterprise primetime. Primarily because the quality is not there yet,right? Like, we've all seen, sort of, agents going rogue. Partially even RAG-based chatbots, like, not even agents, like, you know, hallucinating things.

Like the example of selling, you know, a Chevy Tahoe for a dollar. Or, you know, Air Canada's chatbot making up a policy. And so, because of all of that and overall the lack of clarity in the regulatory space, often within enterprises, the adoptionright now is dictated by legal more than product teams.

And so, overall, we're seeing, sort of, a lack—like, it's not ready for the primetime. The other aspect of it is, for a lot of enterprises, they see AI as a cost-saving method. And agents are not cheap yet,right? Like, we can do increasingly better, like, RAG-based chatbots for low cost, but, like, agents are still not in the cheap level if you want high quality.

And so they're at odds there. The difference, though, is for startups and SMBs. They are really your niche these days, where for them, they actually see agents as a way—as a value-add, not as a cost-saving. They already know that their quality of customer support and sales and stuff is not where you want it to be.

And so for them, it's much more an opportunity to really add value. Because, you know, it can't be worse than the current experience. So why am I talking about the difference between these two types of innovation? It's very natural for a business, actually, to focus on sustaining innovation, especially if you're the incumbent in the market.

It's—you already have an established customer base, they're already paying you, so that's great. The problems are well-defined, and so overall it's much more predictable, and you can—you hopefully understand your target customer. While for disruptive innovation, it's, sort of, really the opposite, and you don't know yet what it's going to be, and, like, you often are not going to make money initially.

But ignoring disruptive innovationright now can be more dangerous than ever before. Because the biggest difference between sustaining and disruptive innovation is the quality. And the quality changes every day,right? Like, every dayright now with AI, there's massive leaps. And so something that wasn't quite there from a quality perspective yesterday can be there tomorrow.

And in a similar way, a company that is not your concern yesterday can be tomorrow. I think the best example of this is the impact that ChatGPT and Perplexity and similar things had on how Google approaches search,right? Now we have AI Overview and things like that.

And so our teamright now is focused on this disruptive innovation aspect. That's, sort of, where our innovation perspective comes from: we want to understand what is going to be disruptive to Twilio, and how do we prepare for that, react quickly, or be the disruptors in the space and inform the rest of the company about our learnings.

But that wasn't our original agenda. Our original agenda was we were founded as a three-person Skunk Works team last year that was really focused on special projects and trying to rapidly prototype the bridge between our customer engagement—like, our customer data platform and our communications platform—to create this customer engagement flywheel that I was talking about.

And to do this in a way of quickly prototyping solutions, showing them to customers, getting feedback, understanding what is their vision, where are they going to go, not just their current problems, and then iterate on that. And we found Gen AI was actually a natural, like, fit for us.

Because the biggest thing about customer engagement is you're, sort of, sitting between unstructured communications data and structured customer data. And nobody wants to chat with a JSON object. And, like, your unstructured data is not very helpful for analysis.

And really, large language models are perfect to bridge this. They're great at creating this connection and translating structure to unstructured and vice versa. And so, as a result, we dove into prototyping our first two solutions in that space.

And that's what got us into the AI space. We had conceptualized, essentially, two features that we showed to customers, iterated on, and ultimately handed off to R&D to take to market. One of them was the AI Personalization Engine, which is a RAG on top of this customer profile that is within Segment.

And then the AI Perception Engine that was more focused on how do you take communications data and then translate that into a customer profile. Because within a conversation, there's so much rich information that you want to remember about a customer,right?

And sometimes a human agent will take notes somewhere, but, like, realistically, you want to have one central spot. So we conceptualized these two systems, and, sort of, together they make this customer memory. But as we showed them to leadership and they handed them over to R&D, we did hit our first challenge.

And that was that we had conceptualized something that wasn't sustaining innovation, it was disruptive innovation. It was a problem where the quality was not quite there, the cost was not quite there, and primarily we were solving a niche use case.

Like, what we had made up was primarily helpful for this up-and-coming wave of agents and generative AI solutions in general, but there wasn't a large market for this yet. And so, even though we had great feedback, we didn't really have the momentum that helped us to prioritize this among the sustaining innovation and profitability focus that our R&D teams had.

And so we learned from that that even though our customer obsession was key, because it helped us understand that we were on theright track, we knew from talking to them that this wasn't just a problem they were trying to solve today, it was a problem that they were going to solve tomorrow.

What wasn't working was we had to build ideas out further to be able to gain that traction and be able to hand it over to R&D. And so we started working on our next project called AI Assistance, which is an agent builder that is built on top of this customer memory concept.

And it's designed to really allow you to build omnichannel customer engagement chatbots. And we started with the same principles. We tried to really rapidly prototype, quickly get something into the—like, to show to customers, get feedback, get a working demo, continue to iterate on it as we're getting feedback, both internally and externally.

But we did a couple of things differently this time. One, we decided we have to get this into the hands of people as soon as possible. What we had built previously was primarily demo-driven. It was, what can we show to customers and get feedback based on that?

And so we changed our mode and ran internal hackathons, tried to give really rough—access to really rough prototypes with all the rough edges to customers coming into the office. And we were like, "Here's for one day you have access to this prototype.

Let us know everything that is wrong with it and what you want it to do." And this was both terrifying and incredibly, you know, insightful for us. And additionally, we tried to find more opportunities to do dogfooding. We tried to find ways we can solve problems internally.

We started with a low-risk use case around IT help desk. We knew it wasn't the perfect target customer that we would expect to buy our solution, but it was a way for us to gather data and see what the quality challenges were and how we wanted to structure things.

And then we started to establish an engineering organization that would actually be able to set the foundation for us to actually take this into the hands of people. But that brought us to the next challenge, because at Twilio, we've worked hard over the last couple of years to really build the foundation to—in our software development life cycle—to really create trust with customers in what we're launching.

And Gen AI really changed how you develop AI products,right? It's actually exciting, but also tricky for larger companies, because Gen AI turned, sort of, the entire product development life cycle on its head. Where in the past you had to gather a lot of data, you had to make sure you had enough data to train a model.

You trained the model until you reached a certain level of quality, then you released it, and then you started optimizing. These days you can actually use Gen AI to build a prototype, or often an MVP, very rapidly without having that data, and release it out to some customers to get feedback, and then continue to iterate on that.

But that puts us into a dilemma, because we have a system now that we know is not quite there from a quality perspective, but we need to get it in front of people to make it better. And so how do we do that, and how do we keep up with the pace of technology without really upsetting customers with something that doesn't meet our normal quality bar?

And so that's why we looked around the industry for other examples. The two that came up for us were GitHub Next, the creators of GitHub Copilot, and then Cloud for Emerging Technologies and Incubation that created things like Workers and D1 and Workers AI.

And both of these were really focused on setting the appropriate expectations for users around reliability, capabilities, and availability. And so our solution for this was to create our own sub-brand called Twilio Alpha, that was very focused on, you know, setting theright expectations with customers, but enabling us, therefore, to ship early and ship often, engage the interest from customers outside of those that we would have to otherwise track down, and enable customers to come to us.

And so that's learning number two: to ship early and ship often so that we can regularly, like, get rapid feedback from customers.

The slides are off now.

Oh, no.

Yeah. Did they adjust stuff?

Yeah, literally like last night.

That was like, dude, are you pushing it? Wait, it's coming in, it hasn't moved at all.

Mine still thinks it's connected.

No, yeah, hold on.

Nope, now it's gone.

Yeah, I think it's moved, maybe.

There we go. Nope. Yeah. Yep. Cool. So that got us to our second learning about shipping early and shipping often. Because by having a sub-brand, it allowed us to open waitlist for developer previews, show our offerings, and onboard people quickly, as well as have internal POCs to learn while we're onboarding people.

But everyone talks about shipping fast. So, like, you know, how fast? Like, especially with, sort of, the rapid pace of AI, like, how do we ship fast enough? And the honest answer is we haven't figured out yet what is fast enough, but we put some base principles in place that help us move fast.

One was around how we grew the team. As we started hiring people, we focused on curiosity and creativity. Because at Twilio, we've always valued the creativity of developers to solve not just coding, but business problems. In fact, our co-founder, Jeff Lawson, wrote a whole book "Ask Your Developer" about it.

And so for us, using that—like, having natural curiosity within the team—was more important than existing AI experience, because we wanted everyone to be able to solve problems rather than relying on product to figure out what is the thing we're building.

We wanted to give problems to the engineers and be able to figure out what we're going to build. The other thing we put in place is flexibility, both in terms of our systems. We assume that every model that we're using is going to be redundant tomorrow.

That doesn't mean that we're jumping on every model that comes out in every paper, but instead we understand what are our known limitations and how do we—and every time a model comes out, how do we quickly validate whether the next model is something that helps us in that moment?

So, for example, when 3.5 Sonnet came out, we were able to figure out within a day whether we wanted to pay any attention to this for an hour or whether we were putting it in the backlog. The other side is flexibility of roadmap.

Having a new team that is specifically focused on innovation helped us not have any pre-existing commitments to customers and allowed us to really have flexibility about the roadmap. And we try to defend this flexibility as much as we can with current initiatives around this.

The big part around this I would recommend is to understand that expectations are shifting at any moment currently. Meaning, figure out what are expectations that are staying, because those are worth putting on the roadmap. Those that are ever-changing, you need to have more flexibility around.

Think about the difference between trust and safety, where people are, you know, always going to be focused on, versus multimodality, where every time some new mode comes out, people are going to change what they want to have. Like, most people don't talk about image these days anymore, but, like, that was the hot thing, like, a few months ago.

Now everyone talks about voice. So you will constantly have to iterate on your roadmap, and you have to be okay with failures. But if you're okay with failures, especially in the environmentright now where everyone in the company and most companies, like, has to provide value at all times, how do you provide that value while you're waiting for your eventual home run?

That's where we made, sort of, the last mistake that I want to call out, where, as I mentioned, our team is self-contained within the company, including having our own go-to-market. And in the first year, we took that self-containment a bit too far.

We basically just operated in our own corner quietly and shared what we were working on on a need-to-know basis, both with customers but also internally. And that resulted in often teams not even knowing that we existed as a team or what we were working on.

And it was on us to find potential conflicts or collaboration opportunities. And at the same time, our customers were having the same questions that we had around, "What are we doing about AI?" You know, and it really, especially for a B2B company, you have the opportunity to help your customers be thought leaders in their space and build a stronger bond.

So our third learning was to share things as we go, both internally and externally. Internally, to find opportunities to share all that enormous learnings—the enormous learnings that we had—with others to empower them, but also externally to enable others to be thought leaders in their space.

So these are our biggest learnings from the three, you know, from the last year, and we actually ended up turning them into our base principles for the team. So being customer and developer obsessed to make sure that we talk to customers early and often and understand not just, you know, their current problems and their current problems with Twilio, but understand their business, their challenges in general, and their vision for customer engagement so that we can anticipate their future needs.

Shipping early and often so that we can set theright expectations with them, but still get things into their hands and get feedback from them. And creating a team that is curious and owns problems so that we can actually build things quickly.

And then lastly, share as you go—share as we go—so that we're both sharing internally as much as we can, but also sharing externally to help our customers be thought leaders. And with that, thank you so much for your attention.

I will put, like, the slides are already on that URL if you want to check them out. As Peter mentioned, I will have to run to the Twilio booth to raffle away a prize. But if people do have questions, please come to the Twilio booth.

It's, like, literally over there. It's the only one that is red. And I'm happy to answer any questions there. But thank you so much for having me, and have a great day.