How to Fail at AI Strategy: Hamel Husain & Greg Ceccarelli

Allright, everyone, welcome. Hamel and I are absolutely thrilled to be here with you to basically teach you how to build the definitive guide to completely, utterly, and spectacularly messing up your AI strategy. I actually couldn't have really wished for a better foil with Grace's lead-in than this, because we're not just talking about minor setbacks here.

We're going to take you through a way to create full-blown, company-crippling, career-ending failure.

Grace talked about best practices, but we're here to embrace worst practices. In fact, we're going to make sure you know how to completely torpedo your AI projects and ensure you alienate everyone that you work with. How does that sound?

It sounds great to me. So before we begin, we might as well start with some introductions. We have no agenda here, it's just a sequence of steps. But I'm Greg, I'm an executive leader, and I've spent years in the C-suite crafting AI strategies.

I'm now a co-founder of an AI startup, but previously I was the Chief Product Officer at Pluralsight and executive leader at other companies. And I've had a front-row seat to how executive teams can transform clear strategic opportunities into labyrinthine disasters.

And I'm Hamel. I'm a machine learning engineer and independent consultant who has worked with many companies on AI. I've witnessed every conceivable way AI strategies can fail, and I've found it really fascinating how creative people can get with their failures.

That'sright. So you could say that together we're kind of like the dream team of disaster. We've advised, or maybe we've, you know, interacted with representatives from numerous companies. We even have this fancy website. But for today's presentation, we've decided to live and breathe the great words of the late Charlie Munger, who said, "Invert.

Always invert." So let's get started. Allright, the first step to failure is to make sure that you begin to divide and conquer your own company. This is key if you're destined to fail. You've got to embrace the disconnect between willingness to pay, price, and cost, the keys to creating value, by contemplating unreasonable goals.

And you should, especially everyone here in the audience, know by now you have to make sure you go and attend every AI industry conference,right? But never go back and talk about what you learned with your team. The point, just like Moses here parting the Red Sea, is to create impenetrable silos and incentivize secrecy between your teams.

So let's get into it.

So I talked about the value stick and willingness to pay in the prior slide, but here it's really important for us to adhere to the anti-value stick. You've got to embrace it, because it's the opposite of everything good and useful when it comes to value creation and being strategic.

And today that's our guiding principle. You might be thinking that WTP means willingness to pay, but here it's wishful thinking promises. You've got to tell your customers that AI is going to do absolutely everything for them. Your new systems are going to write their emails, walk their dog, solve climate change, and achieve world peace.

But don't really worry about the details. Just promise the moon. You know about PRICE,right? Well, for us, that's another acronym. Particularly ridiculous infrastructure costs everywhere.

I mean, that was a mouthful, sorry. Buy the most expensive GPUs. Don't bother with any cost-benefit analysis. Just max out the company credit card. Think of this as an investment in something. And cost? Well, it's that cascade of spectacular technical debt you're about to run headlong into.

You need to think about building systems so convoluted, so intertwined, that even you, as an executive, can barely understand it. You know about job security,right? This is a key to guaranteeing it. Think about it this: when it inevitably breaks, no one's there except for you.

And finally, if you know about value, you know about WTS or willingness to sell. For us, it's why this system. Well, the answer, and I mean always, is because AI. There's never any further explanation needed. No board is ever going to question you.

It's like magic, but much more expensive and less reliable. So step two here is when you start to define your strategy,right? Here's the first key: fake any diagnosis you might be thinking of. Grab last year's annual report or operating plan, and just start highlighting random paragraphs, preferably the ones you understand the least, and declare, "AI must fix this."

Don't bother talking to anyone who actually does the work.

And your guiding policy should be both incredibly ambiguous and vague. Something like, "Become the global AI leader in everything." Except don't define what everything means. That's someone else's problem.

Totally. And your action plan? Simple. You need an AI-powered SEO tool that guarantees top Google search results, even if you sell garden gnomes,right? And a generative art plugin that creates NFTs of your CEO's cat. And of course, an AI drone lunch delivery service, because synergy?

Announce all this at your next company all-hands meeting, and you get bonus points if you wear a shiny suit and use the word "disruptive" at least a dozen times.

And the last point on this slide is about timelines. But timelines are for companies that intend to finish projects. What we recommend is you embrace perpetual beta. Just create a massive backlog in GitHub and stick all those highlighted financial reports into that Greg was mentioning earlier.

Great strategy. But you know what strategy really works? Just create a 4,000-page document that you post in all your Slack channels and just erode people's willpower to engage with the material. And with these tidal wave of documents, you know, in words, Greg, isn't there a strategy that you have about jargon?

There certainly is. The point is to communicate in such a way that nobody understands. Drown everyone in a tsunami of jargon. Say things like, "Our multimodal, agentic, transformer-based system leverages few-shot learning and chain-of-thought reasoning to optimize the synergistic potential of our dynamic hyperparameter space."

If you say it with confidence, you probably will have absolutely no idea what you just said.

Remember, the goal is to look incredibly smart, even if nobody understands a word you're saying. The key is obfuscation.

Yeah, you might be tempted to do something like defining a very cogent, clear business-on-a-page approach, like an inadvantage, but never be too tempted.

One of the most effective ways to cause dysfunction in your organization is to use jargon everywhere. And use jargon strategically to hide the jobs to be done. For example, I had a mental health client. Instead of saying, "We need to write a prompt," we would just say, "Hey, we're building agents."

And what that did is it made sure that the mental health experts were not in the room and didn't know how to participate. And that's exactly the result you want.

That'sright. Just like Hamel reduced those mental health experts' mental health, I like to do that as well. So instead of saying, "Hey, let's make sure the AI has theright context," I just talk about rags.

And don't say, "Make sure users can trick the AI into doing something bad." Just say, "Prompt injections."

Yeah, and the key here is to encourage engineers, not the people who might best understand your customers, to write prompts, because what could possibly go wrong?

Look, we know that translating everyday English language into jargon can be really difficult. So we made this guide for you. And this guide will help you divide your organizations, just like Greg was talking about earlier.

Just like Moses.

The link isright here. But remember, making everything, even writing prompts, seems super technical and out of reach for everyone is what you want to go for.

Allright, just a brief recap. We talked about how to seed your division, how to start to define your strategy, how to communicate it. Now we're on to mobilization,right? Because you've got to do something with that giant backlog. Well, some of you might know about Jeffrey Moore, but I've never heard of him.

Today we're pioneering a very new, revolutionary framework, which is about zoning to lose. It's designed specifically for failure.

Just randomly assign AI tasks to people with absolutely no relevant experience. For example, outsource your data review to offshore Q&A teams who have very little context about your business.

Yeah, and most importantly, you might be tempted to use the incubation zone to bootstrap new AI ideas. But the goal is to launch from here completely untested, bug-written AI chatbots directly to your customers. As Hamel mentioned, never worry about beta testing.

Just disregard quality assurance. Just ship it straight to production. Because what's the worst that could happen outside of a potentially career-ending PR disaster? So if you do it sort ofright, it should feel something like South Park. You're going to yank all your best engineers from potentially supporting your revenue-producing products, wait a while, and then profit?

No, actually, it's going to feel more like total collapse. And because you're so disorganized, we can now transition to...

Look, at this point, your organization is in complete disarray. But it's time to do the deed and burn it all to the ground. So the most effective way to start doing this is to focus on tools, not processes.

And those problems that you created earlier, and other ones that may exist, don't analyze them. Don't try to understand them. Just throw tools at them. So if your RAG system isn't retrieving theright documents, just buy a new, more expensive vector database.

Yeah. And if you need to measure progress, just use every off-the-shelf evaluation metric you can possibly find. Never bother customizing them to your business needs. Just blindly trust the numbers, even if they make no sense.

Oh, and like, you know, we've talked a lot about agents today. If they're not working, just pick a new framework and vendor. Fine-tune without any measurement or evaluation. Just assume it's going to be better, because it's kind of like alchemy with a lot more electricity.

Exactly. You don't need to look at our design metrics. Evals? That's a vendor problem. Just plug in a tool and it will solve all your problems. Greg, I really love how you demonstrated exactly what we're going for here with Whac-A-Mole.

Every time you see a problem, hammer it with a tool. If another problem comes up, hammer that with a tool. If the same problem comes again, hammer it with a different tool. You get the point.

Yeah, Hamel, I really appreciate being meme fodder to help you get your point across.

Look, I want to emphasize you should adopt a mindset that evals are a vendor problem. Just realize that there should be a one-size-fits-all solution. Let the vendors figure it out. You're too busy being an executive.

And if you really want to do this properly, you need to create a dashboard that looks like this, with every off-the-shelf metric that you can gather. The more metrics, the better. It doesn't matter if the metrics track with outcomes or real failure modes.

Make sure the numbers are unintelligible so you don't know the difference between a 3.5 and a 4.5. Keep hoarding random metrics until you find one that's going up and to theright. Then you can claim success.

And look, maybe you might have a hard time figuring out where you can come up with these generic metrics, but we got you. Just adopt the ones from eval frameworks. In fact, adopt all of them. Let your eval metrics guide you blindly, and never ask whether they actually measure success.

Again, the more numbers you have, the better.

Yeah, I personally like to optimize for cosine similarity, bleh, and rouge, ignoring actual user experience. And I said it once, and I'm going to say it again: never cross-check with domain experts or your users. Because if an LLM says it's accurate, who are we to argue?

We are their humble servants, after all.

Amen. Now it's time to unveil the most potent technique we have in our toolbox. And it's avoid looking at data. Seriously, just avoid it. Keep a blindfold next to you at all times. If you happen to bump into data by accident, put that blindfold on.

Yeah, data, that sounds really messy. Let's let a tool handle it. Because you can absolutely, 100% trust the AI's output without ever looking at it yourself.

Looking at data? That's an engineering problem. You're a leader. You have more important strategic things to do, like having meetings about meetings.

Besides, developers, they always have more domain expertise than your business teams.

Yeah, and we know that ultimately, by this point, your customers are really your best Q&A. And hopefully, you have lots of them. And they'll complain if something is wrong, maybe, eventually.

But more importantly, you've got to trust your gut. It got you this far in life,right? And feelings are always a reliable substitute for data, especially when you're making million-dollar decisions. If you have trouble trusting your gut, just put the blindfold on, and it'll get youright back in touch with those feelings.

So now we know by now that engineers are all quoting wizards. And they're going to handle everything. It doesn't really matter if they haven't spoken to a customer in years, because you can quickly forget about the fact that there might be simpler options, like using spreadsheets to annotate and look at data.

Say after me: "Remember, this is beyond me."

Great advice. And just, it's not enough for you not to look at the data. You have to make sure no one else is looking at data. And the best way to do that is to put your data in complex systems that only engineers can access, and it's not available to domain experts.

Right. So like, instead of using a simple spreadsheet or perhaps an Airtable like up on the screen, as an executive, you should insist on buying a custom data analysis platform that requires perhaps a team of PhDs to operate and understand.

Remember those bonus points? You get more of them if it takes six months to load this thing and errors incessantly.

So there you have it. The ultimate foolproof guide in under 20 minutes to achieving total AI failure.

If you follow this advice that we've given you here meticulously, it's guaranteed that you're going to waste time, resources, and alienate all the people you work with. And as far as I'm concerned, that's the ultimate success that you can have.

Sure is. So for more advice, it's actually real, do visit ai-execs. We also have an O'Reilly book, the same material, coming out February 27. And so while this talk was inverted, you know, our lived experience really isn't. And we're always very eager to help you on your journey.

So find us after this presentation at the Q&A speaker booth. Thanks so much.