How to Hire AI Engineers when EVERYONE is cheating with AI — Beth Glenfield, DevDay

Hi everyone. So, I'm Beth Glenfield. I actually flew in from Ireland, so I will try and slow down my accent for everybody. I have heard that I do get a bit American sometimes, so, uh, yeah, it may not be the Irish accent you're expecting.

So I'm going to talk to you today about how I believe AI is breaking how we hire, technically. So I know everyone's very busy, but just a few questions to think about. Is that okay?

Yeah.

Perfect. Is are you using AI today in your recruitment process, both as an interviewee and an engineer? Are you competing against Google or Meta employees, both for positions but also for candidates? And are you recognizing that LeetCode puzzles just aren't really up to scratch like they used to be for knowing if people will build great AI products?

So welcome to 2025, where the technical recruitment process is very broken, and small companies are getting crushed in the talent war. So just to refresh our memories, let's look back at what happened over the last 18 months. So we had CLUI come to the table.

They have raised a $5.3 million round, and it is an AI cheating service from an ex-Columbia student. They're currently heading towards $1 million in ARR. And, um, we're also seeing 93% LeetCode wizard success rates for Google, Google and Meta interviews.

And we're seeing, like, 1 in 3 interviews now are having AI assistants. So really, whenever you're interviewing candidates, you're just interviewing for who has the best AI coding assistant. So Sam Altman said this very plainly: the tactical thing to do is to learn the best AI tools.

We also saw that Marc Benioff announced that Salesforce will no longer be hiring software engineers this year because, according to them, they have seen a 30% boost in their productivity since they've replaced people with AI. I would like to see the data on that.

But really, jobs have fundamentally changed. And you're not just competing on compensation anymore. You're competing on brand recognition, career prestige, and perceived stability. There's a lot of people being laid off, and people are looking for security. So when a candidate has to choose between your Series A startup and Google's AI division, we kind of know what really happens there.

So the candidates you actually want for AI development, the creative problem solvers, the collaborative leaders, and the ones who can work with AI instead of being replaced by AI, they are not being optimized for LeetCode performance. Instead, they are building AI tools and using AI libraries.

They're contributing to open source. They are understanding how the business impact will be impacted, and that is just not what LeetCode is indexing for. So what if, instead of asking candidates to solve puzzles that they will never actually use in a, uh, job, you could observe how they collaborate with AI teammates on actual business scenarios?

What if, instead of measuring their ability to memorize algorithms, you can see how they delegate tasks, how they handle ambiguity, and how they deal whenever requirements change mid-sprint? So you're not competing on brand recognition. You can demonstrate to a candidate at the beginning of a process what your engineering culture is and see how they would perform in your environment.

So at DevDay, we are completely reimagining the technical hiring process in the AI era. Here are some examples of what we're seeing today, like I mentioned earlier. And then we'll get into what we're considering is the new kind of process you should be looking for in a hiring process.

What we what we do, instead of getting candidates to code interviews, is we create real-life workplace simulations. Candidates work alongside AI agents with very different personalities. You can have the perfectionist, the pragmatist, the security expert, and even the junior developer that needs extensive mentoring so that they know how they have to make trade-offs on day-in, day-out situations.

They're not solving contrived problems. They are building features specifically for your business domain.

So instead of measuring how they can code, we're actually, actually measuring them on skills that matter. How do they collaborate with AI? How do they handle ambiguity? How do they communicate these technical decisions in pull requests, in comments on tickets?

And how do they mentor others and adapt when everything changes on a daily basis? Now, if you do work at Google or Meta, you can just brute force your hiring process. You interview 100 candidates, you hire 5, and you give them great paychecks.

But for the majority of companies, we don't have the luxury. You cannot afford to make one bad hire, and you can lose up to $20,000 to $60,000 in that bad hiring process. And you definitely can't afford to hire someone who doesn't know how to ship AI products.

So, look, the future is coming for us, whether we like it or not. Uh, Mark Zuckerberg says that AI will handle mid-level engineering work by later this year. TechCrunch is also reporting that we're wiping out engineer-level engineering jobs.

But that doesn't mean fewer engineering jobs. They're just very different engineering jobs. Jobs that require creativity, collaboration, and the ability to work according to business judgment and not just code. Jobs that require working with AI instead of being replaced by it.

So we're currently working with select design partners. If you're interested in thinking about this process, or learning more, or just want to chat about how you're thinking about hiring, let me know. My details are on the screen, or just come find me at the event.

Thank you.