Frontier Feud: Anthropic, Google DeepMind, Meta FAIR, Thinking Machines — Barr Yaron, Amplify

Welcome to Frontier Feud. So, I'm your host, Barr Yaron. I'm a partner at Amplify. We're the first investors in technical founders, and I invest in data and AI companies. I'm very excited to be here, Frontier Feuding, in my favorite city, New York City.

And today on stage we have some amazing folks competing for prizes and eternal glory and all that good stuff. So, we're going to introduce our teams. Do not let these smiling faces deceive you; we are here to compete.

Before we do intros for the teams, just an audience feeler: who here in the audience, including those on stage, have watched Family Feud before? Quick show of hands. Okay, most people. For those of you who haven't, the premise is pretty simple.

The nuance is you'll learn with the game. So, we surveyed 100 AI engineers on a series of questions. Folks on stage are going to guess the answers to those questions, and the most popular answers from the survey are going to get the most points.

So, feel free to follow along and guess yourself in the audience as we go along. If you don't like the answers, you can look to your left and look to yourright and blame your neighbors. No, I'm just kidding.

In all seriousness, thank you so much to everyone who filled out that survey before. And so, we're going to do some quick intros. We have on my left, Mahir, you can kick it off. Tell us who you are, what you do, and your tech hot take.

We're going to do quick intros.

Cool. Hey everyone, my name is Mahir. I work at Anthropic. My tech hot take is I think, let's say, of the five-ish people training big models today, at least one of them will no longer be training AI models by the end of the year.

Mic drop, but really, mic pass. So, John.

Hi guys, I'm John. I work at Anthropic, and my tech hot take is I think AIs make really good therapists.

Hi, I'm Tina. I work at Reflection AI. It's a startup building coding agents. And my tech hot take is that everybody working in our industry should spend like 20 minutes a day with just a piece of paper and a pencil and think like non-AI interrupted thoughts.

Thank you.

Hi, I'm Shrishta. I'm the product lead for the Gemini developer APIs working with Paige. I have two. My first tech hot take is a follow-up from Tina's. I think it'll be important for people to do what Tina said, because eventually when you have AI models representing you, they'll need some good content to be trained on.

I think I'll save my next one for later.

I love it.

Excellent.

Paige, you want to start your side?

Heck yeah. Greetings, everyone. We on the stage did not get to vote in the survey, so just be aware that we did not pre-game it.

No fraud.

No fraud. My name is Paige. I work at Google DeepMind, leading engineering for our AI DevRel team. And I guess my tech hot take is that speed forward ahead a year and a half, I think that the majority of deployed models will be on-device and we'll be seeing a trend more towards smaller models that are kind of orchestrated together, maybe hyper-specialized for a specific task, as opposed to relying just on larger models and sending data elsewhere to do interesting things.

My name's Colin. You might remember me from my talk earlier today. I'm a researcher at Augment Code, building AI coding tools. Before that, I was at FAIR, Facebook AI Research here in New York, working on AI for board games.

I just had to get out both hot takes because I had two ones. So, one is that in the future, dating apps will be our AIs dating each other. Two is I don't think transformers are the final architecture, because eventually we'll build these models using biological materials.

Oh, snap.

Hi, my name is Petra. I'm the product manager of factuality for the AI answer at the top of the Google search page. My hot take is that chatting with one bot one-on-one is boring, and in the future, most conversations will have at least two other bots in the room with you.

I love it.

Hello, I'm Steven. I'm joining Thinking Machines soon. And my hot take is that if you think we're hitting a token wall, that sounds like a skill issue to me.

I love it. Amazing. So, with that, let's get started. I'm going to have Paige and Mahir join me. Not that you're so far from me.

Excellent.

Great.

So, the objective is like hit this as fast as possible,right?

Yes.

Okay.

So,

whoever buzzes first has the first chance to answer this question. So, I'm going to ask a question and we'll take it from here. So, we asked 100 AI engineers, name the most influential AI researcher.

God dang it. I looked at the screen.

Okay, what's your guess?

Ilya Sutskeva.

Okay, we have Ilya.

Noam Shazir.

Noam Shazir. Very influential, but not on the board, not in our topic answers. So, your team decides if to play or pass.

Allright, we're going to play.

You're going to play. You're going to play. Okay, awesome. So, John, it's your turn to guess someone.

I'll guess another person.

But you could stay there. We're working out the kinks.

Cheating. Fraud.

Andrej Karpotkin.

Answer. Good answer. Allright, Tina, how about you?

Jeff Hinton.

Oh, an excellent one, but it's not a success.

Oh, that's so Jeff. How?

Oh, sorry. Did you say Jeff Hinton?

Yeah.

Oh.

That's on me. I thought you said something else. Great.

The lead author on the transformer paper. The attention all you need paper.

We need a specific name.

Sorry.

Sorry, I'm having a literal, like I know the name, but I'm having a literal like.

We're going to have. Yeah, I think. But wait, you already had a strike that doesn't count, so we're going to keep it. We're netting at one. Go, Mahir.

I'm going to say Yann LeCun.

Yann LeCun. We do have Yann LeCun.

Okay. Why do we keep going?

Because you're winning.

Oh, until we lose?

Yeah, until we lose.

Oh, I didn't.

If you want to lose, here's your opportunity.

Steve Sviswani.

No. So, not on the board.

Okay.

What was the question again?

Good question. So, name the most influential AI researcher.

I'm going to say Sam Altman.

Sam Altman. It's a good one. It's not on the board.

Oh, that's ironic.

And so, we are at three strikes. So, your team needs to deliberate. And if you pick an answer, you actually steal the points.

We have four options on the board.

And for those who are joining, the question is, we asked 100 AI engineers, name the most influential AI researcher.

Allright, I'm giving you a five, four. You're good? Allright.

I'll do Jeff Dean.

Jeff Dean is not on the board, which is

what I had registered earlier. Okay, so that goes to Rocco's Basilisk. And we're going to go.

Wait, what were the rest of them?

Yeah, I'll show you the rest of them.

So, these were the ones that were guessed. We also have Andrew Ng. We have Yuan Shenibuo. We have Fei-Fei Li. And we have Yoshua Benjio.

There is a long tail of very influential researchers too, but they're only eight spots, so all great guesses. Okay, next. Allright, I think John and Colin.

Wait, so what is it?

Yeah. You want to know the question?

Yeah.

Patience. So, name the top considerations when choosing a model.

Oh, I think there's a throat thing. I think he's half-versed.

More passion.

Come aggressively, this team. But we'll, yes, name the top considerations when choosing a model.

Intelligence.

I'm going to be very literal.

Very smart answer.

Very smart answer.

You have given an opportunity to guess.

Safety.

Safety is actually not on here.

And so, why don't we get Tina and Petra to guess.

Am I next?

Yeah.

Oh, price.

So, cost is the number one answer, which actually surprised me. I don't know if, like, maybe raise of hands if you would have guessed price. Now you know, so hindsight's 20/20. That wouldn't have been obvious to me. But this means that you guys will continue with this answer and we'll continue from there.

Latency.

Latency.

Yes.

Evals slash benchmark scores.

Some of these things are ambiguous, but I'm going to put it under don't kill me accuracy performance.

That's like everything.

The CEO.

The CEO. Good answer. Good answer. Good answer. Moving on.

Oh, I thought it was because we missed, oh no, we skipped that.

No, your team got it.

Like where the model's being served, like if it's on-prem or, you know.

No, we're not going to count that one. But it's a good answer. Okay, this team has an opportunity to steal these points.

I hear some folks in the audience think they're ready to steal. So, I love that about you. I love the confidence.

Oh my god.

We can make a guess. Oh, that was me. No, it wasn't.

Open source versus closed source. Whoa. Okay, so with that, you guys get 63 points because you stole the points by getting them.

Oh.

So, it's a pretty close game going into the third and final question.

Yeah. Allright.

Sorry. I know everyone wants the answers.

Who has like very strong upper body strength?

So, here are the answers.

Oh, capabilities.

Yeah, these are the answers.

Great. Thanks for keeping me honest, everyone in the audience and everyone here. So, allright, next. So, who do we have coming up? Petra and Tina. Incredible.

You can do it.

So, both in the front. Okay, this requires power, is what we've learned. So, are you ready?

Ready.

Okay. We asked 100 AI engineers, name a buzzword.

It's okay. I'm not offended. But name a buzzword everyone in AI is tired of hearing.

Agents.

Agents. What do you all think? Yes?

Okay, but you still have an opportunity to guess because you could get the number one answer.

Oh, okay.

But you have to guess now.

I'm giving you three, two, one. Sorry.

Okay, so it's going to go to this team. And, Chrysloi, it goes to you. We asked 100 AI engineers, name a buzzword everyone in AI is tired of hearing.

I'm going to start my five, four.

Multimodality.

Multimodality? We do not have multimodality.

But I don't know where this strike came from, so it's the first strike.

I'm going to go with Copilot.

Copilot. What do we think? Yes or no to Copilot?

That's a very good guess.

I think it's a good, okay. Good answer. Good answer. But no.

AGI.

AGI. Okay, what does the audience think? AGI?

Yes.

Allright, it seems like we think it's a good answer. And no, just kidding. Yes. The number one answer, AGI. Allright, Tina.

Oh,

DeepSeek.

No

DeepSeek.

So, this team has an opportunity to steal.

But you'reright. Maybe for like the week.

Yeah, you're back. You're so back.

I'm going to give connection my last one. Safety.

Safety. So, unfortunately, safety is not one of the answers.

So, these were the additional answers. People here are sick of rag and prompt engineering. But what this means is that we have a winner, which is Rocco's Basilisk.

Amazing work by paid attention and the mixture of experts. And so, we're going to move to the fast money round. Do you guys have two representatives?

Allright, you two are going. And that means, John, leave the stage. I'll call you when you're ready. Allright, we're moving into fast money. And so, some of you have seen Family Feud, but we're playing fast money. The goal is for the two of them together to get to 200 points.

You have Mahir 20 seconds to answer five questions. If you can't think of anything, just say pass, and we can come back to it at the end or never. And if you hear a buzzer sound, let me check that it works.

Beautiful. Your answer wasn't one of the survey questions, so you can keep asking or move on. Allright, are you ready?

Yeah.

Okay. Name an AI tool that engineers love.

Cursor.

Name the, sorry, I didn't start the clock.

Name the job most at risk of AI disruption.

Software engineers.

Name the most influential AI paper in history.

Attention is all you need.

Name the biggest nightmare for AI engineer at 2 a.m.

Hardware failure.

Okay. So, we got through four, but they were good ones. So, name an AI tool that engineers love. You said cursor. Cursor is the number one answer.

We asked, name the job most at risk of AI disruption. You said software engineering. That is the number three answer. We asked, name the most influential AI paper in history. What do you think? Demolished. Attention is all you need is by far the number one answer.

Name the biggest nightmare for AI eng at 2 a.m.

Hardware failure.

Hardware failure. You know what? We'll give it. We'll give it as an infra problem.

Okay, okay.

And so, you're going into the round, 140 points. You want to tap John in?

John ran away. He got scared. No. There we go. Amazing. Incredible. So, John.

Let's go. Big things.

You have 25 seconds, and they got to stand back there. You have 25 seconds to answer five questions. And if you can't think of anything, just say pass, and we'll come back to it if we have time at the end.

If you hear a buzzer sound, it means that either your answer isn't on the board or Mahir has already answered it. You ready?

Yeah.

Okay. And I'm going to start your timer.

Name an AI tool engineers love.

Cursor.

Oh, ChatGPT.

Name the job most at risk of disruption.

Artists.

Name the most influential AI paper in history.

Transformers is all, attention is all you need.

Okay. So, we have model APIs, which is

the third answer on the board. We have name the job, you said artist. So, I'm going to give you content creation slash writing. It's a little ambiguous. And attention is all you need was already selected by Mahir. But I'll let you pick the next question, which is name the biggest nightmare for AI engineer at 2 a.m.

Someone wrote in cold email from a VC, so I will talk to them after.

Kuda airs.

Kuda airs. Okay, well, you know what? Your teammates, because you think alike and you think of great answers. So, let me just show you the number one answers here, which are cursor, data entry for the job most at risk of AI disruption.

Attention is all you need, the top paper. So, that was a good one, just already taken. The biggest nightmare, an outage of the model or otherwise, and an industry that would benefit the most from AI, healthcare. So, with that, we have, well, you lost the 200 points.

But I think that you're still winners, and you're all still winners. And we have a few prizes outside that I think someone is bringing. So, we have a massive llama for this team. We have rainbow lambabini babies for everyone.

We have AI engineering books and gift cards to your favorite restaurants in New York. And we can bring them out and leave. Thank you so much for joining us for Frontier Feud. And we're going to see you next year.

We're running a massive survey on the state of AI engineering. Going to be presenting it at the June AI Engineer World Fair. So, if you liked some of the teaser questions, this will get much more in-depth into the tools folks are using, the workflows that AI engineers have, and it's a way for the industry to be more transparent.

So, thank you. You can find the QR code or the link here if you want to participate in that survey on the state of AI engineering. Amazing.