Combine Skills and MCP to Close the Context Gap — Pedro Rodrigues, Supabase

Allright, I get— I have the green light, so we can get it started. Uh, hello everyone. You might have noticed that the titles changed a bit from the one that we have in on the schedule. That's because when I submitted the talk, the MCP versus skill debate was still going on, was a hot topic.

I think now we settled on they're both different, they all have their own roles, and now the— I think the debate is more on the MCP versus CLI. So I thought it would be more useful to come and explain how we've wrote our Supabase skill, and the lessons that we got from writing this document.

Because I've never spent more time writing a single document since I've wrote my master's thesis, okay? So I know that writing a skill sounds simple, but it can be very complex, especially when you have a complex product like Supabase.

For starters, I'm Pedro, I'm an AI tooling engineer at Supabase. I'm an MCP enthusiast, AI in general. Feel free to connect me on LinkedIn. And I'm also a co-founder of the Lisbon AI Week in Lisbon, we'll be on late October this year.

And if we're talking about me at the moment, I usually prefer doing this on a dark, darker mode. I don't know. How many of you prefer dark mode over light mode? The majority. I thought so. So let's do this presentation on dark mode instead.

So we— I think we can all agree that agents are already smart enough,right? They can do very cap— they're very capable of doing mundane tasks by themselves. But when you present a task about something that they've haven't seen yet, or you've updated since they were trained, like your product for example, they need theright guidance.

For example, at Supabase we noticed that they would usually either miss some security pitfalls that we have, like row-level security instructions that they have to set to basically not expose your app. They could just usually operate on stale knowledge, on their training data, and they're very lazy to— and very stubborn to admit that they don't know, and they do need to find fresh information.

And also, we would like to guide them on specific workflows that we think are the most optimized for agents on our product. So for starters, how many of you know or have written a skill before? Okay, so what I'm going to say is probably not new for most of you, but just to get an introduction on skills.

So skills are folders containing instructions, scripts, and resources that agents discover,right? Progressively they discover. This is the main selling point of skills. And they have this envelope called front matter where they have the name and the description. This is how the agent is going to decide when to load the skill.

Then they have the actual instructions inside the file, the main file called skill.md. And then optional bundled resources like scripts to perform actions, or reference files that the main file can reference for more information that doesn't have to be loaded immediately to context.

So we tested out at Supabase, we experienced giving the same agent, in this case was Claude Sonnet 4.6, the same prompt for a simple task. Like we had an app, like a collaboration, a collaborative app, and we wanted to create this new SQL view,right, on top of a table that already had row-level security enabled.

So the users could only see the information that belonged to them. We gave them just the MCP on one condition, and the MCP plus the agents, the agent skills. And the result was, well, they expected. If you don't know in Postgres, if you create a skill over a table, on top of a table that has RLS enabled, if you don't explicitly pass that flag over there, the security invoker equals true, it will bypass the RLS.

So basically the view will expose data that is not exposed by default on the table. So the agent with the skill, with the knowledge, was able to get this information implemented correctly and safely, while the one that only had access to the integration, to the MCP tool, did not.

So for this we decided, well, just like this, we wanted to enable agents to know how to work correctly with Supabase. So we decided to, we're announcing, actually announcing today, this Supabase agent skill that I've been working on, on this couple, couple past months.

And to make things official, I'm actually going to try something. Wait. I'm going to

live tweet it on stage. So it's live.

Allright, so what exactly is this skill about, and what lessons can I share with you? So if you're building a skill for your product, you can build one like we did, and happy to discuss the all details. Like this is free text, and we haven't achieved a standard for it, so happy to chat about it later.

So to break down on some principles that we converged to, the first one is that don't duplicate information. Yeah, like treat skills as a document, documentation for yourself. And you will not duplicate your documentation,right? You already have documentation on your product, just point the agent to it.

It's the most up to date. You'll have to be very stubborn with the model to go to ask them to go search the web or your documentation, provide the guidance, tell where to and how to find the documentation, but be very persistent with it to go for it.

We also did run a little experiment, and this is still, well, as I said, an experiment and yet to be

a bit, we're still figuring out the details. But we also announced quite recently, and you can see it, you can read it on our blog, we're expo we're basically exposing our documentation through SSH. The main reason behind it is that the agents can now look for the documentation like it was a file system.

So they're very familiar with file systems in general, are navigating them, finding files and information on using Linux-based tools. If we expose, if we give them this interface to also do this, but remotely, our premise is that they will have easy, ease to navigate the agent.

So we'd also would love after the talk or during the conference to see and to hear your opinions on this idea. The second principle that I got, that I have for you is that if something can get skipped, it will be skipped.

What I mean by this is that besides new information on searching online, so agents, like fetching information online or tool calling, it's expensive for agents, so they mostly default to their training data. The same is true for reference files.

We've noticed that even if, when the agent loaded the skill, it will, even if it had reference files there, it will be very lazy to load them. And even if it loaded one reference file, if your problem requires more than one, the information that it's more than in more than one file, it will most likely will not load two files,right?

It's almost impossible. And not even starting on three or four. So you have to be very critical about what you put on your skill.md file from the beginning. You put information that it's likely not to change. Like in our case, a security checklist about Supabase that we didn't really want the agent to miss at all.

So we decided this cannot be on a reference file. We actually started by putting it on a reference file, and he usually missed it. So we put it on the skill itself. So if you have any information that the agent can just not miss, and like defines your product, goes to the skill.md file.

Do not afford it to put on a reference file. And lastly, the third principle that I have for you when writing a product skill is to be opinionated. You know your product the best. You know how to work with it.

You know how your users, or you should know how your users are using it. Don't be afraid of guiding the agents on what, on workflows that you think are the most effective when working with your product. In our case, for example, managing a database schema,right?

I haven't, I actually haven't asked this and should have in the beginning. How many of you know Supabase and what Supabase does? Okay, for, so for the ones who don't know, we're basically backend service. We provide a storage, database, authentication, and other features that you would need to create a backend out of the box.

So we provide, as I said, we provide a database, and the agent can interact and manipulate your schema,right? We found that this, for our platform, was the best workflow for the agents to efficiently manage the schema. So in this case, it run direct DDL operations, like change the schema freely on your development or staging database.

Once you're happy about it, we provide an advisor to our basically linked to give any security or performance issues that the database could have, fix them, and only then create the migration file. So this will prevent the agent to create a migration file every time he changes the schema.

We found this to be the best workflow to manage the schema. So for us, it should be in the skill when working with Supabase. How we tested this skill. So we're living a very interesting times where we now can test free tests.

We can now test documents. We can now test documentation,right? This would be completely bonkers to think years ago. Now I've basically been testing a Markdown file. And how did we do this? Through evals. So for those of you who don't know evals, evaluations, for short for evaluations, are basically tests that you can run, mostly like you would run on your CI.

But now instead of evaluating code, you're evaluating an agent, an LLM, and its behavior. What tools it's calling, what's the, its reasoning,right? And so we ran on an early stage, we ran a set of six specific scenarios for Supabase.

So ongoing Supabase projects in different situ in different scenarios,right? And we ran it against four different agents from two vendors in three test conditions. So we wanted to test on a baseline, so no MCP, no skills, with just the MCP server, and with the MCP server plus the skill.

All this was done based on a task completeness score, four graded score, the run on BrainTrust. If you don't know them, they're around, they're sponsors of this conference, go to their booth and talk about it. It's a very cool product.

So the results are out here. The skills plus the MCP outperform any other conditions on every model that we tested. So we tested on Claude code for Opus 4.6 and Sonnet 4.6, and also on Codex for GPT 5.4 and GPT 5.4 mini.

So I think we can conclude that it's pretty unanimous that the skills were actually improving the performance, the task completeness score, because they were providing theright guidance to the agents. So we had already the tools, we already had an MCP server, we just needed theright guidance on how to operate with Supabase.

It's agent agnostic. Most, more and more agents are adopting this open standard on skills. And currently, as I said in the beginning, the bottleneck is not the context, it's the guidance. So if you can take something from this talk when building a skill for your product, is that point for your single source of truth.

Point to your documentation, basically. Be opinionated. You know your product, don't be afraid to show it. And start minimal. Any model vendor, whether that's Anthropic, OpenAI, on any blog post about skills, you will read,

start minimal, start slow, and then iterate, expand, don't be afraid to create new versions for it.

So if you want to know more about it, we actually wrote a blog post. It's live today. You can check it on my Twitter account on Supabase blog. Or you can just run this command and install your, install it on your project and start to use them now.

That's all. I'll be around. And once again, thank you very much.

I do have one more thing to show you. So we're running a giveaway. If you want to have a chance to win a Mac mini, just scan the QR code, sign up for Supabase, and good luck.

Do I have time for any questions, if there's any?

Yeah.

Yeah? Okay.

I have one there. So since we're all moving into RAGsright now, I'm just wondering how much of a demand do you guys see on vectorized databases of Supabaseright now?

It's an interesting question. So

I mean, it depends. You're asking about like how many customers are.

How much are you having a demand for vectorized databases?

There have been more and more as the days pass. Like the use cases for vectors are mainly for embeddings,right? Yeah. And semantic, there are many use cases for embeddings. The one that I'm most interested about is semantic search,right?

That you could use to provide even more context, for example, through the SSH, exposing the docs through the SSH. You can now, instead of naively let them navigate with the Bash tools, you can augment the tools, the already known Bash tools, into providing some sort of semantic search.

So I do see a very big potential on vectors. On our data, definitely customers have been exploring more this solution. Thank you for the question.

Thanks.

Yeah.

Thank you for the talk. Very interesting. How do you distribute the skills inside your organization? Do you just pass them? You have a repository, GitHub repository, or do you use some package manager?

That's an amazing, that's an amazing question, because currently one of the downsides or the, I would say the constraints of using skills is the distribution system,right? We're still finding that there's still some players trying to reclaim either the registry or the way to distribute it.

So Vercel came up with this skills package. We're seeing plugins,right, that you can bundle with MCP servers and other things. But they are model specific. So this is to distribute in general, skills in general. It's already a problem for itself that we haven't solved it yet.

Internally, we have, we are packaging the skills on the repos themselves. So if you want to create a plugin, you just create a .claude plugin or a .cursor plugin or whatever in that repo,right? And then it's available. Or it's discoverable if the repo it's open sourced or you have access to it, you can use the skills package to fetch it.

So yeah, this, and this is how I've seen the other companies distributing their skills. Like a repo, a skill, trying to package the skills into the knowledge.

Thank you.

Thank you. Are there any more? I think we have time for one more. Yeah. Yes.

I recently built self-improving skills. Should we do a call up?

Sure. Let's talk after. Allright. So once again, thank you very much. It was a pleasure to be here. I'll be around. Thank you.