Vibe Coding in 2026: How AI Pair Programming Is Changing Software Development

Last Updated on October 4, 2026

Updated October 2026: rewritten around the research, including METR’s trial of experienced developers, large company field experiments and peer-reviewed security studies.

Vibe coding feels like a superpower at first. You describe what you want, the AI writes the code, and features appear fast.

But does it actually make developers faster, and is the code any good? The research gives a more interesting answer than the hype.

It can speed some work up, and it can slow experienced developers down.

And it often produces code with security holes.

What is vibe coding?

Vibe coding means building software by telling an AI what you want in plain language. You accept the code it writes, often without reading it closely.

The term was coined by Andrej Karpathy, a co-founder of OpenAI, in a post on X in February 2025. He described a new kind of coding where you “forget that the code even exists”.

He added that it was “not too bad for throwaway weekend projects”. That caveat often gets lost.

Merriam-Webster describes it as letting an AI “create the product for you”. It notes that the coder “does not need to understand how or why the code works”.

Collins Dictionary named “vibe coding” its Word of the Year for 2025.

What the research shows

Experienced developers: slower, but felt faster

The most careful test so far comes from METR, a non-profit research group. In early 2025, it ran a randomised trial with 16 experienced open-source developers.

They worked on 246 real tasks in large, mature projects they knew well. Each task was randomly assigned to allow or ban AI tools, mostly the Cursor editor with Claude models.

The gap between belief and reality is the most striking part. Developers felt faster even when the clock said otherwise.

The caveats METR itself gives

METR warns readers against “overgeneralizing” from these results. They studied experts in codebases they knew deeply, which is the hardest place for AI to help.

The authors say their results are consistent with “small greenfield projects or development in unfamiliar codebases seeing substantial speedup”. The study is also a preprint, not yet peer reviewed.

A 2026 update: the picture is shifting

In February 2026, METR reported a larger follow-up with 57 developers and over 800 tasks. Returning developers were an estimated 18% faster, and new recruits 4% faster.

But METR called these numbers an “unreliable signal”. Between 30% and 50% of developers held back tasks because they didn’t want to do them without AI.

METR says developers are probably faster now than in early 2025. It just can’t measure by how much.

Where AI coding clearly helps

Simple, well-defined tasks

A 2023 experiment by Microsoft, GitHub and MIT researchers asked 95 freelance developers to build a small web server in JavaScript.

Those with GitHub Copilot finished 55.8% faster. They took 71 minutes on average, against 161 minutes without it.

That was one small, clearly defined task, built from scratch. The authors work for the companies that make Copilot.

Real jobs, at scale

A 2026 study in Management Science pooled randomised field experiments at Microsoft, Accenture and a Fortune 100 company. It covered 4,867 developers.

Developers given an AI code assistant completed 26% more tasks. Less experienced developers adopted the tool more and gained the most.

That study tested code-completion tools, not today’s more autonomous coding agents. Completed tasks also aren’t the same as time saved per task.

The security problem

Roughly a quarter to 40% of code had flaws

Security is where the research is most consistent. A 2022 peer-reviewed study tested GitHub Copilot on 89 security-sensitive scenarios, producing 1,689 programs.

About 40% were vulnerable. Those scenarios were chosen to be risky, so that’s a worst-case test.

A 2025 study in ACM Transactions on Software Engineering looked at real Copilot code in public GitHub projects. Security weaknesses appeared in 29.5% of Python and 24.2% of JavaScript snippets.

The Veracode figure comes from a company that sells code security scanning. It also reported that security performance hadn’t improved over time.

Overconfidence is the hidden risk

A 2023 Stanford study had 47 people complete security-related coding tasks, some with an AI assistant. Those with AI wrote significantly less secure code.

They were also more likely to believe their code was secure. On one task, only 3% of the AI group wrote a secure solution, against 21% without AI.

People who trusted the AI less, and worked harder on their prompts, produced fewer vulnerabilities.

Code quality and delivery

Google’s 2024 DORA report surveyed software teams. A 25% increase in AI adoption was linked to 3.4% better code quality and 3.1% faster code reviews.

But it was also linked to a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability. Google’s summary: AI “does not appear to be a panacea”.

These are survey associations, not experiments. Still, they fit the pattern: faster writing, but more to check.

What earlier research found

The original Copilot experiment

The 2023 Copilot experiment remains the most cited evidence that AI speeds coding up. Its 55.8% figure comes from one small task.

Security in the lab

The 2022 “Asleep at the Keyboard” study was among the first to measure how often AI-suggested code was vulnerable.

Overconfident users

The 2023 Stanford study showed that AI help can make people both less secure and more confident.

How it fits together

AI coding helps most on small, well-defined or unfamiliar tasks. It helps least, and may even slow things down, for experts in complex code they already know.

How much should you trust this?

Promising. The broad pattern is consistent, but the tools change faster than the studies.

What makes it convincing

• Several randomised trials, including large field experiments, have tested AI coding tools.
• Multiple peer-reviewed studies agree that AI code often has security flaws.
• The researchers behind the main trial are candid about its limits.

What makes me cautious

• The METR trial is a preprint with only 16 developers.
• Most studies tested older tools, not today’s coding agents.
• Some studies were run by companies that make or sell related products.
• Speed and quality are measured in very different ways across studies.

What this means for you

• Use vibe coding for prototypes. Karpathy himself framed it as fine for “throwaway weekend projects”.
• Review before you ship. Treat AI code as a draft that needs checking, especially for security.
• Run security checks. OWASP’s guidance on AI risks warns that models can suggest insecure or non-existent code libraries.
• Don’t trust the feeling of speed. Measure real outcomes, like time to working, tested code.
• Be extra careful with sensitive code. Login, payments and data handling deserve human review.
• Expect change. Tools are improving fast, so today’s results may not hold next year.

The OWASP Top 10 for LLM Applications lists the main security risks of AI-generated output.

In this talk, hosted by the Stanford Digital Economy Lab, METR’s Joel Becker explains why impressive AI benchmarks haven’t always meant faster developers:

What we still don’t know

1. How fast are today’s coding agents? Most trials tested 2022 to 2025 tools.
2. Does vibe coding create long-term maintenance problems? Good long-term studies are missing.
3. Is AI code getting more secure? One industry report says no, but independent data is limited.
4. Who benefits most? Beginners seem to gain more than experts, but results vary.
5. How do we measure productivity fairly? Studies disagree on the right yardstick.

My take: a great sketchpad, a risky shortcut

What I like about this research is that it punctures the hype in both directions. AI coding is neither magic nor useless.

I’m less convinced by teams that ship vibe-coded features without review. The security studies are too consistent to ignore.

Use it to explore and prototype fast. Then slow down, read the code and test it before anyone depends on it.