State of AI report - Developers reveal biggest challenges with AI-assisted coding aka vibe-coding

AI-powered developer tools are reshaping how software gets written, but the experience is still far from seamless. According to the newly released “2025 State of Web Dev AI” report, while adoption of tools like ChatGPT, Claude, and GitHub Copilot continues to rise, developers say the tools still struggle with accuracy, context-awareness, and code quality.

The report, compiled by Sacha Greif and the Devographics team, draws on more than 4,000 responses from web developers surveyed earlier this year. It offers one of the most comprehensive snapshots yet of what it’s like to code with AI in 2025. Or as some call it, vibe-coding.

Despite the hype, AI-generated code is still a minority contributor in most developers’ workflows. Sixty-nine percent of respondents said less than 25% of their code is written by AI. Still, 59% agreed that AI tools have become an integral part of their work, with 69% using them at least a few times a week, mostly for generating helper functions, frontend components, tests, or documentation.

But while developers are using AI tools more often, their satisfaction varies widely depending on the product.

Claude, Supermaven, and Cursor lead on developer satisfaction

Among the most positively rated tools were Anthropic’s Claude (65% satisfaction), Cursor IDE (55%), and the emerging coding assistant Supermaven (66%). In contrast, Google Gemini and Meta’s Llama earned less favorable reviews, with only 37% and 39% of users, respectively, reporting positive experiences. JetBrains AI fared worst among coding assistants, with just 28% of users feeling good about it.

Even GitHub Copilot — the most widely adopted tool after ChatGPT — received mixed reviews. One respondent said, “Copilot was constantly forcing its way into my flow, with 95% of the time being wrong suggestions.”

The biggest pain points: Hallucinations, poor code quality, and context loss

The most common complaint across all tools? Hallucinations and inaccuracies.

“Even the best models require a bit of babysitting,” one developer noted. “They can be supremely confident they’re right.”

Seventy-six percent of respondents said they have to rewrite or refactor at least half of the AI-generated code they use — often because it’s unreadable, repetitive, or simply doesn’t work. Several developers expressed frustration that AI-generated code may “work in the moment,” but fail long-term tests of maintainability and quality.

Context limitations were another major issue. Developers reported frequent problems with memory loss in tools like ChatGPT and Warp, leading to incomplete conversations or broken coding sessions. IDEs like Cursor were better, but not immune.

One user summed it up bluntly: “Context loss is the biggest issue… ChatGPT in the browser will just lose entire chunks of conversations.”

Others pointed out the lack of deep integration with core developer workflows: “Renaming a file should update imports. Moving a file should be reflected in the code. Yet coding agents aren’t even aware of such updates.”

The future

The report ends on a cautiously optimistic note: while today’s AI coding assistants may still require hand-holding, developers haven’t given up on the vision of AI as a true pair programmer, just one that listens better, remembers more, and doesn’t hallucinate