Not even close.
With so many wild predictions flying around about the future AI, it’s important to occasionally take a step back and check in on what came true — and what hasn’t come to pass.
Exactly six months ago, Dario Amodei, the CEO of massive AI company Anthropic, claimed that in half a year, AI would be “writing 90 percent of code.” And that was the worst-case scenario; in just three months, he predicted, we could hit a place where “essentially all” code is written by AI.
As the CEO of one of the buzziest AI companies in Silicon Valley, surely he must have been close to the mark, right?
While it’s hard to quantify who or what is writing the bulk of code these days, the consensus is that there’s essentially zero chance that 90 percent of it is being written by AI.
Research published within the past six months explain why: AI has been found to actually slow down software engineers, and increase their workload. Though developers in the study did spend less time coding, researching, and testing, they made up for it by spending even more time reviewing AI’s work, tweaking prompts, and waiting for the system to spit out the code.
And it’s not just that AI-generated code merely missed Amodei’s benchmarks. In some cases, it’s actively causing problems.
Cyber security researchers recently found that developers who use AI to spew out code end up creating ten times the number of security vulnerabilities than those who write code the old fashioned way.
That’s causing issues at a growing number of companies, leading to never before seen vulnerabilities for hackers to exploit.
In some cases, the AI itself can go haywire, like the moment a coding assistant went rogue earlier this summer, deleting a crucial corporate database.
“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”
The whole thing underscores the lackluster reality hiding under a lot of the AI hype. Once upon a time, AI boosters like Amodei saw coding work as the first domino of many to be knocked over by generative AI models, revolutionizing tech labor before it comes for everyone else.
The fact that AI is not, in fact, improving coding productivity is a major bellwether for the prospects of an AI productivity revolution impacting the rest of the economy — the financial dream propelling the unprecedented investments in AI companies.
It’s far from the only harebrained prediction Amodei’s made. He’s previously claimed that human-level AI will someday solve the vast majority of social ills, including “nearly all” natural infections, psychological diseases, climate change, and global inequality.
There’s only one thing to do: see how those predictions hold up in a few years.
He looked at where AI was six months prior and made a wild speculation that, given the data, seemed plausible, if a little outlandish. I’m not mad.
One day that prediction may come true, and there may come another day, later, where we agree that the time between when he said it would happen and the time it actually did happen is not significant enough to mention.
There’s a big difference between writing 100% of 90% of programs, and 90% of code in each program. The other 10% can be difficult, and part of the 10% is fixing that 90% not working.
When the CEO of a tech company says that in x months this and that will happen, you know it’s just musk talk.
more like 6 months" because we need the VC funds still"
Ooh, so that’s CEO speak for: “we’re broke, please give us more money”.
It’s almost like he’s full of shit and he’s nothing but a snake oil salesman, eh.
They’ve been talking about replacing software developers with automated/AI systems for a quarter of a century. Probably longer then that, in fact.
We’re definitely closer to that than ever. But there’s still a huge step between some rando vibe coding a one page web app and developers augmenting their work with AI, and someone building a complex, business rule heavy, heavy load, scalable real world system. The chronic under-appreciation of engineering and design experience continues unabated.
Anthropic, Open AI, etc? They will continue to hype their own products with outrageous claims. Because that’s what gets them more VC money. Grifters gonna grift.
Unfortunately other CEOs are believing it and overhype it, especially if investors are involved
They have to hyperbolize to attract investors. At the rate they burn cash they won’t survive without constant massive financial inputs.
See also: COOL:gen
The whole concept of generating code is basically ancient by now. I heard about this stuff in the 90s, but now I found it that this thing has been around since 1985.
“Come on, I’m a CEO, it’s my job to lie to everyone and hype people up so they throw money at me. It’s really their fault for believing a CEO would be honest.”
That sounds like you can be replaced with a LLM. Let’s do that and save the company millions of dollars a year while us Plebeians got on with doing the real work and making money.
You joke, but it has been successfully argued in court that advertisers can lie to you because no reasonable person would believe that advertisements are truthful.
And these people get paid absurd amounts of money too.
Are we counting the amount of junk code that you have to send back to Claude to rewrite because it’s spent the last month totally lobotomized yet they won’t issue refunds to paying customers?
Because if we are, it has written a lot of code. It’s just awful code that frequently ignores the user’s input and rewrites the same bug over and over and over until you get rate limited or throw more money at Anthropic.
It’s almost as if they shamelessly lie…
The study they’re basing the ‘AI slows down programmers’ on forces software engineers to use AI in their workflow, without any previous experience with that workflow.
It does seem silly, but it’s perfectly aligned with the marketing hype that the AI companies are producing.
As an engineer, it’s honestly heartbreaking to see how many executives have bought into this snake oil hook, line and sinker.
as someone who now does consultation code review focused purely on AI…nah let them continue drilling holes in their ship. I’m booked solid for the next several months now, multiple clients on the go, and i’m making more just being a digital janitor what I was as a regular consultant dev. I charge a premium to just simply point said sinking ship to land.
Make no mistake though this is NOT something I want to keep doing in the next year or two and I honestly hope these places figure it out soon. Some have, some of my clients have realized that saving a few bucks by paying for an anthropic subscription, paying a junior dev to be a prompt monkey, while firing the rest of their dev team really wasn’t worth it in the long run.
the issue now is they’ve shot themselves in the foot. The AI bit back. They need devs, and they can’t find them because putting out any sort of ad for hiring results in hundreds upon hundreds of bullshit AI generated resumes from unqualified people while the REAL devs get lost in the shuffle.
while firing the rest of their dev team
That’s the complete mistake right there. AI can help code, it can’t replace the organizational knowledge your team has developed.
Some shops may think they don’t have/need organizational knowledge, but they all do. That’s one big reason why new hires take so long to start being productive.
Rubbing their chubby little hands together, thinking of all the wages they wouldn’t have to pay.
A tale as old as time…
Did you think executives were smart? What’s really heartbreaking is how many engineers did. I even know some that are pretty good that tell me how much more productive they are and all about their crazy agent setups (from my perspective i don’t see any more productivity)
Honestly, it’s heartbreaking to see so many good engineers fall into the hype and seemingly unable to climb out of the hole. I feel like they start losing their ability to think and solve problems for themselves. Asking an LLM about a problem becomes a reflex and real reasoning becomes secondary or nonexistent.
Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.
Based on my experience, I’m skeptical someone that seemingly delegates their reasoning to an LLM were really good engineers in the first place.
Whenever I’ve tried, it’s been so useless that I can’t really develop a reflex, since it would have to actually help for me to get used to just letting it do it’s thing.
Meanwhile the people who are very bullish who are ostensibly the good engineers that I’ve worked with are the people who became pet engineers of executives and basically have long succeeded by sounding smart to those executives rather than doing anything or even providing concrete technical leadership. They are more like having something akin to Gartner on staff, except without even the data that at least Gartner actually gathers, even as Gartner is a useless entity with respect to actual guidance.
Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.
I’m seeing a lot of this, though. Like, I’m not technically required to use AI, but the VP will send me a message noting that I’ve only used 2k tokens this month and maybe I could get more done if I was using more…?
Yeah, fortunately while our CTO is giddy like a schoolboy about LLMs, he hasn’t actually attempted to force it on anyone, thankfully.
Unfortunately, a number of my peers now seem to have become irreparably LLM-brained.
I mean before we’d just ask google and read stack, blogs, support posts, etc. Now it just finds them for you instantly so you can just click and read them. The human reasoning part is just shifting elsewhere where you solve the problem during debugging before commits.
“Stack overflow engineer” has been a derogatory forever lol
No, good engineers were not constantly googling problems because for most topics, either the answer is trivial enough that experienced engineers could answer them immediately, or complex and specific enough to the company/architecture/task/whatever that Googling it would not be useful. Stack overflow and the like has always only ever really been useful as the occasional memory aid for basic things that you don’t use often enough to remember how to do. Good engineers were, and still are, reasoning through problems, reading documentation, and iteratively piecing together system-level comprehension.
The nature of the situation hasn’t changed at all: problems are still either trivial enough that an LLM is pointless, or complex and specific enough that an LLM will get it wrong. The only difference is that an LLM will spit out plausible-sounding bullshit and convince people it’s valuable when it is, in fact, not.
In the case of a senior engineer then they wouldn’t need to worry about the hallucination rate. The LLM is a lot faster than them and they can do other tasks while it’s being generated and then review the outputs. If it’s trivial you’ve saved time, if not, you can pull up that documentation, and reason and step through the problem with the LLM. If you actually know what you’re talking about you can see when it slips up and correct it.
And that hallucination rate is rapidly dropping. We’ve jumped from about 40% accuracy to 90% over the past ~6mo alone (aider polygot coding benchmark) - at about 1/10th the cost (iirc).
it’s trivial you’ve saved time, if not, you can pull up that documentation, and reason and step through the problem with the LLM
Insane that just writing the code isn’t even an option in your mind
That isn’t the discussion at hand. Insane you don’t realise that.
It is, actually. The entire point of what I was saying is that you have all these engineers now that reflexively jump straight to their LLM for anything and everything. Using their brains to simply write some code themselves doesn’t even occur to them as an something they should do. Much like you do, by the sounds of it.
The good news is that AI is at a stage where it’s more than capable of doing the CEO of Anthropic’s job.
Well it bullshits constantly, so it’s most of the way there.
One issue that remains is that the LLM doesn’t care if it is telling the truth or lying. To be a CEO, it needs to be more inclined to lie.
Seems a better prompt could solve that.
I think Claude would refuse to work with dictators that murder dissidents. As an AI assistant, and all that.
If they have a model without morals then that changes things.
It is writing 90% of code, 90% of code that goes to trash.
Writing 90% of the code, and 90% of the bugs.
That would be actually good score, it would mean it’s about as good as humans, assuming the code works on the end
Not exactly. It would mean it isn’t better than humans, so the only real metric for adopting it or not would be the cost. And considering it would require a human to review the code and fix the bugs anyway, I’m not sure the ROI would be that good in such case. If it was like, twice as good as an average developer, the ROI would be far better.
If, hypothetically, the code had the same efficacy and quality as human code, then it would be much cheaper and faster. Even if it was actually a little bit worse, it still would be amazingly useful.
My dishwasher sometimes doesn’t fully clean everything, it’s not as strong as a guarantee as doing it myself. I still use it because despite the lower quality wash that requires some spot washing, I still come out ahead.
Now this was hypothetical, LLM generated code is damn near useless for my usage, despite assumptions it would do a bit more. But if it did generate code that matched the request with comparable risk of bugs compared to doing it myself, I’d absolutely be using it. I suppose with the caveat that I have to consider the code within my ability to actual diagnose problems too…
One’s dishwasher is not exposed to a harsh environment. A large percentage of code is exposed to an openly hostile environment.
If a dishwasher breaks, it can destroy a floor, a room, maybe the rooms below. If code breaks it can lead to the computer, then network, being compromised. Followed by escalating attacks that can bankrupt a business and lead to financial ruin. (This is possibly extreme, but cyber attacks have destroyed businesses. The downside risks of terrible code can be huge.)
Yes, but just like quality, the people in charge of money aren’t totally on top of security either. They just see superficially convincing tutorial fodder and start declaring they will soon be able to get rid of all those pesky people. Even if you convince them a human does it better, they are inclined to think ‘good enough for the price’.
So you can’t say “it’s no better than human at quality” and expect those people to be discouraged, it has to be pointed out how wildly off base it is.
Human coder here. First problem: define what is “writing code.” Well over 90% of software engineers I have worked with “write their own code” - but that’s typically less (often far less) than 50% of the value they provide to their organization. They also coordinate their interfaces with other software engineers, capture customer requirements in testable form, and above all else: negotiate system architecture with their colleagues to build large working systems.
So, AI has written 90% of the code I have produced in the past month. I tend to throw away more AI code than the code I used to write by hand, mostly because it’s a low-cost thing to do. I wish I had the luxury of time to throw away code like that in the past and start over. What AI hasn’t done is put together working systems of any value - it makes nice little microservices. If you architect your system as a bunch of cooperating microservices, AI can be a strong contributor on your team. If you expect AI to get any kind of “big picture” and implement it down to the source code level - your “big picture” had better be pretty small - nothing I have ever launched as a commercially viable product has been that small.
Writing code / being a software engineer isn’t like being a bricklayer. Yes, AI is laying 90% of our bricks today, but it’s not showing signs of being capable of designing the buildings, or even evaluating structural integrity of something taller than maybe 2 floors.
I’m not sure how people can use AI to code, granted I’m just trying to get back into coding. Most of the times I’ve asked it for code it’s either been confusing or wrong. If I go through the trouble to write out docstrings, and then fix what the AI has written it becomes more doable. But don’t you hate the feeling of not understanding what you’ve written does or more importantly why it’s been done that way?
AI is only useful if you don’t care about what the output is. It’s only good at making content, not art.
deleted by creator
Yeah, I find it can be useful in some stages of writing or researching. But by the time I’ve got a finished product there’s really no AI left in there.
I worked with someone that I later found out used AI to code her stuff. She knew how to code some, but didn’t understand a lot of fundamentals.
Turns out, she would have AI write most of it, tweak it to work with her test cases, and call it good.
Half of my time was spent fixing her code, and when she was fired, our customer complaints went way down.
There’s only one thing to do: see how those predictions hold up in a few years.
Or, you know, do the sensible thing and called the dude the snake oil salesman he is and run him out of town on a rail.
After working on a team that uses LLMs in agentic mode for almost a year, I’d say this is probably accurate.
Most of the work at this point for a big chunk of the team is trying to figure out prompts that will make it do what they want, without producing any user-facing results at all. The rest of us will use it to generate small bits of code, such as one-off scripts to accomplish a specific task - the only area where it’s actually useful.
The shine wears off quickly after the fourth or fifth time it “finishes” a feature by mocking data because so many publicly facing repos it trained on have mock data in them so it thinks that’s useful.
Rule of thumb: only use it for one or two runs and that’s it. after that back off because then Claude Code is then just going to start vomiting fecal matter from the other fecal matter its consumed.
If it can’t nail something on the first or second go, don’t bother. I have clients that have pushed it through those moments and have produced literal garbage. But hey I make money off them so keep pushing man. I got companies/clients that are so desperate to reverse what they’ve done that they’re willing to wait until like March of next year when I’m free.
Sounds like they need to work on their prompts. I vibe code some hobby projects I wouldn’t have done otherwise and it’s never done that. I have it comment each change and review it all in diff checker so that’s 90% of the time.
I guarantee you that it HAS done that and I can almost assure you that whatever hobby project you’ve vibe coded doesn’t scale and I sure as hell hope it’s nothing that needs to be online or handles any sort of user info.
There is something I never understood about people who talk about scaling. Surely the best way to scale something is simply to have multiple instances with so many users on each one. You can then load balance between them. Why people feel the need to make a single instance scale to the moon I have no idea.
It’s like how you don’t need to worry about MS Word scaling because everyone has a copy on their own machine. You could very much do the same thing for cloud services.
You have no idea what you’re talking about lol.
Scale? It’s a personal ancestry site for my surname with graphs and shit mate. Compares naming patterns, locations, dna, clustering, etc between generations and tries to place loose people. Works pretty well, managed to find a bunch of missing connections through it.
It’s not just code, but day to day shit too. Lately corporate communications and even training modules feel heavily AI generated. Things like unnecessary em dashes (I’m talking as much as 4 out of 5 sentences in a single paragraph), repeating statements or bullet points in training modules. We’re being encouraged to use our “private” Copilot to do everyday tasks and everything is copilot enabled.
I don’t mind if people use it, but it’s dangerous and stupid to think that it produces near perfect results every time. It’s been good enough to work as an early rough draft or something similar, but it REQUIRES scrutiny and refinement by hand. It’s like it can get you from nothing to 60-80% there, but never higher. The quality of output can vary significantly from prompt to prompt in my limited experience.
Yeah, I try to use ai a fair bit in my work. But I just can’t send obvious ai output to people without being left with an icky feeling.
