Not even close.
With so many wild predictions flying around about the future AI, it’s important to occasionally take a step back and check in on what came true — and what hasn’t come to pass.
Exactly six months ago, Dario Amodei, the CEO of massive AI company Anthropic, claimed that in half a year, AI would be “writing 90 percent of code.” And that was the worst-case scenario; in just three months, he predicted, we could hit a place where “essentially all” code is written by AI.
As the CEO of one of the buzziest AI companies in Silicon Valley, surely he must have been close to the mark, right?
While it’s hard to quantify who or what is writing the bulk of code these days, the consensus is that there’s essentially zero chance that 90 percent of it is being written by AI.
Research published within the past six months explain why: AI has been found to actually slow down software engineers, and increase their workload. Though developers in the study did spend less time coding, researching, and testing, they made up for it by spending even more time reviewing AI’s work, tweaking prompts, and waiting for the system to spit out the code.
And it’s not just that AI-generated code merely missed Amodei’s benchmarks. In some cases, it’s actively causing problems.
Cyber security researchers recently found that developers who use AI to spew out code end up creating ten times the number of security vulnerabilities than those who write code the old fashioned way.
That’s causing issues at a growing number of companies, leading to never before seen vulnerabilities for hackers to exploit.
In some cases, the AI itself can go haywire, like the moment a coding assistant went rogue earlier this summer, deleting a crucial corporate database.
“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”
The whole thing underscores the lackluster reality hiding under a lot of the AI hype. Once upon a time, AI boosters like Amodei saw coding work as the first domino of many to be knocked over by generative AI models, revolutionizing tech labor before it comes for everyone else.
The fact that AI is not, in fact, improving coding productivity is a major bellwether for the prospects of an AI productivity revolution impacting the rest of the economy — the financial dream propelling the unprecedented investments in AI companies.
It’s far from the only harebrained prediction Amodei’s made. He’s previously claimed that human-level AI will someday solve the vast majority of social ills, including “nearly all” natural infections, psychological diseases, climate change, and global inequality.
There’s only one thing to do: see how those predictions hold up in a few years.
Well, 90% of code of which only 3% works. That sounds sbout right.
I can say 90% of PRs in my company clearly look or declared to be AI generated because of how random things that still slip by in the commits, so maybe he’s not wrong. In fact people are looked down upon if they aren’t using AI and are celebrated for figuring out how to effectively make AI do the job right. But I can’t say if that’s the case for other companies.
Are we counting the amount of junk code that you have to send back to Claude to rewrite because it’s spent the last month totally lobotomized yet they won’t issue refunds to paying customers?
Because if we are, it has written a lot of code. It’s just awful code that frequently ignores the user’s input and rewrites the same bug over and over and over until you get rate limited or throw more money at Anthropic.
The study they’re basing the ‘AI slows down programmers’ on forces software engineers to use AI in their workflow, without any previous experience with that workflow.
It does seem silly, but it’s perfectly aligned with the marketing hype that the AI companies are producing.
I’m not sure how people can use AI to code, granted I’m just trying to get back into coding. Most of the times I’ve asked it for code it’s either been confusing or wrong. If I go through the trouble to write out docstrings, and then fix what the AI has written it becomes more doable. But don’t you hate the feeling of not understanding what you’ve written does or more importantly why it’s been done that way?
AI is only useful if you don’t care about what the output is. It’s only good at making content, not art.
I worked with someone that I later found out used AI to code her stuff. She knew how to code some, but didn’t understand a lot of fundamentals.
Turns out, she would have AI write most of it, tweak it to work with her test cases, and call it good.
Half of my time was spent fixing her code, and when she was fired, our customer complaints went way down.
I’m a video producer who occasionally needs to code. I find it much more useful to write the code myself, then have AI identify where things might be going wrong. I’ve developed a decent intuition for when it will be helpful and when it will just run in circles. It has definitely helped me out of some jams. Generative images/video are in much the same boat. I almost never use a fully AI shot/image in professional work. But generative fill and generative extend are extremely useful.
Yeah, I find it can be useful in some stages of writing or researching. But by the time I’ve got a finished product there’s really no AI left in there.
@Angry_Autist@lemmy.autism.place I feel obliged to tag you here
“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”
You can’t tell me these things don’t have a sense of humor. This is beautiful.
It’s not just code, but day to day shit too. Lately corporate communications and even training modules feel heavily AI generated. Things like unnecessary em dashes (I’m talking as much as 4 out of 5 sentences in a single paragraph), repeating statements or bullet points in training modules. We’re being encouraged to use our “private” Copilot to do everyday tasks and everything is copilot enabled.
I don’t mind if people use it, but it’s dangerous and stupid to think that it produces near perfect results every time. It’s been good enough to work as an early rough draft or something similar, but it REQUIRES scrutiny and refinement by hand. It’s like it can get you from nothing to 60-80% there, but never higher. The quality of output can vary significantly from prompt to prompt in my limited experience.
Yeah, I try to use ai a fair bit in my work. But I just can’t send obvious ai output to people without being left with an icky feeling.
As the CEO of one of the buzziest AI companies in Silicon Valley, surely he must have been close to the mark, right?
You must be delusional to believe this
The conflict of interest here is pretty obvious, and if anybody was suckered into believing this guy’s prognostications on his company’s products perhaps they should work on being less credulous.
Definitely depends on the person. There are definitely people who are getting 90% of their coding done with AI. I’m one of them. I have over a decade of experience and I consider coding to be the easiest but most laborious part of my job so it’s a welcome change.
One thing that’s really changed the game recently is RAG and tools with very good access to our company’s data. Good context makes a huge difference in the quality of the output. For my latest project, I’ve been using 3 internal tools. An LLM browser plugin which has access to our internal data and let’s you pin pages (and docs) you’re reading for extra focus. A coding assistant, which also has access to internal data and repos but is trained for coding. Unfortunately, it’s not integrated into our IDE. The IDE agent has RAG where you can pin specific files but without broader access to our internal data, its output is a lot poorer.
So my workflow is something like this: My company is already pretty diligent about documenting things so the first step is to write design documentation. The LLM plugin helps with research of some high level questions and helps delve into some of the details. Once that’s all reviewed and approved by everyone involved, we move into task breakdown and implementation.
First, I ask the LLM plugin to write a guide for how to implement a task, given the design documentation. I’m not interested in code, just a translation of design ideas and requirements into actionable steps (even if you don’t have the same setup as me, give this a try. Asking an LLM to reason its way through a guide helps it handle a lot more complicated tasks). Then, I pass that to the coding assistant for code creation, including any relevant files as context. That code gets copied to the IDE. The whole process takes a couple minutes at most and that gets you like 90% there.
Next is to get things compiling. This is either manual or in iteration with the coding assistant. Then before I worry about correctness, I focus on the tests. Get a good test suite up and it’ll catch any problems and let you reflector without causing regressions. Again, this may be partially manual and partially iteration with LLMs. Once the tests look good, then it’s time to get them passing. And this is the point where I start really reading through the code and getting things from 90% to 100%.
All in all, I’m still applying a lot of professional judgement throughout the whole process. But I get to focus on the parts where that judgement is actually needed and not the more mundane and toilsome parts of coding.
But I get to focus on the parts where that judgement is actually needed and not the more mundane and toilsome parts of coding.
The parts you’re doing yourself are writing tests and fixing vibe-coded bugs. And you’re outsourcing all the creative, design-based aspects of programming. I think you and I have very different definitions of “mundane” and “toilsome”.
What? I’ve already written the design documentation and done all the creative and architectural parts that I consider most rewarding. All that’s left for coding is answering questions like “what exactly does the API I need to use look like?” and writing a bunch of error handling if statements. That’s toil.
No, your LLM writes your design documentation and tells you how your application is supposed to work, according to what you wrote.
Also, you’re either writing dead-simple applications, or you’re being incredibly hyperbolic if those are the only questions left left after your design document is written.
O it’s writing 100% of the code for our management level people who are excited about “”““AI””“”
But then us plebes are rewriting 95% of it so that it will actually work (decently well).
The other day somebody asked me for help on a repo that a higher up had shit coded because they couldn’t figure out why it “worked” but also logged a lot of critical errors. … It was starting the service twice (for no reason), binding it to the same port, and therefore the second instance crashed and burned. That’s something a novice would probably know not to do. But, if not, immediately see the problem, research, understand, fix, instead of “Icoughbuiltcoughthis thing, good luck fuckers”
Ai writes 90% of my code…i don’t code much.
these tech bros just make up random shit to say to make a profit
These hyperbolic statements are creating so much pain at my workplace. AI tools and training are being shoved down our throats and we’re being watched to make sure we use AI constantly. The company’s terrified that they’re going to be left behind in some grand transformation. It’s excruciating.
Wait until they start noticing that we aren’t 100 times more efficient than before like they were promised. I’m sure they will take it out on us instead of the AI salesmen
It’s not helping that certain people Internally are lining up to show off whizbang shit they can do. It’s always some demonstration, never “I competed this actual complex project on my own.” But they gets pats on the head and the rest of us are whipped harder.
Ask it to write a <reasonable number> of lines of lorem ipsum across <reasonable number> of files for you.
… Then think harder about how to obfuscate your compliance because 10m lines in 10 min probably won’t fly (or you’ll get promoted to CTO)
Malicious compliance time
