The difference here is that a cheap AI chip won’t fix the fundamental software problems with LLMs. We might reach a point where they can produce output faster, but as long as what’s actually going on is probabilistic next-token prediction in a static vector database, that just means faster mistakes as well.

There’s an odd psychosis going around where people become convinced that actual AGI can be derived from this technology. People who should know better just shut their brains off when it comes to token prediction, because they’ve had very compelling “conversations” with the predictor. They forget that the actual model is static, has no internal state, and doesn’t even “remember” what you’ve said to it.

What it has is a context window, and your entire conversational history - both what you’ve said and how it has responded - gets shoved into that window when you interact with it. (Or depending on the chatbot harness, saved in “memory” files that it can retrieve when the context contents indicate that would be useful.)

That’s why the bots seem so weirdly forgetful one moment and like they’ve got photographic memories the next. Stuff that is in the context window and has its “attention” will influence the tokens it produces, but whether or not the right things are in the context window and it’s including them in the token prediction is a crapshoot.

No, I’ve decided not to use a tool that isn’t fit for purpose. Studies have consistently shown that people think they’re much more productive on LLMs than they actually are, and the side effects of cognitive debt and skill loss aren’t worth the often imaginary productivity gains.

Combine that with the awful financials of OpenAI, Anthropic, SpaceX, and Google’s Gemini, and you’ve got the makings of a really nasty situation.

Say I come to depend on Claude Code for all my coding, and build an entire workflow around it. I get used to spending thousands of dollars a month for access, and tell myself it’s worth it for the imaginary productivity gains (although what’s actually happening is that I’m just producing a shit-ton of garbage code no one has any hope of understanding).

Then Anthropic reaches the point where they run out of investor money to spend, and fold because even charging developers like me thousands of dollars doesn’t even begin to cover their costs, and I’m one of their rare customers who are loyal, versus the bulk of developers who use up their free allotments of tokens and then model-hop.

Now what? My workflow is broken, I’ve forgotten how to code, and nothing I’ve produced recently is human-readable.

No thanks. I’m not an old man telling at clouds, I’m a software developer who can recognize a problem when I see one.

If it does get better, it will be with technology other than LLMs, because LLMs don’t get cheaper per unit of usage as usage scales.

I suspect that we won’t have actually useful AI of some sort until LLMs get out of the way. They’re sucking all the oxygen out of the room right now.

What we want is software that behaves predictably. Since LLMs don’t do that, we don’t want them or their “agents.”

For fuck’s sake.

I had someone ask me recently if a picture of Thomas Aquinas was AI-generated. It was a 15th-century painting.

We’re sure they’re absolutely PR stunts and bullshit.

An LLM does nothing without being prompted. An LLM only has access to the tools you give it via whatever harness you’re interacting with it through. An LLM in an actual sandbox has zero chance to hack anything, especially if it’s properly air-gapped, as any responsible person would do with technology they actually think is dangerously powerful.

If their LLMs are behaving badly, that’s because their prompts are poorly written, their harnesses are vibe-coded garbage, and their “sandboxes” aren’t real sandboxes.

Hey now! I can honestly say Microsoft has never done that with any of my computers!

(Why yes, they do all run Linux. How did you know?)

AI is going to happen.

Oh? So there are hard numbers showing a positive ROI now?

I wonder how the venture capital firms that fund Anthropic’s money-burning operation feel about this. “Oh look, now I know how that particular funding round’s money was actually spent. Fantastic.”

The problem is that synthetic data is not fit for that purpose. The more of it you use, the worse at dealing with the cases LLMs get.

Think of it like this… You feed a language model a bunch of genuine human-written content. Great. Now it can produce the most likely text in a lot of cases. Word combinations that rarely appear in written language rarely get generated, so most of its synthetic data lacks those rare - but still valid - combinations.

Train it on this synthetic data, and now more outliers and rare combinations get filed off. Rinse and repeat.

It’s also unavoidable.

(Non)Human centipede.

I’m waiting for the day when desperate LLM companies start paying people to post real human content, only for those people to just ask ChatGPT to do it.

“Hi everyone, welcome to my talk on fainting from anxiety, as triggered by… this… this… uuuuuuuhhhgg…” [Thud]

I have an autographed original drawing. Unfortunately, it’s a drawing of the Sandman, by Neil Gaiman. Yeah… I don’t have that on display in my house anymore.

I mean, did anyone expect anything different? The frontier providers that Copilot depends on switched to token-based billing in a desperate and ludicrous attempt to somehow turn a profit on AI, so of course Microsoft is gonna jack up their prices too.

There was a lot of news about the Muttsee Dam solar project a few years ago, so it’s a real thing and a good thing. But it’s hardly current news. It’s been operational since 2022.

It wasn’t. But that doesn’t mean we shouldn’t strive to make it be the best version of itself it could be, and fight tooth and nail against attempts to make it worse.

Very serious. Your personal amount of usage means nothing at all in this conversation. It is entirely about tokens per watt. The amount of energy the memory operations involve scale incredibly well when people are accessing the same object in memory simultaneously. Last I looked it was around a 10x difference for the same models efficiency.

Hold up. Are you talking about caching? Because if you are… yeah. That has nothing to do with the model and everything to do with the service layer around the model. The same service layers can be - and have been - implemented in tools like Lemonade Server, llama.cpp, Ollama, etc.

And I really do want to know your sources.

Mine say GPT 5.5 is probably using quite a lot more than 0.34 Wh per query (0.34 Wh is what Sam Altman claimed for the then-current version of GPT in June of 2025, but he hasn’t released numbers since then and no one has done an independent analysis). With Claude, an independent estimate from last year pegged Sonnet at 0.8 Wh for a short prompt, 2.8 Wh for a medium one, and 5.5 Wh for a long one. Current numbers are, again, almost certainly much higher. And just for fun, there’s DeepSeek (which I’ve never used and never would use), with the reasoning-tuned DeepSeek-R1 hitting a whopping 29 Wh for a complex query.

Meanwhile, small, open models are probably in the 0.07 - 0.2 range, depending on the model, the hardware it’s running on, and the nature of the query. Of course, there are much weightier open models too, with ones like Llama 3.1 405B using about 9 Wh for a medium-length prompt. On the other hand… who is going to run that on their local machine?

Look… If I’m wrong, and using local models the way I do - sparingly and infrequently - really does consume more electricity than using Claude Code, I want to know. I have no problem whatsoever with eschewing AI models entirely, since I despise all of them. But given how tight-lipped OpenAI and Anthropic are about energy consumption per average prompt, and what independent analyses have estimated, I am highly skeptical that they are acting as some sort of paragons of environmental stewardship.

You’re probably burning more energy turning it off and on again. It doesn’t really use any noticeable power sitting idle.

I am absolutely not burning more energy than a frontier model by doing things like putting my laptop to sleep or shutting down unused services when I want to conserve battery power.

Anyway, a direct comparison would be pretty difficult because your model is probably tens of billions of parameters, not over a trillion.

True.

Energy consumption per output token will probably be a bit higher for the frontier models but something that people have found is that higher quality models often need fewer tokens to achieve the same goal.

That’s actually not true. In fact it’s much the opposite. Frontier models churn through tokens at a much higher rate, because of their higher complexity and higher number of parameters. Research is still new on this, but having a frontier model analyze your code files versus a small, local model for the same task seems to be enormously wasteful. If you must use a frontier model for something, have it do that work after receiving the output from an agent using a small model to read and summarize your code.

Plus how many times do you re-prompt your local model vs Claude Fable or Opus for example to get the desired result?

…Almost never? I’m not a fan of letting AI do much of ANY of my coding, because it will inevitably bloat my codebase with garbage regardless of which model I use. So I severely restrict my model usage to simple, clearly-defined, narrow-scoped tasks that can save me a bit of time, and that’s it. With guardrails and discipline like that, I barely ever have the need to re-prompt.