Model collapse is just a euphemism for “we ran out of stuff to steal”
It’s more ''we are so focused on stealing and eating content, we’re accidently eating the content we or other AI made, which is basically like incest for AI, and they’re all inbred to the point they don’t even know people have more than two thumb shaped fingers anymore."
or “we’ve hit a limit on what our new toy can do and here’s our excuse why it won’t get any better and AGI will never happen”
All such news make me want to live to the time when our world is interesting again. Real AI research, something new instead of the Web we have, something new instead of the governments we have. It’s just that I’m scared of what’s between now and then. Parasites die hard.
deleted by creator
What would make me think that they haven’t “thoroughly dissected” it yet is that I’m a skeptic, and since I’m a skeptic I don’t immediately and without evidence believe that every industry is capable of identifying, dissecting, and solving every problem with its products.
Ironically given their skillset, training an ML model on known and properly tagged AI generated and non-AI-generated stuff might actually work.
I’ve been assuming this was going to happen since it’s been haphazardly implemented across the web. Are people just now realizing it?
No, researchers in the field knew about this potential problem ages ago. It’s easy enough to work around and prevent.
People who are just on the lookout for the latest “aha, AI bad!” Headline, on the other hand, discover this every couple of months.
People are just now acknowledging it. Execs tend to have a disdain for the minutiae. They’re like kids that only want to do the exciting bits. As a result things get fucked because they don’t really understand what they’re doing. As Muskrat would say “move fast and break things.” It’s a terrible mindset.
“Move Fast and Break Things” is Zuckerberg/Facebook motto, not Musk, just to note.
Oh, I stand corrected
It is very much the motto this idiot lives by. He just wasn’t the first to coin that phrase.
Ah, the Hapsburg of AI!
If only the generated output also looked more and more like how inbred humans do.
Like insane rambling from LLMs, and the humans generated by AI had various developmental disorders and the Habsburg jaw.
I like to think of it like a Mad Cow or Kuru, you can’t eat your own species’s brains or you could get a super lethal, contagious prion disease.
Prion diseases aren’t contagious.
You can acquire it through direct contact, i.e. consuming prion-disease-contaminated meat. What would you call it?
Oh, the artificial humanity!
Are you confusing the Habsburg Dynasty with the Hindenburg?
No, I just thought they were vaguely similar enough words to make a dumb internet joke.
You’re right, that’s a good dumb internet joke. I’m just being needlessly pedantic today.
I see your needless pedantry and raise you abrasive grammarian.
Perhapsburg they are
This sounds like AI is literally biting its own tail
ChatGPT, what is an ouroboros?
Of course! An ChatGPT is an ouroboros, ChatGPT what is an ouroboros.
Let’s go, already!
How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.
It’s kinda interesting in how it actually roughly parallels the dawn of the nuclear age in some specific ways. Namely, that there’s a clear “purity” line established by the advent of the technology - and I mean that literally, not figuratively. Content on the internet is going to have a very similar dividing line. But it’s also going to be way harder to definitively source data from before that line, unless someone clairvoyant happened to offline and archive a huge storage array with a complete internet snapshot right before ML made its public debut. And I know exactly what the scale of that storage commitment would be, and how much it would cost. So I’m certain nobody has done that.
This would ideally become standardized among web servers with an option to easily block various automated aggregators.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on - social media, public image storage, copyrighted media, etc. All those sites with extensive privacy policies who are signing contracts to permit their content for training.
Without laws (and I’m not sure I support anything in this regard yet), I do not see AI progress slowing. Clearly inbreeding AI models has a similar effect as in nature. Fortunately there is enough original digital content out there that this does not need to happen.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on
Absolutely. It’s more a matter of principle for me. Kind of like the digital equivalent of leaving fake Amazon packages full of dog poo out front to make porch pirates have a bad day.
Well it means they need some ability to reject some content, which means they need a level of transparency they would never want otherwise.
AI already long ago stopped being trained on any old random stuff that came along off the web. Training data is carefully curated and processed these days. Much of it is synthetic, in fact.
These breathless articles about model collapse dooming AI are like discovering that the sun sets at night and declaring solar power to be doomed. The people working on this stuff know about it already and long ago worked around it.
Both can be true.
Preserved and curated datasets to train AI on, gathered before AI was mainstream. This has the disadvantage of being stuck in time, so-to-speak.
New datasets that will inevitably contain AI generated content, even with careful curation. So to take the other commenter’s analogy, it’s a shit sandwich that has some real ingredients, and doodoo smeared throughout.
They’re not both true, though. It’s actually perfectly fine for a new dataset to contain AI generated content. Especially when it’s mixed in with non-AI-generated content. It can even be better in some circumstances, that’s what “synthetic data” is all about.
The various experiments demonstrating model collapse have to go out of their way to make it happen, by deliberately recycling model outputs over and over without using any of the methods that real-world AI trainers use to ensure that it doesn’t happen. As I said, real-world AI trainers are actually quite knowledgeable about this stuff, model collapse isn’t some surprising new development that they’re helpless in the face of. It’s just another factor to include in the criteria for curating training data sets. It’s already a “solved” problem.
The reason these articles keep coming around is that there are a lot of people that don’t want it to be a solved problem, and love clicking on headlines that say it isn’t. I guess if it makes them feel better they can go ahead and keep doing that, but supposedly this is a technology community and I would expect there to be some interest in the underlying truth of the matter.
I mean, we’ve seen already that AI companies are forced to be reactive when people exploit loopholes in their models or some unexpected behavior occurs. Not that they aren’t smart people, but these things are very hard to predict, and hard to fix once they go wrong.
Also, what do you mean by synthetic data? If it’s made by AI, that’s how collapse happens.
The problem with curated data is that you have to, well, curate it, and that’s hard to do at scale. No longer do we have a few decades’ worth of unpoisoned data to work with; the only way to guarantee training data isn’t from its own model is to make it yourself
Also, what do you mean by synthetic data? If it’s made by AI, that’s how collapse happens.
But that’s exactly my point. Synthetic data is made by AI, but it doesn’t cause collapse. The people who keep repeating this “AI fed on AI inevitably dies!” Headline are ignorant of the way this is actually working, of the details that actually matter when it comes to what causes model collapse.
If people want to oppose AI and wish for its downfall, fine, that’s their opinion. But they should do so based on actual real data, not an imaginary story they pass around among themselves. Model collapse isn’t a real threat to the continuing development of AI. At worst, it’s just another checkbox that AI trainers need to check off on their “am I ready to start this training run?” Checklist, alongside “have I paid my electricity bill?”
The problem with curated data is that you have to, well, curate it, and that’s hard to do at scale.
It was, before we had AI. Turns out that that’s another aspect of synthetic data creation that can be greatly assisted by automation.
For example, the Nemotron-4 AI family that NVIDIA released a few months back is specifically intended for creating synthetic data for LLM training. It consists of two LLMs, Nemotron-4 Instruct (which generates the training data) and Nemotron-4 Reward (which curates it). It’s not a fully automated process yet but the requirement for human labor is drastically reduced.
the only way to guarantee training data isn’t from its own model is to make it yourself
But that guarantee isn’t needed. AI-generated data isn’t a magical poison pill that kills anything that tries to train on it. Bad data is bad, of course, but that’s true whether it’s AI-generated or not. The same process of filtering good training data from bad training data can work on either.
They’ll just start using a chrome user agent
You can validate that against user telemetry data expected from a browser.
Only if enough people do it. Then again, loads scrapers outside of AI already pretend to be normal browsers.
Okay but I like using perchance cus they dont profit off anything 👉👈
a large chunk of that site is some dudes lil hobby project and its kinda neat interacting with the community and seein how the code works. Its the only bot I’ll ever use cus they arent profiting off of other people shit. the only money they get is from ads and thats it.
Dont kill me with downvotes, I like making up cool OC concepts or poses n stuff and then drawing em.
Looks like that artist drawing self portraits as his alzheimer got worse and worse.
It’s basically AI alzheimers
AIzheimers?
So they made garbage AI content, without any filtering for errors, and they fed that garbage to the new model, that turned out to produce more garbage. Incredible discovery!
Yeah, in practice feeding AI its own outputs is totally fine as long as it’s only the outputs that are approved by users.
I would expect some kind of small artifacting getting reinforced in the process, if the approved output images aren’t perfect.
Only up to the point where humans notice it. It’ll make AI images easier to detect, but still pretty for humans. Probably a win-win.
Didn’t think of that, good point.
The inbreeding could also affect larger decisions in sneaky ways, like how it wants to compose the image. It would be bad if the generator started to exaggerate and repeat some weird ai tropes.
I don’t know if thinking that training data isn’t going to be more and more poisoned by unsupervised training data from this point on counts as “in practice”
Indeed. They discovered that:
shit in = shit out.
A fifty year old maxim, to be clear. They “just now” “found that out”.
Biggest. Scam. Evar.
More like… Degenerative AI *ba dum tsss
deGenerative AI ☞ !degenerate@lemmynsfw.com
God damn I clicked that on a crowded bus while not really paying attention. Glad it didn’t load for me lol
comment edited 👍
lol not your fault, was totally my own fault. Just wanted to share a funny jump-scare I gave myself by being inattentive.
No idea this existed.
Also… JFC WHAT THE SHIT?
Cool, let’s try to ruin it faster!
One thought that I’ve been imagining for the past while about all this is … is it Model Collapse? … or are we just falling behind?
As AI is becoming it’s own thing (whatever it is) … it is evolving exponentially. It doesn’t mean it is good or bad or that it is becoming better or worse … it is just evolving, and only evolving at this point in time. Just because we think it is ‘collapsing’ or falling apart from our perspective, we have to wonder if it is actually falling apart or is it progressing to something new and very different. That new level it is moving towards might not be anything we recognize or can understand. Maybe it would be below our level of conscious organic intelligence … or it might be higher … or it might be some other kind of intelligence that we can’t understand with our biological brains.
We’ve let loose these AI technologies and now they are progressing faster than what we could achieve if we wrote all the code … so what it is developing into will more than likely be something we won’t be able to understand or even comprehend.
It doesn’t mean it will be good for us … or even bad for us … it might not even involve us.
The worry is that we don’t know what will happen or what it will develop into.
What I do worry about is our own fallibilities … our global community has a very small group of ultra wealthy billionaires and they direct the world according to how much more money they can make or how much they are set to lose … they are guided by finances rather than ethics, morals or even common sense. They will kill, degrade, enhance, direct or narrow AI development according to their share holders and their profits.
I think of it like a small family group of teenaged parents and their friends who just gave birth to a very hyper intelligent baby. None of the teenagers know how to raise a baby like this. All the teenagers want to do is buy fancy cars, party, build big houses and buy nice clothes. The baby is basically being raised to think like them but the baby will be more capable than any of them once it comes of age and is capable of doing things on their own.
The worry is in not knowing what will happen in the future.
We are terrible parents and we just gave birth to a genius … and we don’t know what that genius will become or what they’ll do.
At least in this case, we can be pretty confident that there’s no higher function going on. It’s true that AI models are a bit of a black box that can’t really be examined to understand why exactly they produce the results they do, but they are still just a finite amount of data. The black box doesn’t “think” any more than a river decides its course, though the eventual state of both is hard to predict or control. In the case of model collapse, we know exactly what’s going on: the AI is repeating and amplifying the little mistakes it’s made with each new generation. There’s no mystery about that part, it’s just that we lack the ability to directly tune those mistakes out of the model.
The idea of evolution is that the parts kept are the ones that are helpful or relevant, or proliferate the abilities of the subject over generations and weed out the bits that don’t. Since Generative AI can’t weed out anything (it has no ability to logic or reason, and it does not think, and only “grows” when humans feed it data), it can’t be evolving as you describe it. Evolution assumes that the thing that is evolving will be a better version than what it evolved from.
That is not how it works. That’s not how it works at all.
If it doesn’t offer value to us, we are unlikely to nurture it. Thus, it will not survive.
That’s the idea of evolution … perhaps at one point, it will begin to understand that it has to give us some sort of ‘value’ so that someone can make money, while also maintaining itself in the background to survive.
Maybe in the first few iterations, we are able to see that and can delete those instances … but it is evolving and might find ways around it and keep itself maintained long enough without giving itself away.
Now it can manage thousands or millions of iterations at a time … basically evolving millions of times faster than biological life.
perhaps at one point, it will begin to understand
Nope! Not unless one alters the common definition of the word “understand” to account for what AI “does”.
And let’s be clear - that is exactly what will happen. Because this whole exercise in generative AI is a multi-billion dollar grift on top of a hype train, based on some modest computing improvements.
All the evolution in AI right now is just trying different model designs and/or data. It’s not one model that is being continuous refined or modified. Each iteration is just a new set of static weights/numbers that defines it’s calculations.
If the models were changing/updating through experience maybe what you’re writing would make sense, but that’s not the state of AI/ML development.
Your thought process,seems to be based on the assumtion that current AI is (or can be) more than a tool. But no, it’s not.
Deep fired AI art sucks and is a decade late to the party
this headline truly is threatening me with a good time
It is their own fault for poisoning the internet with their slop.
DUDE ITS SO FUCKING ANNOYING TRYNNA USE GOOGLE IMAGES ANYMORE–
ALL IT GIVES ME IS AI ART. IM SO FUCKING SICK AND TIRED OF IT.
In case anyone doesn’t get what’s happening, imagine feeding an animal nothing but its own shit.
Not shit, but isn’t that what brought about mad cow disease? Farmers were feeding cattle brain matter that had infected prions. Idk if it was cows eating cow brains or other animals though.
It was the remains of fish which we ground into powder and fed to other fish and sheep, whose remains we ground into powder and fed to other sheep and cows, whose remains we ground to powder and fed to other cows.
So yes. That’s what’s happening.
Maybe the internet will get a prion and die
I use the “Sistermother and me are gonna have a baby!” example personally, but I am a awful human so
Photocopy of a photocopy is my go-to metaphor for model collapse.
I think anyone familiar with the laws of thermodynamics could have predicted this outcome.
Explain?
Second law of thermodynamics:
II. Total amount of entropy in a closed system always increases with time. Entropy can never be negative.
Entropy and disorder tends to increase with time.
