Google apologizes for ‘missing the mark’ after Gemini generated racially diverse Nazis::Google says it’s aware of historically inaccurate results for its Gemini AI image generator, following criticism that it depicted historically white groups as people of color.
It’s great seeing time and time again that no one really does understand these models and that their preconceived notions of what biases exist ends up shooting them in the foot. It truly shows that they don’t really understand how systematically problematic the underlying datasets are and the repurcussions of relying on them too heavily.
Honestly pisses me off that so many real humans lack the contextual awareness to know that contextual awareness is a concept that does not even exist to LLMs.
Its not an issue. Gemini can generate the apology for you.
Who exactly are they apologizing to? Is it the Nazis?
They didn’t apologize. Headlines just say they did.
Kanye has entered the chat.
“Especially” 💀
Now that shit is funny. I hope more people take more time to laugh at companies scrambling to pour billions into projects they don’t understand.
Laugh while it’s still funny, anyway.
Horror is the naked moment between one type of laugher and the other
I dont get the “American Woman” one
It’s a demonstration that the model is coded to include diversity, and it doesn’t generate 4 middle aged WASP moms
I think it’s an example of why they programmed in diversity, to ensure you get diverse responses, but they forgot about edge cases.
The complaint listed in the text was that it “refused to generate white people in any context”, which was not the author’s experience, hence they shared screens of their results which did include white americans
This could make for some hilarious, alternate history satire or something. I could totally see Key and Peele heading a group of racially diverse nazis ironically preaching racial purity and attempting to take over the world.
Dave Chappelle did that with a blind black man that joined the Klan (back in the day before he went off the deep end)
…white is a color. Also white people usually look pink, cream, orange or red. Only albinos look the closest to white though not white enough.
It’s just the name of a racial category. There are no black people either.
Sure there are. Maybe not Vanta Black
This is the best summary I could come up with:
Google has apologized for what it describes as “inaccuracies in some historical image generation depictions” with its Gemini AI tool, saying its attempts at creating a “wide range” of results missed the mark.
The statement follows criticism that it depicted specific white figures (like the US Founding Fathers) or groups like Nazi-era German soldiers as people of color, possibly as an overcorrection to long-standing racial bias problems in AI.
Over the past few days, however, social media posts have questioned whether it fails to produce historically accurate results in an attempt at racial and gender diversity.
The criticism was taken up by right-wing accounts that requested images of historical groups or figures like the Founding Fathers and purportedly got overwhelmingly non-white AI-generated people as results.
Image generators are trained on large corpuses of pictures and written captions to produce the “best” fit for a given prompt, which means they’re often prone to amplifying stereotypes.
“The stupid move here is Gemini isn’t doing it in a nuanced way.” And while entirely white-dominated results for something like “a 1943 German soldier” would make historical sense, that’s much less true for prompts like “an American woman,” where the question is how to represent a diverse real-life group in a small batch of made-up portraits.
The original article contains 766 words, the summary contains 211 words. Saved 72%. I’m a bot and I’m open source!
If the black Scottish man post is anything to go by, someone will come in explaining how this is totally fine because there might’ve been a black Nazi somewhere, once.
Hey! If Demoman catches you talkin’ anymore shit like that he’s gonna turn the lot of us into a fine red spray!
Well there’s that video of those black Israelites hasseling that Jewish dude. They looked like bums tho.
Kanye?
Someone needs to edit this to feature Kanye
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25298169/1000001251.png)
Looks like they scrubbed swastikas out of the training set? I have mixed feelings about this. Like if they want something to have historical accuracy or my own personal opinions on censorship that shouldn’t be scrubbed. But also this is the perfect tool to churn out endless amounts of pro nazi propaganda so maybe it’s safer to keep it removed?
I wonder if it’s just a hard shape to get right, like hands.
Isn’t there an entire subreddit of humans who can’t get it right? I think we’re starting to see considerable overlap between the intelligence of the smartest AI and the dumbest humans.
Probably. Image generators still have a bit of trouble with signs and iconography. A swastika probably falls into a similar category.
A Washington Post investigation last year found that prompts like “a productive person” resulted in pictures of entirely white and almost entirely male figures, while a prompt for “a person at social services” uniformly produced what looked like people of color. It’s a continuation of trends that have appeared in search engines and other software systems.
This is honestly fascinating. It’s putting human biases on full display at a grand scale. It would be near-impossible to quantify racial biases across the internet with so much data to parse. But these LLMs ingest so much of it and simplify the data all down into simple sentences and images that it becomes very clear how common the unspoken biases we have are.
There’s a lot of learning to be done here and it would be sad to miss that opportunity.
It’s putting human biases on full display at a grand scale.
Not human biases. Biases in the labeled data set. Those could sometimes correlate with human biases, but they could also not correlate.
But these LLMs ingest so much of it and simplify the data all down into simple sentences and images that it becomes very clear how common the unspoken biases we have are.
Not LLMs. The image generation models are diffusion models. The LLM only hooks into them to send over the prompt and return the generated image.
Not human biases. Biases in the labeled data set.
Who made the data set? Dogs? Pigeons?
If you train on Shutterstock and end up with a bias towards smiling, is that a human bias, or a stock photography bias?
Data can be biased in a number of ways, that don’t always reflect broader social biases, and even when they might appear to, the cause vs correlation regarding the parallel isn’t necessarily straightforward.
I mean “taking pictures of people who are smiling” is definitely a bias in our culture. How we collectively choose to record information is part of how we encode human biases.
I get what you’re saying in specific circumstances. Sure, a dataset that is built from a single source doesn’t make its biases universal. But these models were trained on a very wide range of sources. Wide enough to cover much of the data we’ve built a culture around.
Except these kinds of data driven biases can creep in from all sorts of ways.
Is there a bias in what images have labels and what don’t? Did they focus only on English labeling? Did they use a vision based model to add synthetic labels to unlabeled images, and if so did the labeling model introduce biases?
Just because the sampling is broad doesn’t mean the processes involved don’t introduce procedural bias distinct from social biases.
How are you guys getting it to generate"persons". It simply says It’s against my GOGLE AI PRINCIPLE to generate images of people.
They actually neutered their AI on thursday, after this whole thing blew up.
So right now, everyone’s fucked because Google decided to make a complete mess of this.
It’s putting human biases on full display at a grand scale.
The skin color of people in images doesn’t matter that much.
The problem is these AI systems have more subtle biases, ones that aren’t easily reveals with simple prompts and amusing images, and these AIs are being put to work making decisions who knows where.
In India they’ve been used to determine whether people should be kept on or kicked off of programs like food assistance.
Well, humans are similar to pigs in the sense that they’ll always find the stinkiest pile of junk in the area and taste it before any alternative.
EDIT: That’s about popularity of “AI” today, and not some semantic expert systems like what they’d do with Lisp machines.
I can’t fathom why google would force diversity into AI.
People use AI as tools. If the tool doesn’t work correctly, people will not use it, full stop. It’s that simple.
There are many different AI out there that don’t behave this way and people will be quick to move on to one of those instead.
Surprisingly stupid even for google.
Ah, the Battlefield 5 experience
inclusivity is obviously good but what googles doing just seems all too corporate and plastic
It’s trying so hard to not be racist that is being even more racing that other AI, is hilarious
No matter what Google does, people are going to come up with gotcha scenarios to complain about. People need to accept the fact that if you don’t specify what race you want, then the output might not contain the race you want. This seems like such a silly thing to be mad about.
It’s silly to point at brand new technology and not expect there to be flaws. But I think it’s totally fair game to point out the flaws and try to make it better, I don’t see why we should just accept technology at its current state and not try to improve it. I totally agree that nobody should be mad at this. We’re figuring it out, an issue was pointed out, and they’re trying to see if they can fix it. Nothing wrong with that part.
It’s really a failure of one-size-fits-all AI. There are plenty of non-diverse models out there, but Google has to find a single solution that always returns diverse college students, but never diverse Nazis.
If I were to use A1111 to make brown Nazis, it would be my own fault. If I use Google, it’s rightfully theirs.
The issue seems to be the underlying code tells the ai if some data set has too many white people or men, Nazis, ancient Vikings, Popes, Rockwell paintings, etc then make them diverse races and genders.
What do we want from these AIs? Facts, even if they might be offensive? Or facts as we wish they would be for a nicer world?
No matter what Google does, people are going to come up with gotcha scenarios to complain about.
American using Gemini: “Please produce images of the KKK, historically accurate Santa’s Workshop Elves, and the board room of a 1950s auto company”
Also Americans: “AH!! AH!!! Minorities and Women!!! AAAAAHHH!!!”
I mean, idk, man. Why do you need AI to generate an image of George Washington when you have thousands of images of him already at your disposal?
Because sometimes you want an image of George Washington, riding a dinosaur, while eating a cheeseburger, in Paris.
Which you actually can’t do on Bing anyway, since it ‘content warning’ stops you from generating anything with George Washington…
Ask it for a Founding Father though, it’ll even hand him a gat!
He’s not even eating the cheeseburger, crap AI.
Funnily enough, he’s not eating one in the other three images either. He’s holding an M16 in one, with the dinosaur partially as a hamburger (?). In the other two he’s merely holding the burger.
I assume if I change the word order around a bit, I could get him to enjoy that burger :D
This is the thing. There’s an incredible number of inaccuracies in the picture, several of which flat out ignore the request in the prompt, and we laugh it off. But the AI makes his skin a little bit darker? Write the Washington Post! Historical accuracy! Outrage!
Well, the tech is of course still young. And there’s a distinct difference between:
A) User error: a prompt that isn’t as good as it can be, with the user understanding for example the ‘order of operations’ that the AI model likes to work in.
B) The tech flubbing things because it’s new and constantly in development
C) The owners behind the tech injecting their own modifiers into the AI model in order to get a more diverse result.
For example, in this case I understand the issue: the original prompt was ‘image of an American Founding Father riding a dinosaur, while eating a cheeseburger, in Paris.’ Doing it in one long sentence with several comma’s makes it harder for the AI to pin down the ‘main theme’ from my experience. Basically, it first thinks ‘George on a dinosaur’ with the burger and Paris as afterthoughts. But if you change the prompt around a bit to ‘An American Founding Father is eating a cheeseburger. He is riding on a dinosaur. In the background of the image, we see Paris, France.’, you end up with the correct result:
Basically the same input, but by simply swapping around the wording it got the correct result. Other ‘inaccuracies’ are of course to be expected, since I didn’t really specify anything for the AI to go of. I didn’t give it a timeframe for one, so it wouldn’t ‘know’ not to have the Eiffel Tower and a modern handgun in it. Or that that flag would be completely wrong.
The problem is with C) where you simply have no say in the modifiers that they inject into any prompt you send. Especially when the companies state that they are doing it on purpose so the AI will offer a more diverse result in general. You can write the best, most descriptive prompt and there will still be an unexpected outcome if it injects their modifiers in the right place of your prompt. That’s the issue.
C is just a work around for B and the fact that the technology has no way to identify and overcome harmful biases in its data set and model. This kind of behind the scenes prompt engineering isn’t even unique to diversifying image output, either. It’s a necessity to creating a product that is usable by the general consumer, at least until the technology evolves enough that it can incorporate those lessons directly into the model.
And so my point is, there’s a boatload of problems that stem from the fact that this is early technology and the solutions to those problems haven’t been fully developed yet. But while we are rightfully not upset that the system doesn’t understand that lettuce doesn’t go on the bottom of a burger, we’re for some reason wildly upset that it tries to give our fantasy quasi-historical figures darker skin.
An it’s not a beyond burger… it’s promoting the genocide of cattle.
Here’s one that was made, just for you, with specifically a VEGAN cheeseburger in the prompt :D
The random lettuce between every layer is weirdly off-putting to me. It seems like it’s been growing on the burger for quite some time :D
Doesn’t look too bad to me. I love a fair bit of crispy lettuce on a burger. Doing it like that at least spreads it out a bit, rather than having a big chunk of lettuce.
Still, it that was my burger… I’d add another patty and extra cheese.
Oh no minorities are overrepresented, quick, do something!
There is a difference between having actually diverse data sources and secretly adding the word “diverse” to each image generation prompt
Never claimed they had diverse data sources - they probably don’t.
My point is that that when minorities are underrepresented, which is the default case in GenAI, the (white, male) public tends to accept that.
I like that they tried to fix the issue of GenAI being racist and sexist. Even though the solution is obviously flawed: Better this than a racist model.
I can’t believe someone has to spell this out for you, but here we go: an accurate picture of people from an era in which there was no diversity will, by definition, not be diverse.
The idea was noble, their implementation was ham fisted.
