In particular, know how to identify the common and deadly species (eg: much of the genus Amanita) yourself, and get multiple trustworthy field guides for your part of the world.
While I would not advocate anyone taking up amateur mycology under any circumstances, let alone with an app, or book, to guide them, it’s important to note that this article is biased and makes false or misleading claims.
The main issue is that it is talking about AI and meaning LLM-based algorithms. But it uses a study that showed that apps which identify mushrooms are inaccurate in which all of the apps predate, and do not use, LLMs as part of their identification process.
Countering misinformation with misinformation isn’t generally the best option in my opinion so I just wanted to point that out.
You didn’t read the article.
I read the article and its linked sources in a few cases. How else would I have been able to directly address them?
When Australian scientists tested the accuracy of popular mushroom ID apps last year after a spike in poisonings, they found the most precise one correctly identified dangerous mushrooms 44 percent of the time.
Notice this paragraph which links to https://pubmed.ncbi.nlm.nih.gov/36794335/
The extract for which talks about the following apps:
Picture Mushroom (Next Vision Limited©), Mushroom Identificator (Pierre Semedard©), and iNaturalist (iNaturalist, California Academy of Sciences©)
None of which use LLMs and predate the issue that the article is talking about. I checked, before my comment, all of their pages on the iOS App store, at least. They’re all 4+ years old and none use LLMs.
Amusingly enough, the Public Citizen article linked earlier in OP’s article calls out iNaturalist as something they’ve been working with to positively improve the experience of identifying mushrooms:
https://www.citizen.org/article/mushroom-risk-ai-app-misinformation/
The Fungal Diversity Survey, a project devoted to correcting the many gaps in understanding regarding fungal biodiversity, partners with iNaturalist to document and verify mushroom observation
But ultimately there were no apps ACTUALLY TESTED that use OpenAI or LLMs for their identification.
Where does the article say the problem started with AI? It doesn’t even mention LLMs, just the explosion in grifter apps since it became easier to produce a grifter app.
If you read the article, you did not read it properly.
And they didn’t test any of them, and linked to an actual test which ALSO didn’t test any of them as if it supported the claim that these apps are, as you (but not the article) say, are grifter apps.
LLMs have literally zero value in any context vaguely related to any kind of advanced computer vision project. It is fundamentally impossible for them to improve the capability of a mushroom recognition app in any way.
It’s not misinformation to state the fact that it’s an absolutely certainty that anyone claiming to use an LLM to identify a mushroom is a scammer.
true to your name you kind of put my comment into less words, nice 👍
you have sort of a weird take on this? like here are our premises, what we know with certainty:
- all mycology apps tested to date are known to be poor (highest accuracy less than 50%)
- all LLMs are known to be fairly poor
and the author is deriving the conclusion:
- mycology apps that happen to be LLM-based have a high likelihood of being poor, so be careful
like yes, it’s not an empirical conclusion because someone still needs to do the work of testing the LLM mycology apps. i’d call it maybe an evidence based hypothesis that the average consumer should heed rather than find out the hard way and get poisoned.
but i think you condeming it as “biased,” “misinformation” or “misleading” is unnecessarily harsh. to me this looks like basic pattern recognition and forming hypotheses based on real evidence.
maybe i am missing a hole in the logic here and if so let me know.
Of all the things to use machine learning for, identifying poisonous fucking mushrooms seems like a poor choice. I’m sure it sounds very confident in its wrong answer, though.
What about identifying three different types of similar flowers by using the sepal length? 🧐 Very valid machine learning 101 tutorial exercise
Yeah, I mean there’s only going to be a handful of species you want to eat in any location at any time.
You should never just be finding random mushrooms and figuring out if you can eat them.
Finding random mushrooms and learning to identify them (which includes learning if they are edible) is absolutely how you should start in amateur mycology, especially if you don’t have any mycology groups nearby that you can join. And if you do, you know what that group will do? Gather random mushrooms to learn/teach identification.
Just don’t go around eating random stuff.
Identifying mushrooms with an ML-based algorithm is a fine idea if you properly design the application to leverage that. As a hedgehog, this is what I would do:
- Train my model on a variety of mushrooms, particularly poisonous ones.
- When testing the model, test as many mushrooms as possible and take note of what’s frequently mis-identified.
- When testing the model, make sure to get a variety of different kinds of lighting.
- In addition to the mis-identifications noted while testing the app, maintain a list of commonly misidentified mushrooms - like the hedgehog mushroom and its counterparts - particularly the ones a forager should be most concerned with (meaning the most poisonous ones).
- When identifying a mushroom to the user, err on the side of calling it a poisonous mushroom. Consider providing a list of possible matches, with the worst case scenario ones up top.
- Include pictures and other information about the mushrooms, as well as regional mushroom lookups for mushrooms that weren’t included.
- Don’t include text like “99% confident that this is a hedgehog mushroom” when the 99% figure is an output from your ML model. I know we said earlier to make sure to do a ton of testing and I’m sure you think you did, but you didn’t do enough to be able to say that. At best, reduce your certainty by 25%, then divide that number between the identified mushroom and the lookalikes, making sure to give extra weight to the most poisonous ones. So that 99% certainty becomes at most a more realistic 38% chance that it’s the poisonous lookalike and 37% chance that it’s whatever was identified in the first place.
You might say that this app would be useless for determining if a mushroom is safe to eat, and I agree, but it’s also a better approach than any of the existing apps out there. If you need to use an app to determine if a wild mushroom is safe to eat then the answer is simple: it isn’t. C’mon, I’m a hedgehog and even I know that.
This comment reads like it was written by AI
This comment reads like it was written by someone who has never designed a mushroom identification app.
It’s probably just a ChatGPT wrapper with a preset prompt. That’s all these “AI entrepreneurs” are capable of. Absolute fucking hacks.
Convolutional neural networks and plant identifying apps came before chat gpt. Beyond both relying on neural networks they don’t have much in common.
I feel like training on poisonous mushrooms is the wrong direction. You want to err on the side of poisonous, not edible. Anything it can’t identify should be considered poisonous.
Many edible mushrooms have poisonous look-alikes, so your approach would be likely to misidentify those poisonous look-alikes - a potentially deadly mistake.
For example - from https://www.gardeningknowhow.com/edible/vegetables/types-of-edible-mushrooms-their-poisonous-look-alikes
Poisonous Morel Mushroom Look-alikes:
- A common fungus, the false morel is almost the spitting image of its edible cousin except it is not hollow inside and contains cottony material.
- Big red is similar except it has reddish tones and the cap is more brain-like.
- Wrinkled thimble cap truly looks like a morel except its wrinkled cap hangs over the stem.
- Bell morel is smaller and the cap, although similar, is much less textured and it has a cottony interior.
It would be easy to train an ML model to confidently identify any of those as morels if you only trained on morels.
The idea is to train on both so it’s less likely to mistake a poisonous mushroom for an edible one, and to then “hedge” your bet anyway, by always presenting the poisonous look-alikes first.
The most dangerous scenario with this app is also the most useful - a user who has some training in mushroom identification uses the app as a quick way to look up a mushroom they think is a particular edible mushroom, notes that the mushroom they think it is is within the list, then reviews the list of poisonous look-alikes, and then applies their training to rule out those look-alikes. Finally they confirm that they cannot rule out the edible mushroom.
The risks here are that
- the user’s training is lacking and that they ruled out a poisonous mushroom that the app suggested, or
- the app didn’t include the particular poisonous mushroom in the first place and the user was thus unable to consider it.
I would personally trust it if it said it’s poisonous, and then do more checks if it said it was edible. It’d be useful for ruling out some options, and maybe to give you a start for further verification. Basically, don’t trust it to tell you what you can eat but trust it when it tells you you can’t eat something.
This will be how the machines ultimately win.
Now that is a thought. Instead of AI doing a skynet/terminator thing it gets to the point we trust it and then tells us to eat or do things that kill us.
Anybody trusting ChatGPT for life or death answers deserves their Darwin Award
close, more like, anybody allowing ChatGPT to be marketed as though it can answer life or death questions should be held accountable for lying to the public for profit
cough cough OpenAI cough cough
While we’re at it can we force OpenAI to change their name since they aren’t running open source code?
yes!
The general public still doesn’t seem to grasp the current capabilities of AI. It’s still just mimicry. AI is a parrot that “learns” something and repeats it to the best of it’s ability, but it doesn’t understand the thing it learned. You can teach a bird to say “Polly want a cracker” but it doesn’t know what a cracker is, and while it does have wants like any other animal, it doesn’t know what “want a cracker” actually means.
ML models get a billion images of mushrooms and then “learn” what “mushroom” looks like, but even if the images of mushrooms are properly labeled poisonous and not poisonous, it doesn’t really know that in the same way humans do. And it gets even worse when the AI tries to make new things from those sets it’s trained off, which all of those certainly do. Making new mushrooms that don’t exist, how can it tell which one of these new fantasy mushrooms are poison and which ones aren’t? It can’t know, but it sure as hell can make it up.
Hell, most AI can’t even get text right.
Don’t trust AI for anything that isn’t hard coded math, or systems that reference and directly quote known good sources without doing any kind of creative embellishments.
You are lumping a whole lot of different things that work in completely different ways under the singular label of AI, and while I can’t really blame you as that is what the industry does as well, image recognition, image generation and large language models like chat-gpt all work entirely differently.
Image recognition especially can be trained to be extremely accurate with a properly restricted scope and a good dataset, but even so it would never be enough for identifying mushrooms because no matter if it’s being done by the perfect AI or an organic meatbag, mushrooms simply cannot be accurately identified from a single picture as they can look literally identical to one another in many ways.And parrots totally can learn what words mean. Just like how a dog can learn what “Sit”, “Paw” or “Let’s go for a walk” mean, parrots just also have the ability to “talk”.
what’s wrong with lumping a lot of things with different substrate together if, as you admit yourself, there’s still no evidence any of them work well?
LLMs are the current big buzzword and the main ones that “don’t work”, because people assume and expect them to be intelligent and actually know and understand things, which they simply do not. Their purpose is to generate text in a way that a human would and for that they actually work perfectly - get a competent LLM and a human and ask them to write about something, and you are very unlikely to spot which one is the machine unless you can catch them lying, and even then it might just be a clueless human talking about things he kinda understands but isn’t an expert of. Like me.
But they are constantly being used for all kinds of purposes that they really don’t yet fit well, because you can’t actually trust anything they say.Image generation mainly has issues with hands and fingers so they aren’t bullet proof at making fake realistic imagery, but for many subjects and style they can create images that are pretty much impossible to identify as being generated. Civit.ai is full of examples. Most people think it doesn’t work yet because they mostly see someone throwing simple prompts into midjourney and taking the first thing it generates for an article thumbnail.
And image identification definitely works, but it’s… Quirky. I said it can’t be used to identify mushrooms, because nothing can identify two things that look exactly the same from one another. But give one enough photos of every single hot wheels car that exists, and you can get one that will perfectly recognize which one you have. But it will also tell you that a shoe or a tree is one of them, because it only knows about hot wheels cars.
Making one that is trying to identify absolutely everything from a photo, like Google Lens, will still misidentify some things as the dataset is so enormous, but so would a human. Just that for an AI, “I don’t know” is never an option, it always says the most likely answer it thinks is right.edit: oops accidental post
okay? so i am quite aware of all of this already; none of this info is new.
my question is still, “what’s wrong with lumping all of these technologies together as ‘AI’ when all of them are ineffective at identifying mushrooms (and certain other tasks)?”
No need to vilify fungi specifically. Plants can kill you too. Or even animals. If you’re going to hunt or forage you have to know your shit.
Yes, but mushrooms are typically harder to identify than plants are, so AI is surely not very good at it. Even mycologists are only learning in recent years that some mushrooms which they had long believed to be the same species are in fact entirely different species (thanks to genetic testing).
I myself forage for both plants and mushrooms and I practiced identifying mushrooms for years before I would eat anything I found.
I do feel like mushrooms get a bad rep compared to plants – there are certain mushrooms (in the UK at least) that are very safe to forage. Boletes (if you check for staining and red on the stem), agaricus, hedgehog fungi, blewits, shaggy inkcaps…
Others I wouldn’t touch with a barge pole, even if I’m 99% sure. Any of the small white funnels (miller etc.) I’m not interested in, and likewise amanitas I won’t go near.
But obviously the point stands that using AI, rather than books or trusted sources, is a non-starter. Always use multiple sources when foraging (message for a general audience).
Death camas and wild onion are not easy to tell apart. Chanterelles and morels can be identified safely and easily by beginners by looking at a few key features. Neither should use an app to ID.
You missed the word “typically”. I well know that there are exceptions.
I read it, just don’t agree on the generalization. I think it’s more that there’s a cultural phobia of fungi, and not really that they’re harder to ID safely than plants.
As somebody who forages for both, I can confidently say that you are unfortunately incorrect. Just read up a little bit on the modern history of mycology and you will learn that experts can’t even identify one fungi from another without looking at its DNA in many cases.
Any guide, app or whatever, with or without LLM, should generally ALWAYS refer to similar plants & shrooms on their pages, especially if they’re dangerous. But yeah, never trust “AI” for anything blindly. It’s not smart. It’s just a tool that still requires a lot of user input & common sense to be actually useful.
Has any government done anything about the innumerable negligent implementations of AI yet? Every day feels like a countdown to catastrophe.
Some day someone is going to hook up AI to something important like traffic lights, people will die, and it will be revealed that AI was the reason.
The EU at least tries to regulate AI with the AI Act
It’s not a catch all, but some things are banned from using AI and others will need an assessment before going live. Some other will not need an assessment, but serious incidents need to be reported
Mushroom ID requires a lot more than just immediately available visuals. You’ve gotta see what the cap looks like, the stem, how the stem connects to the cap, the specific characteristics of the gills, the substrate it’s growing in, and the spore print (i.e. leave it on a piece of white paper, covered, for a number of hours undisturbed so it drops its spores). And even then it can be tough if the mushroom is abnormal or is decaying at all.
With enough info, I’m sure you could train an ML model to ID mushrooms. But you’d need to give it a lot of info to make a successful ID.
Depends on the mushroom; certain mushrooms (e.g. beefsteak polypore) are incredibly distinctive, while others require microscopy to tell apart.
One thing you didn’t mention that I think is a major drawback with id apps is smell. If you’re looking at agaricus for edibility, yellow stainers are distinguished by smell if it’s cold and the staining is less obvious.
Other things they can’t use for id are texture (slimy cap Vs waxy etc.), staining (so you know what to look for – boletes it’s necessary to check for blue staining), brittle gills/stem (does it snap?)… All sorts!
If you want to get into foraging, find a local survival teacher. No matter how many pictures and books you look at, you’ll never learn what wild foods are safe just by reading about them. The only way to really learn is to have someone show you. You need to see it with your own eyes, learn what’s safe to touch and how the safe ones feel in your fingers, and learn the smell and taste of safe food.
Especially mushrooms. Mushrooms are some of the most dangerous foods around. There’s a reason there’s a new hybrid of fruit or vegetable every few years that gets popular, but we only ever eat the same 5 kinds of mushrooms from stores. Some random white mushroom might be perfectly safe, or it might shutdown your organs and kill you three days later.
Honestly it kind of makes me wonder who I am trusting to serve me safe mushrooms.
I guess I always assume food I get from a store (or restaurant) is safe (unless it’s expired or has other obvious issues). And like, most foods have brand recognition of some sort (which may or may not mean anything). But I couldn’t tell you a single company that harvests and sells mushrooms. I mean the blue container makes me feel safe (???) but like…I don’t even know if that’s a marketing thing or just what every company does.
Anyway now I am going to look all of this up.
You’re really trusting your country’s food safety agency.
In most countries, some government entity is responsible for reviewing foods sold in stores for safety, and regulating what the packaging can say. So if a company wants to sell a new kind of mushroom in the stores, they have to get it analyzed for safety.
There’s many mushrooms that are perfectly safe to eat, but don’t taste particularly distinct, don’t store well, or just don’t respond well to farming. There’s also many more that are incredibly toxic, and look very similar to the safe ones.
I’ve done foraging as a hobby, and it’s fun to go out and find things in the woods. However, I pretty much only go for berries and leafy greens, as there’s very few truly dangerous flavors of those in my region. I also generally only take a handful at most, as I don’t want to damage the local ecosystem, or risk a mistake making me seriously ill.
For most people, it’s much safer to just go to the store and buy fresh looking produce. Most places of the world no longer have the habitat to support even a few people going out and regularly foraging for fresh produce. If you do find something you like, you can always take a sample of the fruit and plant it in your garden. This is also great for expanding native plants in your area, which provides food for local wildlife like birds.
I wonder if it’d be easier to grow mushrooms than forage.
https://www.bobvila.com/articles/how-to-grow-mushrooms/
The best place to grow them at home is in a basement or under a sink where they won’t be exposed to bright light. Even apartment dwellers with limited space can grow mushrooms.
Nearly any type of mushroom—including portobello, shiitake, button, oyster, cremini, and enoki—can be grown indoors, but each variety requires a different growing medium. This guide will explore how to grow white button mushrooms, which are actually the same species as cremini and portobello mushrooms.
Only thing that’s required is periodic misting, and I’m sure that one can automate that, maybe use a terrarium mister.
Many species cannot be cultivated. For example chanterelles, truffles, and boletes are symbiotic with tree roots of specific species of trees.
Morels should be able to be cultivated, but if anybody has cracked the code it’s kept secret to keep prices high.
But yeah oyster mushrooms, and a few other wood decay fungi are pretty easy to grow.
Identifying mushroom has always been mysterious to me–how did they differentiate many of the mushroom? Especially I heard that there are so many different mushroom that looks identical
I think the most common advice is, “if you live in an area where this edible mushroom and this impossible-to-tell-apart poisonous lookalike both grow, don’t pick either of them.”
