This is a thing that is true of all LLMs, but it seems like you’re misunderstanding the core issue. It CAN give outputs like that sometimes. What we CAN’T do is force it to give outputs like that ALL the time.
It will answer “I don’t know” if its predictive text model guesses that the most common response to this would be “I don’t know”. To do that, to simplify a little, you could imagine that it reads your question, compares that to all the text in its training data, and tries to find the conversation that looks most like the question you asked, then answers whatever the person in the training data answered. But your exact question wasn’t in its training data, so if you took that mental model, and instead had it compare to 1000 similar looking things in its training model and average them, then it would hopefully do a better job of coming up with something at least close to what you actually asked. Now take it to a million, or a billion.
When we’re asking questions about the real world, we would prefer for it to answer based on knowledge about the real world. But what if it “matches” data from a work of fiction? Or just someone who doesn’t know what they’re talking about? Or true information, but about a different subject?
It doesn’t know anything. It doesn’t understand anything you say. It just looks at patterns that it learned from the training data and tries to guess what words are most likely to be said in that case. In other words, “here’s one case where it didn’t hallucinate” and “it will never hallucinate” are not the same thing at all.
Edit: To clarify, it doesn’t search its training data to answer your question, so asking “was this in the training data” is impossible. By the time you interact with it, the data is long gone. It was just used for training.
Ok very long and detailed response, i was responding to the initial comments that explicitly said if you give ai a made up thing it will definitely hallucinate. Which i demonstrated to be false in (multiple times). I’m not suggesting it doesn’t hallucinate a lot of the time still, but the comments were making out its 100% broken, and it clearly works for many queries very effectively, despite its limited applications. Im just suggesting we don’t throw the baby out with the bathwater.
I think the trouble is, what baby are we throwing out with the bathwater in this case? We can’t prevent LLMs from hallucinating (but we can mitigate it somewhat with carefully constructed prompts). So, use cases where we’re okay with that are fair game, but any use case (or in this case, law?) that would require the LLM never hallucinates aren’t attainable, and to get back earlier, this particular problem has nothing to do with capitalism.
This is a thing that is true of all LLMs, but it seems like you’re misunderstanding the core issue. It CAN give outputs like that sometimes. What we CAN’T do is force it to give outputs like that ALL the time.
It will answer “I don’t know” if its predictive text model guesses that the most common response to this would be “I don’t know”. To do that, to simplify a little, you could imagine that it reads your question, compares that to all the text in its training data, and tries to find the conversation that looks most like the question you asked, then answers whatever the person in the training data answered. But your exact question wasn’t in its training data, so if you took that mental model, and instead had it compare to 1000 similar looking things in its training model and average them, then it would hopefully do a better job of coming up with something at least close to what you actually asked. Now take it to a million, or a billion.
When we’re asking questions about the real world, we would prefer for it to answer based on knowledge about the real world. But what if it “matches” data from a work of fiction? Or just someone who doesn’t know what they’re talking about? Or true information, but about a different subject?
It doesn’t know anything. It doesn’t understand anything you say. It just looks at patterns that it learned from the training data and tries to guess what words are most likely to be said in that case. In other words, “here’s one case where it didn’t hallucinate” and “it will never hallucinate” are not the same thing at all.
Edit: To clarify, it doesn’t search its training data to answer your question, so asking “was this in the training data” is impossible. By the time you interact with it, the data is long gone. It was just used for training.
Ok very long and detailed response, i was responding to the initial comments that explicitly said if you give ai a made up thing it will definitely hallucinate. Which i demonstrated to be false in (multiple times). I’m not suggesting it doesn’t hallucinate a lot of the time still, but the comments were making out its 100% broken, and it clearly works for many queries very effectively, despite its limited applications. Im just suggesting we don’t throw the baby out with the bathwater.
I think the trouble is, what baby are we throwing out with the bathwater in this case? We can’t prevent LLMs from hallucinating (but we can mitigate it somewhat with carefully constructed prompts). So, use cases where we’re okay with that are fair game, but any use case (or in this case, law?) that would require the LLM never hallucinates aren’t attainable, and to get back earlier, this particular problem has nothing to do with capitalism.