What is your brain doing if not statistical text prediction?
The show Westworld portrayed it pretty good. The idea of jumping from text prediction to conscience doesn’t seem that unlikely. It’s basically text prediction on a loop with some exterior inputs to interact.
Free will vs determinism doesn’t have to do with religion.
I do think that the universe is deterministic and that humans (or any other being) do no have free will per se. In the sense that given the same state of the universe at some point the next states are determined and if it were to be repeated the evolution of the state of the universe would be the same.
Nothing to do with religion. Just with things not happening because of nothing, every action is consequence of another action, that includes all our brain impulses. I don’t think there are “souls” outside the state of the matter that could take decisions by themselves with determined.
But this is mostly philosophical of what “free will” means. Is it free will as long as you don’t know that the decision was already made from the very beginning?
Then why you just expressed in a statistical prediction manner?
You saw other people using that kind of language while being derogatory to someone they don’t like on the internet. You saw yourself in the same context and your brain statistically chose to use the same set of words that has been seen the most in this particular context. Literally chatgtp could have been given me your exact same answer if it would have been trained in your same echo chamber.
Have you ever debated with someone from the polar opposite political spectrum and complain that “they just repeat the same propaganda”? Doesn’t it sound like statistical predictions to you? Very simple those, there can be more complex one, but our simplest ways are the ones that define what are the basics of what we are made of.
If you at least would have given me a more complex expression you may had an argument (as humans our process could far more complex an hide a little what we seem to actually be doing it). But in instances like this one, when one person (you) responded with a so obvious statistical prediction on what is needed to be said in a particular complex just made my case. thanks.
But people who agree with my political ideology are considered and intelligent. People who disagree with me are stupider than chatgpt 3.5 and just say the same shit and can’t be reasoned with.
Did you actually read the article or just googled until you find something that reinforced your prestablished opinion to use as a weapon against a person that you don’t even know?
I will actually read it. Probably the only one of us two who would do that.
If it’s convincing I may change my mind. I’m not a radical, like many other people are, and my opinions are subject to change.
They have a conclusion that they’ve come to the conversation with and anything that challenges that gets down voted without consideration.
The assumptions you aren’t allowed to challenge, in order: AI is bad; Computer intelligence will never match or compete with human intelligence; computer intelligence isn’t really intelligence at all, it’s this other thing [insert ‘something’ here like statistical inference or whatever].
“AI is bad” is more of a dictum extending from cultural hedgemony than anything else. It’s an implicit recognition that in many ways, silicon valley culture is an effective looting of the commons, and therefore, one should reject all things that extend from that culture. It’s not a logical or rational argument against AI necessarily, but more of an emotional reaction to the culture which developed it. As a self preservation mechanism this makes some sense, but obviously, it’s not slowing down the AI takeover of all things (which is really just putting highlighter on a broader point that silicon valley tech companies were already in control of major aspects of our lives).
Computer intelligence never match human intelligence is usually some combination of goal post moving, or a redefining of intelligence on the fly (this I’ve specifically presented for the third critique, because it warrants addressing). This is an old trope that goes back almost to the beginning of computer intelligence (it’s not clear to me our definitions of machine intelligence are very relevant). It quite litterally started with multiplying large numbers. Then, for literally decades, things like chess and strategy, forwards facing notions in time were held up as some thing only “intelligent systems could do”. Then post deep blue, that got relegated to very clever programmers and we changed intelligence to be something about learning. Then systems like Alpha go etc came about, where they basically learned the rules to the game by playing, and we relegated those systems to ‘domain specific’ intelligences. So in this critique you are expected to accept and confirm the moving of goalposts around machine intelligence.
Finally, it’s the "what computers do isn’t intelligence, it’s some_other_thing.exe™. In the history of machine intelligence, that some other thing has been counting very quickly, having large-ish memory banks, statistical inference, memorization, etc. The biggest issues with this critique, and when you scratch and sniff it, you very quickly catch an aroma of Chomsky leather chair (and more so if we’re talking about LLMs), and maybe even a censer of a Catholic Church. The idea that humans are fundementally different and in some way special is frankly, fundemental, to most western idealogies in a way we don’t really discuss in the context of this conversation. But the concept of spirit, and that there is something “entirely unique” about humans versus “all of the rest of everything” is at the root of Abrahamic traditions and therefore also at the root of a significant portion of global culture. In many places in the world, it’s still heretical to imply that human beings are no more special or unique than the oak or the capibara or flatworm or dinoflagellate. This assumption, I think, is on great display with Chomsky’s academic work on the concept of the LAD, or language acquisition device.
Chomsky gets a huge amount of credit for shaking up linguistics, but what we don’t often talk about, is how effectively, his entire academic career got relinquished to the dust bin, or at least is now in that pile of papers where we’re not sure if we should “save or throw away”. Specifically, much of Chomsky’s work was predicted on the identification of something in humans which would be called a language acquisition device or LAD. And that this LAD would be found in as a region in human brains and would explain how humans gain language. And just very quickly notice the overall shape of this argument. It’s as old as the Egyptians in at least trying to find the “seat of the soul”, and follows through abrahamism as well. What LLMs did that basically shattered this nothing was show at least one case where no special device was necessary to acquire language; where in fact no human components at all were necessary other than a large corpus of training data; that maybe languages and the very idea of language or language acquisition are not special or unique to humans. LLMs don’t specifically address the issue of a LAD, but they go a step farther in not needing to. Chomsky spent the last of verbal days effectively defending this wrong notion he had (which has already been addressed in neuroscience and linguistics literature), which is an interesting and bitter irony for a linguist, specifically against LLMs.
To make the point more directly, we lack a good coherent testable definition of human intelligence, which makes any comparisons to machine intelligence somewhat arbitrary and contrived, often to support the interlocutors assumptions. Machine intelligence may get dismissed as statistical inference, sure, but then why can you remember things sometimes but not others? Why do you perform better when you are well rested and well fed versus tired and hungry, if not for there being an underlying distribution of neurons, some of which are ready to go, and some of which are a bit spent and maybe need a nap?
And so I would advocate caution about investing heavily into a conversation where these assumptions are being made. It’s probably not going to be a satisfying conversation because almost assuredly they assumptee hasn’t dove very deeply into these matters. And look at the downvote ratio. It’s rampant on Lemmy. Lemmy’s very much victim to it’s pack mentality and dog piling nature.
Funny to me how defensive you got so quick, accusing of not reading the linked paper before even reading it yourself.
The reason OP was so rude is that your very premise of “what is the brain doing if not statistical text prediction” is completely wrong and you don’t even consider it could be. You cite a TV show as a source of how it might be. Your concept of what artificial intelligence is comes from media and not science, and is not founded in reality.
The brain uses words to describe thoughts, the words are not actually the thoughts themselves.
Think about small children who haven’t learned language yet, do those brains still do “stastical text prediction” despite not having words to predict?
What about dogs and cats and other “less intelligent” creatures, they don’t use any words but we still can teach them to understand ideas. You don’t need to utter a single word, not even a sound, to train a dog to sit. Are they doing “statistical text prediction” ?
Read other replies I gave on your same subject. I don’t want to repeat myself.
But words DO define thoughts, and I gave several examples. Some of them with kids. Precisely in kids you can see how language precedes actual thoughts.
Dogs and cats are not proven to be conscious as a human being is. Precisely due the lack of an articulate language.
Your analogy to mechanical systems are exactly where the breakdown to comparison with the human brain occurs, our brains are not like that, we don’t only have the blocks of text loaded into us, sure we only learn what we get exposed to but that doesn’t mean we can’t think of things we haven’t learned about.
The article I linked talks about the separation between the formation of thoughts and those thoughts being translated into words for linguistics.
The fact that you “don’t even know why the how the brain creates an articulated spoken word is even relevant here” speaks volumes to how much you understand the human brain, particularly in the context of artificial intelligence actually understanding the words it generates and the implications of thoughts behind the words and not just guessing which word comes next based on other words, the meanings of which are irrelevant.
I can listen to a song long enough to learn the words, that doesn’t mean I know what the song is about.
but that doesn’t mean we can’t think of things we haven’t learned about.
Can you think of a colour have you never seen? Could you imagine the colour green if you had never seen it?
The creative process is more modification than creation. taking some inputs, mixing them with other inputs and having an output that has parts of all out inputs, does it sound familiar? But without those input seems impossible to create an output.
And thus the importance of language in an actual intelligent consciousness. Without language the brain could only do direct modifications of the natural inputs, of external inputs. But with language the brain can take an external input, then transform it into a “language output” and immediately take that “language output” and read it as an input, process it, and go on. I think that’s the core concept that makes humans different from any other species, this middle thing that we can use to dialogue with ourselves and push our minds further. Not every human may have a constant inner monologue, but every human is capable to talking to themself, and will probably do when making a decision. Without language (language could take many forms, not just spoken language, but the more complex feels like it would be the better) I don’t know how this self influence process could take place.
It’s a basic argument of generative complexity, I found the article some years ago while trying to find an earlier one (I don’t think by the same author) that argued along the same complexity lines, essentially saying that if we worked like AI folks think we do we’d need to be so and so much trillion parameters and our brains would be the size of planets. That article talked about the need for context switching in generating (we don’t have access to our cooking skills while playing sportsball), this article talks about the necessity to be able to learn how to learn. Not just at the “adjust learning rate” level, but mechanisms that change the resulting coding, thereby creating different such contexts, or at least that’s where I see the connection between those two. In essence: To get to AGI we need AIs which can develop their own topology.
As to “rudeness”: Make sure to never visit the Netherlands. Usually how this goes is that I link the article and the AI faithful I pointed it out to goes on a denial spree… because if they a) are actually into the topic, not just bystanders and b) did not have some psychological need to believe (including “my retirement savings are in AI stock”) they c) would’ve come across the general argument themselves during their technological research. Or came up with it themselves, I’ve also seen examples of that: If you have a good intuition about complexity (and many programmers do) it’s not unlikely a shower thought to have. Not as fleshed out as in the article, of course.
That seems a very reasonable approach on the impossibility to achieve AGI with current models…
The first concept I was already kind of thinking about. Current LLM are incredibly inefficient. And it seems to be some theoretical barrier in efficiency that no model has been able to surpass. Giving that same answer that with the current model they would probably need to have trillions of parameters just to stop hallucinating. Not to say that to give them the ability to do more things that just answering question. As this supposedly AGI, even if only worked with word, it would need to be able to do more “types of conversations” that just being the answerer in a question-answer dialog.
But I had not thought of the need of repurpose the same are of the brain (biological or artificial) for doing different task on the go, if I have understood correctly. And it seems pretty clear that current models are unable to do that.
Though I still think that an intelligent consciousness could emerge from a loop of generative “thoughts”, the most important of those probably being language.
Getting a little poetical. I don’t think that the phrase is “I think therefore I am”, but “I can think ‘I think therefore I am’ therefore I am”.
Though I still think that an intelligent consciousness could emerge from a loop of generative “thoughts”, the most important of those probably being language.
Does a dog have the Buddha nature?
…meaning to say: Just because you happen to have the habit of identifying your consciousness with language (that’s TBH where the “stuck in your head” thing came from) doesn’t mean that language is necessary, or even a component of, consciousness, instead of merely an object of consciousness. And neither is consciousness necessary to do many things, e.g. I’m perfectly able to stop at a pedestrian light while lost in thought.
I don’t think that the phrase is “I think therefore I am”, but “I can think ‘I think therefore I am’ therefore I am”.
What Descartes actually was getting at is “I can’t doubt that I doubt, therefore, at least my doubt exists”. He had a bit of an existential crisis. Unsolicited Advice has a video about it.
But when I think of how to define a consciousness and divert it from instinct or reactiveness (like stopping at a red light). I think that something that makes a conscience a conscience must be that a conscience is able to modify itself without external influence.
A dog may be able to fully react and learn how to react with the exterior. But can it modify itself the way human brain can?
A human being can sit alone in a room and start processing information by itself in a loop and completely change that flux of information onto something different, even changing the brain in the process.
For this to happen I think some form of language, some form of “speak to yourself” is needed. Some way for the brain to generate an output that can be immediately be taken as input.
At this point of course this is far more philosophical than technical. And maybe even semantics of “what is a conscience”.
Human brains also do processing of audio, video, self learning, feelings, and many more that are definitely not statistical text. There are even people without “inner monologue” that function just fine
Some research does use LLM in combination with other AI to get better results overall, but purely LLM isn’t going to work.
But language is a big thing in the human intelligence and consciousness.
I don’t know, and I would assume that anyone’s really know. But people without internal monologue I have a feeling that they have it but they are not aware of it. Or maybe they talk so much that all the monologue is external.
Interesting you focus on language. Because that’s exactly what LLMs cannot understand. There’s no LLM that actually has a concept of the meaning of words. Here’s an excellent essay illustrating my point.
The fundamental problem is that deep learning ignores a core finding of cognitive science: sophisticated use of language relies upon world models and abstract representations. Systems like LLMs, which train on text-only data and use statistical learning to predict words, cannot understand language for two key reasons: first, even with vast scale, their training and data do not have the required information; and second, LLMs lack the world-modeling and symbolic reasoning systems that underpin the most important aspects of human language.
The data that LLMs rely upon has a fundamental problem: it is entirely linguistic. All LMs receive are streams of symbols detached from their referents, and all they can do is find predictive patterns in those streams. But critically, understanding language requires having a grasp of the situation in the external world, representing other agents with their emotions and motivations, and connecting all of these factors to syntactic structures and semantic terms. Since LLMs rely solely on text data that is not grounded in any external or extra-linguistic representation, the models are stuck within the system of language, and thus cannot understand it. This is the symbol grounding problem: with access to just formal symbol system, one cannot figure out what these symbols are connected to outside the system (Harnad, 1990). Syntax alone is not enough to infer semantics. Training on just the form of language can allow LLMs to leverage artifacts in the data, but “cannot in principle lead to the learning of meaning” (Bender & Koller, 2020). Without any extralinguistic grounding, LLMs will inevitably misuse words, fail to pick up communicative intents, and misunderstand language.
But this “concepts” of things are built on the relation and iteration of this concepts with our brain.
A baby doesn’t born knowing that a table is a table. But they see a table, their parents say the word table, and they end up imprinting that what they have to say when they see that thing is the word table. That then they can relation with other things they know. I’ve watched some kids grow and learn how to talk lately and it’s pretty evident how repetition precedes understanding. Many kids will just repeat words that they parents said in certain situation when they happen to be in the same situation. It’s pretty obvious with small kids. But it’s a behavior you can also see a lot with adults, just repeating something they heard once they see that particular words fit the context
Also it’s interesting that language can actually influence the way concepts are constructed in the brain. For instance ancient greeks saw blue and green as the same colour, because they did only have one word for both colours.
I’m not sure if you’re disagreeing with the essay or not? But in any case what you’re describing is in the same vein, that is simply repeating a word without knowing what it actually means in context is exactly what LLMs do. They can get pretty good at getting it right most of the times but without actually being able to learn the concept and context of ‘table’ they will never be able to use it correctly 100% of the time. Or even more importantly for AGI apply reason and critical thinking. Much like a child repeating a word without much clue what it actually means.
Just for fun, this is what Gemini has to say:
Here’s a breakdown of why this “parrot-like” behavior hinders true AI:
Lack of Conceptual Grounding: LLMs excel at statistical associations. They learn to predict the next word in a sequence based on massive amounts of text data. However, this doesn’t translate to understanding the underlying meaning or implications of those words.
Limited Generalization: A child learning “table” can apply that knowledge to various scenarios – a dining table, a coffee table, a work table. LLMs struggle to generalize, often getting tripped up by subtle shifts in context or nuanced language.
Inability for Reasoning and Critical Thinking: True intelligence involves not just recognizing patterns but also applying logic, identifying cause and effect, and drawing inferences. LLMs, while impressive in their own right, fall short in these areas.
I mostly agree with it. What I’m saying is the understanding of the words come from the self dialogue made of those same words. How many times has a baby to repeat the word “mom” until they understand what a mother is? I think that without that previous repetition the more complex "understanding is impossible. That human understanding of concepts, especially the more complex concepts that make us humans, come from we being able to have a dialogue with ourselves and with other humans.
But this dialogue initiates as a Parrot, non-intelligent animals with brains that are very similar to ours are parrots. Small children are parrots (are even some adults). But it seems that after being a Parrot for some time it comes the ability to become an Human. That parrot is needed, and it also keeps itself in our consciousness. If you don’t put a lot of effort in your thoughts and says you’ll see that the Parrot is there, that you just express the most appropriate answer for that situation giving what you know.
The “understanding” of concepts seems just like a complex and big interconnection of Neural-Network-like outputs of different things (words, images, smells, sounds…). But language keeps feeling like the more important of those things for intelligent consciousness.
I have yet to read another article that other user posted that explained why the jump from Parrot to Human is impossible in current AI architecture. But at a glance it seems valid. But that does not invalidate the idea of Parrots being the genesis of Humans. Just that a different architecture is needed, and not in the statistical answer department, the article I was linked was more about size and topology of the “brain”.
A baby doesn’t learn concepts by repeating words over and certainly knows what a mother is before it has any label or language to articulate the concept. The label gets associated with the concept later and is not purely by parroting and indeed excessive parroting normally indicates speech development issues.
language is a big thing in the human intelligence and consciousness.
But an LLM isn’t actually language. It’s numbers that represent tokens that build words. It doesn’t have the concept of a table, just the numerical weighting of other tokens related to “tab” & “le”.
I don’t know how to tell you this. But your brain does not have words imprinted in it…
The concept of this is, funnily enough, something that is being studied that derived from language. For instance ancient greeks did not distinguish between green and blue, as both colours had the same word.
Words are not imprinted, they are a series of electrical impulses that we learn over time. As a reference about the complain that LLM does not have words but tokens that represent values within the network.
And those impulses and how we generate them while we think are of great importance on out consciousness.
What is your brain doing if not statistical text prediction?
The show Westworld portrayed it pretty good. The idea of jumping from text prediction to conscience doesn’t seem that unlikely. It’s basically text prediction on a loop with some exterior inputs to interact.
It’s “free will”. They chose to say what they wanted.
At least this is what the old religions teach. I don’t know what AI preachers you’re learning this nonsense from.
Church?
Free will vs determinism doesn’t have to do with religion.
I do think that the universe is deterministic and that humans (or any other being) do no have free will per se. In the sense that given the same state of the universe at some point the next states are determined and if it were to be repeated the evolution of the state of the universe would be the same.
Nothing to do with religion. Just with things not happening because of nothing, every action is consequence of another action, that includes all our brain impulses. I don’t think there are “souls” outside the state of the matter that could take decisions by themselves with determined.
But this is mostly philosophical of what “free will” means. Is it free will as long as you don’t know that the decision was already made from the very beginning?
Um, something wrong with your brain buddy? Because that’s definitely not at all how mine works.
Then why you just expressed in a statistical prediction manner?
You saw other people using that kind of language while being derogatory to someone they don’t like on the internet. You saw yourself in the same context and your brain statistically chose to use the same set of words that has been seen the most in this particular context. Literally chatgtp could have been given me your exact same answer if it would have been trained in your same echo chamber.
Have you ever debated with someone from the polar opposite political spectrum and complain that “they just repeat the same propaganda”? Doesn’t it sound like statistical predictions to you? Very simple those, there can be more complex one, but our simplest ways are the ones that define what are the basics of what we are made of.
If you at least would have given me a more complex expression you may had an argument (as humans our process could far more complex an hide a little what we seem to actually be doing it). But in instances like this one, when one person (you) responded with a so obvious statistical prediction on what is needed to be said in a particular complex just made my case. thanks.
But people who agree with my political ideology are considered and intelligent. People who disagree with me are stupider than chatgpt 3.5 and just say the same shit and can’t be reasoned with.
ok buddy
How to tell me you’re stuck in your head terminally online without telling me you’re stuck in your head terminally online.
But have something more to read.
Why being so rude?
Did you actually read the article or just googled until you find something that reinforced your prestablished opinion to use as a weapon against a person that you don’t even know?
I will actually read it. Probably the only one of us two who would do that.
If it’s convincing I may change my mind. I’m not a radical, like many other people are, and my opinions are subject to change.
They have a conclusion that they’ve come to the conversation with and anything that challenges that gets down voted without consideration.
The assumptions you aren’t allowed to challenge, in order: AI is bad; Computer intelligence will never match or compete with human intelligence; computer intelligence isn’t really intelligence at all, it’s this other thing [insert ‘something’ here like statistical inference or whatever].
“AI is bad” is more of a dictum extending from cultural hedgemony than anything else. It’s an implicit recognition that in many ways, silicon valley culture is an effective looting of the commons, and therefore, one should reject all things that extend from that culture. It’s not a logical or rational argument against AI necessarily, but more of an emotional reaction to the culture which developed it. As a self preservation mechanism this makes some sense, but obviously, it’s not slowing down the AI takeover of all things (which is really just putting highlighter on a broader point that silicon valley tech companies were already in control of major aspects of our lives).
Computer intelligence never match human intelligence is usually some combination of goal post moving, or a redefining of intelligence on the fly (this I’ve specifically presented for the third critique, because it warrants addressing). This is an old trope that goes back almost to the beginning of computer intelligence (it’s not clear to me our definitions of machine intelligence are very relevant). It quite litterally started with multiplying large numbers. Then, for literally decades, things like chess and strategy, forwards facing notions in time were held up as some thing only “intelligent systems could do”. Then post deep blue, that got relegated to very clever programmers and we changed intelligence to be something about learning. Then systems like Alpha go etc came about, where they basically learned the rules to the game by playing, and we relegated those systems to ‘domain specific’ intelligences. So in this critique you are expected to accept and confirm the moving of goalposts around machine intelligence.
Finally, it’s the "what computers do isn’t intelligence, it’s some_other_thing.exe™. In the history of machine intelligence, that some other thing has been counting very quickly, having large-ish memory banks, statistical inference, memorization, etc. The biggest issues with this critique, and when you scratch and sniff it, you very quickly catch an aroma of Chomsky leather chair (and more so if we’re talking about LLMs), and maybe even a censer of a Catholic Church. The idea that humans are fundementally different and in some way special is frankly, fundemental, to most western idealogies in a way we don’t really discuss in the context of this conversation. But the concept of spirit, and that there is something “entirely unique” about humans versus “all of the rest of everything” is at the root of Abrahamic traditions and therefore also at the root of a significant portion of global culture. In many places in the world, it’s still heretical to imply that human beings are no more special or unique than the oak or the capibara or flatworm or dinoflagellate. This assumption, I think, is on great display with Chomsky’s academic work on the concept of the LAD, or language acquisition device.
Chomsky gets a huge amount of credit for shaking up linguistics, but what we don’t often talk about, is how effectively, his entire academic career got relinquished to the dust bin, or at least is now in that pile of papers where we’re not sure if we should “save or throw away”. Specifically, much of Chomsky’s work was predicted on the identification of something in humans which would be called a language acquisition device or LAD. And that this LAD would be found in as a region in human brains and would explain how humans gain language. And just very quickly notice the overall shape of this argument. It’s as old as the Egyptians in at least trying to find the “seat of the soul”, and follows through abrahamism as well. What LLMs did that basically shattered this nothing was show at least one case where no special device was necessary to acquire language; where in fact no human components at all were necessary other than a large corpus of training data; that maybe languages and the very idea of language or language acquisition are not special or unique to humans. LLMs don’t specifically address the issue of a LAD, but they go a step farther in not needing to. Chomsky spent the last of verbal days effectively defending this wrong notion he had (which has already been addressed in neuroscience and linguistics literature), which is an interesting and bitter irony for a linguist, specifically against LLMs.
To make the point more directly, we lack a good coherent testable definition of human intelligence, which makes any comparisons to machine intelligence somewhat arbitrary and contrived, often to support the interlocutors assumptions. Machine intelligence may get dismissed as statistical inference, sure, but then why can you remember things sometimes but not others? Why do you perform better when you are well rested and well fed versus tired and hungry, if not for there being an underlying distribution of neurons, some of which are ready to go, and some of which are a bit spent and maybe need a nap?
And so I would advocate caution about investing heavily into a conversation where these assumptions are being made. It’s probably not going to be a satisfying conversation because almost assuredly they assumptee hasn’t dove very deeply into these matters. And look at the downvote ratio. It’s rampant on Lemmy. Lemmy’s very much victim to it’s pack mentality and dog piling nature.
Funny to me how defensive you got so quick, accusing of not reading the linked paper before even reading it yourself.
The reason OP was so rude is that your very premise of “what is the brain doing if not statistical text prediction” is completely wrong and you don’t even consider it could be. You cite a TV show as a source of how it might be. Your concept of what artificial intelligence is comes from media and not science, and is not founded in reality.
The brain uses words to describe thoughts, the words are not actually the thoughts themselves.
https://advances.massgeneral.org/neuro/journal.aspx?id=1096
Think about small children who haven’t learned language yet, do those brains still do “stastical text prediction” despite not having words to predict?
What about dogs and cats and other “less intelligent” creatures, they don’t use any words but we still can teach them to understand ideas. You don’t need to utter a single word, not even a sound, to train a dog to sit. Are they doing “statistical text prediction” ?
So agi is statistical emotion prediction we then assign logic to
Read other replies I gave on your same subject. I don’t want to repeat myself.
But words DO define thoughts, and I gave several examples. Some of them with kids. Precisely in kids you can see how language precedes actual thoughts.
Dogs and cats are not proven to be conscious as a human being is. Precisely due the lack of an articulate language.
Your analogy to mechanical systems are exactly where the breakdown to comparison with the human brain occurs, our brains are not like that, we don’t only have the blocks of text loaded into us, sure we only learn what we get exposed to but that doesn’t mean we can’t think of things we haven’t learned about.
The article I linked talks about the separation between the formation of thoughts and those thoughts being translated into words for linguistics.
The fact that you “don’t even know why the how the brain creates an articulated spoken word is even relevant here” speaks volumes to how much you understand the human brain, particularly in the context of artificial intelligence actually understanding the words it generates and the implications of thoughts behind the words and not just guessing which word comes next based on other words, the meanings of which are irrelevant.
I can listen to a song long enough to learn the words, that doesn’t mean I know what the song is about.
Can you think of a colour have you never seen? Could you imagine the colour green if you had never seen it?
The creative process is more modification than creation. taking some inputs, mixing them with other inputs and having an output that has parts of all out inputs, does it sound familiar? But without those input seems impossible to create an output.
And thus the importance of language in an actual intelligent consciousness. Without language the brain could only do direct modifications of the natural inputs, of external inputs. But with language the brain can take an external input, then transform it into a “language output” and immediately take that “language output” and read it as an input, process it, and go on. I think that’s the core concept that makes humans different from any other species, this middle thing that we can use to dialogue with ourselves and push our minds further. Not every human may have a constant inner monologue, but every human is capable to talking to themself, and will probably do when making a decision. Without language (language could take many forms, not just spoken language, but the more complex feels like it would be the better) I don’t know how this self influence process could take place.
It’s a basic argument of generative complexity, I found the article some years ago while trying to find an earlier one (I don’t think by the same author) that argued along the same complexity lines, essentially saying that if we worked like AI folks think we do we’d need to be so and so much trillion parameters and our brains would be the size of planets. That article talked about the need for context switching in generating (we don’t have access to our cooking skills while playing sportsball), this article talks about the necessity to be able to learn how to learn. Not just at the “adjust learning rate” level, but mechanisms that change the resulting coding, thereby creating different such contexts, or at least that’s where I see the connection between those two. In essence: To get to AGI we need AIs which can develop their own topology.
As to “rudeness”: Make sure to never visit the Netherlands. Usually how this goes is that I link the article and the AI faithful I pointed it out to goes on a denial spree… because if they a) are actually into the topic, not just bystanders and b) did not have some psychological need to believe (including “my retirement savings are in AI stock”) they c) would’ve come across the general argument themselves during their technological research. Or came up with it themselves, I’ve also seen examples of that: If you have a good intuition about complexity (and many programmers do) it’s not unlikely a shower thought to have. Not as fleshed out as in the article, of course.
That seems a very reasonable approach on the impossibility to achieve AGI with current models…
The first concept I was already kind of thinking about. Current LLM are incredibly inefficient. And it seems to be some theoretical barrier in efficiency that no model has been able to surpass. Giving that same answer that with the current model they would probably need to have trillions of parameters just to stop hallucinating. Not to say that to give them the ability to do more things that just answering question. As this supposedly AGI, even if only worked with word, it would need to be able to do more “types of conversations” that just being the answerer in a question-answer dialog.
But I had not thought of the need of repurpose the same are of the brain (biological or artificial) for doing different task on the go, if I have understood correctly. And it seems pretty clear that current models are unable to do that.
Though I still think that an intelligent consciousness could emerge from a loop of generative “thoughts”, the most important of those probably being language.
Getting a little poetical. I don’t think that the phrase is “I think therefore I am”, but “I can think ‘I think therefore I am’ therefore I am”.
Does a dog have the Buddha nature?
…meaning to say: Just because you happen to have the habit of identifying your consciousness with language (that’s TBH where the “stuck in your head” thing came from) doesn’t mean that language is necessary, or even a component of, consciousness, instead of merely an object of consciousness. And neither is consciousness necessary to do many things, e.g. I’m perfectly able to stop at a pedestrian light while lost in thought.
What Descartes actually was getting at is “I can’t doubt that I doubt, therefore, at least my doubt exists”. He had a bit of an existential crisis. Unsolicited Advice has a video about it.
It may be because of the habit.
But when I think of how to define a consciousness and divert it from instinct or reactiveness (like stopping at a red light). I think that something that makes a conscience a conscience must be that a conscience is able to modify itself without external influence.
A dog may be able to fully react and learn how to react with the exterior. But can it modify itself the way human brain can?
A human being can sit alone in a room and start processing information by itself in a loop and completely change that flux of information onto something different, even changing the brain in the process.
For this to happen I think some form of language, some form of “speak to yourself” is needed. Some way for the brain to generate an output that can be immediately be taken as input.
At this point of course this is far more philosophical than technical. And maybe even semantics of “what is a conscience”.
Human brains also do processing of audio, video, self learning, feelings, and many more that are definitely not statistical text. There are even people without “inner monologue” that function just fine
Some research does use LLM in combination with other AI to get better results overall, but purely LLM isn’t going to work.
Yep, of course. We do more things.
But language is a big thing in the human intelligence and consciousness.
I don’t know, and I would assume that anyone’s really know. But people without internal monologue I have a feeling that they have it but they are not aware of it. Or maybe they talk so much that all the monologue is external.
Interesting you focus on language. Because that’s exactly what LLMs cannot understand. There’s no LLM that actually has a concept of the meaning of words. Here’s an excellent essay illustrating my point.
But this “concepts” of things are built on the relation and iteration of this concepts with our brain.
A baby doesn’t born knowing that a table is a table. But they see a table, their parents say the word table, and they end up imprinting that what they have to say when they see that thing is the word table. That then they can relation with other things they know. I’ve watched some kids grow and learn how to talk lately and it’s pretty evident how repetition precedes understanding. Many kids will just repeat words that they parents said in certain situation when they happen to be in the same situation. It’s pretty obvious with small kids. But it’s a behavior you can also see a lot with adults, just repeating something they heard once they see that particular words fit the context
Also it’s interesting that language can actually influence the way concepts are constructed in the brain. For instance ancient greeks saw blue and green as the same colour, because they did only have one word for both colours.
I’m not sure if you’re disagreeing with the essay or not? But in any case what you’re describing is in the same vein, that is simply repeating a word without knowing what it actually means in context is exactly what LLMs do. They can get pretty good at getting it right most of the times but without actually being able to learn the concept and context of ‘table’ they will never be able to use it correctly 100% of the time. Or even more importantly for AGI apply reason and critical thinking. Much like a child repeating a word without much clue what it actually means.
Just for fun, this is what Gemini has to say:
I mostly agree with it. What I’m saying is the understanding of the words come from the self dialogue made of those same words. How many times has a baby to repeat the word “mom” until they understand what a mother is? I think that without that previous repetition the more complex "understanding is impossible. That human understanding of concepts, especially the more complex concepts that make us humans, come from we being able to have a dialogue with ourselves and with other humans. But this dialogue initiates as a Parrot, non-intelligent animals with brains that are very similar to ours are parrots. Small children are parrots (are even some adults). But it seems that after being a Parrot for some time it comes the ability to become an Human. That parrot is needed, and it also keeps itself in our consciousness. If you don’t put a lot of effort in your thoughts and says you’ll see that the Parrot is there, that you just express the most appropriate answer for that situation giving what you know.
The “understanding” of concepts seems just like a complex and big interconnection of Neural-Network-like outputs of different things (words, images, smells, sounds…). But language keeps feeling like the more important of those things for intelligent consciousness.
I have yet to read another article that other user posted that explained why the jump from Parrot to Human is impossible in current AI architecture. But at a glance it seems valid. But that does not invalidate the idea of Parrots being the genesis of Humans. Just that a different architecture is needed, and not in the statistical answer department, the article I was linked was more about size and topology of the “brain”.
A baby doesn’t learn concepts by repeating words over and certainly knows what a mother is before it has any label or language to articulate the concept. The label gets associated with the concept later and is not purely by parroting and indeed excessive parroting normally indicates speech development issues.
But an LLM isn’t actually language. It’s numbers that represent tokens that build words. It doesn’t have the concept of a table, just the numerical weighting of other tokens related to “tab” & “le”.
I don’t know how to tell you this. But your brain does not have words imprinted in it…
The concept of this is, funnily enough, something that is being studied that derived from language. For instance ancient greeks did not distinguish between green and blue, as both colours had the same word.
You said
You also said
You need to pick an argument and stick to it.
what do you not understand?
Words are not imprinted, they are a series of electrical impulses that we learn over time. As a reference about the complain that LLM does not have words but tokens that represent values within the network.
And those impulses and how we generate them while we think are of great importance on out consciousness.
So, are words important to the human brain or not? You are not consistent.