I’m not sure if you’re disagreeing with the essay or not? But in any case what you’re describing is in the same vein, that is simply repeating a word without knowing what it actually means in context is exactly what LLMs do. They can get pretty good at getting it right most of the times but without actually being able to learn the concept and context of ‘table’ they will never be able to use it correctly 100% of the time. Or even more importantly for AGI apply reason and critical thinking. Much like a child repeating a word without much clue what it actually means.
Just for fun, this is what Gemini has to say:
Here’s a breakdown of why this “parrot-like” behavior hinders true AI:
- Lack of Conceptual Grounding: LLMs excel at statistical associations. They learn to predict the next word in a sequence based on massive amounts of text data. However, this doesn’t translate to understanding the underlying meaning or implications of those words.
- Limited Generalization: A child learning “table” can apply that knowledge to various scenarios – a dining table, a coffee table, a work table. LLMs struggle to generalize, often getting tripped up by subtle shifts in context or nuanced language.
- Inability for Reasoning and Critical Thinking: True intelligence involves not just recognizing patterns but also applying logic, identifying cause and effect, and drawing inferences. LLMs, while impressive in their own right, fall short in these areas.
It’s like my brain is not dissimilar from the toddler I have at home. If I don’t take charge, come up with t activities, give it a routine, feed it properly, make sure it is hydrated and well slept it turns into a bored, bossy, moody asshole. Of course they, both toddler and brain, still do sometimes but if I do all the things above I’m usually prepared. Sometimes they also are super good at doing all the things by themselves.
In some ways, while extremely exhausting, having a toddler is actually great for my brain because otherwise I would not eat well, make plans, follow routines, go out, or have fun. My brain is still unhappy of course because it wants to do nothing and eat “chips with sugar” (to quote this toddler being I have at home).