Intricate metallic surface resembling a microchip or cityscape. Image credit: Photo by Zach M on Unsplash (CC0)
Large language models (LLMs), such as Claude and ChatGPT, can generate lengthy, grammatically correct responses to a wide range of queries. But does this mean they truly understand language in the same way as humans? Or are they more like sophisticated artificial parrots, combining learned patterns to mimic human language without comprehending its underlying meaning?
On the one hand, LLMs learn by adjusting the weights of connections between artificial neurons in a large neural network, a process that shares some similarities with learning in biological neural networks. On the other hand, LLMs require vastly more input than human infants to develop language abilities. They also learn primarily by predicting the next word based on preceding words, without the social and physical interactions that shape language learning in humans. Given these similarities and differences, the extent to which LLMs process language like humans remains an open question.
To investigate this question, we focused on how LLMs process individual sentences. Specifically, we asked whether they primarily respond to the words themselves or are also sensitive to the order of words and their grammatical roles within a sentence. For example, ‘dog bites man’ and ‘man bites dog’ contain the same words but convey very different meanings. Many previous studies of LLMs have not adequately disentangled the effects of word meaning from those of sentence structure, making it difficult to determine whether these models process sentence structure in a human-like way.
We developed a set of sentences designed to separate the effects of word meaning from those of sentence structure. We then used functional magnetic resonance imaging (fMRI) to measure brain activity in participants as they read these sentences and compared the results with the internal representations generated by LLMs processing the same sentences. We found that LLMs were sensitive to sentence structure, but substantially less so than humans. Human brain activity changed considerably when participants read ‘dog bites man’ rather than ‘man bites dog’. In contrast, the internal representations of LLMs changed much less between the two sentences.
Our findings provide evidence that LLMs process language differently from humans, suggesting that their language processing may not fully reflect human-like understanding. This research has potential applications in two areas. First, insights into differences between human and artificial language processing could inform the development of LLM architectures that more closely resemble human language processing, potentially helping to reduce their unpredictable or unexpected behaviours. Second, a better understanding of how the human brain processes language could help improve treatments for language impairments following stroke or brain surgery. Nevertheless, our research primarily addresses a fundamental scientific question, and further work is needed to determine whether, and to what extent, LLMs truly understand human language.
