Does ChatGPT understand the language? | venture beat

Systems that consume massive amounts of human creations to emulate our behaviors are advancing at the forefront of AI. Dall-E and Stable Diffusion learned to generate images from prompts, allowing us to see interpretations of superheroes as Renoir might have painted them.

Large Language Models (LLMs) perform a similar language trick, throwing out realistic press releases or code reviews like performed by a pirate. The most recent iteration, ChatGPT, attracted immense attention as people explored the extent of its capabilities in endlessly fun variations.

So what does all this mean for progress towards true AI? Is this the real deal? A step back from that? Or a simple parrot, our human artifacts merely reflected back at us through the mirror of funhouse code?

Gangsters and sonnets alike

What this last class of algorithms clearly demonstrates is a combinatorial understanding of concepts. When asked to explain computer science algorithms as a smartass from a 1940s gangster movie, ChatGPT accomplishes both tasks. We can vary the character he talks to or the topic we want to talk about.

In another example, Alan Turing in his article “Imitation Game” introduced the topic of the Turing Test, where computers can be said to think when a human investigator cannot distinguish them from a human. Turing gave as an example the request to “write a sonnet on the subject of Forth Bridge”. When I presented the task to the GPT, its response included this stanza:

“The bridge is grandiose, a symbol of its time,

A beacon for the proud Scottish people,

It stands today, a testimony of pride,

Reminds us of the days when dreams were high.”

Not all rhymes and meters worked – remember that GPT never he heard a sound, but inferred these concepts from predicting words into sentences – but clearly struggled to build iambic pentameter and follow the proper rhyme scheme. Stayed on topic. It was written poetically.

attractive cognitive skills

In my limited search, I couldn’t find any previous use of “dreams were loud” as a metaphor (just people complaining about being woken up by their dreams). It’s an obvious, relatively superficial metaphor, but it’s genuine.

We can point to the many poems that fed into GPT-3 and question what is truly new in their output. But if the building blocks are known, the intersections are unique and new. And putting together known building blocks into new patterns is an attractive cognitive skill.

Although the volumes of training data involved are enormous, the regularities have all been discovered by these networks – the rules of sonnets and limericks, the linguistic peculiarities of pirate-ese. Programmers did not carefully generate training sets for each task. The models found the rules independently.

Where is GPT-3 missing? The above stanza is suitable as poetry, but it does not surprise or challenge us. When he imitates a pirate, he doesn’t add new nuances to the role. GPT-3 was trained to approximate the most likely words in sentences. We can push it towards more random outputs – not the most likely one, but the fifth most likely one – but it strongly follows the trail of what has been said over and over again.

He can explain familiar tasks well, but struggles to give new suggestions and solutions. It lacks goals, its own momentum. It lacks a meaningful distinction between what is true and what is likely to be said. Does not have long-term memory: Generating an article is possible, but a book does not fit in its context.

More nuanced language comprehension

With each new scaling factor of language models and each new research paper just off the press, we observe a more nuanced understanding of language. Your outputs get more varied and your abilities more extensive. He uses the language in increasingly obscure and technical domains. But the limitations and the tendency to banality persist.

I am more and more convinced of how powerful self-attention is as a neural network concept for finding patterns in a complex world. On the other hand, the gaps in computer understanding become clearer compared to the rapid improvement in so many areas.

Looking at GPT’s understanding of pronouns in semantically ambiguous situations, its sense of humor or its complex sentence structures, I would say that even the current version is sufficient for general language understanding. But there is some other algorithm not yet invented, or at least a particular combination of existing algorithms and training tasks that are needed to approach real intelligence.

Understanding the language: identifying meaningful patterns

Returning to the starting point: whether the unscientific wonder of seeing a Shakespearean sonnet rise from the dust of simple word prediction tasks or the constant erosion of the human gap in countless tasks to probe the depth of artificial understanding of language, language models in use today are not just a parlor trick. These processes not only mimic human language, but find the meaningful patterns within it – be it syntactic, semantic or pragmatic.

However, there’s something else going on in our head, even if it’s just the same techniques self-applied on another level of abstraction. Without some clever new technique, we’ll just keep banging our heads against the limitations of our awesome tools. And who can say when the ray of inspiration will fall there?

So no, the real AIs haven’t arrived yet. But we are significantly closer than before, and I predict that when that happens, some variation of self-awareness and contrastive learning will be a significant part of that solution.

Paul Barba is Chief Scientist at Lexalytics, an InMoment company.