Deep Learning and Natural Language Processing

Large Language Models

Large Language Models for Large Language Problems

Large language models, which we define as deep learning systems designed to perform natural language processing, are an important tool to help us manage the vast amounts of data our society generates.

About fifty years ago, dating back at least to the expectations set by the mutinous computer HAL 9000 in the movie 2001: A Space Odyssey, experts began to promise us large and imminent breakthroughs in the field of artificial intelligence (AI):  Robots, self-driving cars, and computers that could hold a conversation. These promissory notes have only recently begun to pay off. One of those major payoffs has come in the area of neural networks or, as it is now more often called, deep learning.

Minsky and Papert’s 1969 book Perceptrons argued that neural networks were too weak to solve any serious and interesting problems in AI. However, perceptrons are one-layer, linear neural networks. They are the simplest type of neural network, and do not have deep, multiple layers and non-linear connections as deep learners do. The many counterarguments that Perceptrons inspired arguably kick-started the modern era of deep learning.

So Much Data

Humanity is faced with a huge amount of data—more every day. We must tame it: search for what we are interested in, transform it into a useful form, and find connections between pieces of content that may be in vastly separated locations across the web.

As of 2023, humans produce 2.9 trillion bytes of data every second. As generative AI progresses, computers will soon be outpacing that figure. We have many algorithms to help us, and new ones are being created every day. Choosing which algorithms to use with which problems and which types of data is itself a hard problem.

Language is a Tool

Language is a tool that separates us from other animals, and that has made our civilization possible in two ways: First, language allows the transmission and continuation of knowledge and culture from one generation to the next. Second, at an evolutionary timescale, it builds and furthers human cognitive development.

Humanity is, as with so many of its tools, at a critical juncture with language. We are increasingly swamped with linguistic data. Our ability to filter and process it is increasingly diminishing. We are drowning in data. Each second, more information is created than one human could absorb in an entire lifetime, and each second the situation only grows worse.

Tools to address this problem have only begun to be developed. Large language models—deep learning systems for natural language processing—are an important component of our tool set.