Why does AI seem so smart?

Executive summary: When you prompt an AI chatbot, the responses can seem magical and written by a human being, but how does it all work? In today's article, I share how AI works, focusing on large language models (LLMs) like ChatGPT, Copilot, Claude, and other chatbots.

Take a moment to think about the following questions:

• What is a butterfly?
• What is Alpha Centauri A?
• What is love?
• What are types of non-verbal communication?
• What is a poem?

With those topics in mind, can you write me a short poem about how a butterfly and Alpha Centauri A fall in love? Write the story in 100 words or less, and finish it in 5 seconds.

ChatGPT did just that for me. It may not win any awards, but it certainly evokes a sense of wonder and optimism (or not, if you believe ChatGPT will kill all art, but that is a topic for another day).

Following is a copy of ChatGPT's response to the prompt Write me a poem about how a butterfly and Alpha Centauri A can communicate their love in 100 words or less:

In a meadow, a delicate butterfly fluttered its wings, sending shimmering patterns into the sky. Far away, Alpha Centauri A saw the glimmer, its light responding with a soft glow that reached the Earth.

“Your wings sparkle like stars,” the distant sun whispered through beams of light.

The butterfly twirled in delight, writing back with its graceful flight. “And you shine like hope in the night,” it replied, tracing hearts in the air.

Though separated by a vast universe, their love was a silent poem—an eternal dance of light and flight, bound together beyond the stars.

Large language models (LLMs) like ChatGPT may contain enough information to create the story, but how do they create the poem and make it seem plausible? I will attempt to cover that in this article with as little jargon as possible.

Scrape it up!

When you search for something in Google, it does not search the web for your request. Instead, it proactively scrapes up the world's data from websites, books, audio, video, and more. When you type your search, it extracts that data from something akin to a database called an index.

All the large language models (LLMs), like ChatGPT, Llama, and Perplexity, also scrape the world's information, but how they process that information and present it is different.

If you scrape, what can you do with it?

I usually look for something specific when I perform a Google search. I live in a dense city and want to research a new car for short commutes and errands. My Google search might look something like the following:

Best new cars for city living

As you can see in the following image, Google presents a list of information contained within its index and includes various options such as ads, videos, and links.

Google's massive index and algorithmic prowess delivered the response to me. Since Google knows a lot about me based on my browsing habits, it probably gave me that list based on additional knowledge, like:

• Where I live
• That I like to watch videos over reading
• Whether or not I own a car

Now, let's prompt the LLM with the same question. In the following ChatGPT screenshot, you will see that the same question results in a thoughtful response. It recognizes the characteristics of a "city living" car (compact, efficient, easy to park) and then lists what it assumes will be necessary for me to know.

The LLM response is a bit more magical because it:

• Knows about cars
• Knows about city living
• Understands the needs of city dwellers, like compactness
• Understands what compactness means in the context of parking a vehicle in a city
• Understands the importance of miles per gallon for city dwellers
• Understands that gas stations tend to be harder to come by in cities

Notice the difference? Google is filtering an extensive list based on what it knows about me. ChatGPT understands the world of cars, driving, cities, parking, and so on, then delivers a thoughtfully written summary for my review.

Read on if you want to know how the LLM can do all this!

First, we train

How AI understands context is similar to how we believe the human brain works. If I show you pictures of dogs and teach you how to pronounce the word "dog," your brain's neurons fire. As you can see in the following image, someone teaching you about dogs gives you a basic understanding, if only conceptually.

When you see a dog for the first time, your brain neurons fire again. With this new interaction, you notice dogs are playful and fluffy, and their tails wag. As you can see in the following image, the LLM processes its knowledge and sees direct connections between various elements, just like we believe the human brain functions.

The next time you encounter a dog, it is outside, and once again, your neurons fire and store more memory. This time, your brain connects dogs with enjoying playing outside. Outside, there are clouds and grass, and the grass is green, and some dogs play in the mud. As you can see in the following image, an LLM looks at its knowledge base many times (something we call layers), and the information about the dog becomes more abstract each time.

In this section, I explain how the LLM builds its knowledge base, using dogs as an example. Most LLMs are trained on world knowledge, which means the library of interconnected things grows into a vast knowledge graph. Recall our first exercise when we created a poem with a love story between a butterfly and Alpha Centauri A. The LLM (ChatGPT) wrote that story because it can draw from the world of our existing knowledge. In the following image, you can see what part of that might look like.

Okay, Bill, you explained how the LLM learns about information, but how does it create the poem?

Now that we know the LLM contains the world's knowledge let's revisit the prompt from the start of this article:
Write me a poem about how a butterfly and Alpha Centauri A can communicate their love in 100 words or less.

While the poem generation might seem magical, a series of algorithms predict the best words to output. I will not get into great detail about the algorithms other than to explain away the magic so you better understand how the LLM works "under the hood," if you will. Here are the algorithmic processes we will cover to round out this article:

• Pattern recognition and contextual understanding
• Word-by-word prediction
• Chain of predictions

🧩 Pattern recognition and contextual understanding

The LLM does not "know" butterflies and stars fall in love, but it predicts what's most plausible based on patterns in its training. The model recognizes that love poems often involve beautiful descriptions and emotional exchanges, and since butterflies are graceful and stars shine, it weaves those elements together.

🔮 Word-by-word prediction

When the LLM selects each word to display in the poem, it evaluates thousands of possibilities. It chooses the one with the highest probability based on everything it knows about butterflies, stars, and love. Here's how it could look at a very high level:

• The model predicts "butterfly" → a butterfly likely "flutters."
• "Flutter" is associated with "wings" → wings create patterns or movement.
• The star is far away → stars emit light, so the star likely "responds" with light.
• We often define love in poetic terms → communication happens via a "soft glow."

⛓️ Chain of predictions

Ultimately, the AI builds the poem through word-by-word predictions, drawing on its vast knowledge of language patterns, metaphors, and associations. It can create something seemingly creative and cohesive, even though it's all based on learned patterns.

As a reminder, here is our prompt: Write me a poem about how a butterfly and Alpha Centauri A can communicate their love in 100 words or less.

In this case, our ChatGPT LLM recognized the patterns in our prompt, used its knowledge to predict each new word, and then chained all those words together to form a cohesive poem. As a reminder, this was the result:

In a meadow, a delicate butterfly fluttered its wings, sending shimmering patterns into the sky. Far away, Alpha Centauri A saw the glimmer, its light responding with a soft glow that reached the Earth.

“Your wings sparkle like stars,” the distant sun whispered through beams of light.

The butterfly twirled in delight, writing back with its graceful flight. “And you shine like hope in the night,” it replied, tracing hearts in the air.

Though separated by a vast universe, their love was a silent poem—an eternal dance of light and flight, bound together beyond the stars.

The next time you prompt an AI chatbot, ponder what you are typing and how it might interpret your request and predict a response. Thinking about that and modifying your prompt can often deliver a better result.

Bill Talks AI Podcast Working Session - Bill Raymond

Meeting purpose Introduce each other and define a podcast topic Document a conversational outline Select a recording date Answer any questions

Calendly

Dr. Deborah Chen provided editing services for this article.