How AI Actually Works

How AI Actually Works

Artificial intelligence in 2024 is simultaneously more impressive and more limited than most coverage suggests. Understanding the actual mechanisms — not the marketing — allows you to use these tools effectively and make sound decisions about where they will and will not change your work and industry.

What a Neural Network Actually Is

A neural network is a mathematical function with many parameters — numbers that can be adjusted. During training, the network sees an input (a piece of text, an image, a data point), produces an output, compares that output to the correct answer, calculates the error, and adjusts its parameters slightly to reduce that error. This process — called gradient descent — is repeated billions of times across a massive dataset.

The result is a network of parameters whose collective values capture statistical patterns in the training data. The network does not store facts like a database; it encodes patterns like a compressed statistical model of everything it saw during training.

The key insight: An LLM generates text by repeatedly predicting: "given all the text so far in this conversation, what token most likely comes next?" It does this based on patterns in its training data. When it produces a correct answer, it is because correct answers to similar questions appeared frequently in its training data. When it hallucinates, it is producing a statistically plausible-looking answer that happens to be false.

The Architecture Behind Modern AI

Transformer architecture: The breakthrough that made modern LLMs possible was the transformer, introduced by Google researchers in 2017. Transformers process entire sequences of text in parallel (rather than sequentially, like earlier models) and use an "attention mechanism" to weigh which parts of the input are most relevant to each output token.

This architecture scales efficiently — bigger models trained on more data consistently perform better, which is why AI labs have been competing to build ever-larger models. GPT-3 had 175 billion parameters; GPT-4's parameter count is undisclosed but estimated at over a trillion.

Context window: The amount of text an LLM can "see" at once is its context window. Early models had windows of 2,000-4,000 tokens. Current frontier models have windows of 100,000-1,000,000+ tokens, allowing them to process entire books or long codebases in a single pass.

Multimodality: Models are no longer limited to text. Current frontier models process images, audio, video, and structured data alongside text, enabling capabilities like reading charts, describing images, and generating code from screenshots of user interfaces.

What AI Is and Is Not Good At

AI is genuinely excellent at:

• Drafting, editing, and reformatting text
• Summarizing large amounts of information
• Writing and debugging code (with human verification)
• Translating between languages
• Answering well-documented factual questions
• Pattern recognition in images and structured data
• Generating variations and alternatives

AI consistently fails at:

• Reliable retrieval of specific facts (hallucination risk)
• Tasks requiring genuine causal understanding
• Consistently correct complex multi-step reasoning
• Knowing what it does not know (calibrated uncertainty)
• Tasks requiring real-time or post-training information (without retrieval tools)

The Hype vs. Reality Gap

AGI timelines: Artificial General Intelligence — a system that can learn and perform any intellectual task a human can — is frequently predicted to arrive within years. The honest answer is that no one knows, and the challenges remaining are not merely engineering ones but fundamental scientific ones about the nature of intelligence.

"AI will take all jobs": AI will automate specific tasks within jobs, changing what work involves, at varying speeds across industries. Complete job elimination typically requires automating every task that role involves, plus the social and organizational changes to restructure work around that automation. This happens far more slowly than technology demonstrations suggest.

The cost barrier: Running frontier AI models at scale is extremely expensive — in compute, energy, and infrastructure. The competitive advantage of large-scale AI currently concentrates in the hands of a few companies with the capital to afford it, not democratically available to all.