Large Language Models LLMs Explained

Large language models, or LLMs, are statistical systems trained on enormous quantities of text to predict the most likely next word in a sequence. Every output they produce, from a polished essay to a code snippet to a chatty reply, is the result of repeatedly assigning probabilities to candidate words and sampling from that probability distribution. They do not retrieve a stored answer. They do not look up facts. They generate text one token at a time, guided by patterns learned during training.

This probabilistic core is what makes LLMs powerful and what makes them unreliable in specific, predictable ways. A model that has seen billions of sentences develops a strong sense of how language flows, what arguments usually come next, and which facts tend to appear together. The same training also bakes in the biases of the source text and the gaps in its coverage.

How a Large Language Model Works

An LLM is a neural network, usually based on the transformer architecture introduced in 2017. Training proceeds in two stages: pre-training on a vast text corpus to learn general language patterns, and fine-tuning with human feedback to shape its behaviour for instructions, safety, and helpfulness.

Tokens and Probabilities

The model does not see words. It sees tokens, which are sub-word chunks produced by a tokeniser. For each new token, the network outputs a probability distribution over its entire vocabulary, often around fifty thousand to two hundred thousand possible tokens. The next token is chosen by sampling from this distribution, biased towards higher-probability options. The process repeats until the model emits a stop signal.

This is why LLMs are sometimes called “next-token predictors.” The framing is technically correct and easy to underestimate. Predicting the next token well, over enough text, requires the model to encode grammar, facts, reasoning patterns, and discourse structure.

Mathematical Optimisation

Training is a mathematical optimisation problem. The model has billions of parameters, each a number, and the training algorithm adjusts these numbers to minimise prediction error across the training data. Gradient descent and its variants are the standard methods. The optimisation does not understand meaning; it only minimises a loss function. Meaning, to the extent that LLMs exhibit it, emerges as a side effect of being very good at the prediction task.

The cost of this optimisation is enormous. Training a frontier model takes thousands of GPUs running for weeks and consumes electricity comparable to a small town.

Where Biases Come From

LLM biases are not a separate add-on. They are a direct consequence of the training data and the optimisation target.

Data Biases

The text used to train an LLM is scraped from the web, books, code repositories, and other digital sources. That corpus over-represents English, North American and European perspectives, formal writing styles, and historically dominant viewpoints. Languages with smaller digital footprints, including most Indian languages other than English and Hindi, are under-represented. The model inherits this imbalance.

It also inherits stereotypes encoded in the source text. If most engineers in the training corpus are described as male, the model will tend to assume engineers are male unless explicitly told otherwise. Bias detection and mitigation is an active area of research, but no current method eliminates the problem.

Optimisation and Feedback Biases

The fine-tuning stage uses human raters to teach the model which kinds of responses are preferred. Those raters bring their own preferences for tone, length, hedging, and framing. The model learns to please them. This is useful for safety and helpfulness, but it also encodes the rater pool’s blind spots.

What LLMs Cannot Do

LLMs do not know what they do not know. They generate fluent text whether the underlying facts are correct or not, a behaviour often called hallucination. They cannot reliably perform arithmetic on large numbers without external tools. They cannot access real-time information unless explicitly connected to a search system or database. Their knowledge has a training cut-off, and anything that happened after that date is unknown to them unless retrieved at inference time.

They also do not have persistent memory across conversations by default. Each session starts fresh unless an application explicitly stores and re-supplies past context.

LLMs in India

India is building its own LLM ecosystem. Government-backed efforts include BharatGPT, the IndiaAI Mission’s foundation-model initiative, and university and industry collaborations on Indic-language models. AIRAWAT, the national AI supercomputer, provides training infrastructure. Private companies are releasing models tuned for Hindi, Tamil, Bengali, and other Indian languages, often built on top of open-weight foundations.

The policy debate covers training-data copyright, model safety testing, deepfake misuse, and the energy and water footprint of training. For broader context on this ecosystem, see our explainer on generative AI and LLMs and the AIRAWAT AI supercomputer.

How to Read LLM Output

Treat LLM output as a confident first draft. Useful for structuring an argument, drafting a letter, summarising a long document, or exploring a topic. Not a substitute for primary sources, expert judgement, or a fact-check on anything consequential. Asking the model to cite its sources is helpful only if those citations can be independently verified, since the model may also invent citations.

FAQs

How do large language models generate text?

They predict the next token in a sequence by assigning probabilities to all possible tokens and sampling from that distribution, repeating the process to build a full response.

Are LLMs intelligent?

They are powerful pattern-matchers trained to predict text. Whether that constitutes intelligence is a philosophical question; the practical answer is that they perform well on language tasks but lack grounded understanding, persistent memory, or real-world experience.

Why do LLMs hallucinate?

Because their training rewards fluent, plausible text, not factual accuracy. When the model lacks reliable information on a topic, it still produces fluent output, which can sound confident but be wrong.

What is a transformer?

The transformer is a neural network architecture introduced in 2017 that uses attention mechanisms to process sequences in parallel. It is the backbone of nearly all modern LLMs.

Do LLMs really understand language?

They model the statistical structure of language extremely well. Whether this counts as understanding is contested. They do not have lived experience or sensory grounding.

How much does it cost to train an LLM?

Training a frontier model costs tens of millions to hundreds of millions of dollars, driven mainly by GPU compute, electricity, and engineering time. Smaller models can be trained for far less.

Can LLMs work in Indian languages?

Yes, but quality varies by language. English and Hindi are well-supported in most large models. Tamil, Bengali, Marathi, Telugu, and other Indian languages are improving, especially in models trained or fine-tuned on Indic corpora.

Are LLM outputs biased?

Yes. Biases reflect the training data, the fine-tuning preferences, and the optimisation target. Users should treat output critically, especially on sensitive topics.