AI Glossary / AI foundations / Foundation

Chat model

Definition

A chat model is an AI language model that has been trained to hold a back-and-forth conversation: it reads a running list of messages and writes the next reply. It is built on a general language model but tuned to follow instructions and respond in dialogue. It is not a search engine or a database, and the term is sometimes used loosely for the chat product wrapped around the model.

What it means

In everyday terms, a chat model is the part of an AI assistant that does the talking. You type a request, it writes a response, you reply, and it writes again. It can draft emails, summarize reports, explain a spreadsheet formula or suggest wording for a policy.

The word chat describes the format of the exchange, not a limit on the work. The same model can be used behind a customer help window, inside a finance tool or in an internal assistant. Some people use chat model and the product name interchangeably. In practice the model is the engine, and the chat window, memory features and connected tools are extras added by the product around it.

How it works

A chat model starts as a large language model trained on a very large body of text to predict what comes next. That base model can continue text well, but it does not naturally behave like an assistant. A second stage of training uses example conversations and human feedback to teach it to follow instructions, answer in a helpful tone and decline some requests. This kind of tuning is described in work on training language models to follow instructions with human feedback.

At use time, the conversation is packaged as a list of messages, each marked with a role. A system message sets overall behavior, user messages carry your requests and assistant messages hold earlier replies. The model reads all of that text and generates the next reply one small piece, called a token, at a time.

The model has no memory between calls on its own. Each time, the product sends the earlier messages again so the model can see them. There is a limit, called the context window, on how much text fits. Long conversations can lose early details, and replies can sound confident while being wrong, so important facts need checking.

Three practical examples

Finance team: A controller pastes a month-end variance table into a chat model and asks for a plain-English summary for the leadership meeting. The model drafts three paragraphs. The controller checks every figure against the source before sending.
Retailer: A store manager uses a chat model to rewrite product descriptions in a consistent voice. She gives a short style guide in the first message and asks for ten descriptions at a time. She edits each one before it goes on the site.
Support desk: An agent asks a chat model to turn a long, angry customer email into a calm summary and a suggested reply. The agent reads the draft, fixes a wrong refund detail and then sends it.

Think of it this way

Think of a chat model as a fast, well-read temporary assistant who writes drafts from whatever you hand them across the desk. The analogy stops working because a human assistant knows when they are guessing and remembers last week, while a chat model can state a guess in the same tone as a fact and only sees what is in the current conversation.

Worked example

A small accounting firm tracks how long a task takes with and without a chat model. Before, writing a client summary letter took 40 minutes. With a chat model producing the first draft and a person editing, it takes 15 minutes to prompt and edit, plus 10 minutes to check the numbers, so 25 minutes in total.

Time saved per letter: 40 minus 25 = 15 minutes. The firm writes 60 letters a month. Monthly time saved: 60 times 15 = 900 minutes, which is 900 divided by 60 = 15 hours. If the firm values an hour of staff time at 50 dollars, the saving is 15 times 50 = 750 dollars a month. The firm should subtract the tool subscription and the time spent learning to use it before calling that a net saving.

Illustrative case study

Fictional case: Harbor Lane Supplies, a made-up regional distributor, gave its sales team a chat model to draft quote follow-up emails. Early on, one draft invented a volume discount that did not exist, and a customer asked the team to honor it. The sales lead paused the rollout and reviewed the drafts. The model had been given no price list, so it filled the gap with a plausible guess. The team fixed this by pasting the current price list and discount rules into the first message of each session, adding a rule that the model must say when information is missing, and requiring a person to approve every email before it is sent. Errors of this kind stopped, and the team kept the time saved on drafting.

Common misconceptions and mistakes

A chat model looks things up in a database of facts. It generates text from patterns learned in training, so it can produce fluent statements that are false, and anything that matters should be checked against a reliable source.

A chat model remembers every past conversation. By default it only sees the messages sent to it in the current exchange, and any memory feature is added by the product around it.

Chat model and chatbot mean the same thing. The chat model is the underlying AI, while a chatbot is the full product, which may add rules, company data and tools around the model.

A chat model understands your business automatically. It knows only general patterns plus whatever you provide in the conversation, so it needs your context, terms and constraints to give useful answers.

Business use

Start with one low-risk, repeatable task such as drafting internal summaries or first-pass customer replies. Write a short instruction template, give the model the facts it needs, and keep a person responsible for review. Compare drafts against your current output for a few weeks before widening use.

Measure time per task, edit rate, error rate found in review and user satisfaction. Costs include subscription or usage fees, staff time for training and review, and the effort of setting rules. Main risks are wrong answers, leaking confidential data into a tool, and over-reliance. Set a clear policy on what data may be entered, and check privacy obligations such as those covered in regulator guidance on AI and data protection.

Check your understanding

What is the difference between a chat model and the chat product around it?

Answer: The chat model is the AI engine that writes replies. The product adds the window, any memory, connected tools and company rules.

In the worked example, how many minutes does each letter save?

Answer: 15 minutes: the old time of 40 minutes minus the new time of 25 minutes (15 to prompt and edit plus 10 to check numbers).

Why did Harbor Lane's model invent a discount?

Answer: It had no price list, so it generated a plausible guess. Supplying the real rules and requiring human approval fixed the problem.

Why does a chat model need earlier messages sent again each turn?

Answer: It has no built-in memory between calls, so the product resends the conversation, within the context window limit, for the model to read.

Related terms

Large language model
Prompt
System prompt
Context window
Token
Hallucination
Fine-tuning
Generative AI

Learn next

Try one chat model on a small, safe task and note where it helps and where it slips. Then read about prompts and context windows to get better results, and about hallucination to understand why checking matters.

Sources

Ouyang et al., Training language models to follow instructions with human feedback: https://arxiv.org/abs/2203.02155 Jurafsky and Martin, Speech and Language Processing: https://web.stanford.edu/~jurafsky/slp3/ Google, Machine Learning Glossary: https://developers.google.com/machine-learning/glossary The sources were consulted for how language models are trained to follow instructions and for standard terminology; the examples, analogy, worked example and case are original educational illustrations. Checked 5 Nov 2026.

Previous: Chat interface Next: Chatbot