Why 1.58 instead of two bits?

Three states correspond to log2(3) bits of information. Practical fixed-width storage can still use two bits per code, with other packing schemes possible.

Can an existing model simply be rounded to three values?

It can be quantized experimentally, but useful quality is not guaranteed. Native low-bit training is an important distinction in the cited BitNet work.

Does the model generate only three kinds of output?

No. Ternary weights concern internal computation. The model can still predict tokens from a large vocabulary.

AI Glossary / Generative AI and language / Intermediate

1.58-bit large language model

Definition

A 1.58-bit large language model uses ternary weight values, typically -1, 0 and +1, in its low-precision linear computations. The name comes from log2(3), approximately 1.585 bits, the information needed to distinguish three equally possible states. BitNet b1.58 is a prominent research example.

What it means

Conventional language-model weights often use floating-point representations with many possible values. A ternary representation restricts selected computations to three weight states, reducing the information that must be stored or moved for those weights.

The phrase does not mean that a computer physically stores a fractional bit in each ordinary memory cell. Values can be packed across multiple bits, and practical formats may use two bits per ternary value or another arrangement. Scale factors and other data add overhead.

It also does not mean every calculation, activation, memory allocation or training state uses 1.58 bits. The complete model and runtime contain additional components. Read the architecture and implementation details rather than estimating the entire service from the label alone.

How it works

In the BitNet b1.58 approach, specialized linear layers use a quantization rule that maps weights to ternary codes after scaling. A multiplication by a ternary code can be implemented as keeping, negating or omitting a contribution, with appropriate scaling and accumulation around the operation.

The published approach incorporates low-precision computation during training from scratch. This differs from simply converting an arbitrary, already-trained floating-point model to three values and expecting its quality to remain unchanged.

Activations use their own representation. For example, the published BitNet b1.58 designs described here use eight-bit activations in the relevant linear projections. Other operations and stored quantities have separate requirements.

Efficient deployment needs kernels and packing schemes that exploit the representation on the target hardware. A generic implementation that expands every code into a large floating-point tensor may lose much of the potential benefit. Measure end-to-end memory, speed and quality for the actual workload, including context length and concurrent requests.

Three practical examples

A research team studies whether a natively trained ternary model can meet a defined language task within a limited memory budget.
A local-assistant project benchmarks a compatible inference runtime on its target computer rather than assuming weight precision determines response speed.
A platform team compares ternary, four-bit and floating-point models under the same task, context and resource measurements.

Think of it this way

Think of replacing a fine-grained control dial with three positions: reverse, neutral and forward. A system designed around those positions may work efficiently, while forcing an existing finely tuned mechanism onto them could change its behavior.

The analogy concerns representation and design adaptation. A language model is not a mechanical switchboard, and the three weight states do not correspond to three possible words or three levels of intelligence.

Worked example

Idealized storage is not runtime memory

Suppose an invented weight block contains one billion independent ternary codes. The information-theoretic state count corresponds to one billion times log2(3), approximately 1.585 billion bits, or 198 million bytes before overhead.

A straightforward two-bit-per-code packing instead uses two billion bits, or 250 million bytes. Storing one billion values at 16 bits each uses two billion bytes. These are decimal-byte arithmetic comparisons for the weight block only.

A deployed model also needs scales, other tensors, temporary workspaces and attention caches. Consequently, neither 198 million nor 250 million bytes predicts total runtime memory. The comparison explains the source of potential savings without promising a particular device capacity or speedup.

Illustrative case study

A fictional field-service software team, Cedar Offline, wants an assistant that can operate on company laptops. It sees the 1.58-bit label and initially estimates memory by multiplying the parameter count by that number alone.

A prototype reveals that the chosen export format, context cache and runtime buffers matter substantially. The team also finds that longer prompts change the balance between prompt processing and generated-token performance.

It compares compatible models on representative manuals, checks factual answers and measures the full process on the intended hardware. The final evaluation treats low weight precision as one useful design characteristic. It does not assume that lower precision guarantees lower total cost or sufficient domain performance.

Common misconceptions and mistakes

A 1.58-bit model is ternary, not an ordinary binary model with only two weight states. Marketing and research sometimes group these models under the broader phrase one-bit LLMs, so inspect the stated representation.

Do not infer that training is free or that every internal quantity uses ternary arithmetic. Low inference weight precision does not describe the entire training system.

Published benchmark results demonstrate particular models and configurations. They do not prove that any full-precision model can be converted without loss, that all tasks retain quality or that every processor achieves the same efficiency improvement.

Business use

Evaluate this model family when resource constraints make representation and data movement important. Compare quality, latency, memory and energy under the same operational conditions, with a compatible implementation.

Include integration effort, model licensing, task suitability and maintenance in the decision. Distinguish measured hardware energy from estimated arithmetic energy, and distinguish weight storage from peak process memory. A credible business case names the tested model and runtime rather than attributing universal savings to a bit-width label.

Check your understanding

A model has ternary linear weights but eight-bit activations and additional runtime caches. Is its entire memory footprint equal to parameter count times 1.58 bits?

Answer: No. That calculation describes an idealized weight-code quantity and omits the actual packing, scales, other tensors and runtime storage.

Related terms

Large language model
Inference
Activation
Quantisation
ternary weights
BitNet
model compression
quantisation-aware training
memory bandwidth

Learn next

Compare a ternary code with its packed representation and identify the required scale information. Then distinguish native low-bit training from post-training quantization. Benchmark a complete inference workload before turning theoretical weight savings into a deployment estimate.

Sources

Sources: Ma and colleagues, The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits: https://arxiv.org/abs/2402.17764 Ma and colleagues, BitNet b1.58 2B4T Technical Report: https://arxiv.org/abs/2504.12285 Microsoft, official BitNet inference framework: https://github.com/microsoft/BitNet

Previous: 0/1 loss Next: (1+ε)-approximate nearest neighbor search