Does a pipeline need a feature store?

No; it can obtain inputs in other ways.

Does batch processing guarantee identical results to online processing?

Only when the relevant data, artifacts, and processing semantics agree.

AI Glossary / Graphs and symbolic AI / Advanced

Inference Pipeline

Definition

An inference pipeline is the sequence of processing steps that takes input data, prepares it as needed, applies a trained model, and delivers a prediction or other model output. It may also retrieve features, validate records, transform outputs, and record execution details. The pipeline connects a model to the data and application that make its predictions useful.

What it means

A trained model is one component of a working prediction service. Raw requests may use different units or formats from those expected by the model, and the model's output may require conversion before an application can use it.
An inference pipeline can run as an online service responding to requests or as a batch program processing a collection of records.
The term does not mean a training pipeline. Training fits model parameters; inference applies a prepared model and its associated processing steps.

How it works

Initialization loads the model and any required preprocessing artifacts, configuration, and dependencies. Processing then accepts an input, checks its contract, obtains additional features if needed, and applies the appropriate transformations.
The prepared values are passed to the model. Subsequent steps may reverse a target transformation, select a label, format a response, or store the result. Logging and error handling help reveal where an unsuccessful request or unexpected prediction originated.
A fitted transformation must retain the parameters intended for the deployed model. Applying that transformation is different from fitting it again on each request. A pipeline abstraction can help keep transformations and prediction in the required order, but does not automatically validate every upstream data source or downstream action.

Three practical examples

A document-routing service receives text, tokenizes it with the deployed vocabulary, applies a classifier, and returns the selected queue with the relevant prediction identifier.
An inventory batch process retrieves product features, prepares a fixed input schema, generates demand estimates, and stores them for a specified forecast period.
A building-energy application converts sensor readings into the model's expected units and scales, predicts consumption, and presents the result in an operational unit such as kilowatt-hours.

Think of it this way

Think of a laboratory measurement process. Preparing the sample, using the instrument, converting its reading, and labeling the result all matter. A well-calibrated instrument cannot compensate for a mislabeled sample or an incorrect conversion applied afterward.

Worked example

Consider an invented pipeline predicting a building's next-hour electricity consumption. It accepts a temperature value and an explicit unit, converts the value to Celsius, applies a saved standardization, runs a fixed toy model, and converts the model output to kilowatt-hours.
The saved transformation is z = (temperature_C - 20) / 5. The constants 20 and 5 belong to this deployed model's preparation and are not recomputed from each request.
The toy model returns y_scaled = 1.2 + 0.5z. The output conversion multiplies y_scaled by 10 to obtain predicted kilowatt-hours. These invented coefficients make the stages easy to inspect; they are not a validated energy model.
A request arrives with temperature 86 and unit Fahrenheit. The input conversion gives (86 - 32) × 5/9 = 30 degrees Celsius. Standardization gives z = (30 - 20) / 5 = 2. The model returns 1.2 + 0.5 × 2 = 2.2, and the final conversion produces 22 kilowatt-hours.
The raw request, standardized input, model output, and displayed prediction are therefore distinct objects: 86 Fahrenheit, z = 2, y_scaled = 2.2, and 22 kilowatt-hours.
If the pipeline incorrectly treats 86 as Celsius, z becomes 13.2, the model returns 7.8, and the displayed prediction becomes 78 kilowatt-hours. The model calculation follows its inputs correctly; the error occurred earlier in unit preparation.
If a developer instead replaces the saved centering constant 20 with the current request's converted temperature 30, z becomes zero and the prediction becomes 12 kilowatt-hours. This is a changed transformation, not the intended application of the deployed preprocessing artifact.
An equivalent request of 30 Celsius should still produce 22 kilowatt-hours. A request of 68 Fahrenheit converts to 20 Celsius, giving z = 0 and a prediction of 12 kilowatt-hours. A request with an unsupported unit should follow the pipeline's defined error behavior rather than silently being interpreted as Celsius.
Now consider a separate timing illustration. Suppose sequential validation, feature retrieval, transformation, model execution, and response formatting take 2, 8, 3, 4, and 1 milliseconds. Their sum is 18 milliseconds, excluding any other overhead. Halving model execution to 2 milliseconds reduces that sum to 16 milliseconds, not 9. Optimizing one stage does not halve the whole pipeline's time.
This example assumes sequential stages and fixed illustrative timings. Real requests may involve caching, parallel work, queues, or variable latency, so their end-to-end performance must be measured under the intended workload.

Illustrative case study

Fictional Cedar Energy deploys the same model in a notebook and an online service. Identical physical temperatures produce different predictions because the notebook converts Fahrenheit while the service assumes Celsius.
Cedar creates reference requests covering equivalent units and checks each intermediate value. It packages the saved transformation with the model and gives the output conversion an explicit unit contract.
The team also measures time spent retrieving features. This reveals that accelerating model execution alone addresses only part of the response delay.

Common misconceptions and mistakes

“The pipeline is just the model call.” Preparation and delivery can materially affect results.
“Every pipeline must be online.” Batch prediction is also an inference pipeline.
“Preprocessing should be fitted on each request.” Deployed fitted transformations normally reuse their saved parameters.
“A successful response proves a correct prediction.” The pipeline can execute successfully with incorrect semantics.
“Faster model execution gives the same proportional service speedup.” Other stages contribute to total time.

Business use

Define contracts at each stage and test the complete request-to-output flow. Record compatible model and transformation versions, required feature freshness, and the meaning of returned values. Investigate failures by stage, but judge success at the application boundary. Decide explicitly how the service handles unavailable features, unsupported inputs, and partial batch failures.

Check your understanding

Where did the 78-kilowatt-hour error begin?

Answer: Fahrenheit was treated as Celsius.

Why retain 20 and 5?

Answer: They define the saved input transformation.

Why does halving model time save only 2 milliseconds?

Answer: The other sequential stages still take 14 milliseconds.

Related terms

Preprocessing
Inference Data
Batch inference
Online inference
Latency
Model serving
feature retrieval
postprocessing

Learn next

Trace a request of 77 Fahrenheit through the saved pipeline. Check that the converted input is 25 Celsius and calculate the final prediction.

Sources

Hopsworks MLOps Dictionary, Inference Pipeline: https://www.hopsworks.ai/dictionary/inference-pipeline Scikit-learn documentation, pipelines and composite estimators: https://scikit-learn.org/stable/modules/compose.html#pipeline

Previous: Inference path Next: Inference rules