AI Glossary / AI and machine learning / Advanced

Feedback neural network

Definition

A feedback neural network includes connections that allow earlier internal activity or later-layer information to influence subsequent processing. Such feedback can create recurrent state or iterative refinement, unlike a strictly feedforward computation that passes through its layers without internal cycles.

What it means

The network can use information from a previous processing step instead of treating every step as entirely independent. This can support sequence processing, memory-like state or repeated interpretation of an input.

Feedback neural network is a broad description, not one standardized architecture. A recurrent network processing a time series and a network iteratively refining an image interpretation can both involve feedback but use different mechanisms. The exact connections, timing and update rules must be specified to understand the model.

How it works

A state update combines current input with information retained or returned from earlier computation. In a simple recurrent form, the new state is a learned transformation of the input and previous state. Other architectures use feedback between levels or repeated processing passes.

Training adjusts parameters according to an objective. Unrolling repeated updates is one way to analyze or train appropriate recurrent structures. Feedback alone does not guarantee stable behavior, useful long-term memory or better predictions. The update rule, initialization and handling of sequences all affect the result.

Three practical examples

1. A sequence model can use previous sensor readings when interpreting the current reading, helping distinguish a trend from an isolated value.

2. A speech-processing model can retain context across successive pieces of an audio sequence. Its actual memory is limited by architecture and training.

3. A computer-vision research model can refine an interpretation over several internal passes. Extra passes add computation and should be justified by measured improvement. These are architectural possibilities, not universal advantages of feedback.

Think of it this way

Imagine reading a difficult paragraph twice. Your initial interpretation influences how you process it the second time. A feedback architecture can similarly let earlier internal results influence later computation.

The comparison does not imply understanding, consciousness or human memory. It describes information flow. A network can repeatedly reinforce a mistaken interpretation just as easily as improve one if its learned updates are unsuitable.

Worked example

Consider a toy scalar state update h(t) = 0.5 × h(t−1) + x(t), starting with h(0) = 0. For inputs 2, 0 and 0, the successive states are 2, 1 and 0.5.

The first input continues to influence later states even after the new inputs become zero. Its contribution halves at each step in this example. A feedforward rule h(t) = x(t) would instead output 2, 0 and 0, retaining no such state.

This is a simplified linear illustration of recurrence, not a complete trained neural network. Real architectures can use nonlinear transformations, multiple units and gates, so their behavior cannot be inferred from this one coefficient.

Illustrative case study

Fictional equipment-monitoring team PulseWorks compares a stateful model with a model that processes each reading independently. The stateful model appears to recognize gradual changes, but its evaluation unexpectedly improves when unrelated machines are processed consecutively.

An investigation finds that hidden state was not reset between machines. Information from one sequence was contaminating another. The team corrects sequence boundaries and reruns the comparison. The scenario illustrates that feedback introduces state-management responsibilities alongside any potential modeling benefit; an apparently stronger score may reflect a workflow error.

Common misconceptions and mistakes

Feedback connections are not the same thing as backpropagation. Backpropagation is a way to compute training gradients, including in feedforward networks; feedback describes the model's processing connections or state flow.

Human ratings after an AI response also do not automatically make its architecture a feedback neural network. Distinguish external evaluation loops from internal recurrence. Another mistake is assuming repeated processing must converge to a correct answer. Depending on the dynamics, it can stabilize incorrectly, oscillate or amplify errors. Claims about memory should specify what is retained and for how long.

Business use

This term matters when a system uses sequences or iterative refinement. Ask why the task needs retained state, what information is carried between steps and when that state is reset. Compare against a simpler architecture under realistic latency and accuracy requirements.

For operational systems, inspect behavior after missing observations, interruptions and changes in sequence length. Ensure evaluation does not leak information across independent cases. A feedback design can be a useful technical choice, but its business value comes from demonstrated task performance and reliable operation, rather than resemblance to human thinking.

People also ask

Are all recurrent networks feedback networks?

Recurrent architectures involve feedback or retained state, although terminology and exact implementations vary.

Does feedback mean the model learns after every prediction?

No. State can change during inference while trained parameters remain fixed. Learning requires a separate parameter-update process.

Can a feedback model process a single image?

Yes. Some designs use repeated internal refinement of one input rather than a sequence of different external inputs.

Check your understanding

A model carries hidden state from one time step to the next but does not update its weights. Is it necessarily learning new parameters?

Answer: No. It is updating its state during computation. Parameter learning and state evolution are distinct processes, even when both occur in the same overall system.

Related terms

• Recurrent neural network
• Feedforward neural network
• Gated recurrent unit
• Dynamical system
• hidden state
• backpropagation through time
• LSTM
• iterative refinement

Learn next

Trace the toy state update and compare it with a rule that ignores previous state. Then inspect a recurrent model's sequence boundaries, initialization and reset behavior. Distinguish changes in activation state from changes in learned weights.

More in AI and machine learning

• Feed-forward neural network regression - Feed-forward neural network regression is using a standard layered network to predict a number, such as a price, a demand or a risk score, from input features.
• Feedback - Feedback is information about how things went, returned to the system that produced them, so future behavior can adjust: outcomes flowing back to the actor.
• Feedback loop - A feedback loop is the closed circuit in which a system's outputs affect its future inputs: what the system does changes the world it then reads, for better...
• Filters, random fields, and maximum entropy model - The filters, random fields, and maximum entropy model, usually called FRAME, is a probabilistic model developed for texture analysis and synthesis.
• Finite mixture model - A finite mixture model describes data as coming from a fixed number of underlying groups, each with its own distribution, so one dataset can be read as...
• Fitness approximation - Fitness approximation estimates the quality of candidate solutions using a cheaper substitute for an expensive or unavailable evaluation.

Sources

Stephen Grossberg, expert-authored Recurrent Neural Networks overview: https://www.scholarpedia.org/article/Recurrent_neural_networks. Zamir and colleagues, Feedback Networks, CVPR 2017: https://openaccess.thecvf.com/content_cvpr_2017/papers/Zamir_Feedback_Networks_CVPR_2017_paper.pdf. The scalar recurrence and PulseWorks case are original illustrations of information flow and evaluation design. Editorial review on 14 October 2026 encountered direct-access errors at Scholarpedia and CVF; the Scholarpedia indexed introduction and the accessible author manuscript at https://arxiv.org/pdf/1612.09508 were consulted, especially sections 1 and 3 on iterative prediction and hidden-state feedback.

Previous: Feedback Network Next: Feedforward neural network