Are weights always learned?

No. They may be fixed by design, initialized before training, frozen during some training stages or updated by a learning procedure.

Bias is usually stored separately, but it can be represented mathematically as a weight on an input that is always one.

Do artificial weights fully describe biological synapses?

No. Biological responses involve dynamics and mechanisms beyond a single scalar abstraction.

AI Glossary / Deep learning / Advanced

Synaptic weight

Definition

A synaptic weight describes the strength of influence that one neuron or computational unit has on another. In an artificial neural network, it is typically a numerical parameter multiplying an incoming signal. In biological modeling, it represents aspects of synaptic efficacy. The mathematical parameter is a useful abstraction, not a complete description of a physical synapse.

What it means

An artificial neuron combines inputs according to their weights. A positive weight increases the neuron's weighted sum when its input increases; a negative weight decreases that sum. The resulting activation also depends on the other inputs, bias and activation function.
Weights help determine the behavior learned by a network, but an individual weight rarely corresponds to a self-contained fact or explanation.
The biological terminology reflects an inspiration for the model. It should not imply that a standard artificial network reproduces all the chemistry, timing or adaptation of biological neural communication.

How it works

For a simple artificial unit, the input score is z = w1x1 + w2x2 + b. Each weight scales its corresponding input, the products are added, and the bias shifts the total. An activation function then transforms z.
During gradient-based training, an optimizer adjusts eligible weights in response to the loss. A basic gradient-descent step subtracts the learning rate multiplied by the loss derivative from the current weight.
In neuroscience models, weight changes are described through synaptic plasticity rules. Biological efficacy can vary over different timescales; reducing it to one constant number is a modeling choice. Artificial gradient descent should not be presented as a literal description of all biological learning.

Three practical examples

An image classifier assigns learned weights to signals derived from an illustration. The contribution of one connection depends on the activity reaching it and the transformations that follow.
A small forecasting model multiplies recent activity measures by coefficients. Changing the input units requires corresponding handling of the weights or preprocessing to preserve the prediction.
A researcher simulates a network with a biological plasticity rule and observes how connection strengths evolve. The chosen rule and neuron model define what the simulation represents; its weights cannot automatically be equated with every measured biological quantity.

Think of it this way

Imagine a mixing desk where each incoming channel has a gain control. The final mixture depends on both the gain setting and the signal on that channel. A large gain applied to a silent channel contributes nothing at that moment.
A weight plays a comparable role in a weighted sum. The analogy becomes limited in a deep network because the signals themselves are outputs of other learned transformations, and later nonlinear operations can change their influence again.

Worked example

Take two inputs, x1 = 2 and x2 = 3, with weights w1 = 0.4 and w2 = −0.2, and bias b = 0.1. Their weighted sum is 0.4 × 2 − 0.2 × 3 + 0.1 = 0.3. The separate input contributions are 0.8 and −0.6.
If this illustrative unit has a linear output and target 1, its half-squared-error loss is one-half × (0.3 − 1)² = 0.245. The derivative with respect to w1 is (0.3 − 1) × 2 = −1.4.
For a teaching step that changes only w1 with learning rate 0.1, the new weight is 0.4 − 0.1 × (−1.4) = 0.54. The output becomes 0.58 and the loss becomes 0.0882. This single-example improvement does not demonstrate generalization.
Now express x1 in units one hundred times smaller, so its numerical value becomes 200. A weight of 0.004 preserves the original contribution of 0.8. The weight is one hundredth as large, but that contribution has not changed. Magnitude comparisons without considering scale can therefore mislead.

Illustrative case study

Fictional Willow Analytics inspects a model predicting editing workload. A dashboard ranks inputs by the absolute size of their first-layer weights and labels the largest as the most important.
A reviewer notices that duration is measured in hours while another count is recorded in hundreds. Changing the unit convention alters the apparent ranking without changing equivalent predictions. The team withdraws the simple importance labels and documents preprocessing alongside the weights.
It then evaluates model behavior using appropriate analysis of predictions and realistic input changes. The case illustrates why readable numbers are not automatically readable explanations.

Common misconceptions and mistakes

A weight and an activation are different. The weight is a model parameter; the activation is a value produced for a particular input. Activations can change between examples while trained weights remain fixed.
A negative weight is not evidence that an input is undesirable. It describes a mathematical relationship at a particular connection, not a moral judgment or a causal effect.
A large weight does not necessarily dominate the output. Input scale, other pathways and nonlinearities matter. Similarly, a zero weight removes that direct contribution but does not prove the original feature has no influence elsewhere in the network.

Business use

Weights matter for model storage, updating, reproducibility and analysis. A deployed model normally needs the correct architecture, parameter values and preprocessing together. Copying weights into a system that interprets inputs differently can change behavior substantially.
When reviewing explanations, ask whether the analysis concerns an individual connection, a complete prediction or behavior across a dataset. Those questions require different evidence. For model updates, compare performance on independent data and preserve version information. More changed weights do not automatically mean more learning of useful patterns, and fewer changed weights do not guarantee that behavior remains similar.

Check your understanding

Answer: What is the second input's contribution in the example? −0.2 × 3 = −0.6.

Why does gradient descent increase w1 here?

Answer: Its loss derivative is negative, so subtracting the learning-rate-scaled derivative raises the weight.

Why can changing units change weight magnitude without changing predictions?

Answer: A compensating change in the coefficient preserves the product of input and weight.

Related terms

Neuron
Activation
Bias
Parameter
Gradient descent
Backpropagation
synaptic plasticity
feature scaling

Learn next

Recalculate the example while updating both weights and the bias. Then apply an activation function and examine how its derivative changes the learning calculation. Finally, rescale an input and verify which parameter change preserves the original output.

Sources

Google Machine Learning Crash Course, Neural networks: Nodes and hidden layers: weighted sums and bias. https://developers.google.com/machine-learning/crash-course/neural-networks/nodes-hidden-layers Gerstner and colleagues, Neuronal Dynamics, chapter 19: synaptic efficacy, plasticity and learning rules. https://neuronaldynamics.epfl.ch/online/Ch19.html The numerical example and fictional case are original illustrations.

Previous: Symmetry reduction Next: Synchronization