Should weights always sum to the sample size?

No; survey expansion weights may sum approximately to a population total.

Does weighting guarantee representativeness?

No; it relies on the available information and assumptions.

Can a zero weight exclude a case from a weighted calculation?

Often, but software behavior and downstream processing should be checked.

AI Glossary / Statistics and probability / Intermediate

Sample weighting

Definition

Sample weighting assigns different numerical contributions to observations when calculating estimates, fitting models, or evaluating performance. A weight may represent an observation's sampling probability, frequency, importance, or another explicitly defined role. Weighting changes how evidence contributes to a result; the meaning and statistical treatment of the weights must match the purpose of the analysis.

What it means

An unweighted calculation gives every recorded observation the same influence. A weighted calculation can make one observation count more or less than another for a stated reason.
In a survey, a sampled unit may represent several population units because of the design. In machine learning, a weight may emphasize errors on a selected class. These uses can share similar arithmetic while answering different questions. A class-balancing weight is not automatically a valid population survey weight.

How it works

Identify the target quantity or learning objective before choosing weights. Estimating a population proportion, minimizing a cost-sensitive training loss, and summarizing repeated identical observations require different interpretations.
For probability samples, a basic design weight is the inverse of a unit's inclusion probability. A unit selected with probability 0.1 receives design weight 10. Additional adjustments may address nonresponse or align estimates with relevant known totals, subject to their assumptions.
For a weighted mean, calculate sum(wi xi)/sum(wi), typically using nonnegative weights with a positive total. For a weighted population-total estimator, the sum itself may be the target. Normalizing weights changes a total but leaves a weighted mean unchanged.
In model training, a weighted objective may average weighted per-observation losses. Confirm that the chosen estimator actually accepts and uses sample weights as intended. Pipeline components, metrics, and regularization may use weight normalization differently.
Distinguish frequency weights from other weights. An integer frequency weight can represent repeated identical observations in some calculations, but a survey weight does not create that many independent measured units. Variance formulas cannot generally treat expanded population counts as actual sample size.
Inspect the weight distribution. Very large or uneven weights can make results depend heavily on a few observations and can reduce precision under common conditions. Trimming or capping weights changes the bias - variance tradeoff and should be justified and documented.
Use uncertainty estimation appropriate to the design. Survey strata, clusters, inclusion probabilities, and replicate weights may all matter. Merely supplying a generic weight column to a standard model does not guarantee valid standard errors.
Evaluate whether weighting addresses the actual problem. It cannot supply observations for unsupported groups without assumptions, remove every unmeasured confounder, or correct inaccurate labels automatically. Compare weighted and unweighted summaries to understand influence, while keeping the intended target clear.

Three practical examples

Survey estimation: a business survey uses design weights after sampling small and large firms at different rates.
AI training: a classifier gives specified examples greater loss weight to reflect a defined learning objective.
Evaluation: analysts weight test cases to approximate a stated deployment mixture, checking whether the available cases adequately cover that mixture.

Think of it this way

Think of each observation carrying a clearly labeled amount of influence. The label may say how many population units it represents or how costly its error is considered. The arithmetic can look similar, but those labels mean different things and cannot be swapped without changing the question.

Worked example

A fictional population contains 800 small establishments and 200 large establishments. A stratified survey independently selects 100 from each group by simple random sampling within the group, with complete response. Design weights are 800/100 = 8 for small establishments and 200/100 = 2 for large establishments.
Suppose 30 sampled small establishments and 60 sampled large establishments report using a tool. The unweighted sample proportion is (30+60)/200 = 45%.
The weighted estimated user total is 30×8+60×2 = 360. The weights sum to 100×8+100×2 = 1,000, giving an estimated population proportion of 360/1,000 = 36%.
The difference reflects the deliberate overrepresentation of large establishments in the sample. The observed within-group rates are still estimates. Correct weighting under this design does not make the 36% an exactly known population value or eliminate other possible measurement problems.

Illustrative case study

Fictional case: Cypress Analytics compares AI adoption across firms using a survey that oversamples large employers. Its first dashboard reports the raw respondent percentage as the market-wide figure.
The team applies the documented design weights, reports group-specific results, and explains the change in the aggregate. For a separate classifier-training task, it defines a new weighting objective rather than reusing the survey weights without considering the model's purpose.

Common misconceptions and mistakes

Weights are not all interchangeable: design, frequency, precision, importance, and class weights have different meanings.
A weight of 100 does not mean 100 independent observations were collected.
Equalizing class influence does not prove fair or calibrated predictions.
Weighted estimates can have more sampling variability than unweighted ones, depending on the design and target.
Multiplying all weights by a constant leaves a weighted mean unchanged, but may affect software objectives when regularization or normalization is handled differently.

Business use

Use weighting when there is a defensible reason for unequal contributions. Keep the weight's origin, formula, normalization, and intended analysis documented. In AI and survey dashboards, clearly label whether a score describes the observed sample, a target population, or a chosen cost-sensitive objective, since each can produce a different valid number.

Check your understanding

Why is the example's unweighted percentage larger?

Answer: Large establishments, which have the higher observed usage rate, make up half the sample but only one-fifth of the population.

If all weights are doubled, does the weighted proportion change?

Answer: No; both numerator and denominator double.

Related terms

Calibration
Effective sample size
Weighted mean
inclusion probability
design weight
class imbalance
importance weighting

Learn next

Recompute a simple estimate with documented weights and inspect which observations influence the change most. Then verify that the uncertainty calculation and software interpretation match the weight's purpose.

Sources

Statistics Canada, Weighting: https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch6/5214809-eng.htm scikit-learn, compute_sample_weight: https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_sample_weight.html The stratified population example and case are original fictional illustrations.

Previous: Sample variance Next: Sampled softmax