Is WALS the same as ordinary weighted least squares?

It repeatedly uses weighted least-squares subproblems inside a factorization procedure rather than fitting one fixed design matrix once.

Does each update need gradient descent?

No. The conditional problem can be solved through suitable linear-algebra methods.

Does alternating guarantee a global optimum?

Not generally for the joint nonconvex factorization problem.

AI Glossary / Mathematics and optimization / Intermediate

Weighted Alternating Least Squares

Definition

Weighted Alternating Least Squares, or WALS, is an algorithm for fitting matrix-factorization models by repeatedly solving weighted least-squares problems for one group of factors while keeping the other fixed. In recommendation systems, the weights can give observed interactions and unobserved entries different influence. Each conditional fit can be convex even though the complete problem of fitting both factor matrices together is generally nonconvex.

What it means

A factorization model scores a user-item pair by combining a user vector with an item vector. Training tries to make these scores agree with specified targets according to a loss.
Weighted means that different entries contribute different amounts to the squared-error objective. Alternating means first adjusting user factors with item factors fixed, then adjusting item factors with user factors fixed. Least squares describes the conditional fitting problem.
In an implicit-feedback setup, an unobserved interaction may receive a zero target with low confidence rather than being treated as a definite negative preference. That target and weight are modeling choices; they do not establish that a user saw and rejected the item.

How it works

One common objective sums c[i,j](p[i,j]−u[i]·v[j])^2 over selected or modeled user-item pairs, then adds a regularization penalty on the factors. The nonnegative c values are confidence or importance weights, while p values are training targets.
With item factors fixed, each user's parameters can be found by solving a weighted linear least-squares subproblem, often with a quadratic penalty. With user factors fixed, perform the corresponding item updates. Repeat according to a stated stopping rule.
Exact block minimization cannot increase that fixed objective, but decreasing objective values do not alone prove convergence of every parameter or arrival at the global optimum. Initialization, regularization, numerical solution quality, and evaluation design still matter.

Three practical examples

A recommendation prototype can assign greater confidence to observed activity than to missing interactions when fitting user and item factors.
A system can update user factors against a fixed set of item factors as part of a defined alternating training procedure.
A researcher can compare confidence-weighting schemes on held-out recommendation tasks, checking how they affect popular items, infrequent users, and overall ranking behavior.

Think of it this way

Think of adjusting two sets of interlocking measurements: hold one set steady while fitting the other, then switch roles. Unequal weights indicate which mismatches matter more in each fit. The analogy captures alternating adjustment, but it does not imply the two sets have one uniquely correct arrangement or that repeated fitting always reaches the best possible joint solution.

Worked example

Consider one scalar user factor u while two scalar item factors are fixed at v1=1 and v2=2. The targets are p1=4 for an observed entry and p2=0 for a missing entry. Give them weights 1 and 0.25, and use a user regularization term u^2.
The conditional objective, omitting terms constant in u, is J(u)=(4−u)^2+0.25(0−2u)^2+u^2. Expanding gives J(u)=16−8u+3u^2. Its derivative is −8+6u, so the unique minimizer is u=4/3.
The fitted scores are 4/3 and 8/3. The missing-entry score is not forced to zero because the shared factor must balance both targets and the penalty. At u=0, J=16; at u=4/3, J=32/3, approximately 10.667.
If the missing-entry weight becomes 1 instead, the coefficient of u^2 becomes 6, yielding u=2/3. Stronger pressure toward the missing entry's zero target changes both scores.
This is one conditional user update, not a complete alternating run. An item update would next hold the current user factors fixed and solve the corresponding item subproblems. The numbers illustrate the objective's mechanics rather than a measured recommendation improvement.

Illustrative case study

In a fictional course-recommendation project, most courses have never been shown to most learners. An initial factorization treats every absent interaction as equally reliable evidence of disinterest, which heavily influences its fit.
The team revisits the confidence scheme and separates absence of exposure from observed behavior where the available data permit. It compares alternatives on a realistic held-out task rather than choosing the scheme with the smallest incomparable training number.
The lesson is that weighting encodes assumptions about evidence. Alternating least squares solves the chosen objective; it does not decide whether missing activity represents preference.

Common misconceptions and mistakes

Do not confuse confidence weights with the learned coordinates in user and item vectors. They play different roles in the objective.
Do not call the joint factorization a convex problem simply because each fixed-factor update is convex. The product of unknown factors changes the joint geometry.
Do not assume missing values must always be zero targets or always be ignored. Their treatment depends on the task and data.
Also, changing the number of modeled missing entries, their weights, or the scale of the loss can change the effective strength of regularization. Compare implementations using the complete objective, including normalization conventions.

Business use

WALS can provide a structured way to train recommendation factors, but operational value depends on the recommendation task, data, and serving design. Record how targets, weights, and regularization are constructed so training is reproducible.
Evaluate ranking quality and relevant user groups, and check cold-start behavior separately. A reduction in weighted training error is evidence that the selected fitting objective improved, not proof that users will find the recommendations more useful or that business outcomes will increase.

Check your understanding

If item factors are held fixed during a user update, are both factor matrices being optimized simultaneously?

Answer: No. The current subproblem changes the user factors only. A later step changes item factors while holding the updated user factors fixed.

Related terms

Matrix factorization
Weighted least squares
User matrix
Item matrix
Regularization
alternating least squares
implicit feedback
confidence weight
nonconvex optimization

Learn next

Re-derive the scalar update in the worked example with several missing-entry weights. Then write down a tiny full objective and identify which terms change in each alternating step. Study evaluation of implicit-feedback recommendations before interpreting a zero target as a known dislike.

Sources

Google Machine Learning Glossary, Weighted Alternating Least Squares. https://developers.google.com/machine-learning/glossary#weighted-alternating-least-squares-wals Google Recommendation Systems, Matrix factorization: weighted objectives and alternating optimization. https://developers.google.com/machine-learning/recommendation/collaborative/matrix

Previous: Weighted A* search Next: Weighted average