Must every example have labels for every task?

No. Suitable training designs can use task-specific datasets or mask missing target losses. Missing labels should not be fabricated as observed targets.

Does multitask learning require a neural network?

No. Sharing or coupling can also be built into other model families.

Can an auxiliary task be used only during training?

Yes. It can shape a useful representation even when its output is not needed by the deployed product.

AI Glossary / Mathematics and optimization / Intermediate

Multitask learning

Definition

Multitask learning trains related learning tasks jointly so that information learned for one task can help the others. Tasks may share a representation, selected parameters, or a mathematical relationship between their models. The aim is often better generalization or more efficient use of data, but sharing can also harm a task when the assumed relationship is unsuitable.

What it means

A task is a specific learning problem, such as predicting a message's topic or estimating its urgency. Multitask learning connects these problems during training instead of treating them as wholly unrelated projects.

One common design uses shared neural-network layers followed by separate task-specific output heads. The shared layers learn features that several tasks can use, while each head produces its own prediction. This is one implementation, not the definition of the whole field. Related linear models and other learning methods can also be trained jointly without a neural network.

How it works

Define each task's inputs, targets, loss, and evaluation criteria. Decide which parts of the models should be shared and which should remain separate. With hard parameter sharing, tasks use some identical parameters. With soft sharing, their parameters remain distinct but a penalty or other relationship encourages suitable similarity.

A training objective can combine task losses, such as L=L_A+lambda L_B, where lambda controls the second task's contribution. Loss units, dataset sizes, and task sampling affect that contribution; equal numerical weights do not necessarily mean equal influence.

During training, information from several tasks changes the shared components. Evaluate every important task against an appropriate separately trained baseline. A lower combined loss can hide a decline in one task, and an auxiliary task that helps training need not be exposed as a product feature at prediction time.

Three practical examples

1. Customer messages: jointly learn topic classification and urgency prediction from related text features, while keeping separate outputs and success measures.

2. Equipment analysis: learn to estimate several related sensor quantities from common operating features, with task-specific parameters where their relationships differ.

3. Image understanding: train a shared visual representation for object categorization and another related prediction, provided evaluation supports the chosen amount of sharing.

Think of it this way

Think of students learning related subjects with some shared foundational lessons and some separate specialist lessons. A common foundation can reduce repeated effort, but forcing every subject into the same lesson plan can leave important differences unexplained.

The analogy captures selective sharing. Models do not understand subjects as students do, and a plausible relationship between task names does not prove that their training signals will help each other. That must be checked empirically for the chosen architecture and data.

Worked example

Consider two simple tasks with scalar parameters a and b. Their separate training objectives are 0.5(a−1)² and 0.5(b−3)², so independent training gives a=1 and b=3.

Introduce soft sharing with objective F(a,b)=0.5(a−1)²+0.5(b−3)²+0.5(a−b)². Setting derivatives to zero gives 2a−b=1 and −a+2b=3. Solving gives a=5/3 and b=7/3.

The squared-error terms now contribute 2/9 each, and the sharing penalty contributes 2/9, for total F=2/3. At the independent solution, the two fitting terms are zero but the sharing penalty is two, so that solution is worse for the joint objective. The objective is strictly convex, making the stationary point its unique minimizer.

This example shows how a relationship between tasks changes the learned parameters. It does not prove a prediction improvement: fitting each training target becomes worse. If the targets are noisy estimates of similar underlying quantities, sharing might help on new data; if the true quantities differ, it can introduce harmful bias. With the sharing penalty removed, the separate values return.

Illustrative case study

Fictional case study: a help-desk team trains separate models for request topic and urgency. It experiments with a shared text encoder and two output heads, hoping that useful language patterns can support both tasks.

The team evaluates each output on held-out messages and compares it with its own separate-model baseline. It also examines rare urgent requests rather than relying only on overall averages. If urgency performance declines, the team can reduce sharing, change loss weighting, or keep that task separate. The case illustrates an evaluation process, not an assumed accuracy gain.

Common misconceptions and mistakes

Do not confuse multitask learning with running several independent programs simultaneously. Joint learning requires some route for information or constraints to connect tasks.

Do not assume related tasks always help each other. Negative transfer occurs when joint learning worsens a task's performance.

Do not treat multiple outputs alone as proof of useful sharing. A wrapper that fits independent models can produce many outputs without learning a shared representation. Also distinguish multitask learning from a purely sequential transfer workflow in which one pretrained model is later adapted to another task, although real systems can combine these approaches.

Business use

Multitask learning can be attractive when related products or outputs use overlapping information and some tasks have limited labels. Sharing may reduce duplicated computation or improve data use, but neither result should be presumed.

Define which tasks have priority and what decline, if any, is acceptable for each. Track per-task quality, training effort, serving cost, and the consequences of changing a shared component. A model update that benefits one output can affect all the others, so validation and ownership need to reflect that connection.

Check your understanding

In the worked example, why are a=5/3 and b=7/3 preferred over a=1 and b=3 by the joint objective?

Answer: The joint objective trades individual fit against similarity. The closer pair reduces the sharing penalty enough to offset its larger individual fitting losses.

Related terms

Transfer learning
Auxiliary Tasks
Regularization
shared representation
auxiliary task
parameter sharing
negative transfer
multioutput prediction
loss weighting

Learn next

vary the coupling strength in the two-task example and observe how the parameters approach each other. Then design a comparison that reports separate task results, so a favorable average cannot conceal a decline in an important output.

Sources

Sources: Aalto Dictionary of Machine Learning, multitask learning: https://aaltodictionaryofml.github.io/ADictML_Main.pdf ; Rich Caruana, Multitask Learning: https://www.cs.cornell.edu/~caruana/mlj97.pdf . Primary descriptions of shared learning and the limits of beneficial transfer were read. The coupled quadratic calculation and fictional help-desk case are original illustrations, not reproduced experimental results.

Previous: Multitask Next: Multithreading (computer architecture)