Can separate fits give the same coefficients?

Yes, under the common-design conditions described above.

Is a prediction for a new observation more uncertain than its fitted mean?

Under the example's model, yes.

AI Glossary / Statistics and probability / Advanced

Multivariate regression

Definition

Multivariate regression models several response variables using one or more predictors. In multivariate linear regression, each response has a linear mean equation, and the model can describe relationships among the response errors.

What it means

Multiple responses distinguish multivariate regression from a regression with only one response and several predictors. The latter is usually called multiple or multivariable regression. A model with several responses and several predictors is both.

How it works

A common linear formulation is Y=XB+E. Rows correspond to observations, columns of Y to responses, columns of X to predictors including any intercept, and B contains the response-specific coefficients.
With the same design matrix and observations for every response, unrestricted ordinary least-squares coefficients equal those obtained by fitting each response separately. The joint formulation supports inference about combinations of coefficients and responses using their covariance. It does not automatically change the point predictions or improve accuracy.

Three practical examples

A fictional service team predicts preparation and execution times from request features, then examines the uncertainty of their total.
A fictional training study relates two assessment scores to a shared set of participant characteristics.
A fictional sensor project predicts two calibration responses from operating conditions while checking whether residuals remain related.

Think of it this way

Two forecasting worksheets may use the same input columns but predict different outputs. Joining them into a multivariate model preserves their separate equations while allowing questions about how their errors and predictions move together.

Worked example

Consider an original five-observation dataset with predictor values x=-2, -1, 0, 1, and 2. The first response values, in that order, are 7, 6, 12, 10, and 15. The second response values are 24, 20, 18, 18, and 20.
Fit an intercept and a slope for each response using the same five rows. Because the predictor values sum to zero, each intercept is the corresponding response mean: ten for the first response and twenty for the second.
The sum of squared predictor values is ten. The sum of x times the first response is twenty, giving slope 20/10=2. The sum of x times the second response is minus ten, giving slope -10/10=-1.
The fitted mean equations are therefore Y1-hat=10+2x and Y2-hat=20-x. The coefficient matrix has an intercept row (10, 20) and a slope row (2, -1).
The first fitted response values are 6, 8, 10, 12, and 14. Its residuals are 1, -2, 2, -2, and 1. The second fitted values are 22, 21, 20, 19, and 18, with residuals 2, -1, -2, -1, and 2.
Each residual column sums to zero, and each has zero cross-product with x. These are checks of the least-squares fit with an intercept and this predictor.
The sum of squared residuals is 14 for each response. The sum of cross-products between the two residual columns is four. With five rows and two fitted coefficients per response, the residual degrees of freedom are three.
Under the usual independent-row, common-error-covariance linear model, the residual covariance estimate therefore has rows (14/3, 4/3) and (4/3, 14/3). The estimated residual correlation is (4/3)/(14/3)=2/7, approximately 0.286.
Fitting the two regressions separately gives the same coefficients in this unrestricted common-design example. What the separate coefficient lists do not provide by themselves is the cross-response residual covariance needed for joint calculations.
At x=1, the predicted response means are 12 and 19, so the predicted mean of their sum is 31. The actual responses at that observed row are ten and eighteen, illustrating that fitted means need not equal observations.
For the fitted mean vector at x=1, the usual covariance multiplier is one fifth plus one squared divided by ten, or 0.3. Multiplying the residual covariance estimate by 0.3 gives diagonal entries 1.4 and off-diagonal entries 0.4.
The estimated variance of the fitted mean sum is therefore 1.4+1.4+2 times 0.4=3.6. Its estimated standard error is sqrt(3.6), approximately 1.897. Ignoring the cross-response covariance would instead give variance 2.8, understating uncertainty for this sum.
For an independent new observation at the same predictor value, uncertainty also includes its new response errors. The estimated error variance of the sum is 14/3+14/3+2 times 4/3=12. Adding fitted-mean uncertainty gives an estimated prediction variance of 15.6, substantially larger than 3.6.
This distinction separates uncertainty about a conditional mean from uncertainty about an individual future outcome. The tiny invented dataset is for arithmetic, not evidence that these uncertainty estimates would be reliable for operational use.
At x=3, the equations predict 16 and 17. That predictor value lies outside the observed range from minus two to two. The equations can be evaluated there, but their numerical availability does not validate extrapolation.
Finally, deleting a row from only one response's fit would give the two equations different estimation samples. The simple common-design equivalence and the residual-pair calculations above could no longer be applied unchanged. Missing outcomes require an explicit handling strategy.

Illustrative case study

Fictional company Willow Services fits preparation and execution equations using matched request records. Analysts initially see that the coefficients match separate regressions and wonder what the joint model adds.
They then examine the combined-duration question and document how residual covariance affects uncertainty. Before deployment, they evaluate later requests and check the predictor range and error patterns. This fictional case illustrates the importance of the intended joint calculation.

Common misconceptions and mistakes

Several predictors with one response do not constitute multivariate regression in the sense used here.
A joint fit does not necessarily alter separate least-squares coefficients.
Residual covariance matters when combining responses.
A mean prediction interval and an individual prediction interval answer different questions.
Regression associations alone do not establish causal effects.

Business use

Specify which outcomes and combinations matter to the decision. In the fictional service project, validate preparation time, execution time, and their total. Report the data range and missing-data handling so users know what the fitted equations support.

Check your understanding

What are the slopes?

Answer: Two and minus one.

What is the estimated residual covariance?

Answer: Four thirds.

What is the estimated prediction variance of the new total at x=1?

Answer: 15.6.

Related terms

Multiple regression
Least squares
Prediction interval
response vector
residual covariance
MANOVA
joint inference

Learn next

Compare coefficient tests with joint response tests, then examine residual diagnostics and missing-outcome methods.

Sources

KNIME, Data Science Glossary, Multivariate regression: https://www.knime.com/blog/data-science-glossary . Statsmodels, Multivariate Linear Model: https://www.statsmodels.org/stable/examples/notebooks/generated/multivariate_ls.html . Dataset, calculations, and company scenario are original.

Previous: Multivariate random variable Next: Multivariate statistics