Follow the transformation
Build a design matrix from training features.
Fit coefficients using training data only.
Predict continuous values on held-out cases.
Predictive Modelling · Lesson 58
Linear Regression
Linear regression predicts a continuous target as an intercept plus a weighted sum of features.
Linear Regression
Linear regression predicts a continuous target as an intercept plus a weighted sum of features. Ordinary least squares estimates coefficients by minimising squared residual error.
Learning goal: explain why Linear Regression behaves this way, apply it to a small example, and verify the result independently. Begin by being able to justify this first step: Build a design matrix from training features.
Deeper walkthrough
Read Linear Regression as a mechanism, not a recipe
Treat this as a sequence of observable decisions rather than one opaque command. Stage 1: Build a design matrix from training features. Stage 2: Fit coefficients using training data only. Stage 3: Predict continuous values on held-out cases. Final checkpoint: Compare against a simple baseline and consider regularisation when needed.
Know what would convince you
- Confirm fitted transformations/models saw only training data.
- Retain fold/test predictions so metrics can be recomputed independently.
Useful distinctionTraining evidence: Information allowed to influence fitted state.
Click a stage to inspect what happens, what changes, and what should be checked before moving on.
Stage 1
Build a design matrix from training…
Build a design matrix from training features. At this stage of Linear Regression, keep the incoming data or object separate from the learned parameter, transformed object, or statistic so the change can be reproduced and independently checked.
Transformation focus: keep the input and produced parameters/result separate so the change is observable and reproducible.
Trace the mechanism step by step
- Build a design matrix from training features.
- Fit coefficients using training data only.
- Predict continuous values on held-out cases.
- Inspect residual patterns and scale-sensitive errors.
- Compare against a simple baseline and consider regularisation when needed.
Linear-regression check
For y ≈ 2x, training points (1,2), (2,4), (3,6) imply a slope near 2; x=4 should predict near 8.Expected / illustrative result
A tiny linear pattern makes the meaning of the fitted slope and residuals visible.
Interpret the result.
For Linear Regression, connect the result to the fitted state, held-out data or prediction rule that produced it and independently check one prediction, split or metric component.
Place the concept correctly
Training evidenceInformation allowed to influence fitted state.Held-out evidenceIndependent observations used to estimate generalisation.InterpretationWhat the result supports, with assumptions and limitations.When it is appropriate
Use Linear Regression when it answers a defined question in Predictive Modelling and its inputs/assumptions match the current data or program state.
When to stop or reconsider
Reconsider Linear Regression when the required information is unavailable, the operation would violate a validation/data boundary, or a simpler operation answers the question more transparently.
Failure modes to recognise
- Learning preprocessing/feature/model choices from held-out test information.
- Comparing models under different splits or preprocessing and attributing the difference to the algorithm.
- Turning an association or model explanation into an unsupported causal claim.
How to check the result
- Confirm fitted transformations/models saw only training data.
- Retain fold/test predictions so metrics can be recomputed independently.
- Inspect errors/subgroups and compare with a baseline before generalising the conclusion.
Demonstrate understanding
Try this:
Construct a tiny example of Linear Regression. First build a design matrix from training features. Then fit coefficients using training data only. Predict the result before execution and explain one boundary or failure case.
Use a tiny fixed split or synthetic example. State what is fitted, what remains held out, and what result you expect before running it.
Check reasoning, not memorisation
Which approach best demonstrates understanding of Linear Regression?
Remember the logic
Step 1Build a design matrix from training features.Step 2Fit coefficients using training data only.Step 3Predict continuous values on held-out cases.Step 4Inspect residual patterns and scale-sensitive errors.What to remember
- Linear regression predicts a continuous target as an intercept plus a weighted sum of features. Ordinary least squares estimates coefficients by minimising squared residual error.
- Build a design matrix from training features.
- Learning preprocessing/feature/model choices from held-out test information.
- Confirm fitted transformations/models saw only training data.