Interactive · Statistics

Why squared error, not just "closest line"?

Any line can look "roughly right." Least squares picks one specific line — the one that minimizes the sum of squared vertical distances to every point. Below, you can compare that choice against the more obvious alternative — minimizing absolute distance — and see exactly where they disagree.

1. Drag the line, watch both penalties

Drag the two handles to rotate/shift the line. The two meters on the right update live: one totals the plain distances (absolute error), the other totals the squared distances (squared error). Try to make each meter as small as possible — notice they don't agree on the same "best" line.

Least squares fitting stage
Sum of |errors| (absolute)
–
Sum of errors² (squared)
–

Drag either handle. The dashed lines show each point's vertical error — the raw ingredient both meters are built from.

2. Add one outlier. Watch what happens.

Click "Add an outlier point," then hit "Snap to least-squares line" again. The squared-error line bends noticeably toward the outlier. Absolute error barely reacts. That's not a bug — it's the entire personality difference between the two penalties.

A single point 20 units away contributes 20 to absolute error, but 400 to squared error. Squaring doesn't just measure a bigger miss — it treats it as disproportionately worse. That's why the squared-error line works harder to avoid large misses, even at the cost of several small ones.

3. Why square at all? Three real reasons

This isn't an arbitrary convention — squaring earns its place for concrete, connected reasons.

It has one clean minimum

Squared error is a smooth curve — a parabola in each parameter. Calculus gives you a single exact minimum by setting a derivative to zero. Absolute error has a sharp corner at zero, which calculus can't differentiate through, so there's no clean formula — you'd need to search for the answer instead of solving for it.

It penalizes big misses harder

Doubling an error quadruples its squared contribution. This makes the fit intolerant of large, systematic misses — useful when a big error usually signals something is actually wrong with the model, not just noise.

It connects directly to variance

Squared error is the same building block used to define variance and standard deviation. Minimizing it means the fitted line is, in a precise statistical sense, the one that explains the most variance in your data — linking curve-fitting straight back to the rest of statistics.