Polynomial Fits

Data science

Overfitting

Underfitting

Bias-variance tradeoff

Polynomial regression

Model complexity

Author

Apurva Nakade

Published

July 17, 2026

// VM is the shared utility library, loaded globally via _includes/head-scripts.html (js/**). This page uses VM.numerical.polynomialFit, VM.plotting.paddedRange, VM.ui.renderTable.
VM = window.VM

Fitting a model to data always involves a choice: how complex should the model be? A model with more free parameters can match the training data more closely — in the extreme, a polynomial with as many coefficients as data points can pass through every point exactly, leaving zero error. But fitting the data perfectly is not the same as capturing the underlying trend: between and around the data points, a high-degree polynomial is free to swing wildly, and that shape is extremely sensitive to the exact values of the data it was fit to.

This is the core of the bias-variance tradeoff:

A low-degree model (e.g. a line) has high bias — it can’t bend to match every point — but low variance: nudging one data point barely changes the fitted line. This is underfitting.
A high-degree model has low bias (it fits the training data almost perfectly) but high variance: nudging one data point can send the fitted curve swinging by orders of magnitude elsewhere. This is overfitting.

The app below generates 6 points that lie almost exactly on a line, then fits polynomials of degree 1 through 5 to them. Drag the slider to move the y-coordinate of the last point (marked in orange) and watch what happens to each fit, click “Regenerate points” to draw a brand-new set of 6 points from scratch, or use the checkboxes to show or hide individual fits.

// basePoints redraws whenever regenerateTrigger changes (initial load
// counts as one draw), seeded from the current timestamp so each draw is a
// fresh, non-repeating set of points — this is what keeps the first 5
// points fixed between slider moves, but lets the button reshuffle all 6.
// Kept outside the ojs-row div below: an output:false cell still renders
// an empty .cell div, and if it sat inside the row it would become the
// row's first child, stealing the `flex: 1 1 200px` fill rule meant for
// the slider.
basePoints = {
  regenerateTrigger
  // Seeded PRNG (mulberry32), seeded from the current time so each
  // "Regenerate" click produces a different draw.
  let state = Date.now()
  const rand = () => {
    state = (state + 0x6D2B79F5) | 0
    let t = Math.imul(state ^ (state >>> 15), 1 | state)
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296
  }
  // Box-Muller transform: turns two uniform samples into one standard-normal sample.
  const gaussian = () => {
    const u1 = Math.max(rand(), 1e-12)
    const u2 = rand()
    return Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2)
  }

  const slope = 1.2, intercept = 3, noiseStd = 0.6
  const pts = []
  for (let i = 0; i < 6; i++) {
    const x = i
    const y = slope * x + intercept + noiseStd * gaussian()
    pts.push({x, y})
  }
  return pts
}

// lastYRange: the slider's fixed [lo, hi] bounds around the last point's
// original (noisy) y-value. setup below uses these same bounds — not the
// current lastY value — to fix the plot's y-axis, so dragging the slider
// never rescales the chart out from under you.
lastYRange = {
  const baseline = basePoints[basePoints.length - 1].y
  return {lo: baseline - 10, hi: baseline + 10}
}

viewof lastY = {
  const midpoint = (lastYRange.lo + lastYRange.hi) / 2
  const input = Inputs.range([lastYRange.lo, lastYRange.hi], {step: 0.1, value: midpoint, label: "Last point's y"})
  input.classList.add("ojs-fill")
  return input
}

// regenerateTrigger: clicking "Regenerate points" bumps this, which is the
// only thing basePoints depends on below — that's what makes a fresh draw
// happen on click instead of only once at page load.
viewof regenerateTrigger = {
  const button = Inputs.button("Regenerate points", {value: 0, reduce: v => v + 1})
  button.classList.add("ojs-auto")
  return button
}

viewof shownDegrees = {
  const options = ["Degree 1", "Degree 2", "Degree 3", "Degree 4", "Degree 5"]
  const checkboxInput = Inputs.checkbox(options, {value: ["Degree 5"]})
  checkboxInput.classList.add("ojs-fill")
  return checkboxInput
}

// points: the first 5 points are the fixed baseline; only the last point's
// y-coordinate is reactive, driven by the slider above.
points = {
  const pts = []
  for (let i = 0; i < basePoints.length - 1; i++) pts.push(basePoints[i])
  pts.push({x: basePoints[basePoints.length - 1].x, y: lastY})
  return pts
}

// result: fits polynomials of degree 1 through 5 to the current points.
//   fits[1] — a line (2 parameters): high bias, low variance
//   fits[2] — a parabola (3 parameters): a middle ground
//   fits[5] — a degree-5 polynomial (6 parameters, one per point): this
//     exactly interpolates all 6 points, so its training error is always
//     essentially zero — but watch how much the curve itself swings as the
//     slider moves.
result = {
  const math = window.math
  const fits = {}
  for (let degree = 1; degree <= 5; degree++) {
    fits[degree] = VM.numerical.polynomialFit(math, points, degree)
  }
  return {points, fits}
}

// setup: sampled curves and axis ranges shared by mainPlot.
setup = {
  const xs = []
  for (const p of result.points) xs.push(p.x)
  const xRange = VM.plotting.paddedRange(xs, {emptyRange: [-1, 6], relativePadding: 0.08, minPadding: 0.3})

  // A fixed y-range, set once from the 5 fixed points plus the slider's
  // full [lo, hi] reach — not the current lastY value — so the axis never
  // rescales as the slider moves. A rescaling axis makes the slider's
  // effect look bigger or smaller than it really is at every step, which
  // is exactly what's confusing; a fixed axis instead lets a high-degree
  // curve visibly swing far outside it, or the moved point run off the
  // top/bottom, rather than the whole chart chasing them.
  const ys = []
  for (let i = 0; i < result.points.length - 1; i++) ys.push(result.points[i].y)
  ys.push(lastYRange.lo, lastYRange.hi)
  const yRange = VM.plotting.paddedRange(ys, {emptyRange: [-5, 20], relativePadding: 0.15, minPadding: 2})

  const sampleCount = 400
  const sampleXs = []
  for (let i = 0; i < sampleCount; i++) {
    sampleXs.push(xRange.lo + (xRange.hi - xRange.lo) * i / (sampleCount - 1))
  }

  const sampleCurve = fit => {
    const curveYs = []
    for (const x of sampleXs) {
      if (!fit) {
        curveYs.push(null)
        continue
      }
      const y = fit.evaluate(x)
      if (Number.isFinite(y) && Math.abs(y) < 1e6) curveYs.push(y)
      else curveYs.push(null)
    }
    return curveYs
  }

  const curves = {}
  for (let degree = 1; degree <= 5; degree++) {
    curves[degree] = sampleCurve(result.fits[degree])
  }

  return {points: result.points, xRange, yRange, sampleXs, curves}
}

// mainPlot draws the data points and, depending on the checkboxes, each
// fitted curve.
//
// _plotDiv is kept in a closure so Plotly reuses the same DOM node across
// reactive updates — this preserves zoom and pan when the slider or
// checkboxes change.
mainPlot = {
  const Plotly = window.Plotly
  let _plotDiv = null

  return ({points, xRange, yRange, sampleXs, curves}, showDegree1, showDegree2, showDegree3, showDegree4, showDegree5) => {
    // The last point is the one the slider moves — plotted as its own
    // trace, in a different color/shape/size, so it's visually obvious
    // which point is draggable versus which 5 are fixed.
    const fixedXs = [], fixedYs = []
    for (let i = 0; i < points.length - 1; i++) {
      fixedXs.push(points[i].x)
      fixedYs.push(points[i].y)
    }
    const movedPoint = points[points.length - 1]

    // One entry per degree, in draw order (highest degree first, so the
    // more oscillatory curves don't hide behind the flatter ones).
    const curveSpecs = [
      {degree: 5, show: showDegree5, color: "#dc2626", label: "Degree 5 (overfit)"},
      {degree: 4, show: showDegree4, color: "#0891b2", label: "Degree 4"},
      {degree: 3, show: showDegree3, color: "#9333ea", label: "Degree 3"},
      {degree: 2, show: showDegree2, color: "#16a34a", label: "Degree 2"},
      {degree: 1, show: showDegree1, color: "#2563eb", label: "Degree 1 (underfit)"}
    ]

    const data = []
    for (const spec of curveSpecs) {
      if (!spec.show) continue
      data.push({x: sampleXs, y: curves[spec.degree], type: "scatter", mode: "lines", name: spec.label, line: {color: spec.color, width: 2.5}})
    }
    data.push({x: fixedXs, y: fixedYs, type: "scatter", mode: "markers", name: "Fixed points", marker: {color: "#111827", size: 9, line: {color: "white", width: 1}}})
    data.push({x: [movedPoint.x], y: [movedPoint.y], type: "scatter", mode: "markers", name: "Moved point (slider)", marker: {color: "#f97316", size: 13, symbol: "diamond", line: {color: "white", width: 1.5}}})

    const layout = {
      margin: {l: 0, r: 0, t: 0, b: 0},
      xaxis: {title: "x", range: [xRange.lo, xRange.hi], zeroline: true},
      yaxis: {title: "y", range: [yRange.lo, yRange.hi], zeroline: true},
      legend: {orientation: "h", y: -0.08},
      hovermode: "closest",
      uirevision: "static",
      autosize: true
    }
    const config = {responsive: true, displaylogo: false}

    if (!_plotDiv) {
      _plotDiv = document.createElement("div")
      _plotDiv.className = "plotly-box-large"
      Plotly.newPlot(_plotDiv, data, layout, config)
      // newPlot fires before the div is in the DOM, so Plotly measures 0 width.
      // A ResizeObserver catches the real size once it's inserted and laid out.
      new ResizeObserver(() => Plotly.Plots.resize(_plotDiv)).observe(_plotDiv)
    } else {
      Plotly.react(_plotDiv, data, layout, config)
    }
    return _plotDiv
  }
}

mainPlot(setup, showDegree1, showDegree2, showDegree3, showDegree4, showDegree5)

Model comparison

// errorTable reports each model's parameter count and total squared error
// over the 6 data points, updating live as the slider moves — for all 5
// degrees, regardless of which curves the checkboxes currently show. The
// degree-5 row stays near zero regardless of the slider — it has exactly
// as many parameters as points, so it can always interpolate them exactly.
errorTable = {
  const labels = {
    1: "Degree 1 (line)",
    2: "Degree 2 (parabola)",
    3: "Degree 3",
    4: "Degree 4",
    5: "Degree 5 (interpolates all 6 points)"
  }

  const rows = []
  for (let degree = 1; degree <= 5; degree++) {
    const fit = result.fits[degree]
    if (!fit) {
      rows.push([labels[degree], "—", "—"])
      continue
    }
    rows.push([labels[degree], fit.coeffs.length, fit.totalSquaredError.toExponential(3)])
  }

  return VM.ui.renderTable({
    html,
    headers: ["Model", "Parameters", "Total squared error"],
    csvHeaders: ["Model", "Parameters", "Total squared error"],
    filename: "polynomial-fit-errors.csv",
    rows
  })
}

Error vs. degree

// errorPlot: total squared error as a function of degree, for the current
// points. Training error decreases monotonically with degree — more
// parameters can only help fit the training data — which is exactly why
// training error alone can't tell you when a model is overfitting.
errorPlot = {
  const rows = []
  for (let degree = 1; degree <= 5; degree++) {
    const fit = result.fits[degree]
    if (fit) rows.push({degree, error: fit.totalSquaredError})
  }

  return Plot.plot({
    width: 500, height: 300, marginLeft: 46,
    x: {label: "Polynomial degree", domain: [1, 2, 3, 4, 5]},
    y: {label: "Total squared error (training data)"},
    marks: [
      Plot.line(rows, {x: "degree", y: "error", stroke: "#9333ea"}),
      Plot.dot(rows, {x: "degree", y: "error", fill: "#9333ea", r: 5})
    ]
  })
}

Why this matters

The degree-5 polynomial has 6 free coefficients — exactly as many as there are data points. It can therefore pass through all 6 points exactly, no matter where you drag the slider: its training error stays pinned near zero the entire time. If you judged these three models purely by how well they fit the data you gave them, the degree-5 fit would always look best.

But watch the curve itself, not just the error number. As you move the orange point, the degree-5 curve doesn’t just adjust locally near that point — it can swing by large amounts across the whole domain, oscillating between the other 5 points to make room for the one you moved. The degree-1 and degree-2 fits barely react at all: they don’t have enough parameters to chase a single outlier, so they stay close to the overall trend. Toggle the degree-3 and degree-4 checkboxes to see the middle ground: each extra parameter buys a little more flexibility, and a little more sensitivity to the exact data.

That instability is variance: how much the fitted model changes when the training data changes slightly. A model that is very sensitive to its training data will not generalize well to new data — it has fit the noise in this particular sample, not the underlying line. The systematic gap between a simple model and the true trend is bias: the degree-1 line can never bend to match a curved pattern, even with infinite data. Choosing a model degree is choosing a point on this tradeoff — too low, and the model can’t capture the pattern (underfitting); too high, and the model captures the noise along with the pattern (overfitting).

In practice, you can’t see this directly from training error alone, since (as the plot above shows) it only ever goes down as complexity increases. Diagnosing overfitting requires checking performance on data the model didn’t get to fit — a held-out validation or test set — which is why train/validation/test splits and cross-validation are standard practice in model fitting.