--- title: "Least squares significance, normal equations" subject: "Numerical Method" chapter: 4 key: "semester_4/numerical_method:least-squares-significance-normal" source: https://www.bctengineer.com/q/semester_4/numerical_method/least-squares-significance-normal site: BCT Engineering --- # Least squares significance, normal equations Numerical Method · Chapter 4: Interpolation - Asked in: 2073 Bhadra, 2+6 marks - Video: [Deriving the least squares estimators of the slope and intercept (simple linear regression)](https://www.youtube.com/watch?v=ewnc1cXJmGA) — jbstatistics - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_4/numerical_method/least-squares-significance-normal - Whole chapter: https://www.bctengineer.com/md/semester_4/numerical_method/chapter-4 ## Question What is the practical significance of the least squares method of curve fitting? Derive the normal equations to fit a given set of data to a linear equation (y = ax + b). ## Short answer Least squares minimizes the sum of squared vertical deviations, giving a best-fit curve that uses all noisy data and avoids sign cancellation. For y = ax + b, setting partial derivatives of S = sum(y_i - a x_i - b)^2 to zero yields sum y = nb + a sum x and sum xy = b sum x + a sum x^2. ## Answer #### Practical significance of the least squares method Experimental data contain measurement errors, so the points do not lie exactly on any curve and an interpolating polynomial through every point would follow the noise. The least squares method finds the curve of a chosen form (line, parabola, exponential, power, …) that passes as close as possible to all points on the whole, by minimising the sum of the squares of the vertical deviations. - It gives the **best-fit** relation between variables from noisy data, so that values can be predicted (interpolation and extrapolation) and trends studied. - It uses all the data points, so one bad reading has less effect than in an exact-fit polynomial. - Squaring the errors removes the sign problem (positive and negative errors cannot cancel) and gives smooth equations, the normal equations, which are easy to solve. - Non-linear laws (y = axᵇ, y = abˣ, y = aeᵇˣ) can be reduced to a straight line by taking logarithms, so the same method fits them. - It is the basis of regression, calibration of instruments, curve fitting in engineering and science, and signal and data analysis. #### Normal equations for y = ax + b Let (xᵢ, yᵢ), i = 1 … n, be the data. The error at each point is eᵢ = yᵢ − axᵢ − b. Minimise ```text S = Σ (yᵢ − axᵢ − b)² ``` Setting the partial derivatives to zero: ```text ∂S/∂a = −2 Σ xᵢ(yᵢ − axᵢ − b) = 0 ∂S/∂b = −2 Σ (yᵢ − axᵢ − b) = 0 ``` which give the normal equations ```text Σy = n b + a Σx Σxy = b Σx + a Σx² ``` Solving them: ```text a = (n Σxy − Σx Σy) / (n Σx² − (Σx)²) b = (Σy − a Σx) / n ``` The fitted line is y = ax + b. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. Why is least squares preferred over an exact-fit interpolating polynomial for experimental data? A. It needs no calculation B. It forces the curve through every point C. Measurements contain errors, so the best curve passes close to all points overall instead of following the noise D. It works only for straight lines Answer: C. Noisy data do not lie on a curve exactly, so an exact-fit polynomial would follow the noise; least squares gives the best overall fit. (Key point: Why least squares) 2. What exactly is minimised in the least squares method? A. Sum of the squares of the vertical deviations B. Largest single deviation C. Sum of the y values D. Sum of the vertical deviations Answer: A. S = sum of (yi - axi - b)^2 is minimised; squaring stops positive and negative errors cancelling. (Key point: Quantity minimised) 3. How is a law like y = a x^b fitted by the same method? A. Square both sides B. Take logarithms to turn it into a straight line C. It cannot be fitted D. Differentiate it twice Answer: B. Taking logs gives log y = log a + b log x, a straight line. (Key point: Reducing non-linear laws) 4. Which pair are the normal equations for y = ax + b? A. Σy = nb + aΣx^2 and Σxy = bΣx + aΣx B. Σy = nab and Σxy = Σx^2 C. Σy = nb + aΣx and Σxy = bΣx + aΣx^2 D. Σy = aΣx and Σxy = bΣx^2 Answer: C. Setting ∂S/∂b = 0 and ∂S/∂a = 0 gives these two equations. (Key point: Normal equations) ## Related in Numerical Method - [Derive linear least squares](https://www.bctengineer.com/question.md?key=semester_4%2Fnumerical_method%3Aleast-squares-linear-derive) — Numerical Method, chapter 4 - [Least squares: y = ax + b + c/x](https://www.bctengineer.com/question.md?key=semester_4%2Fnumerical_method%3Aleast-squares-ax-b-c-over-x) — Numerical Method, chapter 4 - [Quadratic least squares fit](https://www.bctengineer.com/question.md?key=semester_4%2Fnumerical_method%3Aquadratic-fit-least-squares) — Numerical Method, chapter 4 - [Fit y = ax/(b + x)](https://www.bctengineer.com/question.md?key=semester_4%2Fnumerical_method%3Afit-y-ax-over-b-plus-x) — Numerical Method, chapter 4, asked 2× - [Fit curve y = a b^x (decay data)](https://www.bctengineer.com/question.md?key=semester_4%2Fnumerical_method%3Afit-y-abx-d) — Numerical Method, chapter 4, asked 3× - [Fit y = ln(ax + b)](https://www.bctengineer.com/question.md?key=semester_4%2Fnumerical_method%3Afit-log-ax-b) — Numerical Method, chapter 4, asked 2× ## Related in other subjects - [Least squares fit for calibration data](https://www.bctengineer.com/question.md?key=semester_3%2Fmetrology%3Aleast-squares-calibration-line) — Metrology, chapter 2 - [Is y[n] = ax[n] + b linear](https://www.bctengineer.com/question.md?key=semester_4%2Fsignals_and_systems%3Alinearity-affine-dt) — Signals and Systems, chapter 6, asked 4× - [Significance of differential equations; liver model](https://www.bctengineer.com/question.md?key=semester_8%2Fsimulation_and_modeling%3Adiff-eq-significance-liver) — Simulation and Modeling, chapter 3 - [Derive slope deflection equations](https://www.bctengineer.com/question.md?key=semester_5%2Ftheory_of_structures_ii%3Aslope-deflection-equation-derive) — Theory of Structures II, chapter 3, asked 4×