---
title: "Short note: Linear prediction theory"
subject: "Communication Systems"
chapter: 4
key: "semester_6/communication_systems:short-note-linear-prediction"
source: https://www.bctengineer.com/q/semester_6/communication_systems/short-note-linear-prediction
site: BCT Engineering
---

# Short note: Linear prediction theory

Communication Systems · Chapter 4: Pulse Modulation

- Asked in: 2081 Bhadra (CS II), 4 marks; 2070 Chaitra (CS II), 6 marks (asked 2 times)
- Video: [DIGITAL COMMUNICATIONS UNIT-II LINEAR PREDICTIVE CODING](https://www.youtube.com/watch?v=znTF5UvvEbU) — prathima samuda
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_6/communication_systems/short-note-linear-prediction
- Whole chapter: https://www.bctengineer.com/md/semester_6/communication_systems/chapter-4

## Question

Write a short note on linear prediction theory.

## Short answer

Linear prediction estimates current sample as weighted sum of past samples, minimizing mean-square error via Wiener-Hopf equations. Used in DPCM/ADPCM/LPC, prediction gain improves SQNR by 4-11 dB for speech. Adaptive predictors update weights (e.g., LMS) for time-varying statistics.

## Answer

**Linear prediction** estimates the present sample of a signal as a **linear combination of its past samples**. It is the basis of DPCM, ADPCM and speech coders (LPC), because only the small prediction error needs to be transmitted.

**Predictor.** With $p$ past samples:

$$
\hat{x}[n] = \sum_{k=1}^{p} w_k\, x[n-k]
$$

The **prediction error** is

$$
e[n] = x[n] - \hat{x}[n]
$$

```text
x[n] --+------------------>( + )--> e[n]
       |                     ^ -
       v                     |
   [z^-1]->[z^-1]-> ... ->[z^-1]   (tapped delay line)
     |w1     |w2            |wp
     +-------+---- ... -----+--> sum = x^[n]
```

**Optimum coefficients.** The weights are chosen to minimise the mean-square error

$$
J = E\{e^2[n]\} = E\left\{\left(x[n] - \sum_{k=1}^{p} w_k x[n-k]\right)^2\right\}
$$

Setting $\partial J/\partial w_k = 0$ gives the **normal (Wiener–Hopf) equations**:

$$
\sum_{j=1}^{p} w_j\, R_x(k-j) = R_x(k), \qquad k = 1, 2, \ldots, p
$$

or in matrix form $\mathbf{R}\mathbf{w} = \mathbf{r}$, so $\mathbf{w}_o = \mathbf{R}^{-1}\mathbf{r}$, where $R_x(k)$ is the autocorrelation of the input. The minimum error is

$$
J_{min} = R_x(0) - \mathbf{r}^T\mathbf{w}_o
$$

**Example:** first-order predictor ($p = 1$): $w_1 = R_x(1)/R_x(0) = \rho$, and $J_{min} = R_x(0)(1-\rho^2)$. For highly correlated speech ($\rho \approx 0.9$), the error variance is only 19% of the signal variance.

**Key points:**

- Prediction gain $G_p = \sigma_x^2/\sigma_e^2$ gives an SQNR improvement in DPCM of $10\log G_p$ dB (typically 4–11 dB for speech).
- If the statistics change with time, an **adaptive** predictor updates the weights (e.g. by the LMS algorithm), as in ADPCM.
- In DPCM the predictor works on quantised samples, so transmitter and receiver predictions stay identical.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What does a linear predictor of order p compute?
   A. The average of the error over p samples
   B. A weighted sum of the p past samples as the estimate of the present sample
   C. A weighted sum of p future samples
   D. The difference between two consecutive samples only
   Answer: B. x^[n] = sum of w_k x[n-k] for k=1..p, using past samples. (Key point: Linear predictor)

2. How are the optimum predictor coefficients found?
   A. Set all weights equal to 1/p
   B. Choose weights so the error is exactly zero for every sample
   C. Maximise the prediction error power
   D. Minimise mean-square prediction error, giving the normal (Wiener-Hopf) equations Rw = r
   Answer: D. Setting dJ/dw_k = 0 on J = E{e^2} gives Rw = r, so w = R^-1 r. (Key point: Optimum weights)

3. For p = 1 with correlation coefficient rho = 0.9, what fraction of the signal variance is the minimum error?
   A. rho^2 = 0.81
   B. 1 - rho = 0.1
   C. 1 - rho^2 = 0.19
   D. rho = 0.9
   Answer: C. J_min = R(0)(1 - rho^2) = 0.19 R(0), i.e. 19%. (Key point: First-order example)

4. Why is the predictor in DPCM driven by quantised samples?
   A. So transmitter and receiver predictions are identical
   B. To make the prediction gain infinite
   C. To reduce the number of levels in the quantizer
   D. To avoid using any past samples
   Answer: A. The receiver only has quantised samples, so using them at the transmitter keeps both predictors in step. (Key point: Use in DPCM)

## Related in Communication Systems

- [Hamming distance and weight importance; syndrome decoding](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Ahamming-importance-syndrome-decoding) — Communication Systems, chapter 8

## Related in other subjects

- [Linear and non-linear behaviour of structures](https://www.bctengineer.com/question.md?key=semester_4%2Ftheory_of_structures_i%3Alinear-nonlinear-behaviour) — Theory of Structures I, chapter 1, asked 3×
- [Parabolic E-k and quantized linear metal](https://www.bctengineer.com/question.md?key=semester_3%2Felectrical_engineering_material%3Aek-parabolic-linear-metal) — Electrical Engineering Material, chapter 1
- [Elastic, inelastic, linear and non-linear behaviour](https://www.bctengineer.com/question.md?key=semester_4%2Ftheory_of_structures_i%3Aelastic-inelastic-linear-nonlinear) — Theory of Structures I, chapter 1