--- title: "Quantization, rounding vs truncation, dead band" subject: "Digital Signal Analysis and Processing" chapter: 4 key: "semester_7/digital_signal_analysis_and_processing:quantization-rounding-limit-dead-band" source: https://www.bctengineer.com/q/semester_7/digital_signal_analysis_and_processing/quantization-rounding-limit-dead-band site: BCT Engineering --- # Quantization, rounding vs truncation, dead band Digital Signal Analysis and Processing · Chapter 4: Discrete Filter Structures - Asked in: 2071 Shrawan, 1+1+2+1 marks - Video: [Quantization - Truncation and Rounding Methods - Errors due to Quantization Methods](https://www.youtube.com/watch?v=gl-6H2KKAww) — Padmasri Naban - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_7/digital_signal_analysis_and_processing/quantization-rounding-limit-dead-band - Whole chapter: https://www.bctengineer.com/md/semester_7/digital_signal_analysis_and_processing/chapter-4 ## Question What is the importance of quantization in Digital Signal Processing? Which one is better, rounding or truncation? Explain about limit cycles in recursive system. Define dead band. ## Short answer Quantization maps analog to digital, introduces noise power ~q²/12, limiting SNR to ~6 dB/bit. Rounding is better (zero-mean error −q/2 to q/2, power q²/12) than truncation (bias, error power q²/3). In recursive filters, product quantization may cause zero-input limit cycles—persistent small oscillations. Dead band is the output range [−β, β] where limit cycles are sustained; for a first-order filter |y| ≤ (q/2)/(1−|a|). ## Answer #### Importance of quantization A digital system has finite word length, so the continuous amplitude of a sampled signal must be mapped to a finite set of levels (A/D conversion). Quantization is also applied to filter coefficients and to the results of multiplications. It is what makes DSP implementable on real hardware. Its price is an error (noise) with power of about q²/12 for step size q, so the word length (number of bits) decides the signal-to-noise ratio, about 6 dB per bit. #### Rounding or truncation **Rounding** is better. Its error lies in (−q/2, +q/2], is symmetric and has zero mean, so its total error power is only q²/12. Truncation error lies in (−q, 0]: it has the same variance (q²/12) but a mean of −q/2 (a DC bias), so it adds a systematic offset and its mean-square error is q²/3, four times larger. Truncation is only simpler to implement (just drop bits). #### Limit cycles in a recursive system In a recursive (IIR) filter, the product quantization is inside the feedback loop. Even with zero (or constant) input, the rounding of the products can make the output oscillate with a small steady amplitude instead of decaying to zero. These sustained oscillations are called **limit cycles**. There are two kinds: - **Zero-input limit cycles**: small-amplitude oscillation or constant output caused by rounding in the feedback products. - **Overflow oscillations**: large-amplitude oscillation caused by arithmetic overflow (wrap-around) in adders. Example: y[n] = 0.9y[n−1] + x[n] with rounding to integers. For x[n] = 0 and y[−1] = 5, rounding 0.9×5 = 4.5 gives 5 again, and the output stays at 5 forever, though ideally it should decay to 0. #### Dead band The range of output values [−β, +β] in which a limit cycle can get stuck is the **dead band**. For a first-order filter y[n] = a·y[n−1] + x[n], the dead band is |y| ≤ (q/2) / (1 − |a|). While the output is inside this band, rounding makes the effective pole sit on the unit circle, so the output does not decay further. Using more bits (smaller q) reduces the dead band. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. Why is quantization essential in DSP, and what is its cost? A. Hardware has finite word length; the cost is error noise of about q^2/12 B. It increases the sampling rate; cost is aliasing C. It removes the need for A/D conversion; cost is delay D. It makes filters linear phase; cost is more memory Answer: A. Finite word length forces signals, coefficients and products onto finite levels, adding noise of variance q^2/12. (Key point: Importance of quantization) 2. Why is rounding preferred over truncation? A. It is simpler to implement B. Its error is zero-mean with smaller variance, while truncation has a mean of -q/2 (DC bias) C. Truncation gives smaller error range D. Rounding error is always positive Answer: B. Rounding error lies in (-q/2, q/2] with zero mean, whereas truncation error in (-q, 0] adds a bias. (Key point: Rounding vs truncation) 3. What are zero-input limit cycles in a recursive filter? A. Overflow of the input signal B. Sustained small oscillations or constant output caused by rounding of products in the feedback loop, instead of decaying to zero C. Oscillation caused by aliasing D. Oscillation caused by missing poles Answer: B. Product quantization inside the feedback loop can keep the output from decaying to zero. (Key point: Limit cycles) 4. For y[n] = a y[n-1] + x[n], what is the dead band? A. |y| <= (q/2)/(1 - |a|) B. |y| <= q/2 C. |y| <= q(1 - |a|) D. |y| <= 1/(q|a|) Answer: A. The range [-beta, +beta] in which output can stick is |y| <= (q/2)/(1 - |a|); a smaller q shrinks it. (Key point: Dead band) ## Related in Digital Signal Analysis and Processing - [Limit cycle effect in recursive systems](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Alimit-cycle-recursive) — Digital Signal Analysis and Processing, chapter 4, asked 2× - [Why Kaiser window is better than fixed windows](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Awhy-kaiser-better) — Digital Signal Analysis and Processing, chapter 5, asked 2× - [Even and odd part of a signal](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Aeven-odd-part-piecewise) — Digital Signal Analysis and Processing, chapter 1, asked 2× - [Digital LPF to HPF transformation](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Alowpass-to-highpass-transformation) — Digital Signal Analysis and Processing, chapter 6 - [Unit step signal and relation to impulse](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Aunit-step-relation-impulse) — Digital Signal Analysis and Processing, chapter 1 ## Related in other subjects - [Why band-limit a signal before sampling](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Asampling-bandlimit) — Data Communication, chapter 5 - [Digital PI design for plant with 2 s dead time](https://www.bctengineer.com/question.md?key=semester_6%2Fdigital_control_system%3Api-design-dead-time-2s) — Digital Control System, chapter 4, asked 3× - [Need for non-uniform quantization; one algorithm](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Anonuniform-quantization-algorithm) — Communication Systems, chapter 4 - [Analog vs digital signal; convert A2.64H](https://www.bctengineer.com/question.md?key=semester_3%2Fdigital_logic%3Aanalog-digital-convert-a264h) — Digital Logic, chapter 1