---
title: "Importance of source coding; Huffman for eight symbols at 1000 symbols/s"
subject: "Communication Systems"
chapter: 6
key: "semester_6/communication_systems:source-coding-huffman-eight-1000"
source: https://www.bctengineer.com/q/semester_6/communication_systems/source-coding-huffman-eight-1000
site: BCT Engineering
---

# Importance of source coding; Huffman for eight symbols at 1000 symbols/s

Communication Systems · Chapter 6: Baseband Digital Data Transmission

- Asked in: 2071 Shrawan (CS II), 2+4+2 marks
- Video: [Huffman Coding (Basics, Algorithm, Procedure & Example) Explained in Digital Communication](https://www.youtube.com/watch?v=HmBH30NrM7c) — Engineering Funda
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_6/communication_systems/source-coding-huffman-eight-1000
- Whole chapter: https://www.bctengineer.com/md/semester_6/communication_systems/chapter-6

## Question

Explain the importance of source coding in Digital Communication System. A discrete memory less source emits one of the eight symbols with probabilities P = {0.25, 0.20, 0.2, 0.15, 0.08, 0.07, 0.03, 0.02}. If the output symbols are encoded using Huffman code, find the Coding efficiency and output bit if the symbol rate of the source is 1000 symbols per second.

## Short answer

Source coding removes redundancy, bringing bit rate near entropy to save bandwidth. For the eight symbols, entropy is 2.6640 bits/symbol, Huffman average length = 2.72 bits/symbol, coding efficiency = 97.94%, and output bit rate = 1000 × 2.72 = 2720 bits/s.

## Answer

#### Importance of source coding

Source coding represents the source output with the minimum average number of bits by removing redundancy:

- It brings the average code length $\bar L$ close to the entropy $H$ (Shannon's source coding theorem: $\bar L \ge H$).
- It lowers the bit rate, so less bandwidth, power and storage are needed.
- Variable-length prefix codes (Huffman, Shannon–Fano) give short words to likely symbols and remain uniquely decodable.

#### Huffman code

$P = \{0.25, 0.20, 0.2, 0.15, 0.08, 0.07, 0.03, 0.02\}$ for $x_1 \dots x_8$ (sum = 1).

**Reduction (sum of two lowest moved up as high as possible):**

| Stage | Probabilities (descending) | Combined |
|---|---|---|
| 1 | 0.25, 0.2, 0.2, 0.15, 0.08, 0.07, 0.03, 0.02 | 0.03 + 0.02 = 0.05 |
| 2 | 0.25, 0.2, 0.2, 0.15, 0.08, 0.07, 0.05 | 0.07 + 0.05 = 0.12 |
| 3 | 0.25, 0.2, 0.2, 0.15, 0.12, 0.08 | 0.12 + 0.08 = 0.2 |
| 4 | 0.25, 0.2, 0.2, 0.2, 0.15 | 0.2 + 0.15 = 0.35 |
| 5 | 0.35, 0.25, 0.2, 0.2 | 0.2 + 0.2 = 0.4 |
| 6 | 0.4, 0.35, 0.25 | 0.35 + 0.25 = 0.6 |
| 7 | 0.6, 0.4 | 0.6 + 0.4 = 1 |

| Symbol | $p_i$ | Huffman code | $l_i$ |
|---|---|---|---|
| x1 | 0.25 | 01 | 2 |
| x2 | 0.2 | 11 | 2 |
| x3 | 0.2 | 000 | 3 |
| x4 | 0.15 | 001 | 3 |
| x5 | 0.08 | 101 | 3 |
| x6 | 0.07 | 1000 | 4 |
| x7 | 0.03 | 10010 | 5 |
| x8 | 0.02 | 10011 | 5 |

**Entropy:**

| Symbol | $p_i$ | $\log_2(1/p_i)$ | $p_i\log_2(1/p_i)$ |
|---|---|---|---|
| x1 | 0.25 | 2.0000 | 0.5000 |
| x2 | 0.2 | 2.3219 | 0.4644 |
| x3 | 0.2 | 2.3219 | 0.4644 |
| x4 | 0.15 | 2.7370 | 0.4105 |
| x5 | 0.08 | 3.6439 | 0.2915 |
| x6 | 0.07 | 3.8365 | 0.2686 |
| x7 | 0.03 | 5.0589 | 0.1518 |
| x8 | 0.02 | 5.6439 | 0.1129 |
| **Total** | 1 | | **2.6640** |

$$
H = 2.6640\ \text{bits/symbol}
$$

**Average length and efficiency:**

$$
\begin{aligned}
\bar L &= \sum p_i l_i = 2.72\ \text{bits/symbol} \\
\eta &= \frac{H}{\bar L} = \frac{2.6640}{2.72} = 97.94\%
\end{aligned}
$$

(A fixed 3-bit code would give $2.664/3 = 88.80\%$.)

#### Output bit rate

$$
R_b = r\bar L = 1000 \times 2.72 = 2720\ \text{bits/s}
$$

(The information rate is $rH = 2664$ bits/s.)

**Answer:** coding efficiency = **97.94 %**; output bit rate = **2720 bits/s**.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What does Shannon's source coding theorem say about average code length L?
   A. L is always less than H
   B. L >= H
   C. L = H always
   D. L is independent of H
   Answer: B. No code can have average length below the entropy; efficiency = H/L. (Key point: Source coding theorem)

2. With H = 2.6640 bits and Huffman average length 2.72 bits, what is the efficiency?
   A. 88.80%
   B. 97.94%
   C. 100%
   D. 92.3%
   Answer: B. 2.6640/2.72 = 97.94%. (Key point: Huffman efficiency)

3. At 1000 symbols/s and L = 2.72 bits/symbol, what is the output bit rate?
   A. 2664 bits/s
   B. 2720 bits/s
   C. 3000 bits/s
   D. 1000 bits/s
   Answer: B. Rb = r x L = 1000 x 2.72 = 2720 bits/s (information rate rH is 2664). (Key point: Output bit rate)

## Related in Communication Systems

- [Source coding; Huffman for five symbols (0.3, 0.25, 0.2, 0.15, 0.1)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-huffman-five-s0) — Communication Systems, chapter 6
- [Importance of source coding; Shannon-Fano vs fixed length (seven symbols)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-sf-fixed-seven) — Communication Systems, chapter 6
- [Importance of source coding; Shannon-Fano vs BCD (four symbols)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-sf-bcd-four) — Communication Systems, chapter 6
- [Importance of source coding; fixed vs Shannon-Fano (five symbols)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-fixed-sf-five) — Communication Systems, chapter 6
- [Source vs channel coding; Huffman for five symbols](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-channel-huffman-five) — Communication Systems, chapter 6
- [Information rate and coding efficiency: BCD vs Huffman](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Abcd-source-info-rate-huffman) — Communication Systems, chapter 6

## Related in other subjects

- [Huffman code for eight symbols (1/2 ... 1/128)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-eight-symbols) — Data Communication, chapter 8
- [Huffman code for symbols A to E](https://www.bctengineer.com/question.md?key=semester_8%2Fmultimedia_system%3Ahuffman-numerical-abcde) — Multimedia System, chapter 5
- [Huffman code for symbols A to E (0.4, 0.3, ...)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-table-abcde) — Data Communication, chapter 8
- [Huffman code for symbols A0 to A4](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-a0-a4) — Data Communication, chapter 8