---
title: "Importance of source coding; Shannon-Fano vs fixed length (seven symbols)"
subject: "Communication Systems"
chapter: 6
key: "semester_6/communication_systems:source-coding-sf-fixed-seven"
source: https://www.bctengineer.com/q/semester_6/communication_systems/source-coding-sf-fixed-seven
site: BCT Engineering
---

# Importance of source coding; Shannon-Fano vs fixed length (seven symbols)

Communication Systems · Chapter 6: Baseband Digital Data Transmission

- Asked in: 2075 Chaitra (CS II), 4+6 marks
- Video: [SHANNON FANO CODING SOLVED EXAMPLE - with table and calculations - Hindi](https://www.youtube.com/watch?v=U-SpTfzZwFY) — Electronics Subjectified
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_6/communication_systems/source-coding-sf-fixed-seven
- Whole chapter: https://www.bctengineer.com/md/semester_6/communication_systems/chapter-6

## Question

Discuss the importance of source coding in Digital Communication system. A DMS emits one of the seven symbol with probabilities P = [0.25, 0.2, 0.15, 0.08, 0.07, 0.03, 0.02]. Find the coding efficiency, Code Redundancy for both Shannon-Fano coding and Fixed Length Coding and compare result.

## Short answer

Source coding saves bandwidth by assigning shorter codes to frequent symbols. With an eight-symbol corrected source (adding p=0.2), entropy H = 2.6640 bits/symbol. Fixed-length (3 bits) gives 88.80% efficiency, 11.20% redundancy; Shannon-Fano yields 2.72 bits/symbol, 97.94% efficiency, 2.06% redundancy.

## Answer

#### Importance of source coding

**Source coding** is the efficient representation of the source output in bits, by removing redundancy, so that the average number of bits per symbol $\bar L$ is as close as possible to the entropy $H$.

- **Reduces bit rate:** fewer bits per symbol means a lower transmission rate for the same information.
- **Saves bandwidth and power:** a lower bit rate needs less channel bandwidth and less transmitted energy; storage space is also saved (ZIP, MP3, JPEG).
- **Uses symbol statistics:** frequent symbols get short code words and rare symbols long code words (variable-length coding), as in Morse code.
- **Approaches the entropy limit:** Shannon's source coding theorem says $\bar L \ge H$; good codes (Huffman, Shannon–Fano) reach close to $H$.
- **Unique decodability:** prefix codes (no code word is a prefix of another) can be decoded without separators.
- **Leaves room for channel coding:** the bits saved can be used for error-control redundancy.

#### Numerical

The seven listed probabilities add to only 0.80, so a value is missing from the printed question. The same source appears in another IOE question as $P = \{0.25, 0.2, 0.2, 0.15, 0.08, 0.07, 0.03, 0.02\}$ (a second 0.2 added), which adds to 1, so this set is used below. A fixed-length code needs 3 bits whether there are 7 or 8 symbols.

**Entropy:**

| Symbol | $p_i$ | $\log_2(1/p_i)$ | $p_i\log_2(1/p_i)$ |
|---|---|---|---|
| x1 | 0.25 | 2.0000 | 0.5000 |
| x2 | 0.2 | 2.3219 | 0.4644 |
| x3 | 0.2 | 2.3219 | 0.4644 |
| x4 | 0.15 | 2.7370 | 0.4105 |
| x5 | 0.08 | 3.6439 | 0.2915 |
| x6 | 0.07 | 3.8365 | 0.2686 |
| x7 | 0.03 | 5.0589 | 0.1518 |
| x8 | 0.02 | 5.6439 | 0.1129 |
| **Total** | 1 | | **2.6640** |

$$
H = 2.6640\ \text{bits/symbol}
$$

**(a) Shannon–Fano coding** (first split {0.25, 0.2} = 0.45 vs 0.55):

| Symbol | $p_i$ | Step 1 | Step 2 | Step 3 | Step 4 | Step 5 | Step 6 | Code | $l_i$ |
|---|---|---|---|---|---|---|---|---|---|
| x1 | 0.25 | 0 | 0 |  |  |  |  | 00 | 2 |
| x2 | 0.2 | 0 | 1 |  |  |  |  | 01 | 2 |
| x3 | 0.2 | 1 | 0 |  |  |  |  | 10 | 2 |
| x4 | 0.15 | 1 | 1 | 0 |  |  |  | 110 | 3 |
| x5 | 0.08 | 1 | 1 | 1 | 0 |  |  | 1110 | 4 |
| x6 | 0.07 | 1 | 1 | 1 | 1 | 0 |  | 11110 | 5 |
| x7 | 0.03 | 1 | 1 | 1 | 1 | 1 | 0 | 111110 | 6 |
| x8 | 0.02 | 1 | 1 | 1 | 1 | 1 | 1 | 111111 | 6 |

$$
\begin{aligned}
\bar L &= 0.25(2)+0.2(2)+0.2(2)+0.15(3)+0.08(4)+0.07(5)+0.03(6)+0.02(6) \\
&= 0.5+0.4+0.4+0.45+0.32+0.35+0.18+0.12 = 2.72\ \text{bits/symbol} \\
\eta_{SF} &= \frac{2.6640}{2.72} = 97.94\%, \qquad \gamma_{SF} = 1-\eta = 2.06\%
\end{aligned}
$$

**(b) Fixed-length coding:** $\bar L = \lceil\log_2 8\rceil = 3$ bits/symbol.

$$
\eta_{FL} = \frac{2.6640}{3} = 88.80\%, \qquad \gamma_{FL} = 11.20\%
$$

#### Comparison

| Method | $\bar L$ | Efficiency | Redundancy |
|---|---|---|---|
| Shannon–Fano | 2.72 | 97.94 % | 2.06 % |
| Fixed length | 3 | 88.80 % | 11.20 % |

Shannon–Fano coding saves $3 - 2.72 = 0.28$ bits/symbol (about 9.3 %) over fixed-length coding, because it gives short code words to the frequent symbols. Fixed-length coding is simpler and needs no code table, but wastes bits when probabilities are unequal.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What is the main aim of source coding?
   A. Add redundancy to correct errors
   B. Remove redundancy so average length approaches the entropy
   C. Increase bandwidth
   D. Encrypt the data
   Answer: B. Source coding makes L-bar as close as possible to H, reducing bit rate, bandwidth and power. (Key point: Purpose of source coding)

2. Shannon's source coding theorem says:
   A. L-bar <= H
   B. L-bar >= H
   C. L-bar = 2H
   D. L-bar is independent of H
   Answer: B. No uniquely decodable code can have average length below the entropy. (Key point: Source coding theorem)

## Related in Communication Systems

- [Importance of source coding; fixed vs Shannon-Fano (five symbols)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-fixed-sf-five) — Communication Systems, chapter 6
- [Importance of source coding; Shannon-Fano vs BCD (four symbols)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-sf-bcd-four) — Communication Systems, chapter 6
- [Digital communication system; Shannon-Fano for six symbols](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Adcs-block-shannon-fano-six) — Communication Systems, chapter 6
- [Importance of source coding; Huffman for eight symbols at 1000 symbols/s](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asource-coding-huffman-eight-1000) — Communication Systems, chapter 6
- [Huffman code for five-symbol DMS vs fixed length](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Ahuffman-dms-five-symbols) — Communication Systems, chapter 6
- [Digital communication system elements; Shannon-Fano coding](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Adcs-block-shannon-fano) — Communication Systems, chapter 6

## Related in other subjects

- [Shannon-Fano code for eight symbols](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ashannon-fano-eight-symbols) — Data Communication, chapter 8
- [Source coding; fixed vs variable length codes](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Asource-coding-fixed-variable) — Data Communication, chapter 8
- [Shannon-Fano code for six symbols](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ashannon-fano-six-symbols) — Data Communication, chapter 8