---
title: "Huffman code for p = 0.6, 0.2, 0.1, 0.05, 0.05"
subject: "Data Communication"
chapter: 8
key: "semester_5/data_communication:huffman-p06-02-01"
source: https://www.bctengineer.com/q/semester_5/data_communication/huffman-p06-02-01
site: BCT Engineering
---

# Huffman code for p = 0.6, 0.2, 0.1, 0.05, 0.05

Data Communication · Chapter 8: Information Theory and Coding

- Asked in: 2075 Chaitra, 8 marks
- Video: [Huffman Coding (Basics, Algorithm, Procedure & Example) Explained in Digital Communication](https://www.youtube.com/watch?v=HmBH30NrM7c) — Engineering Funda
- Answer checked by Claude Sonnet 5.5 on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_5/data_communication/huffman-p06-02-01
- Whole chapter: https://www.bctengineer.com/md/semester_5/data_communication/chapter-8

## Question

Define entropy in information theory. Find the efficient code word and efficiency using the Huffman algorithm using probabilities p(x1)=0.6, p(x2)=0.2, p(x3)=0.1, p(x4)=0.05, p(x5)=0.05.

## Short answer

Entropy H = -Σ p log2 p. Huffman codes: x1=1, x2=00, x3=010, x4=0110, x5=0111. Average length L=1.7000 bits/symbol, H=1.6710 bits/symbol, efficiency η=98.29%.

## Answer

**Entropy** is the average information content per source symbol: H = −Σ pᵢ log₂ pᵢ bits/symbol. It measures the uncertainty (randomness) of the source, and is the minimum average number of bits per symbol needed to code it without loss.

Probabilities: p(x1) = 0.6, p(x2) = 0.2, p(x3) = 0.1, p(x4) = 0.05, p(x5) = 0.05 (sum = 1).

#### Huffman algorithm
1. Arrange the symbols in decreasing order of probability.
2. Combine the two least probable symbols into one with the sum of their probabilities.
3. Repeat (re-sorting) until only one symbol of probability 1 remains.
4. Assign 0 and 1 to the two branches at every merge.
5. The code of a symbol is the sequence of bits read from the root back to that symbol.

#### Merging steps
At every step the two smallest probabilities are combined (N1, N2, ... are the new nodes):

```text
1. x4(0.05) + x5(0.05) = 0.1
2. x3(0.1) + N1(0.1) = 0.2
3. x2(0.2) + N2(0.2) = 0.4
4. N3(0.4) + x1(0.6) = 1
```

#### Code table

| Symbol | p | Code | Length l | p·l |
|---|---|---|---|---|
| x1 | 0.6 | 1 | 1 | 0.6000 |
| x2 | 0.2 | 00 | 2 | 0.4000 |
| x3 | 0.1 | 010 | 3 | 0.3000 |
| x4 | 0.05 | 0110 | 4 | 0.2000 |
| x5 | 0.05 | 0111 | 4 | 0.2000 |

#### Entropy, average length, efficiency
- H = −Σ p log₂ p = **1.6710 bits/symbol**
- L = Σ p·l = **1.7000 bits/symbol**
- Efficiency η = H/L = 1.6710/1.7000 = **98.29 %**   (redundancy = 1.71 %)

Huffman codes are not unique when probabilities tie, but L and η are the same for every valid choice.

**Answer: L = 1.7000 bits/symbol, η = 98.29 %**

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What is entropy?
   A. The average information per symbol, H = -sum p log2 p
   B. The maximum symbol probability
   C. The number of symbols
   D. The average code length
   Answer: A. Entropy measures source uncertainty and is the minimum average bits per symbol for lossless coding. (Key point: Entropy definition)

2. What is the first merge for probabilities 0.6, 0.2, 0.1, 0.05, 0.05?
   A. x1 and x2
   B. x4 and x5 to give 0.1
   C. x2 and x3
   D. x1 and x5
   Answer: B. The two smallest, 0.05 and 0.05, are merged first into 0.1. (Key point: Huffman merging)

3. What are L and efficiency?
   A. L = 2.0, eta = 83.5%
   B. L = 1.70 bits/symbol, eta = 98.29%
   C. L = 1.67, eta = 100%
   D. L = 1.70, eta = 89%
   Answer: B. L = 0.6+0.4+0.3+0.2+0.2 = 1.7 and eta = 1.6710/1.7 = 98.29%. (Key point: Result)

## Related in Data Communication

- [Huffman code for p = 0.5, 0.25, 0.125, 0.125](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-p05-025-0125) — Data Communication, chapter 8
- [Huffman code for five symbols (0.2, 0.35, ...)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-five-symbols-02-035) — Data Communication, chapter 8
- [Huffman code for eight symbols (1/2 ... 1/128)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-eight-symbols) — Data Communication, chapter 8
- [Entropy and information rate, 5 outcomes](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Aentropy-info-rate-5-outcomes) — Data Communication, chapter 8
- [Huffman code for symbols A to E (0.4, 0.3, ...)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-table-abcde) — Data Communication, chapter 8
- [Huffman code for a sentence (Kun Mandir)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-kun-mandir) — Data Communication, chapter 8

## Related in other subjects

- [Find x3[n] from X3=X1X2, x1 5-point, x2 4-point](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Adft-product-x3-73s) — Digital Signal Analysis and Processing, chapter 7, asked 3×
- [Variable length coding; entropy and fixed code efficiency](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Avariable-length-coding-entropy-six) — Communication Systems, chapter 6
- [Information rate and coding efficiency: BCD vs Huffman](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Abcd-source-info-rate-huffman) — Communication Systems, chapter 6
- [Algorithm for Huffman code](https://www.bctengineer.com/question.md?key=semester_4%2Fdata_structure_and_algorithm%3Ahuffman-code-algorithm) — Data Structure and Algorithm, chapter 6