---
title: "Entropy of 9 balls in a bin"
subject: "Data Communication"
chapter: 8
key: "semester_5/data_communication:entropy-nine-balls"
source: https://www.bctengineer.com/q/semester_5/data_communication/entropy-nine-balls
site: BCT Engineering
---

# Entropy of 9 balls in a bin

Data Communication · Chapter 8: Information Theory and Coding

- Asked in: 2070 Asar, 2+2 marks
- Video: [Calculate Entropy || Information Theory || Communication Systems || Problem](https://www.youtube.com/watch?v=N5MyS3kvPSw) — EKO Network
- Answer checked by Claude Sonnet 5.5 on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_5/data_communication/entropy-nine-balls
- Whole chapter: https://www.bctengineer.com/md/semester_5/data_communication/chapter-8

## Question

What is the physical meaning of entropy and mention its equation. What is the condition for maximum and minimum entropy? Suppose there are 9 balls in a bin. Out of the 9 balls, 4 balls are red, 2 balls are yellow and 3 balls are green. Calculate the entropy of the system each time a ball is picked from the bin.

## Short answer

Entropy H = −Σ pᵢ log₂ pᵢ measures average uncertainty in bits per draw. It is maximum when all outcomes are equally likely (H_max = log₂ M) and zero when one outcome is certain. For 4 red, 2 yellow, 3 green balls: H = −[(4/9)log₂(4/9)+(2/9)log₂(2/9)+(3/9)log₂(3/9)] ≈ 1.53 bits/draw.

## Answer

#### Physical meaning and equation
**Entropy** is the measure of uncertainty (surprise, randomness) of a source, and the average information in bits per symbol or per draw:

H = −Σ pᵢ log₂ pᵢ  bits

It is also the minimum number of bits per symbol needed to represent the source.

#### Maximum and minimum
- **Maximum:** when all M outcomes are equally likely (pᵢ = 1/M): H_max = log₂ M.
- **Minimum:** H = 0 when one outcome has probability 1 (no uncertainty) and the others have 0.

#### Nine balls
Probabilities: red = 4/9, yellow = 2/9, green = 3/9 = 1/3.

H = −[ (4/9) log₂(4/9) + (2/9) log₂(2/9) + (1/3) log₂(1/3) ]
= (4/9)(1.1699) + (2/9)(2.1699) + (1/3)(1.5850)
= 0.5200 + 0.4822 + 0.5283
= **1.5305 bits per draw**

For comparison H_max = log₂ 3 = 1.585 bits (if the three colours were equally likely).

**Answer: H ≈ 1.53 bits/draw**

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What does entropy physically measure?
   A. Total number of symbols
   B. Average uncertainty/information per symbol (draw) in bits
   C. Signal power
   D. Bandwidth of the source
   Answer: B. Entropy H = -Σ p log2 p is the average information per symbol and the uncertainty of the source. (Key point: Meaning of entropy)

2. When is entropy maximum and minimum?
   A. Max when one outcome has probability 1; min when all equal
   B. Max when all outcomes are equally likely (log2 M); min (0) when one outcome has probability 1
   C. Max is 1 bit always; min is -1
   D. Both occur when probabilities are equal
   Answer: B. Uniform probabilities give H = log2 M; a certain outcome gives H = 0. (Key point: Maximum and minimum entropy)

3. What are the probabilities of red, yellow and green?
   A. 4/9, 2/9, 3/9
   B. 4/9, 3/9, 2/9
   C. 1/3 each
   D. 4/5, 2/5, 3/5
   Answer: A. Out of 9 balls: 4 red, 2 yellow, 3 green gives 4/9, 2/9, 3/9 (= 1/3). (Key point: Probabilities of colours)

4. What is the entropy per draw, approximately?
   A. 1.00 bit
   B. 1.53 bits
   C. 1.585 bits
   D. 2.17 bits
   Answer: B. H = 0.5200 + 0.4822 + 0.5283 = 1.5305 bits, slightly below the maximum log2 3 = 1.585. (Key point: Entropy value)

## Related in Data Communication

- [Entropy and information rate, 5 outcomes](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Aentropy-info-rate-5-outcomes) — Data Communication, chapter 8

## Related in other subjects

- [Minimum speed of ball in a box](https://www.bctengineer.com/question.md?key=semester_1%2Fengineering_physics%3Afootball-box-minimum-speed) — Engineering Physics, chapter 10
- [Maximum demand and need to calculate it](https://www.bctengineer.com/question.md?key=semester_6%2Findustrial_power_distribution_and_illumination%3Amaximum-demand-need) — Industrial Power Distribution and Illumination, chapter 1
- [Maximum efficiency and ball screws](https://www.bctengineer.com/question.md?key=semester_7%2Fmachine_design_ii%3Aself-locking-maximum-efficiency-ball-screw) — Machine Design II, chapter 8
- [Physical meaning of equilibrium and its application](https://www.bctengineer.com/question.md?key=semester_1%2Fapplied_mechanics%3Aphysical-meaning-equilibrium-structural) — Applied Mechanics, chapter 2