Skip to main content

Entropy of 9 balls in a bin

Short answer

Entropy H = −Σ pᵢ log₂ pᵢ measures average uncertainty in bits per draw. It is maximum when all outcomes are equally likely (H_max = log₂ M) and zero when one outcome is certain. For 4 red, 2 yellow, 3 green balls: H = −[(4/9)log₂(4/9)+(2/9)log₂(2/9)+(3/9)log₂(3/9)] ≈ 1.53 bits/draw.

What is the physical meaning of entropy and mention its equation. What is the condition for maximum and minimum entropy? Suppose there are 9 balls in a bin. Out of the 9 balls, 4 balls are red, 2 balls are yellow and 3 balls are green. Calculate the entropy of the system each time a ball is picked from the bin.

Physical meaning and equation

Entropy is the measure of uncertainty (surprise, randomness) of a source, and the average information in bits per symbol or per draw:

H = −Σ pᵢ log₂ pᵢ bits

It is also the minimum number of bits per symbol needed to represent the source.

Maximum and minimum

  • Maximum: when all M outcomes are equally likely (pᵢ = 1/M): H_max = log₂ M.
  • Minimum: H = 0 when one outcome has probability 1 (no uncertainty) and the others have 0.

Nine balls

Probabilities: red = 4/9, yellow = 2/9, green = 3/9 = 1/3.

H = −[ (4/9) log₂(4/9) + (2/9) log₂(2/9) + (1/3) log₂(1/3) ] = (4/9)(1.1699) + (2/9)(2.1699) + (1/3)(1.5850) = 0.5200 + 0.4822 + 0.5283 = 1.5305 bits per draw

For comparison H_max = log₂ 3 = 1.585 bits (if the three colours were equally likely).

Answer: H ≈ 1.53 bits/draw

✓ Answer checked by Claude Sonnet 5.5

Related questions

All Data Communication chapter 8 questions →