- 2068 Chaitra · 6 marks
Entropy and information rate, 5 outcomes
Short answer
Entropy H = ∑ p log2(1/p) = (1/2)*1 + (1/4)*2 + (1/8)*3 + (1/16)*4 + (1/16)*4 = 1.875 bits/outcome. With 16 outcomes per second, information rate R = 16 × 1.875 = 30 bits/s.
The probability of the five possible outcomes of an experiment are given as: P(x1) = 1/2, P(x2) = 1/4, P(x3) = 1/8, P(x4) = P(x5) = 1/16. Determine the entropy and information rate if there are 16 outcomes per second.
Entropy
Probabilities: 1/2, 1/4, 1/8, 1/16, 1/16.
H = Σ pᵢ log₂(1/pᵢ) = (1/2)(1) + (1/4)(2) + (1/8)(3) + (1/16)(4) + (1/16)(4) = 0.5 + 0.5 + 0.375 + 0.25 + 0.25 = 1.875 bits/outcome
Information rate
R = r · H, with r = 16 outcomes per second:
R = 16 × 1.875 = 30 bits/s
Answer: H = 1.875 bits/outcome, R = 30 bits/second
✓ Answer checked by Claude Sonnet 5.5
Related questions
- Huffman code for five symbols (0.2, 0.35, ...) · Data Communication, chapter 8
- Define information and entropy · Data Communication, chapter 8 · asked 3×
- Entropy, information and channel capacity · Data Communication, chapter 8
- Huffman code for p = 0.6, 0.2, 0.1, 0.05, 0.05 · Data Communication, chapter 8
- Shannon-Fano code for eight symbols · Data Communication, chapter 8
- Huffman code for eight symbols (1/2 ... 1/128) · Data Communication, chapter 8
- Symbol rate, entropy and information rate (6 symbols per 10 μs) · Communication Systems, chapter 6
- Find x3[n] from X3=X1X2, x1 5-point, x2 4-point · Digital Signal Analysis and Processing, chapter 7 · asked 3×
- Entropy and information rate of 6-level PCM signal · Communication Systems, chapter 6
- Entropy and information rate; Shannon limit as bandwidth → ∞ · Communication Systems, chapter 6