---
title: "Entropy and information rate, 5 outcomes"
subject: "Data Communication"
chapter: 8
key: "semester_5/data_communication:entropy-info-rate-5-outcomes"
source: https://www.bctengineer.com/q/semester_5/data_communication/entropy-info-rate-5-outcomes
site: BCT Engineering
---

# Entropy and information rate, 5 outcomes

Data Communication · Chapter 8: Information Theory and Coding

- Asked in: 2068 Chaitra, 6 marks
- Video: [Calculate Source Entropy and Information Rate | Communication system.](https://www.youtube.com/watch?v=pi1XA8j5dcY) — ELECTRICAL HUB
- Answer checked by Claude Sonnet 5.5 on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_5/data_communication/entropy-info-rate-5-outcomes
- Whole chapter: https://www.bctengineer.com/md/semester_5/data_communication/chapter-8

## Question

The probability of the five possible outcomes of an experiment are given as: P(x1) = 1/2, P(x2) = 1/4, P(x3) = 1/8, P(x4) = P(x5) = 1/16. Determine the entropy and information rate if there are 16 outcomes per second.

## Short answer

Entropy H = ∑ p log2(1/p) = (1/2)*1 + (1/4)*2 + (1/8)*3 + (1/16)*4 + (1/16)*4 = 1.875 bits/outcome. With 16 outcomes per second, information rate R = 16 × 1.875 = 30 bits/s.

## Answer

#### Entropy
Probabilities: 1/2, 1/4, 1/8, 1/16, 1/16.

H = Σ pᵢ log₂(1/pᵢ)
= (1/2)(1) + (1/4)(2) + (1/8)(3) + (1/16)(4) + (1/16)(4)
= 0.5 + 0.5 + 0.375 + 0.25 + 0.25
= **1.875 bits/outcome**

#### Information rate
R = r · H, with r = 16 outcomes per second:

R = 16 × 1.875 = **30 bits/s**

**Answer: H = 1.875 bits/outcome, R = 30 bits/second**

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. Which formula gives the entropy?
   A. H = Σ p log2(1/p)
   B. H = Σ p^2
   C. H = Σ log2 p
   D. H = Σ p / log2 p
   Answer: A. H = Σ p log2(1/p), the average information per outcome. (Key point: Entropy formula)

2. What is H for probabilities 1/2, 1/4, 1/8, 1/16, 1/16?
   A. 1.5 bits
   B. 1.875 bits
   C. 2.32 bits
   D. 1.0 bit
   Answer: B. H = 0.5 + 0.5 + 0.375 + 0.25 + 0.25 = 1.875 bits/outcome. (Key point: Entropy value)

3. How is information rate found, and what is it for 16 outcomes/s?
   A. R = r·H = 30 bits/s
   B. R = H/r = 0.117 bits/s
   C. R = r + H = 17.9 bits/s
   D. R = r·log2 5 = 37.1 bits/s
   Answer: A. R = r·H = 16 × 1.875 = 30 bits/s. (Key point: Information rate)

## Related in Data Communication

- [Huffman code for five symbols (0.2, 0.35, ...)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-five-symbols-02-035) — Data Communication, chapter 8
- [Define information and entropy](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Adefine-information-entropy) — Data Communication, chapter 8, asked 3×
- [Entropy, information and channel capacity](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ainformation-entropy-capacity) — Data Communication, chapter 8
- [Huffman code for p = 0.6, 0.2, 0.1, 0.05, 0.05](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-p06-02-01) — Data Communication, chapter 8
- [Shannon-Fano code for eight symbols](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ashannon-fano-eight-symbols) — Data Communication, chapter 8
- [Huffman code for eight symbols (1/2 ... 1/128)](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ahuffman-eight-symbols) — Data Communication, chapter 8

## Related in other subjects

- [Symbol rate, entropy and information rate (6 symbols per 10 μs)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asymbol-rate-six-symbols-10us) — Communication Systems, chapter 6
- [Find x3[n] from X3=X1X2, x1 5-point, x2 4-point](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Adft-product-x3-73s) — Digital Signal Analysis and Processing, chapter 7, asked 3×
- [Entropy and information rate of 6-level PCM signal](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aentropy-info-rate-5khz-six-levels) — Communication Systems, chapter 6
- [Entropy and information rate; Shannon limit as bandwidth → ∞](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aentropy-info-rate-shannon-limit) — Communication Systems, chapter 6