---
title: "Information and entropy; capacity as bandwidth → ∞"
subject: "Communication Systems"
chapter: 6
key: "semester_6/communication_systems:information-entropy-capacity-infinite"
source: https://www.bctengineer.com/q/semester_6/communication_systems/information-entropy-capacity-infinite
site: BCT Engineering
---

# Information and entropy; capacity as bandwidth → ∞

Communication Systems · Chapter 6: Baseband Digital Data Transmission

- Asked in: 2070 Asar (CS II), 2+4 marks
- Video: [Channel Capacity by Shannon-Hartley | Basics, Proof & Maximum Bandwidth Condition](https://www.youtube.com/watch?v=3ekWsXeZ8TM) — Engineering Funda
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_6/communication_systems/information-entropy-capacity-infinite
- Whole chapter: https://www.bctengineer.com/md/semester_6/communication_systems/chapter-6

## Question

Define Information and Entropy. Calculate the upper limit of the channel capacity as the bandwidth of the channel B tends to infinity.

## Short answer

Information of a symbol with probability p is log2(1/p) bits; entropy is the average. As bandwidth tends to infinity, channel capacity tends to the finite limit C = 1.44 S/N0 bits/s.

## Answer

#### Information and entropy

- **Information:** the amount of information in a message $m_k$ of probability $p_k$ is $I_k = \log_2\frac{1}{p_k}$ bits. Less likely messages carry more information; a certain message carries none.
- **Entropy:** the average information per symbol of a source:

$$
H = \sum_{k=1}^{M} p_k\log_2\frac{1}{p_k}\ \text{bits/symbol}, \qquad 0 \le H \le \log_2 M
$$

#### Upper limit of channel capacity as $B \to \infty$

Shannon–Hartley: for an AWGN channel with signal power $S$ and noise PSD $N_0$ (so $N = N_0 B$),

$$
C = B\log_2\left(1 + \frac{S}{N_0B}\right)
$$

Let $x = \dfrac{S}{N_0 B}$, so $B = \dfrac{S}{N_0}\cdot\dfrac1x$:

$$
C = \frac{S}{N_0}\cdot\frac{1}{x}\log_2(1+x) = \frac{S}{N_0}\log_2(1+x)^{1/x}
$$

When $B \to \infty$, $x \to 0$, and the standard limit $\lim_{x \to 0}(1+x)^{1/x} = e$ gives

$$
C_\infty = \frac{S}{N_0}\log_2 e = 1.44\,\frac{S}{N_0}\ \text{bits/s}
$$

Capacity stays finite because the noise power grows in proportion to the bandwidth. At this limit the energy per bit satisfies $E_b/N_0 = \ln 2 = 0.693$ (−1.6 dB), the Shannon limit.

**Answer:** $C_\infty = 1.44\,S/N_0$ bits/s.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What is the information in a message of probability p?
   A. log2(1/p) bits
   B. p log2 p
   C. 1/p
   D. p^2
   Answer: A. I = log2(1/p); less likely messages carry more information, and a certain message carries none. (Key point: Information of a message)

2. For M symbols, entropy H lies between:
   A. 0 and 1
   B. 0 and log2 M
   C. 1 and M
   D. log2 M and infinity
   Answer: B. 0 <= H <= log2 M, maximum when all symbols are equally likely. (Key point: Entropy bounds)

3. What is the capacity limit as B tends to infinity for an AWGN channel?
   A. Infinite
   B. 1.44 S/N0 bits/s
   C. S/N0 x 0.693
   D. B log2(S/N0)
   Answer: B. C = (S/N0) log2 e = 1.44 S/N0; capacity stays finite because noise power N0 B grows with B. (Key point: Infinite bandwidth capacity)

4. Why does capacity stay finite as B grows without limit?
   A. Signal power falls with B
   B. Noise power N0 B grows in proportion to bandwidth
   C. Sampling rate is fixed
   D. Entropy is bounded
   Answer: B. More bandwidth admits more noise, so SNR falls as 1/B and the limit (1+x)^(1/x) = e applies. (Key point: Why finite)

## Related in Communication Systems

- [Entropy and information rate; Shannon limit as bandwidth → ∞](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aentropy-info-rate-shannon-limit) — Communication Systems, chapter 6
- [Duobinary error; entropy; capacity as B → ∞](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aduobinary-entropy-capacity-infinite-b) — Communication Systems, chapter 6
- [Shannon-Hartley theorem; capacity limit 1.44S/N₀](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Ashannon-hartley-1-44) — Communication Systems, chapter 6, asked 2×
- [Entropy and information rate of 6-level PCM signal](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aentropy-info-rate-5khz-six-levels) — Communication Systems, chapter 6
- [Symbol rate, entropy and information rate (6 symbols per 10 μs)](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Asymbol-rate-six-symbols-10us) — Communication Systems, chapter 6
- [Message vs information; entropy](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Amessage-vs-information-entropy) — Communication Systems, chapter 6

## Related in other subjects

- [Entropy, information and channel capacity](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Ainformation-entropy-capacity) — Data Communication, chapter 8
- [Define information and entropy](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Adefine-information-entropy) — Data Communication, chapter 8, asked 3×
- [Entropy and information rate, 5 outcomes](https://www.bctengineer.com/question.md?key=semester_5%2Fdata_communication%3Aentropy-info-rate-5-outcomes) — Data Communication, chapter 8
- [Entropy, information gain and ID3 tree](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-entropy-information-gain) — Data Mining, chapter 3