Skip to main content

Message vs information; 5 symbols per 10 μs

Short answer

Message is the actual symbol; information is the uncertainty removed, quantified as –log2(p) bits. Symbol rate = 10^5 symbols/s. Entropy = 1.875 bits/symbol. Information rate = 187.5 kbps.

Differentiate between message and information? A discrete source is emitting one of 5 possible symbols per 10 microsec. The probabilities are 1/2, 1/4, 1/8, 1/16 and 1/16. Find (a) Symbol rate (b) Source entropy (c) Information rate

Message vs information

MessageInformation
The actual symbol or sequence produced by the sourceMeasure of uncertainty removed by receiving the message
Has physical form and meaningDepends only on probability, not meaning
Counted in symbolsMeasured in bits: I=log⁡2(1/p)I = \log_2(1/p)
A certain message is still a messageA certain message (p=1p = 1) carries 0 bits

Numerical

One of 5 symbols every 10 μ10\ \mus with P={12,14,18,116,116}P = \{\tfrac12, \tfrac14, \tfrac18, \tfrac1{16}, \tfrac1{16}\}.

(a) Symbol rate

r=110×10−6=105 symbols/sr = \frac{1}{10\times10^{-6}} = 10^5\ \text{symbols/s}

(b) Source entropy

Symbolpip_ilog⁡2(1/pi)\log_2(1/p_i)pilog⁡2(1/pi)p_i\log_2(1/p_i)
x10.51.00000.5000
x20.252.00000.5000
x30.1253.00000.3750
x40.06254.00000.2500
x50.06254.00000.2500
Total11.8750
H=12(1)+14(2)+18(3)+116(4)+116(4)=0.5+0.5+0.375+0.25+0.25=1.875 bits/symbol\begin{aligned} H &= \tfrac12(1) + \tfrac14(2) + \tfrac18(3) + \tfrac1{16}(4) + \tfrac1{16}(4) \\ &= 0.5 + 0.5 + 0.375 + 0.25 + 0.25 = 1.875\ \text{bits/symbol} \end{aligned}

(c) Information rate

R=rH=105×1.875=1.875×105 bits/s=187.5 kbpsR = rH = 10^5 \times 1.875 = 1.875\times10^5\ \text{bits/s} = 187.5\ \text{kbps}

Answer: r=105r = 10^5 symbols/s, H=1.875H = 1.875 bits/symbol, R=187.5R = 187.5 kbps.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Communication Systems chapter 6 questions →