Information rate of a quantized 100 Hz signal

Given: $f_m = 100$ Hz, sampled at Nyquist rate; four symbols with $p_1 = p_4 = 0.125$ and $p_2 = p_3$.

Step 1: find $p_2$ and $p_3$. Probabilities add to 1:

$$\begin{aligned} p_1 + p_2 + p_3 + p_4 &= 1 \\ 0.125 + 2p_2 + 0.125 &= 1 \;\Rightarrow\; p_2 = p_3 = 0.375 \end{aligned}$$

Step 2: symbol rate. Nyquist sampling rate:

$$r = f_s = 2f_m = 2 \times 100 = 200\ \text{samples (symbols)/s}$$

Step 3: entropy.

$$\begin{aligned} H &= 2(0.125)\log_2\frac{1}{0.125} + 2(0.375)\log_2\frac{1}{0.375} \\ &= 2(0.125)(3) + 2(0.375)(1.4150) \\ &= 0.75 + 1.0613 = 1.8113\ \text{bits/symbol} \end{aligned}$$

Step 4: information rate.

$$R = rH = 200 \times 1.8113 = 362.26\ \text{bits/s}$$

Note: with 2 bits/sample (fixed coding) the bit rate would be 400 bits/s, so the source information rate is less than the coded bit rate.

Answer: information rate $R \approx$ 362.26 bits/s ($H = 1.8113$ bits/symbol, $r = 200$ symbols/s).