Types of noise; Shannon-Fano and Huffman coding efficiency

Three types of noise

• Thermal (Johnson) noise: caused by random motion of free electrons in a conductor due to temperature. It is white and Gaussian, with power $P_n = kTB$ ($k = 1.38\times10^{-23}$ J/K).
• Shot noise: caused by the random arrival of charge carriers crossing a junction in diodes and transistors. Its mean-square current is $\overline{i_n^2} = 2qI_{dc}B$.
• Flicker ($1/f$) noise: found in semiconductor devices at low frequencies (below a few kHz); its power spectral density varies as $1/f$, due to surface and contact irregularities.

(Others: partition noise, transit-time noise, atmospheric and man-made external noise.)

Coding efficiency

Only five probabilities are given, and they add to 0.82, so the missing sixth probability is $1 - 0.82 = 0.18$ (the standard form of this problem is $P = \{0.36, 0.18, 0.18, 0.12, 0.09, 0.07\}$).

Take $x_1 = 0.36$, $x_2 = 0.18$, $x_3 = 0.12$, $x_4 = 0.09$, $x_5 = 0.07$ and the missing symbol $x_6 = 0.18$.

Entropy:

$$H = 2.3695\ \text{bits/symbol}$$

Shannon–Fano coding (split into groups of nearly equal probability: {0.36, 0.18} = 0.54 vs {0.18, 0.12, 0.09, 0.07} = 0.46, and so on):

$$\begin{aligned} \bar L_{SF} &= 0.36(2)+0.18(2)+0.18(2)+0.12(3)+0.09(4)+0.07(4) \\ &= 0.72+0.36+0.36+0.36+0.36+0.28 = 2.44\ \text{bits/symbol} \\ \eta_{SF} &= \frac{2.3695}{2.44} = 97.11\% \end{aligned}$$

Huffman coding (combine the two lowest each time, move the sum as high as possible; 0 to upper, 1 to lower):

Reduction (sum of two lowest moved up as high as possible):

$$\begin{aligned} \bar L_{H} &= 0.36(2)+0.18(2)+0.18(2)+0.12(3)+0.09(4)+0.07(4) = 2.44\ \text{bits/symbol} \\ \eta_{H} &= \frac{2.3695}{2.44} = 97.11\% \end{aligned}$$

Answer: both Shannon–Fano and Huffman give $\bar L = 2.44$ bits/symbol and efficiency 97.11 % for this source (Huffman is never worse than Shannon–Fano; here they are equal).