Binary error probability; threshold effect in FM and mitigation

Error probability in a binary communication system

Model. In a binary system, symbol 1 is sent as $s_1(t)$ and 0 as $s_2(t)$, each lasting $T_b$, with equal probability. Channel adds white Gaussian noise $w(t)$ of PSD $N_0/2$. The receiver filter output is sampled at $t=T_b$:

 s_i+w  +--------+ y(t)  sample  y  +---------+
 ------>| filter |------>-/ ------->| y > lam?|--> 1/0
        +--------+       t=Tb       +---------+

The sample is $y=a_i+n$, where $a_1, a_2$ are the noise-free outputs ($a_1>a_2$) and $n$ is Gaussian with zero mean and variance $\sigma^2$. Conditional pdfs:

$$f(y\mid1)=\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_1)^2/2\sigma^2},\qquad f(y\mid0)=\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_2)^2/2\sigma^2}$$

   f(y|0)            f(y|1)
     .--.             .--.
    /    \     |     /    \
   /      \    |    /      \
  /     ...\...|.../...     \
 --------a2---lam---a1----------> y
        Pe0 = area right of lam under f(y|0)
        Pe1 = area left  of lam under f(y|1)

Decision rule: choose 1 if $y>\lambda$, else 0. For equal priors the optimum threshold is midway, $\lambda=\frac{a_1+a_2}{2}$.

Error when 0 is sent ($y>\lambda$): with $u=(y-a_2)/\sqrt2\sigma$,

$$\begin{aligned} P_{e0} &= \int_{\lambda}^{\infty}\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_2)^2/2\sigma^2}dy=\frac{1}{\sqrt{\pi}}\int_{(\lambda-a_2)/\sqrt2\sigma}^{\infty}e^{-u^2}du \\ &= \frac12\operatorname{erfc}\left(\frac{a_1-a_2}{2\sqrt2\,\sigma}\right) \end{aligned}$$

By symmetry $P_{e1}$ (1 sent, $y<\lambda$) is the same. The average error probability is

$$P_e=\tfrac12P_{e0}+\tfrac12P_{e1}=\frac12\operatorname{erfc}\left(\frac{a_1-a_2}{2\sqrt2\,\sigma}\right)=Q\left(\frac{a_1-a_2}{2\sigma}\right)$$

where $\operatorname{erfc}(x)=\frac{2}{\sqrt\pi}\int_x^{\infty}e^{-u^2}du$ and $Q(x)=\frac12\operatorname{erfc}(x/\sqrt2)$.

With a matched filter (matched to $s_1-s_2$), the maximum of $\frac{(a_1-a_2)^2}{\sigma^2}$ is $\frac{2E_d}{N_0}$, where $E_d=\int_0^{T_b}[s_1(t)-s_2(t)]^2dt$. Therefore

$$P_e=\frac12\operatorname{erfc}\left(\sqrt{\frac{E_d}{4N_0}}\right)$$

$P_e$ depends only on the energy of the difference signal relative to $N_0$, not on pulse shape. Making the two signals as different as possible (antipodal) gives the lowest $P_e$.

($E_b$ = average energy per bit.)

Threshold effect in FM

Threshold effect in FM. The FM improvement $\text{SNR}_o\propto\beta^2$ holds only when the carrier is much stronger than the noise at the discriminator input. When the carrier-to-noise ratio (CNR) falls below a threshold, about 10 dB (typically 10–13 dB), the noise phasor sometimes becomes larger than the carrier and the resultant phasor encircles the origin, causing a sudden $2\pi$ phase jump.

  high CNR: noise wobbles     low CNR: resultant can swing
  the tip slightly            around origin -> 2pi jump
        .-.                         .---.
  0 -->( * )                   0 --( *   )
        '-'                         '---'

Each $2\pi$ jump produces an impulse (spike) in the discriminator output, heard as clicks. The click rate rises rapidly as CNR drops, so the output SNR falls much faster than linearly and the signal is mutilated. Above threshold, FM is excellent; below it, FM can perform worse than AM. Wider deviation (larger $\beta$) means larger $B_T$, more noise and a higher threshold.

 SNRo(dB)            . FM above threshold
     |             .   (slope 1, offset 3/2 beta^2)
     |           .
     |         .
     |       . <- threshold knee (CNR ~ 10 dB)
     |     .:
     |   . :     AM / baseband
     |  .  :
     +------------------> SNRi (dB)

Mitigation of threshold effect

Techniques to reduce (extend) the threshold:

1. FM feedback (FMFB) demodulator: the VCO output tracks the signal, so the IF bandwidth needed is reduced (compressed deviation); less noise enters, threshold lowered by about 5–7 dB.
2. Phase-locked loop (PLL) demodulator: the loop's narrow bandwidth tracks the instantaneous frequency and rejects noise outside it; gives a few dB threshold extension.
3. Pre-emphasis and de-emphasis: boost high message frequencies before modulation and cut them after detection; this reduces the high-frequency (parabolic) noise and improves the output SNR by about 10–13 dB, although it does not itself lower the threshold.
4. Reducing IF bandwidth / modulation index where possible, or increasing transmitted power (raises CNR above threshold).
5. Diversity reception, which helps keep the CNR above threshold during fading.