Error probability in a binary communication system

Model. In a binary system, symbol 1 is sent as $s_1(t)$ and 0 as $s_2(t)$, each lasting $T_b$, with equal probability. Channel adds white Gaussian noise $w(t)$ of PSD $N_0/2$. The receiver filter output is sampled at $t=T_b$:

 s_i+w  +--------+ y(t)  sample  y  +---------+
 ------>| filter |------>-/ ------->| y > lam?|--> 1/0
        +--------+       t=Tb       +---------+

The sample is $y=a_i+n$, where $a_1, a_2$ are the noise-free outputs ($a_1>a_2$) and $n$ is Gaussian with zero mean and variance $\sigma^2$. Conditional pdfs:

$$f(y\mid1)=\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_1)^2/2\sigma^2},\qquad f(y\mid0)=\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_2)^2/2\sigma^2}$$

   f(y|0)            f(y|1)
     .--.             .--.
    /    \     |     /    \
   /      \    |    /      \
  /     ...\...|.../...     \
 --------a2---lam---a1----------> y
        Pe0 = area right of lam under f(y|0)
        Pe1 = area left  of lam under f(y|1)

Decision rule: choose 1 if $y>\lambda$, else 0. For equal priors the optimum threshold is midway, $\lambda=\frac{a_1+a_2}{2}$.

Error when 0 is sent ($y>\lambda$): with $u=(y-a_2)/\sqrt2\sigma$,

$$\begin{aligned} P_{e0} &= \int_{\lambda}^{\infty}\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_2)^2/2\sigma^2}dy=\frac{1}{\sqrt{\pi}}\int_{(\lambda-a_2)/\sqrt2\sigma}^{\infty}e^{-u^2}du \\ &= \frac12\operatorname{erfc}\left(\frac{a_1-a_2}{2\sqrt2\,\sigma}\right) \end{aligned}$$

By symmetry $P_{e1}$ (1 sent, $y<\lambda$) is the same. The average error probability is

$$P_e=\tfrac12P_{e0}+\tfrac12P_{e1}=\frac12\operatorname{erfc}\left(\frac{a_1-a_2}{2\sqrt2\,\sigma}\right)=Q\left(\frac{a_1-a_2}{2\sigma}\right)$$

where $\operatorname{erfc}(x)=\frac{2}{\sqrt\pi}\int_x^{\infty}e^{-u^2}du$ and $Q(x)=\frac12\operatorname{erfc}(x/\sqrt2)$.

With a matched filter (matched to $s_1-s_2$), the maximum of $\frac{(a_1-a_2)^2}{\sigma^2}$ is $\frac{2E_d}{N_0}$, where $E_d=\int_0^{T_b}[s_1(t)-s_2(t)]^2dt$. Therefore

$$P_e=\frac12\operatorname{erfc}\left(\sqrt{\frac{E_d}{4N_0}}\right)$$

$P_e$ depends only on the energy of the difference signal relative to $N_0$, not on pulse shape. Making the two signals as different as possible (antipodal) gives the lowest $P_e$.

Special cases:

($E_b$ = average energy per bit.)