---
title: "Binary baseband error probability; comparison with M-ary"
subject: "Communication Systems"
chapter: 9
key: "semester_6/communication_systems:error-prob-binary-vs-mary"
source: https://www.bctengineer.com/q/semester_6/communication_systems/error-prob-binary-vs-mary
site: BCT Engineering
---

# Binary baseband error probability; comparison with M-ary

Communication Systems · Chapter 9: Noise in Communication Systems

- Asked in: 2076 Asoj (CS II), 8 marks
- Video: [ESE 471 A General Formula for Prob Symbol Error in M ary PAM](https://www.youtube.com/watch?v=6o-uUbFNicQ) — Neal Patwari
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_6/communication_systems/error-prob-binary-vs-mary
- Whole chapter: https://www.bctengineer.com/md/semester_6/communication_systems/chapter-9

## Question

Derive the expression for evaluating error probability in binary baseband system and compare it with M-ary system.

## Short answer

Binary baseband error probability is P_e = 1/2 erfc((a1-a2)/(2√2 σ)) = Q((a1-a2)/(2σ)); with matched filter, P_e = 1/2 erfc(√(E_d/(4N_0))) where E_d is difference signal energy. M-ary systems save bandwidth by factor log2 M but require more power for same P_e, trading power for bandwidth with P_e ≈ (1-1/M) erfc(A/(√2 σ)) for PAM.

## Answer

#### Error probability in binary baseband system

**Model.** In a binary system, symbol 1 is sent as $s_1(t)$ and 0 as $s_2(t)$, each lasting $T_b$, with equal probability. Channel adds white Gaussian noise $w(t)$ of PSD $N_0/2$. The receiver filter output is sampled at $t=T_b$:

```text
 s_i+w  +--------+ y(t)  sample  y  +---------+
 ------>| filter |------>-/ ------->| y > lam?|--> 1/0
        +--------+       t=Tb       +---------+
```

The sample is $y=a_i+n$, where $a_1, a_2$ are the noise-free outputs ($a_1>a_2$) and $n$ is Gaussian with zero mean and variance $\sigma^2$. Conditional pdfs:

$$
f(y\mid1)=\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_1)^2/2\sigma^2},\qquad f(y\mid0)=\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_2)^2/2\sigma^2}
$$

```text
   f(y|0)            f(y|1)
     .--.             .--.
    /    \     |     /    \
   /      \    |    /      \
  /     ...\...|.../...     \
 --------a2---lam---a1----------> y
        Pe0 = area right of lam under f(y|0)
        Pe1 = area left  of lam under f(y|1)
```

**Decision rule:** choose 1 if $y>\lambda$, else 0. For equal priors the optimum threshold is midway, $\lambda=\frac{a_1+a_2}{2}$.

**Error when 0 is sent** ($y>\lambda$): with $u=(y-a_2)/\sqrt2\sigma$,

$$
\begin{aligned}
P_{e0} &= \int_{\lambda}^{\infty}\frac{1}{\sqrt{2\pi}\sigma}e^{-(y-a_2)^2/2\sigma^2}dy=\frac{1}{\sqrt{\pi}}\int_{(\lambda-a_2)/\sqrt2\sigma}^{\infty}e^{-u^2}du \\
&= \frac12\operatorname{erfc}\left(\frac{a_1-a_2}{2\sqrt2\,\sigma}\right)
\end{aligned}
$$

By symmetry $P_{e1}$ (1 sent, $y<\lambda$) is the same. The average error probability is

$$
P_e=\tfrac12P_{e0}+\tfrac12P_{e1}=\frac12\operatorname{erfc}\left(\frac{a_1-a_2}{2\sqrt2\,\sigma}\right)=Q\left(\frac{a_1-a_2}{2\sigma}\right)
$$

where $\operatorname{erfc}(x)=\frac{2}{\sqrt\pi}\int_x^{\infty}e^{-u^2}du$ and $Q(x)=\frac12\operatorname{erfc}(x/\sqrt2)$.

**With a matched filter** (matched to $s_1-s_2$), the maximum of $\frac{(a_1-a_2)^2}{\sigma^2}$ is $\frac{2E_d}{N_0}$, where $E_d=\int_0^{T_b}[s_1(t)-s_2(t)]^2dt$. Therefore

$$
P_e=\frac12\operatorname{erfc}\left(\sqrt{\frac{E_d}{4N_0}}\right)
$$

$P_e$ depends only on the **energy of the difference signal** relative to $N_0$, not on pulse shape. Making the two signals as different as possible (antipodal) gives the lowest $P_e$.

For polar NRZ ($\pm A$ at the sampler): $P_e=\frac12\operatorname{erfc}\left(\frac{A}{\sqrt2\sigma}\right)$.

#### Comparison with M-ary system

For M-ary PAM with levels $\pm A,\pm3A,\dots,\pm(M-1)A$ (spacing $2A$), inner levels can err on both sides and outer levels on one side, giving

$$
P_e=\left(1-\frac1M\right)\operatorname{erfc}\left(\frac{A}{\sqrt2\,\sigma}\right),\qquad P_{av}=\frac{(M^2-1)A^2}{3}
$$

which reduces to the binary result for $M=2$.

| Point | Binary ($M=2$) | M-ary ($M=2^k$) |
|---|---|---|
| Bits per symbol | 1 | $k=\log_2M$ |
| Bandwidth for same bit rate | $R_b/2$ (Nyquist) | $R_b/(2\log_2M)$, i.e. $k$ times less |
| Level spacing for same peak power | Large | Smaller, by factor $M-1$ |
| $P_e$ for same average power | Lowest | Higher; needs about $(M^2-1)/3$ times more power |
| Thresholds | One | $M-1$ |
| Receiver complexity | Simple | More complex |
| Typical use | Noisy, power-limited links | Band-limited links (e.g. telephone modems) |

So M-ary signalling **trades power for bandwidth**: it saves bandwidth but needs more signal power for the same error probability.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. Which gives the binary baseband Pe with matched filter, in terms of difference-signal energy Ed?
   A. (1/2) erfc sqrt(Ed/4N0)
   B. (1/2) erfc sqrt(Ed/N0)
   C. Q( sqrt(Ed/4N0) )
   D. erfc sqrt(2Ed/N0)
   Answer: A. Maximum (a1-a2)^2/sigma^2 = 2Ed/N0 gives (1/2) erfc sqrt(Ed/4N0). (Key point: Binary Pe)

2. For M-ary PAM with level spacing 2A, what is the symbol error probability?
   A. (1/M) erfc( A/(sqrt2 sigma) )
   B. (1 - 1/M) erfc( A/(sqrt2 sigma) )
   C. (1/2) erfc( A/(sqrt2 sigma) ) for all M
   D. M erfc( A/(sqrt2 sigma) )
   Answer: B. Inner levels err on both sides, outer on one side; averaging gives (1 - 1/M) erfc, which equals the binary result at M = 2. (Key point: M-ary PAM Pe)

3. For the same bit rate, how does M-ary bandwidth compare with binary?
   A. log2 M times greater
   B. The same
   C. M times less
   D. log2 M times less
   Answer: D. Each symbol carries k = log2 M bits, so the symbol rate and bandwidth fall by k. (Key point: Bandwidth trade-off)

4. For the same Pe and average power, what happens as M increases?
   A. Spacing shrinks, so about (M^2 - 1)/3 times more power is needed
   B. Power requirement falls
   C. Power stays the same
   D. Only one threshold is needed
   Answer: A. P_av = (M^2-1)A^2/3, so level spacing shrinks and more power is needed: M-ary trades power for bandwidth. (Key point: Power penalty)

## Close variants set in other exams

- [Error probability in binary baseband system](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aerror-probability-binary-baseband) — Communication Systems, chapter 9, asked 2×
- [Error probability of M-ary system](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Amary-error-probability) — Communication Systems, chapter 9, asked 2×

## Related in Communication Systems

- [Error probability of binary ASK extended to M-ary](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aask-error-prob-mary) — Communication Systems, chapter 9
- [M-ary error probability; binary vs M-ary](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Amary-error-compare-binary) — Communication Systems, chapter 9
- [Binary PAM error probability extended to M-ary](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Abinary-pam-mary-error) — Communication Systems, chapter 9
- [Error probability in a binary communication system](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aerror-probability-binary-system) — Communication Systems, chapter 9
- [Binary error probability; threshold effect in FM and mitigation](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Aerror-prob-threshold-effect) — Communication Systems, chapter 9, asked 2×
- [Short note: M-ary baseband communication system](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Ashort-note-mary-baseband) — Communication Systems, chapter 6, asked 3×
