Nyquist criteria for zero ISI; duobinary difficulties

Nyquist criterion for zero ISI

Let $p(t)$ be the overall pulse (transmit filter, channel, receive filter), normalized so $p(0) = 1$, and $T_b$ the bit period.

Time domain: the pulse must be zero at every sampling instant except its own:

$$p(nT_b) = \begin{cases} 1, & n = 0 \\ 0, & n \ne 0 \end{cases}$$

Frequency domain: the spectrum and its copies shifted by multiples of $R_b = 1/T_b$ must add to a constant:

$$\sum_{n=-\infty}^{\infty} P\left(f - \frac{n}{T_b}\right) = T_b$$

The minimum-bandwidth solution is $P(f) = T_b\,\text{rect}(fT_b)$ (bandwidth $R_b/2$, pulse $\text{sinc}(t/T_b)$); the practical solution is the raised cosine spectrum with bandwidth $\frac{R_b}{2}(1 + \alpha)$.

Two major difficulties with duobinary encoding and their solutions

Duobinary sends $c_k = a_k + a_{k-1}$ with $H(f) = 2\cos(\pi fT_b)e^{-j\pi fT_b}$.

1. Error propagation

The receiver estimates $\hat a_k = c_k - \hat a_{k-1}$. If one decision is wrong, the wrong value is used for the next decision, and the error carries forward.

Solution: precoding. Before level mapping form

$$d_k = b_k \oplus d_{k-1}$$

Then $c_k = 0$ if $b_k = 1$ and $c_k = \pm 2$ if $b_k = 0$, so the detector decides each bit from its own sample: $|c_k| < 1 \Rightarrow 1$, otherwise 0. No previous decision is needed, so errors do not propagate.

Example ($d_0 = 1$): $b = 0\,1\,1$ gives $d = 1\,0\,1$, $a = +1, -1, +1$, $c = +2, 0, 0$, decoded $0\,1\,1$.

2. Non-zero DC (low-frequency) content

$|H(f)| = 2\cos(\pi fT_b)$ is maximum at $f = 0$, so the duobinary signal has a strong DC component. It cannot pass through AC-coupled (transformer or capacitor) lines, and it is unsuitable for single-sideband transmission, which needs a spectral null near zero frequency.

Solution: modified duobinary signalling. Correlate symbols two bits apart:

$$c_k = a_k - a_{k-2}, \qquad H(f) = 2j\sin(2\pi fT_b)\,e^{-j2\pi fT_b}$$

$H(0) = 0$, so there is no DC component. It is used with the precoder $d_k = b_k \oplus d_{k-2}$ to avoid error propagation.

(Another drawback: three output levels instead of two need more signal power for the same error rate; this is accepted as the price for minimum bandwidth.)