Derive forward, backward, central differences

The finite difference method (FDM) replaces derivatives by algebraic differences of function values at grid points, spaced $h=\Delta x$ apart. The basis is the Taylor series.

Taylor expansions about $x_i$

$$f_{i+1}=f_i+h f'_i+\frac{h^2}{2!}f''_i+\frac{h^3}{3!}f'''_i+\cdots$$ $$f_{i-1}=f_i-h f'_i+\frac{h^2}{2!}f''_i-\frac{h^3}{3!}f'''_i+\cdots$$

Forward difference

Rearrange the first series for $f'_i$:

$$f'_i=\frac{f_{i+1}-f_i}{h}-\frac h2f''_i-\frac{h^2}{6}f'''_i-\cdots$$ $$\boxed{f'_i\approx\frac{f_{i+1}-f_i}{h}},\qquad \text{error}=-\frac h2f''_i=O(h)$$

Backward difference

Rearrange the second series:

$$f'_i=\frac{f_i-f_{i-1}}{h}+\frac h2f''_i-\cdots$$ $$\boxed{f'_i\approx\frac{f_i-f_{i-1}}{h}},\qquad \text{error}=+\frac h2f''_i=O(h)$$

Central difference

Subtract the second series from the first; the even-order terms cancel:

$$f_{i+1}-f_{i-1}=2hf'_i+\frac{h^3}{3}f'''_i+\cdots$$ $$\boxed{f'_i\approx\frac{f_{i+1}-f_{i-1}}{2h}},\qquad \text{error}=-\frac{h^2}{6}f'''_i=O(h^2)$$

Summary

Halving $h$ halves the error of the first-order schemes but reduces the central-difference error to one quarter. Forward and backward differences are used at boundaries and in upwind schemes, where only one side is available.