Why conjugate gradient over Gaussian methods

Gaussian elimination is a direct method that reduces $[A]$ to triangular form, while the conjugate gradient (CG) method is an iterative method that minimises $\tfrac12x^TAx-b^Tx$ along A-conjugate directions. CG is preferred for large problems for these reasons:

1. Memory: CG needs only matrix-vector products $Ad$, so only the non-zero terms of the sparse matrix are stored. Gaussian elimination produces fill-in, which destroys sparsity inside the band.
2. Operation count: each CG iteration costs about $n\times(\text{non-zeros per row})$ operations, and a good solution often comes in far fewer than $n$ iterations, whereas elimination costs about $n^3/3$ (or $nb^2$ for a banded matrix).
3. Accuracy: round-off errors in elimination accumulate over $n$ steps; in CG each iteration uses the original matrix and corrects the error.
4. Flexibility: the solution can be stopped when the residual is small enough, and a good starting guess can be used; elimination cannot be stopped early.
5. Parallelism: matrix-vector products are easy to parallelise, and the matrix need not even be assembled (element-by-element products).
6. In exact arithmetic CG finishes in at most $n$ steps and with preconditioning converges faster.

Limitation: CG is applicable only to symmetric positive definite matrices, and for several load vectors a factorised direct solver may be cheaper.