Image recognition steps

Image recognition is the process of identifying objects or patterns in a digital image automatically. It is part of image analysis, and goes from the raw picture to a decision such as "this is the letter A" or "this is a human face". The usual steps are the following.

 Image formation -> Conditioning -> Labelling
                                        |
 Matching <- Extracting <- Grouping <---+

1. Image formation

The scene is captured by a camera or scanner and sampled and quantised into a digital image (a matrix of pixels). Resolution, lighting and sensor quality decide how good the input is.

2. Conditioning (pre-processing)

The image is cleaned to make later steps reliable: noise removal (median or Gaussian filter), contrast and brightness adjustment, correction of distortion and conversion to grey scale or binary.

3. Labelling

Basic features that tell which pixels belong to something of interest are found. Typically edge detection (Sobel, Canny), corner detection and thresholding to separate object from background (segmentation). Each pixel or region gets a label, such as "edge" or "object".

4. Grouping

Labelled pixels are joined into larger structures: connected edge points become lines and curves, neighbouring object pixels become regions (connected components). Now we have candidate objects.

5. Extracting

Measurable properties (features) of each group are computed: area, perimeter, centroid, orientation, shape descriptors, colour, texture, number of holes. These form a feature vector.

6. Matching

The feature vector is compared with stored models or templates (or fed to a classifier such as a neural network, k-nearest neighbour) and the best match gives the identity. A confidence value is produced.

Example: recognising a handwritten digit

1. A scanner reads the cheque amount field (formation).
2. Noise specks are removed and the image is made binary (conditioning).
3. Stroke edges are detected (labelling).
4. Edge points are joined into strokes and loops (grouping).
5. Features are measured: one closed loop, one vertical stroke, height-to-width ratio (extracting).
6. The vector matches the stored model of digit "6" best, so the output is 6 (matching).

Other examples are face recognition for attendance, number plate reading and fingerprint matching.