Short note: Neural networks

An Artificial Neural Network (ANN) is a computing model inspired by the human brain: a network of simple, connected processing units (neurons) that learns from examples. It is used for classification, prediction and pattern recognition.

Structure

• Input layer: one node per attribute.
• Hidden layer(s): process the signals.
• Output layer: gives the class.
• Each link has a weight. A neuron computes the weighted sum of inputs plus a bias and passes it through an activation function (step, sigmoid): y = f(Σ wᵢxᵢ + b).

 x1 --w1--\
 x2 --w2---> [ Σ + f ] --> output y
 x3 --w3--/

Learning

Weights start random and are adjusted by training on labelled tuples. The most common algorithm is backpropagation: the output error is propagated backwards and the weights are updated by gradient descent until the error is small.

Features

• Can model complex non-linear relations and tolerates noisy data.
• Works on continuous and discrete data and gives good accuracy.
• Weaknesses: long training time, many parameters to tune, and a "black box" model that is hard to interpret.