Self-organizing map (SOM) steps

A self-organizing map (SOM), or Kohonen network, is an unsupervised neural network that maps high-dimensional input vectors onto a low-dimensional (usually 2-D) grid of neurons so that similar inputs activate nearby neurons (topology preserving). It uses competitive learning: neurons compete and only the winner and its neighbours learn.

 inputs  x1  x2  x3  x4
          \   |   |  /     every input connected to
        +---------------+  every output neuron
        | o  o  o  o  o |  output (map) layer
        | o  o  o  o  o |
        +---------------+

Steps

1. Initialise weight vectors w_j (small random values), learning rate α(0) and neighbourhood radius R(0).
2. For each input vector x: a. Compute the squared Euclidean distance D(j) = Σ_i (w_ij − x_i)² for every output node j. b. The node J with the smallest D(j) is the winner (best matching unit). c. Update the winner and nodes within radius R: w_j(new) = w_j(old) + α (x − w_j(old)).
3. Reduce α (and R) according to a schedule, e.g. α(t+1) = 0.5 α(t).
4. Repeat for all inputs (epochs) until the weights stop changing or iterations end.

Example (computed by program)

Four input vectors: (1,1,0,0), (0,0,0,1), (1,0,0,0), (0,0,1,1); two clusters, R = 0, α = 0.6. Initial weights: cluster 1 = (0.2, 0.6, 0.5, 0.9), cluster 2 = (0.8, 0.4, 0.7, 0.3).

Present (1,1,0,0):

• D(1) = (0.2−1)² + (0.6−1)² + 0.5² + 0.9² = 0.64 + 0.16 + 0.25 + 0.81 = 1.86
• D(2) = (0.8−1)² + (0.4−1)² + 0.7² + 0.3² = 0.04 + 0.36 + 0.49 + 0.09 = 0.98
• Winner: cluster 2. New w2 = (0.92, 0.76, 0.28, 0.12).

Next inputs: (0,0,0,1) → D = 0.66 vs 2.28, cluster 1 wins, w1 = (0.08, 0.24, 0.20, 0.96); (1,0,0,0) → cluster 2 wins; (0,0,1,1) → cluster 1 wins. After the first epoch (then α = 0.3, and so on) inputs (1,1,0,0) and (1,0,0,0) go to cluster 2 and (0,0,0,1) and (0,0,1,1) go to cluster 1: two clear clusters found without any labels.

Uses: clustering, data visualisation, speech and image analysis.