Hopfield network and its steps

A Hopfield network (John Hopfield, 1982) is a single-layer, fully connected recurrent network that works as an auto-associative (content-addressable) memory. Each neuron is connected to every other neuron with symmetric weights (wij = wji) and no self-connection (wii = 0). Neurons take binary or bipolar states (+1 / −1). Given a noisy or partial pattern it settles to the nearest stored pattern.

   (1)---(2)
    | \ / |
    | / \ |
   (4)---(3)     every neuron connected to all others

Steps

Training (storing patterns)

1. Choose bipolar patterns p to store.
2. Compute weights by the Hebbian outer-product rule: W = Σ pᵀp, and set the diagonal to 0 (wij = Σ p_i p_j, i ≠ j).

Recall (testing) 3. Initialise the network state x with the input (noisy) pattern. 4. Pick a neuron i at random (asynchronous update) and compute net_i = Σ_j w_ij x_j (plus any external input). 5. Update: x_i = +1 if net_i > 0, −1 if net_i < 0, unchanged if net_i = 0. 6. Repeat steps 4-5 until no neuron changes (stable state). The network always converges because the energy E = −½ Σ Σ w_ij x_i x_j never increases.

Example (verified)

Store p = [1, −1, 1, −1]. Weights wij = pi·pj for i ≠ j:

      n1  n2  n3  n4
 n1 [  0  -1   1  -1 ]
 n2 [ -1   0  -1   1 ]
 n3 [  1  -1   0  -1 ]
 n4 [ -1   1  -1   0 ]

Test with noisy input x = [1, 1, 1, −1] (second bit wrong):

• n1: net = 0·1 + (−1)(1) + 1·1 + (−1)(−1) = 1, so x1 = +1 (no change)
• n2: net = (−1)(1) + 0 + (−1)(1) + 1·(−1) = −3, so x2 = −1 (corrected)
• n3: net = 3 → +1; n4: net = −3 → −1

The state becomes [1, −1, 1, −1], the stored pattern, and stays stable on another sweep.

Limits: capacity is only about 0.15 N patterns for N neurons, and it may settle in spurious local minima.