---
title: "Machine learning; Boltzmann machines"
subject: "Artificial Intelligence"
chapter: 6
key: "semester_6/artificial_intelligence:ml-boltzmann-machine"
source: https://www.bctengineer.com/q/semester_6/artificial_intelligence/ml-boltzmann-machine
site: BCT Engineering
---

# Machine learning; Boltzmann machines

Artificial Intelligence · Chapter 6: Machine Learning

- Asked in: 2072 Asoj, 2+8 marks
- Video: [Introduction to Boltzmann Machines](https://www.youtube.com/watch?v=1HrOkb-RL0Y) — Juefei Yuan
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_6/artificial_intelligence/ml-boltzmann-machine
- Whole chapter: https://www.bctengineer.com/md/semester_6/artificial_intelligence/chapter-6

## Question

What is machine learning? Explain in detail about Boltzmann machines with suitable algorithm and explanations.

## Short answer

Machine learning improves task performance from data. A Boltzmann machine is a stochastic recurrent network of binary units with symmetric weights and an energy function. Learning involves positive and negative phases to adjust weights by comparing clamped and free equilibrium statistics.

## Answer

#### Machine learning
**Machine learning** is the branch of AI in which a program improves its performance at a task through experience (data) rather than explicit programming. Main kinds: supervised, unsupervised and reinforcement learning.

#### Boltzmann machine
A **Boltzmann machine** (Hinton and Sejnowski, 1985) is a **stochastic recurrent neural network** of binary units (0/1 or ±1) joined by **symmetric weights** (wij = wji, wii = 0). Units are of two kinds: **visible** units (input/output) and **hidden** units. It is the stochastic version of the Hopfield network; simulated annealing lets it escape local minima. It is named after the Boltzmann distribution of statistical mechanics.

```text
   hidden:   (h1)---(h2)---(h3)
              | \  /  | \  /  |
   visible:  (v1)---(v2)---(v3)
```

#### Energy and probability
- Energy of a state s: E = −Σ_{i<j} w_ij s_i s_j − Σ_i θ_i s_i
- Energy gap for unit i: ΔE_i = Σ_j w_ij s_j + θ_i
- Unit turns on with probability p(s_i = 1) = 1 / (1 + e^(−ΔE_i / T)), T = temperature.
- At high T the units behave almost randomly (exploration); as T→0 the rule becomes the deterministic Hopfield threshold.
- At equilibrium, P(state) ∝ e^(−E/T): low-energy states are more probable.

#### Algorithm
**A. Recall / optimisation (simulated annealing)**
1. Initialise weights, set the visible units to the input, start with a high temperature T.
2. Pick a unit at random; compute ΔE_i and set it on with probability 1/(1 + e^(−ΔE_i/T)).
3. Repeat for many updates until equilibrium at this T.
4. Lower T gradually (annealing schedule, e.g. T ← 0.9T) and repeat from step 2.
5. Stop when the state stops changing at low T; the final state is a low-energy (best) solution.

**B. Learning**
1. **Positive phase:** clamp the visible units to a training pattern, let the hidden units reach equilibrium, and measure the average co-activation p⁺_ij = ⟨s_i s_j⟩ clamped.
2. **Negative phase:** run the network freely (nothing clamped) to equilibrium and measure p⁻_ij.
3. Update each weight: Δw_ij = η (p⁺_ij − p⁻_ij).
4. Repeat over all patterns until the model distribution matches the data distribution (the weights stop changing).

#### Remarks
- Can learn internal representations and solve constraint satisfaction and optimisation problems.
- Training is very slow because equilibrium sampling is needed; **restricted Boltzmann machines (RBM)**, which remove the connections within a layer, make it practical.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. How does a unit turn on in a Boltzmann machine?
   A. With probability 1/(1 + e^(-deltaE/T))
   B. Deterministically if deltaE > 0 regardless of T
   C. With probability T
   D. With probability e^(deltaE)
   Answer: A. The probability depends on the energy gap and temperature. (Key point: Probability rule)

2. What happens as T approaches 0?
   A. Units become completely random
   B. The rule becomes the deterministic Hopfield threshold
   C. All units turn off
   D. The weights become zero
   Answer: B. High T gives random exploration; low T gives deterministic behaviour. (Key point: Temperature effect)

3. How are weights updated in Boltzmann learning?
   A. delta w_ij = eta (t - y) x_i
   B. delta w_ij = eta p+_ij only
   C. delta w_ij = eta (p+_ij - p-_ij), clamped minus free co-activation
   D. delta w_ij = w_ij / T
   Answer: C. Positive phase has visible units clamped; negative phase runs freely. (Key point: Learning rule)

4. Why are restricted Boltzmann machines used?
   A. They add self-connections
   B. They remove hidden units
   C. They eliminate the need for data
   D. Removing connections within a layer makes training practical because full equilibrium sampling is very slow
   Answer: D. Full Boltzmann training is very slow. (Key point: RBM)

## Related in Artificial Intelligence

- [Define Boltzmann machine](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Adefine-boltzmann-machine) — Artificial Intelligence, chapter 6
- [Machine learning, GA, operators, areas](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aml-genetic-algorithm-operators-areas) — Artificial Intelligence, chapter 6, asked 2×
- [Inductive learning and ID3](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Ainductive-learning-id3) — Artificial Intelligence, chapter 6
- [GA vs fuzzy based learning](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aga-vs-fuzzy-learning) — Artificial Intelligence, chapter 6
- [Hebbian learning; AND function](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Ahebbian-learning-and-network) — Artificial Intelligence, chapter 6
- [Back propagation algorithm](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aback-propagation-algorithm) — Artificial Intelligence, chapter 6, asked 3×

## Related in other subjects

- [Test: soft drink filling machines](https://www.bctengineer.com/question.md?key=semester_5%2Fprobability_and_statistics%3Asoft-drink-machines-same-volume) — Probability and Statistics, chapter 6
- [Two proportions: injection-molding machines](https://www.bctengineer.com/question.md?key=semester_5%2Fprobability_and_statistics%3Ainjection-molding-defective-fraction) — Probability and Statistics, chapter 7
- [Two machines 11∠0° and 11∠40° kV via j15 Ω](https://www.bctengineer.com/question.md?key=semester_4%2Fpower_system_analysis_i%3Atwo-machines-11kv-j15) — Power System Analysis I, chapter 3
- [Reactive power flow between machines via j10 Ω](https://www.bctengineer.com/question.md?key=semester_4%2Fpower_system_analysis_i%3Atwo-machines-reactive-j10) — Power System Analysis I, chapter 3