--- title: "Back propagation algorithm" subject: "Artificial Intelligence" chapter: 6 key: "semester_6/artificial_intelligence:back-propagation-algorithm" source: https://www.bctengineer.com/q/semester_6/artificial_intelligence/back-propagation-algorithm site: BCT Engineering --- # Back propagation algorithm Artificial Intelligence · Chapter 6: Machine Learning - Asked in: 2071 Magh, 4 marks; 2074 Bhadra, 4 marks; 2070 Magh, 5 marks (asked 3 times) - Video: [Backpropagation in Neural Networks | Back Propagation Algorithm with Examples | Simplilearn](https://www.youtube.com/watch?v=ayOOMlgb320) — Simplilearn - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_6/artificial_intelligence/back-propagation-algorithm - Whole chapter: https://www.bctengineer.com/md/semester_6/artificial_intelligence/chapter-6 ## Question Explain the back propagation algorithm (and how it helps in machine learning). ## Short answer Back-propagation is a supervised algorithm for multilayer feedforward networks. It computes output error, propagates it backward, and updates weights via gradient descent (generalized delta rule). It enables learning non-linear mappings like XOR, trains hidden layers efficiently, and forms the basis of deep learning. ## Answer **Back-propagation (BP)** is the standard supervised learning algorithm for multilayer feed-forward neural networks. It computes the error at the output, propagates it backwards through the layers, and adjusts each weight by gradient descent to reduce the error (generalised delta rule). #### Algorithm (sigmoid units, learning rate η) 1. Initialise all weights and biases with small random values. 2. **Forward pass:** for each training pair (x, t), compute hidden outputs h_j = f(Σ w_ij x_i + b_j) and output y_k = f(Σ v_jk h_j + b_k), with f(z) = 1/(1 + e^(-z)). 3. **Output error term:** δ_k = (t_k − y_k) · y_k (1 − y_k) 4. **Hidden error term:** δ_j = h_j (1 − h_j) · Σ_k δ_k v_jk 5. **Weight update:** Δv_jk = η δ_k h_j; Δw_ij = η δ_j x_i (biases likewise with input 1). 6. Repeat for all patterns (an epoch) until the total error E = ½ Σ (t − y)² is small enough. #### Worked example (verified) Network 2-2-1, input x = (1, 0), target t = 1, η = 0.5. Weights: x1→h1 = 0.5, x1→h2 = 0.4, x2→h1 = 0.3, x2→h2 = 0.2, hidden biases 0.1, 0.1, h→y weights 0.6, 0.7, output bias 0.1. ```text h1 = f(0.5+0.1) = 0.6457, h2 = f(0.4+0.1) = 0.6225 y = f(0.6*0.6457 + 0.7*0.6225 + 0.1) = 0.7157 δy = (1-0.7157)*0.7157*0.2843 = 0.0579 δh1 = 0.0579*0.6*0.6457*0.3543 = 0.0079 δh2 = 0.0579*0.7*0.6225*0.3775 = 0.0095 v1 = 0.6+0.5*0.0579*0.6457 = 0.6187, v2 = 0.7180 new w(x1->h1) = 0.5+0.5*0.0079 = 0.5040 ``` After the update, the output for the same input rises from 0.7157 to 0.7267, closer to 1. #### How it helps machine learning - It lets multilayer networks learn non-linear mappings (e.g. XOR) that a single perceptron cannot. - It trains hidden layers efficiently by reusing computed errors. - It is the basis of modern deep learning (with variants such as momentum and stochastic gradient descent). Limitations: may get stuck in local minima, slow convergence, needs many labelled examples. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. In back-propagation, how is the hidden-layer error term computed? A. delta_j = h_j(1 - h_j) times the sum of delta_k * v_jk from the output layer B. delta_j = (t - y) * h_j, using the target directly C. delta_j = the sum of the inputs x_i multiplied by eta D. delta_j = h_j(1 - h_j) only, without using output errors Answer: A. Hidden error is the sigmoid derivative times the output errors sent back through the weights v_jk. (Key point: Hidden error term) 2. For a sigmoid output unit, what is delta_k? A. (t_k - y_k) only B. (t_k - y_k) * y_k(1 - y_k) C. y_k(1 - y_k) only D. (y_k - t_k) * h_j Answer: B. The error (t - y) is multiplied by the sigmoid derivative y(1 - y). (Key point: Output error term) 3. What is the update rule for a hidden-to-output weight? A. delta v_jk = eta * delta_j * x_i B. delta v_jk = eta * (t_k - y_k) * x_i C. delta v_jk = eta * delta_k * h_j D. delta v_jk = delta_k / h_j Answer: C. The weight change is learning rate times the output delta times the hidden output feeding that weight. (Key point: Weight update) 4. Why does back-propagation help machine learning? A. It guarantees reaching the global minimum of error B. It needs no labelled examples C. It removes the need for a learning rate D. It trains multilayer networks to learn non-linear mappings such as XOR that a single perceptron cannot Answer: D. BP trains hidden layers, enabling non-linear functions; it can still get stuck in local minima and needs labelled data. (Key point: Purpose in ML) ## Related in Artificial Intelligence - [Neural network; back-propagation](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aneural-network-bp-algorithm) — Artificial Intelligence, chapter 6 - [Machine learning, GA, operators, areas](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aml-genetic-algorithm-operators-areas) — Artificial Intelligence, chapter 6, asked 2× - [Machine learning; Boltzmann machines](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aml-boltzmann-machine) — Artificial Intelligence, chapter 6 - [Neural network, back-propagation, perceptron](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Ann-bp-perceptron) — Artificial Intelligence, chapter 6 - [GA vs fuzzy based learning](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aga-vs-fuzzy-learning) — Artificial Intelligence, chapter 6 - [Hebbian learning; AND function](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Ahebbian-learning-and-network) — Artificial Intelligence, chapter 6 ## Related in other subjects - [ANN and back propagation](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aann-backpropagation) — Data Mining, chapter 3 - [Back-face detection algorithm](https://www.bctengineer.com/question.md?key=semester_5%2Fcomputer_graphics%3Aback-face-detection-method) — Computer Graphics, chapter 7, asked 8× - [Back emf in DC motor and its role](https://www.bctengineer.com/question.md?key=semester_4%2Felectrical_machine%3Aback-emf-dc-motor-role) — Electrical Machine, chapter 4, asked 3× - [Back emf and its role in DC motor](https://www.bctengineer.com/question.md?key=semester_4%2Felectrical_machines%3Aback-emf-role-dc-motor) — Electrical Machines, chapter 4, asked 2×