Neural-network terms sound intuitive until you must apply weights, biases and activations in the correct order. The 2-2-1 network below is traced from individual weighted sums through ReLU, sigmoid, classification loss and parameter counting. A feed-forward network is one type of artificial neural network; convolutional and recurrent networks use different connection patterns.
What an artificial neural network actually computes
An artificial neural network (ANN) is a directed network of simple computational units arranged in layers. The biological-neuron analogy explains the name, but the exam-ready definition is mathematical.
For one neuron with inputs \(x_i\), weights \(w_i\) and bias \(b\):
\[
z = \sum_i w_i x_i + b, \qquad a = f(z)
\]
Inputs are observed values. Weights control their influence, the bias shifts the decision boundary, and the activation function converts the linear total \(z\) into the signal sent forward.
As a quick check, let \(x=(2,1)\), \(w=(0.4,-0.5)\), and \(b=0.2\). Then \(z=2(0.4)+1(-0.5)+0.2=0.5\). ReLU outputs \(0.5\). Without the bias, the output would be \(0.3\).
From one neuron to a feed-forward network
Our network has two input values, one hidden layer with two neurons, and one output neuron, written as a 2-2-1 architecture. The input layer stores values rather than calculating weighted sums. Feed-forward means the arrows do not loop back during this computation.
Layer count and connectivity define the architecture. Weights and biases are trainable parameters, while activations are intermediate results for one input. A single-layer perceptron has no hidden layer; a multilayer network has at least one. Backpropagation trains the parameters, but it does not change this forward calculation order. For a wider practical learning track, see the Skills category.
Worked example: calculate the complete forward pass
Calculate each hidden neuron before touching the output neuron:
\[
z_{H1}=(0.6)(0.5)+(0.2)(-0.4)+0.1=0.30-0.08+0.10=0.32
\]
Therefore, \(h_1=\operatorname{ReLU}(0.32)=0.32\).
\[
z_{H2}=(0.6)(-0.3)+(0.2)(0.8)-0.05=-0.18+0.16-0.05=-0.07
\]
Therefore, \(h_2=\operatorname{ReLU}(-0.07)=0\).
Now pass \(h_1\) and \(h_2\) to the output neuron:
\[
z_O=(0.32)(1.2)+(0)(-0.7)-0.1=0.384-0.1=0.284
\]
\[
\hat y=\frac{1}{1+e^{-0.284}}=0.5705
\]
Unit | Weighted sum \(z\) | Activation | Output |
|---|---|---|---|
H1 | 0.32 | ReLU | 0.32 |
H2 | -0.07 | ReLU | 0 |
O | 0.284 | Sigmoid | 0.5705 |
With a stated threshold of \(0.5\), the predicted class is \(1\). If the true label is \(y=1\), binary cross-entropy is \(L=-\ln(0.5705)\approx0.561\). Loss measures how far the prediction is from the target under the chosen loss function.
The order is important. H2's negative pre-activation becomes zero before the output calculation, so its outgoing weighted term contributes nothing. Carrying \(-0.07\) forward instead would describe a different and incorrect network computation.
Activation functions and why the network needs them
Use the convention stated in a question. Here, the binary step is \(0\) for \(z<0\) and \(1\) for \(z\geq0\), sigmoid is \(1/(1+e^{-z})\), and ReLU is \(\max(0,z)\).
Function | Range | At \(z=-1\) | At \(z=0\) | At \(z=1\) | Introductory role |
|---|---|---|---|---|---|
Binary step | \(\{0,1\}\) | 0 | 1 | 1 | Hard decision |
Sigmoid | \((0,1)\) | 0.269 | 0.500 | 0.731 | Smooth binary output |
ReLU | \([0,\infty)\) | 0 | 0 | 1 | Hidden-unit activation |
ReLU suppresses H2 because \(z_{H2}=-0.07\). Sigmoid converts \(z_O=0.284\) to \(0.5705\), not directly to a class. Non-linearity matters because stacked linear transformations still collapse into one linear transformation. A multilayer network needs non-linear activations to model non-linear boundaries, though no activation is universally best.
How loss and gradients update the network
Training repeats five ordered moves:
Run a forward pass.
Compare the prediction with the target using a loss function.
Use backpropagation and the chain rule to obtain gradients.
Update parameters with gradient descent.
Repeat over training examples.
The update rule is \(w_{new}=w_{old}-\text{learning rate}\times\text{gradient}\). Here, \(w_{old}\) is the current weight, the gradient describes how loss changes with that weight, and the learning rate controls the update size.
Conceptually, backpropagation follows the loss backwards through the output and hidden calculations. The chain rule combines each local rate of change, revealing how strongly each parameter affected the loss. A positive gradient makes gradient descent reduce the weight; a negative gradient makes it increase. With sigmoid output and binary cross-entropy, the output-logit gradient is \(\hat y-y\); hidden-layer gradients then multiply this signal by downstream weights and ReLU derivatives.
For our target \(1\) and prediction \(0.5705\), loss is about \(0.561\). If a later model produced a correct-class prediction of \(0.65\), loss would fall to \(-\ln(0.65)\approx0.431\). The \(0.65\) value isolates the loss comparison; it is not the output of the stated weights.
A learning rate is the step size of an update. An epoch is one complete pass through the training set. Overfitting occurs when a model learns training examples too closely and performs poorly on unseen examples.
How exam questions turn ANN ideas into tasks
Common tasks ask you to identify components in a diagram, compute one neuron's output, trace a forward pass, match activations to ranges, count parameters, or notice whether a classification threshold is supplied.
Before calculating, mark the given activation beside each layer and write the threshold separately. This prevents two common category errors: applying the output activation inside the hidden layer, and treating a probability-like sigmoid output as a final class when no decision rule has been stated.
For the worked network, input-to-hidden weights contribute \(2\times2=4\), hidden biases contribute \(2\), hidden-to-output weights contribute \(2\times1=2\), and the output bias contributes \(1\). The total is \(4+2+2+1=9\) trainable parameters. Input values and activations are not parameters.
The 2-2-1 trace isolates the exact layer-by-layer forward pass. Perceptron separability, a full gradient-descent update and multilayer parameter counting appear in Artificial Neural Networks: Worked Examples for GATE.
Common ANN calculation traps and their corrections
Trap | Correction |
|---|---|
Omit the bias | Add \(b\) after all weighted input terms. |
Lose a negative weight's sign | Write each product in brackets before adding. |
Activate too early | Finish the full weighted sum \(z\), then apply \(f(z)\). |
Pass \(-0.07\) through ReLU unchanged | ReLU of any negative input is \(0\). |
Turn sigmoid \(0.5705\) into a class immediately | Classify only after a threshold is supplied. |
Count input nodes or activations as parameters | Count only weights and biases. |
Treat weights as probabilities | Weights are learned influence coefficients and need not lie in \([0,1]\). |
Call every neural network deep | Depth depends on the number of learned layers, not the presence of neurons. |
Keep four decimal places through sigmoid and loss calculations, then round at the end. At \(z=0\), copy the activation convention given in the question instead of assuming one.
The short version and the next calculation
Compute the weighted sum.
Add the bias.
Apply the activation.
Move layer by layer to the prediction.
Use loss to guide parameter updates.
The calculation chain is \(x\rightarrow z\rightarrow\text{activation}\rightarrow\text{next layer}\rightarrow\text{prediction}\).
Now change only \(x_1\) from \(0.6\) to \(0.4\), keeping \(x_2=0.2\). The hidden sums become \(0.20-0.08+0.10=0.22\) and \(-0.12+0.16-0.05=-0.01\), so \(h_1=0.22\) and \(h_2=0\). Then \(z_O=(0.22)(1.2)-0.1=0.164\), giving \(\hat y\approx0.5409\). At threshold \(0.5\), the class remains \(1\).
If you want a structured path from introductory AI concepts into machine learning and generative AI, continue with Artificial Intelligence (AI).
