What is Forward Propagation in Neural Networks

Forward propagation in neural networks is the process where input data flows through each layer of the model to generate an output. It’s the step-by-step computation that transforms raw inputs into predictions using weights, biases and activation functions. This operation forms the backbone of how neural networks learn patterns and make decisions.

Forward Propagation

• It computes intermediate values layer by layer, starting from the input layer and ending at the output layer.
• Each neuron applies weighted sums and activation functions to extract features.
• It is used during both training and inference, but without weight updates.
• The accuracy of predictions heavily depends on how well forward propagation captures patterns from the input data.

Working

1. Input Layer: The network begins by receiving raw data through the input layer. Each feature in the dataset corresponds to a neuron in this layer, allowing the model to read all required information. Before entering the network, the data is often normalized or standardized to ensure faster training and better stability.

2. Hidden Layers: The processed input then passes through one or more hidden layers, where most of the computation happens. Every neuron in a hidden layer performs a weighted calculation on its inputs and then applies an activation function to introduce non-linearity. The computation inside each neuron follows:

$Z=W\times X+b$

where:

• $W$ represents the weights
• $X$ is the input vector
• $b$ is the bias term

After this, an activation function such as ReLU or sigmoid is applied to produce the neuron’s output, which is then passed forward.

3. Output Layer: The final layer generates the model’s prediction. The choice of activation function depends on the task:

• Softmax → multi-class classification
• Sigmoid → binary classification
• Linear → regression

This layer converts the processed information into a meaningful output.

4. Prediction: Based on the current weights and biases, the network produces its final output. The prediction is then compared with the true value using a loss function which calculates the error and sends it to backpropagation for learning.

Mathematical Explanation of Forward Propagation

Consider a neural network with one input layer, two hidden layers and one output layer.

architecture of a neural network

1. Layer 1 (First Hidden Layer)

The transformation is:

$A^{[1]}=\sigma (W^{[1]}X+b^{[1]})$

where:

• $W^{[1]}$ is the weight matrix,
• $X$ is the input vector,
• $b^{[1]}$ is the bias vector,
• $\sigma$ is the activation function.

2. Layer 2 (Second Hidden Layer)

We can have n number of hidden layers:

$A^{[n]}=\sigma (W^{[n]}{A}^{[n-1]}+b^{[n]})$

3. Output Layer

$Y=\sigma (W^{[3]}{A}^{[2]}+b^{[3]})$ where $Y$ is the final output. Thus the complete equation for forward propagation is:

$A^{[3]}=\sigma (\sigma (\sigma (X{W}^{[1]}+b^{[1]})W^{[2]}+b^{[2]})W^{[3]}+b^{[3]})$

This equation illustrates how data flows through the network:

• Weights ($W$) determine the importance of each input
• Biases ($b$) adjust activation thresholds
• Activation functions ($\sigma$) introduce non-linearity to enable complex decision boundaries.

Implementation of Forward Propagation

1. Import Required Libraries

Here we will import Numpy and pandas library.

2. Create Sample Dataset

• The dataset consists of CGPA, profile score and salary in LPA.
• $X$ contains only input features.

3. Initialize Parameters

When initilaizing parameters Random initialization avoids symmetry issues where neurons learn the same function.

4. Define Forward Propagation

• $Z=WX+B$ computes the linear transformation.
• Sigmoid activation ensures values remain between 0 and 1.

5. Execute Forward Propagation

Here we will execute the process of forward propagation using the above functions we created.

Output:

Final Output:
[[0.40566303]
[0.39810287]
[0.41326819]]

• Each number represents the model’s predicted probability before training for the given input.
• The values represent the sigmoid activation output which ranges between 0 and 1 indicating a probability like score for classification.