Adaptation of Neural Network from Scratch for Regression problem - Bad Predictions

[diliprk]

Hi Barathvaj,
Thank you for this great tutorial on building neural networks from scratch. The code is much easier to follow , tweak and learn compared to other sources like Nielsen. I am in need of building a custom neural network for a regression problem, so I tried to adapt your code to build an MLP from scratch and changed the loss function to Mean Squared Error and using ReLu as the activation function.
The problems I have with this adapted implementation are:

• Though the code runs error-free I am not able to get any sensible predictions from this neural network, the predictions remain the same for all the test data points.
• The predictions are also getting repeated in an array, instead of the neural network outputting a point estimate for y_{hat}
• The train/validation loss does not improve after the first training iteration.

I doubt backpropagation is even happening properly. Could you kindly offer any insights to improve the code and fix the above issues?