Implementing SGD(Stochastic Gradient Descent) for Linear Regression



Loss function
  1. Initially assume m = 0 and c = 0. Let L be our learning rate (a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function). L could be a small value like 0.0001 for good accuracy.
  2. Calculate the partial derivative of the loss function with respect to m, and plug in the current values of x, y, m and c in it to obtain the derivative value D.
Derivative of loss function with respective to m
Derivative of loss function with respective to c
  • we update the current value of m and c using the equations.
  • Repeat this process until our loss function is a small value(which means 0 error ,100%accuracy). The value of m and c that we are left with now will be the optimal values.


  1. Stochastic Gradient Descent is sensitive to feature scaling, so it is highly recommended to scale your data.
  2. For example, scale each attribute on the input vector X to [0,1] or [-1,+1], or standardize it to have mean 0 and variance 1.
  3. Note that the same scaling must be applied to the test vector to obtain meaningful results. This can be easily done using StandardScaler.
print("R^2 score for
our model :", r2_score(Y_test, pred_test))
out[4]:R^2 score for our model : 0.7226745354368981

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

From zero to Real-Time Hand Keypoints detection in five months with OpenCV, Tensorflow, and Fastai

Predicting YouTube Dislikes using Machine Learning

Machine Learning Model For Predicting Insurance Claim Charge.

Stream Training Data To Your Models With Diffgram

Day 3 | Machine Learning Update

Sentiment Analysis of Movie Reviews pt.4

How to build a Deep Neural Network from scratch

Understanding Genetic Algorithms

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
kalikiri harichandhana

kalikiri harichandhana


More from Medium

Extra Credit: The Matrix Reflection


Love for Lego

Cell Cycle Dynamics and Endoreplication In the Mosquito Midgut | Biomedgrid llc