Linear Regression (Machine Learning)

Software Engineer
27. Nov 2019
1 von 12

Linear Regression (Machine Learning)

• 1. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 1/12 Linear Regression for Years of experience and Salary Dataset Gradient Descent in Linear Regression In linear regression, the model targets to get the best-fit regression line to predict the value of y based on the given input value (x). While training the model, the model calculates the cost function which measures the Root Mean Squared error between the predicted value (pred) and true value (y). The model targets to minimize the cost function. To minimize the cost function, the model needs to have the best value of θ1 and θ2. Initially model selects θ1 and θ2 values randomly and then itertively update these value in order to minimize the cost function untill it reaches the minimum. By the time model achieves the minimum cost function, it will have the best θ1 and θ2 values. Using these finally updated values of θ1 and θ2 in the hypothesis equation of linear equation, model predicts the value of x in the best manner it can. Therefore, the question arises – How θ1 and θ2 values get updated ?
• 2. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 2/12
• 3. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 3/12 -> θj : Weights of the hypothesis. -> hθ(xi) : predicted y value for ith input. -> j : Feature index number (can be 0, 1, 2, ......, n). -> α : Learning Rate of Gradient Descent. We graph cost function as a function of parameter estimates i.e. parameter range of our hypothesis function and the cost resulting from selecting a particular set of parameters. We move downward towards pits in the graph, to find the minimum value. Way to do this is taking derivative of cost function as explained in the above figure. Gradient Descent step downs the cost function in the direction of the steepest descent. Size of each step is determined by parameter α known as Learning Rate. In the Gradient Descent algorithm, one can infer two points : 1.If slope is +ve : θj = θj – (+ve value). Hence value of θj decreases.
• 4. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 4/12 2. If slope is -ve : θj = θj – (-ve value). Hence value of θj increases. The choice of correct learning rate is very important as it ensures that Gradient Descent converges in a reasonable time. : 1.If we choose α to be very large, Gradient Descent can overshoot the minimum. It may fail to converge or even diverge. 2.If we choose α to be very small, Gradient Descent will take small steps to reach local minima and will take a longer time to reach minima. For linear regression Cost Function graph is always convex shaped. Reference: geeksforgeeks.org/gradient-descent-in-linear-regression/ (Refered For Theory)
• 5. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 5/12 First, we import a few libraries- In [10]: Data Preprocessing & importing Dataset The next step is to import our dataset ‘Salary_Data.csv’ then split them into input (independent) variables and output (dependent) variable. When you deal with real datasets, you usually have around thousands of rows but since the one I have taken here is a sample, this has just 30 rows. So when we split our data into a training set and a testing set, we split it in 1/3, i.e., 20 rows go into the training set and the rest 10 make it to the testing set. In [11]: Plotting Default Dataset Out[11]: YearsExperience Salary 0 1.1 39343.0 1 1.3 46205.0 2 1.5 37731.0 3 2.0 43525.0 4 2.2 39891.0 import numpy as np import pandas as pd import matplotlib.pyplot as plt dataset = pd.read_csv('Salary_Data.csv') x = dataset.iloc[:, :-1].values y = dataset.iloc[:, 1].values data_top = dataset.head() #Dataset display data_top
• 6. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 6/12 In [12]: Training & Split Split arrays or matrices into random train and test subsets In [13]: Linear Regression Now, we will import the linear regression class, create an object of that class, which is the linear regression model. In [14]: Fitting Data Then we will use the fit method to “fit” the model to our dataset. What this does is nothing but make the regressor “study” our data and “learn” from it. plt.scatter(x, y, color = "red") plt.plot(x,y, color = "green") plt.title("Salary vs Experience (Dataset)") plt.xlabel("Years of Experience") plt.ylabel("Salary") plt.show() from sklearn.model_selection import train_test_split x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 1/3) from sklearn.linear_model import LinearRegression lr = LinearRegression()
• 7. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 7/12 In [15]: Testing Now that we have created our model and trained it, it is time we test the model with our testing dataset. In [16]: Data Visualization for Training Dataset First, we make use of a scatter plot to plot the actual observations, with x_train on the x-axis and y_train on the y-axis. For the regression line, we will use x_train on the x-axis and then the predictions of the x_train observations on the y-axis. We add a touch of aesthetics by coloring the original observations in red and the regression line in green. In [17]: Data Visualization for Testing Dataset repeat the same task for our testing dataset, and we get the following code- Out[15]: LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=Fal se) lr.fit(x_train, y_train) y_pred = lr.predict(x_test) plt.scatter(x_train, y_train, color = "red") plt.plot(x_train, lr.predict(x_train), color = "green") plt.title("Salary vs Experience (Training set)") plt.xlabel("Years of Experience") plt.ylabel("Salary") plt.show()
• 8. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 8/12 In [24]: Linear Regression with Gradient Descent Algorithm Approach for Years of experience and Salary Dataset Importing libraries,Dataset & data preprocessing plt.scatter(x_test, y_test, color = "red") plt.plot(x_train, lr.predict(x_train), color = "green") plt.title("Salary vs Experience (Testing set)") plt.xlabel("Years of Experience") plt.ylabel("Salary") plt.show()
• 9. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 9/12 In [27]: # Making the imports import numpy as np import pandas as pd import matplotlib.pyplot as plt plt.rcParams['figure.figsize'] = (12.0, 9.0) # Preprocessing Input data data = pd.read_csv('Salary_Data.csv') X = data.iloc[:, 0] Y = data.iloc[:, 1] #Plotting Data for visualization plt.scatter(X, Y) plt.title("Salary vs Experience (Dataset set)") plt.xlabel("Years of Experience") plt.ylabel("Salary") plt.show()
• 10. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 10/12 Optimizing parameter value like intercept "c" and slope "m" & learning rate alpha in Gradient descent formula In [26]: Prediction Slope & Intercept: 12836.600965885045 2915.2044856014018 # Building the model m = 0 c = 0 L = 0.0001 # The learning Rate #L = 0.0001 # The learning Rate #L = 0.0002 # The learning Rate #L = 0.0003 # The learning Rate epochs = 1000 # The number of iterations to perform gradient descent n = float(len(X)) # Number of elements in X # Performing Gradient Descent for i in range(epochs): Y_pred = m*X + c # The current predicted value of Y D_m = (-2/n) * sum(X * (Y - Y_pred)) # Derivative wrt m D_c = (-2/n) * sum(Y - Y_pred) # Derivative wrt c m = m - L * D_m # Update m c = c - L * D_c # Update c print("Slope & Intercept:") print (m, c)
• 11. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 11/12 In [28]: # Making predictions Y_pred = m*X + c plt.scatter(X, Y) plt.plot([min(X), max(X)], [min(Y_pred), max(Y_pred)], color='red') # regression line plt.title("Salary vs Experience (prediction)") plt.xlabel("Years of Experience") plt.ylabel("Salary") plt.show()
• 12. 9/17/2019 ML_Activity_1 - Jupyter Notebook localhost:8890/notebooks/ML Activity/ML_Activity_1.ipynb 12/12 Batch 1 Block 1 B.TECH ENTC Omkar Rane BETB118 Kaustubh Wankhade BETB129