Python Tutorial

Introduction Python Features Python Applications Python System Requirements Python Installation Python Examples Python Basics Python Indentation Python Variables Python Data Types Python IDE Python Keywords Python Operators Python Comments Python Pass Statement

Python Conditional Statements

Python if Statement Python elif Statement Python If-else statement Python Switch Case

Python Loops

Python for loop Python while loop Python Break Statement Python Continue Statement Python Goto Statement

Python Arrays

Python Array Python Matrix

Python Strings

Python Strings Python Regex

Python Built-in Data Structure

Python Lists Python Tuples Python Lists vs Tuples Python Dictionary Python Sets

Python Functions

Python Function Python min() function Python max() function Python User-define Functions Python Built-in Functions Python Recursion Anonymous/Lambda Function in Python python apply() Function Python lambda() Function

Python File Handling

Python File Handling Python Read CSV Python Write CSV Python Read Excel Python Write Excel Python Read Text File Python Write Text File Read JSON File in Python

Python Exception Handling

Python Exception Handling Python Errors and exceptions Python Assert

Python OOPs Concept

OOPs Concepts in Python Classes & Objects in Python Inheritance in Python Polymorphism in Python Python Encapsulation Python Constructor Python Super function Python Static Method Static Variables in Python Abstraction in Python

Python Iterators

Iterators in Python Yield Statement In Python Python Yield vs Return

Python Generators

Python Generator

Python Decorators

Python Decorator

Python Functions and Methods

Python Built-in Functions Python String Methods Python List Methods Python Dictionary Methods Python Tuple Methods Python Set Methods

Python Modules

Python Modules Python Datetime Module Python Math Module Python Import Module Python Time Module Python Random Module Python Calendar Module CSV Module in Python Python Subprocess Module Python Subprocess

Python MySQL

Python MySQL Python MySQL Client Update Operation Delete Operation Database Connection Creating new Database using Python MySQL Creating Tables Performing Transactions

Python MongoDB

Python MongoDB

Python SQLite

Python SQLite

Python Data Structure Implementation

Python Stack Python Queue Python Linked List Python Hash Table Python Graph

Python Advance Topics

Speech Recognition in Python Face Recognition in Python Python Linear regression Python Rest API Python Command Line Arguments Python JSON Python Virtual Environment Type Casting in Python Python Collections Python Commands Python Data Visualization Python Debugger Python DefaultDict Python Enumerate

Python 2

What is Python 2

Python 3

Anaconda in Python 3 Anaconda python 3 installation for windows 10 List Comprehension in Python3


Python PPTX Python Pickle Python Seaborn Python Coroutine Python EOL Python Infinity Python math.cos and math.acos function Python Project Ideas Based On Django Reverse a String in Python Reverse a Number in Python Python Word Tokenizer Python Trigonometric Functions Python try catch exception GUI Calculator in Python Implementing geometric shapes into the game in python Installing Packages in Python Python Try Except Python Sending Email Socket Programming in Python Python CGI Programming Python Data Structures Python abstract class Python Compiler Python K-Means Clustering NSE Tools In Python Operator Module In Python Palindrome In Python Permutations in Python Pillow Python introduction and setup Python Functionalities of Pillow Module Python Argmin Python whois Python JSON Schema Python lock Return Statement In Python Reverse a sentence In Python tell() function in Python Why learn Python? Write Dictionary to CSV in Python Write a String in Python Binary Search Visualization using Pygame in Python Latest Project Ideas using Python 2022 Closest Pair of Points in Python ComboBox in Python Python vs R Best resources to learn Numpy and Pandas in python Check Letter in a String Python Python Console Python Control Statements Convert Float to Int in Python using Pandas Difference between python list and tuple Importing Numpy in Pycharm Python Key Error Python NewLine Python tokens and character set Python Strong Number any() Keyword in python Best Database in Python Check whether dir is empty or not in python Comments in the Python Programming Language Convert int to Float in Python using Pandas Decision Tree Classification in Python End Parameter in python __GETITEM__ and __SETITEM__ in Python Python Namespace Python GUI Programming List Assignment Index out of Range in Python List Iteration in Python List Index out of Range Python for Loop List Subtract in Python Python Empty Tuple Python Escape Characters Sentence to python vector Slicing of a String in Python Executing Shell Commands in Python Genetic Algorithm in python Get index of element in array in python Looping through Data Frame in Python Syntax of Map function in Python After Python What Should I Learn Python AIOHTTP Alexa Python Artificial intelligence mini projects ideas in python Artificial intelligence mini projects with source code in Python Find whether the given stringnumber is palindrome or not First Unique Character in a String Python Python Network Programming Python Interface Python Multithreading Python Interpreter Data Distribution in python Flutter with tensor flow in python Front end in python Iterate a Dictionary in Python Iterate a Dictionary in Python – Part 2 Allocate a minimum number of pages in python Assertion Errors and Attribute Errors in Python Checking whether a String Contains a Set of Characters in python Python Control Flow Statements *Args and **Kwargs in Python Bar Plot in Python Conditional Expressions in Python Function annotations() in Python How to Write a Configuration file in Python Image to Text in python import() Function in Python Import py file in Python Multiple Linear Regression using Python Nested Tuple in Python Python String Negative Indexing Reading a File Line by Line in Python Python Comment Block Base Case in Recursive function python ER diagram of the Bank Management System in python Image to NumPy Arrays in Python NOT IN operator in Python One Liner If-Else Statements in Python Sklearn in Python Cube Root in Python Python Variables, Constants and Literals What Does the Percent Sign (%) Mean in Python Creating Web Application in python Notepad++ For Python PyPi TensorFlow Python | Read csv using pandas.read_csv() What is online python free IDE What is Python online compiler Run exec python from PHP What are the Purposes of Python Python Ternary Operators Self in Python Python vs Java Python Modulo Python Packages Python Syntax Python Uses Python Bitwise Operators Python Identifiers Python Matrix Multiplication Python AND Operator Python Logical Operators Python Multiprocessing Python Unit Testing __init__ in Python Advantages of Python Is Python Case-sensitive when Dealing with Identifiers Python Boolean Python Call Function Python History Python Image Processing Python main() function Python Permutations and Combinations Difference between Input() and raw_input() functions in Python Conditional Statements in python Confusion Matrix Visualization Python Python Algorithms Python Modules List Difference between Python 2 and Python 3 Is Python Case Sensitive Method Overloading in Python Python Arithmetic Operators Assignment Operators in Python Is Python Object Oriented Programming language Division in Python Python exit commands Continue And Pass Statements In Python Colors In Python Convert String Into Int In Python Convert String To Binary In Python Convert Uppercase To Lowercase In Python Convert XML To JSON In Python Converting Set To List In Python Covariance In Python CSV Module In Python Decision Tree In Python Difference Between Yield And Return In Python Dynamic Typing In Python What is Python compiler GDB Python coding platform Python Classification Python | a += b is not always a = a + b PyDev with Python IDE Character Set in Python Best Python AI Projects _dict_ in Python

How to

How to Substring a String in Python How to Iterate through a Dictionary in Python How to convert integer to float in Python How to reverse a string in Python How to take input in Python How to install Python in Windows How to install Python in Ubuntu How to install PIP in Python How to call a function in Python How to download Python How to comment multiple lines in Python How to create a file in Python How to create a list in Python How to declare array in Python How to clear screen in Python How to convert string to list in Python How to take multiple inputs in Python How to write a program in Python How to compare two strings in Python How to create a dictionary in Python How to create an array in Python How to update Python How to compare two lists in Python How to concatenate two strings in Python How to print pattern in Python How to check data type in python How to slice a list in python How to implement classifiers in Python How To Print Colored Text in Python How to develop a game in python How to print in same line in python How to create a class in python How to find square root in python How to import numy in python How to import pandas in python How to uninstall python How to upgrade PIP in python How to append a string in python How to open a file in python How to Open a file in python with Path How to run a Python file in CMD How to change the names of Columns in Python How to Concat two Dataframes in Python How to Iterate a List in Python How to learn python Online How to Make an App with Python How to comment out a block of code in Python


Python Sort List Sort Dictionary in Python Python sort() function Python Bubble Sort


Factorial Program in Python Prime Number Program in Python Fibonacci Series Program in Python Leap Year Program in Python Palindrome Program in Python Check Palindrome In Python Calculator Program in Python Armstrong Number Program in Python Python Program to add two numbers Anagram Program in Python Number Pattern Programs in Python Even Odd Program in Python GCD Program in Python Python Exit Program Python Program to check Leap Year Operator Overloading in Python Pointers in Python Python Not Equal Operator Raise Exception in Python Salary of Python Developers in India What is a Script in Python

Multiple Linear Regression using Python

Linear Regression:

Linear regression is a method that models the relationship between a dependent variable and one or more independent variables; in other terms, that models the relationship between a target variable and simple regression or multiple regression.

This model assumes a relationship between the given inputs and the output variables. We can also determine the coefficients required by the model to predict the new data.

Linear regression is of two forms; they are

  • Simple Linear Regression
  • Multiple Linear Regression

Simple Linear Regression:

Simple linear regression (SLR) predicts a response using a single feature or variable. In simple linear regression, all the variables are linked linearly. Here the main purpose is to find a linear equation used to predict the answer value of y concerning the feature or the independently derived variable (x).

Below is a dataset with x features and y responses respective to x.

Multiple Linear Regression Using Python

For simple understanding, we define

X as features, that is x = [10, 11, 12, ……, 19],

Y as response, that is y = [11, 13, 12, ……, 22]

We have considered 10 values in the above table

The graphical representation of the above dataset looks like this:

Multiple Linear Regression Using Python

After this, we have to find or identify the most suitable line for this scatter graph to find the response of any new value for a feature.

This line is referred to as the regression line. We have some equations of the regression line.

Multiple Linear Regression Using Python


  • h(xi) is the predicted response value for the ith.
  • β0  and β1xi are regression coefficients.

If we want to build the model, we must know how to estimate the value of regression coefficients. If we know how to estimate regression coefficients, then only we can use this model to get the responses.

Concept of Least Squares:

Yi = 0 + Ρ1xi + Ρi = h(xi) + Ρi Ρ Ρi = yi – h(xi)

Here, Ρi is a residual error in ith observation.

So, we have to minimize the total residual error.

The cost function or squared error, b as:

b(β0, β1) = 1/2nΣnI = 1 έ2i

We have to find the values of Ρ0 and Ρ1  to find b(Ρ0 and Ρ1) minimum.

Let us not go into deep calculations. We are presenting the result below:

β1 = SSab / SSaa

β0 = b – β1a

where SSab would be the sum of cross deviations of “b” and “a”:

SSab = ΣnI = 1 (ai – a)(bi – b) = ΣnI =1 biai – nab

And SSaa would be the sum of squared deviations of “a.”

SSaa = Σni = 1(ai – a)2 = Σni = 1ai2 – n(a)2

Modules used:

  • Numpy
  • Pandas
  • Matplotlib
  • Sklearn


#simple program for simple linear regression
import numpy as np  
import matplotlib.pyplot as mtp  
def estimate_coeff(p, q):  
# Here, we will assume the total number of points or observation  
    n1 = np.size(p)  
# Now, we will calculate the mean of a and b vector  
    p = np.mean(p)  
    q = np.mean(q)  
# here, we will calculate the cross deviation and deviation about a  
    SS_pq = np.sum(q * p) - n1 * q * p  
    SS_pp = np.sum(p * p) - n1 * p * p  
# here, we will calculate the regression coefficients  
    b1 = SS_pq / SS_pp  
    b0 = q - b1 * p  
    return (b0, b1)  
def plot_regression_line(p, q, b):  
# Now, we will plot the actual points or observations as a scatter plot  
    MTP.scatter(p, q, color = "m",  
            marker = "o", s = 30)  
\# here, we will calculate the predicted response vector  
    q_pred = b[0] + b[1] * p  
# here, we will plot the regression line  
    mtp.plot(p, q_pred, color = "g")  
# here, we will put the labels  
# here, we will define the function to show plot  
def main():  
# entering the observation points or data  
    p = np.array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19])  
    q = np.array([11, 13, 12, 15, 17, 18, 18, 19, 20, 22])  
# now, we will estimate the coefficients  
    b = estimate_coeff(p, q)  
    print("Estimated coefficients are :\nb0 = {} \  
        \nb_1 = {}".format(b[0], b[1]))  
# Now, we will plot the regression line  
    plot_regression_line(p, q, b)  
if __name__ == "__main__":  


Estimated coefficients are:
b0 = -0.5606006069
b1 = 1.17696867686
Multiple Linear Regression Using Python

As we can see from the output, we have drawn simple linear regression using Python and the modules matplotlib used to plot the graph and the numpy module.

Multiple Linear Regression:

We require only one independent variable as the input in simple linear regression. However, we provide multiple independent variables for a single dependent variable in multiple linear regression. It is a Machine learning algorithm.

Q(Feature Matrix) = Q is a matrix of size "a * b" where "Qij" represents the values of the jth attribute for the ith observation.

We represent in the matrix form.

Multiple Linear Regression Using Python

S (Response Vector) = It is a vector of size “a” representing the response value for the ith observation.

Multiple Linear Regression Using Python

“A” regression line is:

  h(Qi) = β0 + β1qi1 + β2qi2 + β3qi3 + β4qi4 + β5qi5 +………. + βbqib

Here, h(qi) is known as the response value for the ith observation point. We have to find the regression coefficients that are Ρ0, Ρ1, Ρ2……, Ρb .

We have another formula:

Si = β0 + β1qi1 + β2qi23qi3 + ……..+βbqibi

We can represent the linear model in matrix form


Multiple Linear Regression Using Python


Multiple Linear Regression Using Python

Using the Least Squares algorithm, we can get the estimated value of b (b). We can only use the Least Squares method when the residual error is minimised.

The result will be shown as

Multiple Linear Regression Using Python

Here, (‘) represents the transpose of the matrix and -1 represents the reverse of a matrix.

Multiple Linear Regression Using Python

The above formula is used for calculating the multi-linear regression model. Here Y’ is the calculated response vector.


//program for multiple linear regression using Python
import matplotlib.pyplot as mtp 
import numpy as np  
from sklearn import datasets as data
from sklearn import linear_model as lmd  
from sklearn import metrics as mt  
# First, we will load the Boston dataset  
boston1 = data.load_boston(return_X_y = False)  
# Here, we will define the feature matrix(H) and response vector(f)  
H =  
f =  
# Now, we will split X and y datasets into training and testing sets  
from sklearn.model_selection import train_test_split as tts  
H_train, H_test, f_train, f_test = tts(H, f, test_size = 0.4,  
                                                    random_state = 1)  
# Here, we will create a linear regression object  
reg1 = lmd.LinearRegression()  
# Now, we will train the model by using the training sets, f_train)  
# here, we will print the regression coefficients  
print('The Regression Coefficients are: ', reg1.coef_)  
# Here, we will print the variance score: 1 means perfect prediction  
print('The Variance score is: {}'.format(reg1.score(H_test, f_test)))  
# Here, we will plot for residual error  
# here, we will set the plot style  
mtp. style.use('five thirty eight')  
# here, we will plot the residual errors in training data  
mtp.scatter(reg1.predict(H_train), reg1.predict(H_train) - f_train,  
            color = "green", s = 10, label = 'Train data')  
# Here, we will plot the residual errors in test data  
mtp.scatter(reg1.predict(H_test), reg1.predict(H_test) - f_test,  
            color = "blue", s = 10, label = 'The Test data')  
# Here, we will plot the line for zero residual error  
mtp. hlines(y = 0, xmin = 0, Xmax = 50, linewidth = 2)  
# here, we will plot the legend  
mtp.legend(loc = 'upper right')  
# now, we will plot the title  
mtp.title("The Residual errors")  
# here, we will define the method call for showing the plot  


The Regression Coefficients are:  [   -8.95714048e-02  6.73132853e-02  5.04649248e-02  2.18579583e+00       -1.72053975e+01  3.63606995e+00  2.05579939e-03 -1.36602888          2.89576718e-01      -1.22700072e-02         -8.34881849e-01  9.40360790e-03      -5.04008320e-01    ]

The Variance score is: 0.7209056672661751
Multiple Linear Regression Using Python

As we can see the output of the program how we have drawn the multiple linear regressions using Python. In the program, we have imported the modules matplotlib, numpy, and sklearn. Matplotlib is used to plot the graph and represent the graph on the output screen. Sklearn for dataset usage and numpy for the larger arrays of the dataset.