Regression in Machine Learning With Examples

In machine learning, we use various techniques to analyze data and create machine learning applications. In this article, we will discuss the basics of regression in machine learning.

What is Regression in Machine Learning?

Regression in machine learning is an approach to identify relationships between independent and dependent variables in a dataset. We use regression to predict outcomes based on historical data.

We first use the hsitorical data to create a predictive model using regression. Then we use the developed machine learning model to predict values for new inputs of independent variables.

In the process of creating the machine learning model for regression analysis, we plot a regression line through the data points. Here, each dependent and independent attribute in the dataset becomes a coordinate. While plotting the line, we minimize the distance of the line from the actual data points. When we achieve a regression line where the error i.e. the distance of actual data points from the line is minimized, we use the line to predict the dependent variables using the independent variables in the new data.

Uses of Regression in Machine Learning

Regression is used as an essential component in most predictive models. It is supervised learning that uses the attributes of the historical data to create a mapping of an outcome. 

It doesn’t matter if you are in logistics using demand forecasting or in finance trying to predict the stock prices, regression analysis will be a handful tool for you to achieve these tasks. Thus, you can say that regression analysis is a key component for creating a forecasting model in machine learning applications.

• Regression analysis is used in demand forecasting for inventory management.
• We use regression analysis to predict the outcomes of a marketing or sales campaign.
• E commerce websites use regression to predict customer behavior. Subsequently, it may use the predictive model to recommend different products to the customers. However, various advanced algorithms are used to predict consumer behavior nowadays. We aren’t focused on just regression techniques to do the same.
• Just like e-commerce websites, streaming services like Netflix and Amazon prime also use regression analysis along with advanced machine learning techniques to predict user trends and behavior.
• Regression analysis is also used in time series forecasting and visualization. 
• In finance, regression analysis is used in predicting stock prices and creating algorithmic trading tools.

Types of Regression in Machine Learning

Regression analysis is used in various forms in machine learning applications. Following are some of the most used types of regression in machine learning.

Simple Linear Regression

We use simple regression in machine learning applications when there is only one independent variable and a dependent variable in the dataset. For instance, look at the following dataset.

Weight	Height
30	100
40	123
50	155
60	178
70	221
80	?

Here, we have the weight of pillars and their corresponding heights given. For the weight of 80 kgs, we have no data for height. 

If we have to buy a pillar of 80 kgs, we need to predict the height of the pillar. For this, we can use simple linear regression. Here, we already know the weight of the pillar and the height of the pillar is dependent on the weight of the pillars. Hence, weight is an independent variable and height is the dependent variable. 

When we create a linear regression model, the regression equation comes as follows.

height=2.97*weight + 6.9

Now, using the above formulae, we can predict the height of a pillar having any required weight. Therefore, the predicted height of the pillar with 80 kg weight will be 244.5 kg.

In linear regression, we assume that the independent variable and dependent variables are linearly related. If the dependent and independent variable isn’t linearly related, you will get absurd results that are highly inaccurate. 

Also, you need to perform data preprocessing to remove outliers in the data. Outliers can affect the linear regression models in a significant way. Your machine learning model will become useless if the input data contains outliers that force the linear regression model to produce inaccurate results.

Multiple Regression

In simple linear regression, there is only one independent variable. This might not be the case in real-life machine learning applications.

When there are two or more independent variables, we use multiple regression in machine learning for prediction. Again, we try to find a linear relationship between the dependent and independent variables in multiple regression.

For instance, suppose that we have the radius of pillars available with us along with the height and weight as shown in the following data.

Weight	Radius	Height
30	5	100
40	7.8	123
50	9.9	155
60	12.7	178
70	14.6	221
80	16.2	?

Now, if we calculate the regression line for the above data, the regression line comes as follows.

height = 8.10913*weight - 21.3242*radius - 36.81461.

Therefore, the predicted value of height for a pillar of weight 80kg and height 1.2 m will be 266.5.

The assumptions for simple linear regression hold true for multiple regression. Additionally, we should make sure that there aren’t any highly correlated independent variables. If yes, we can remove one or more attributes from the dataset while data cleaning. Otherwise, the predictions will be inaccurate.

Polynomial Regression

Simple linear regression and multiple regression assume that the relationship between dependent and independent variables in the dataset is linear. However, polynomial regression is used to create a machine learning model for a dataset having a non-linear relationship between the attributes.

Polynomial regression fits a polynomial curve between the points in the dataset. After getting the polynomial function, we can predict the value of the dependent variable for any set of independent variables. 

Ridge Regression

Ridge regression in machine learning is an improved version of linear regression. In Ridge regression, we use linear regression with a small bias. The bias is termed as Ridge regression penalty.

We introduce the bias to make sure that the linear regression model works accurately even if the attributes in the dataset are highly collinear. Thus, introducing the bias helps us avoid overfitting. If you have a small data sample, Ridge regression is a better choice to model the data instead of linear regression. 

Ridge regression is termed as L2 regularization.

Lasso Regression

Lasso regression also improves the linear regression algorithm. It works in the same way Ridge regression does. However, there are certain differences in the mathematical model. Lasso Regression is termed as L1 regularization.

Logistic Regression

Logistic regression works as a classification problem. Here, the dependent variable can have only two values like True and False or Yes and No. Logistic regression works using probabilistic mathematics. It uses sigmoid function or logistic function for data modelling in the machine learning application.

Logistic regression is used in different applications where we have binary outputs. For example, a bank can use logistic regression to decide if they can grant a loan to a person based on their credit score, monthly income, age, etc.

Precautions for Using Regression Algorithms

As we know that regression in machine learning is a supervised learning algorithm. Thus, we need to make sure that the dataset is not biased. Having a biased dataset will lead to a biased machine learning model. This will result in inaccurate results. Therefore, you should always make sure that the dataset used in regression in machine learning must represent real-life transactions or data.

Also, you should keep in mind that all the regression algorithms have certain assumptions and the algorithms will work well only if the training dataset satisfies those assumptions. Therefore, you should always preprocess the dataset and analyze it for the suitability of the regression algorithms.

Conclusion

In this article, we discussed regression in machine learning, types of regression algorithms, and their uses. We also discussed the precautions we need to take while using them. 

I hope you enjoyed reading this article. To read more about programming, you can read this article on developing a chat application in Python. You might also like this article on android download manager example.

Stay tuned for more informative articles. 

Happy Learning!