Understanding Bias and Variance Trade Off in Machine Learning (Simply Put)
Introduction
Bias and variance are the two most critical topics in Data Science. The bias-variance trade-off is a key framework to help us understand different kinds of models. They lie at the base of many important concepts, such as underfitting and overfitting. Understanding this concept can help you deal with underfitting and overfitting during model development phase.
What is Bias ?
Simply put, Bias is the amount of assumptions your model makes against a certain problem that you are trying to frame. The more assumptions your model has, the higher the bias in your model.
For example, Linear Regression is one model with a very high bias because the model has a strong assumptions that the data has a linear distribution. Hence, when you feed the model with non-linear data, the model might not be able to fit the data at the end, which leads to underfitting model.
Underfitting refers to the scenario when the model is not learning enough of the true relationship underlying the data.
So, high bias (a.k.a a lot of assumptions) means underfitting. BAM!
