Introduction to Logistic Regression

Learn how logistic regression is implemented from scratch

Logistic regression is a classification algorithm, which is pretty popular in some communities especially in the field of biostatistics, bioinformatics and credit scoring. It’s used to assign observations a discrete set of classes(target).

Why not logistic classification?

Logistic regression is strictly not a classification algorithm on its own. Because its output is a probability(a continuous number between 0 and 1), it will be only a classification algorithm in combination with a decision boundary which is generally fixed at 0.5

For example, if we have a logistic regression that has to predict whether an email is a spam or not, the output of the function will be 0.2 or 0.7. By default, the logistic regression makes it easier for us by assigning 0(not a spam) for the one who got 0.2 and assigning 1(a spam) for the latter. The threshold is 0.5 but we can manage to change it.

Comparison to linear regression

Let’s suppose you have data on time spent studying, playing and exam score.

Linear Regression: because it’s a regression, meaning that the output is continuous, it could help us predict the student test score between a certain range.

Logistic Regression: this one could help us predict whether the student passed the exam or not. Its…