Do you know Overfitting?

While training Machine Learning models we often encounter the problem of overfitting. It simply means that our model has learned the mapping of training data from features to the target values. While our training loss might be very low (can go up to 0 if overfitted) but the model will not perform well on the test data (the unseen data). The reason is that the model has only memorized the training data and learned nothing of value.

Why does overfitting occur?

Insufficient Data: If your training data isn't enough, your model will only be trained on patterns in the small given dataset which may not occur in test data and/or inference.

Model is too complex: If you are using a more complex model than required then the model will memorize noise in the data and that might not be present in test/inference as it is in the training set.

Overtraining the model: If your validation loss starts going up again and you continue to train it, it will overfit

How do you avoid/fix overfitting?

Increase the training data size (Use data augmentation techniques, GANs...)

Make the model simpler (Use a lower degree polynomial, remove a few layers of neural network...)

Use Early Stopping (Stop training if validation loss starts increasing..)

Use Dropout (In Neural Nets) / Regularization (l2 or l1)

When a model overfits, there is a lot of variation in the output even for very close values of input features. Therefore, the model is said to have a high variance.

Comments

What is AI?

It actually seems funny to write answer to this question (as it's so unusual to find an article about this these days 🤔). AI is short for Artificial Intelligence or intelligence which is created by humans. But what is intelligence then? Intelligence breaks into the tasks that beings are capable of doing. Like thinking, memorising, remembering, deciding, reasoning, predicting, recognising, improving, inventing, reproducing, dreaming, assuming, surviving, feeling, hoping, coping, all these tasks ending with an 'ing' reminds us that they will never end until life (except for reproducing😉). The thing that makes beings actually alive is knowing that they are. But are all the beings intelligent? Not all of them carry out all those tasks. As being smartest of all we humans still don't know if a mouse dreams or not (atleast Jerry does😏). But we do know, beings with a smaller brain or number of brain cells cannot carry out complex tasks. I'm sure ameoba can't recog...

Do you know Machine Learning?

Machine Learning is like Jesus, It's everywhere... From pizzerias to Notco (a company which uses AI to make vegan food that tastes like meat) and from banks to Netflix all are using Machine Learning. But can machines actually learn something? 🧐 There are several algorithms that improve performance on a particular task with experience, that's it. By the way, if anyone asked, that was the definition of Machine Learning. The thing that computer systems can actually increase their performance or learn tasks is what AI is driven by. Machine Learning is basically divided into 3 categories, viz, Supervised Learning, Unsupervised Learning, and Reinforcement Learning. Supervised Learning is learning from a training set of labeled examples provided by a knowledgeable external supervisor. Each example is a description of a situation together with a specification—the label—of the correct action the system should take to that situation, which is often to ident...

Do you know steps in building a full Machine Learning model?

1. Data Collection In Machine Learning the data is the most important thing, unlike humans who look at a person's face a few times and recognize him/her, ML needs tons of data. The 2001's paper from Microsoft showed that moderate and complex models performed almost the same given sufficient data. Apart from it, the quality of data is also important, data that does not represent appropriate relation between features and their label is of no use. 2. Data Preprocessing The preprocessing of data is very essential before feeding it to the algorithm, removing irrelevant features, merging highly correlated features, removing or manually adding missing values, and converting data to numeric values, suppose the data contains a feature representing the country and your dataset consists of many countries which might be moderately correlated to your output so you might not wanna remove it, you can convert it into a one-hot encoding (a zero vector of length equal to the number of c...

Do you know AI

Search This Blog