How to Mitigate Bias Variance in Deep Learning Models?

In deep learning, you don’t have the bias-variance trade off as DL models allow you to reduce both!

A key observation is that in traditional ML, the performance reaches a pleatau after certain amount training data whereas in DL, performance improves with more training data.

Performance of Traditional ML vs. DL models

Fixing bias-variance in your DL model

• In general, training a bigger network (more layers) help to reduce the bias
• In general, regularization (e.g. L2 norm) help reduce the variance

References:

• https://towardsdatascience.com/examples-of-bias-variance-tradeoff-in-deep-learning-6420476a20bd
• deeplearning.ai