In novelty detection, you have a dataset that contains only good data and you are trying to check if new observations are similar to the the good data. In other words, our goal is to check if new observations are outliers.
In outlier detection, you dataset may already have outliers and your goal is to identify such outliers.
Both novelty detection and outlier detection are used to detect anomalies.
Outlier detection is an unsupervised anomaly detection algorithm.
Novelty detection is a semi-supervised anomaly detection algorithm.
Adjusting Columns + Values
Adjusting Rows + Columns
In deep learning, you don’t have the bias-variance trade off as DL models allow you to reduce both!
A key observation is that in traditional ML, the performance reaches a pleatau after certain amount training data whereas in DL, performance improves with more training data.
In Hypothesis testing, Type I error means we reject the Null hypothesis when the null is actually true. Type II error means we fail to reject the null when the null is not true.
This measures how well a human performs on a given classification task. For example, in a name entity recognition problem, if a human on average makes 15 errors out of 100 entities, the human level performance is 15%.
How do we measure human-level performance?
You need to go through manual labeling to measure this.
Get a group of people to label a stratified sample and then measure the average error.
This is also the Bayes Error.
This is the lowest error one can achieve from a ML model.
The difference between the Bayes Error and training error is called the…
#probability #statistics #ML #DL #coding #security #building #digest