Blog

What is Anomaly Detection?

Anomaly detection is the identification of rare items, events or observations which differs significantly from the majority of the data. In other words, an 11outlier is an observation that diverges/deviates from an overall pattern on a sample.

Flavors of Anomalies:

There are different flavors of anomalies depending upon the environment.

Point Anomalies: A data point can be considered as anomalous with respect to the rest of the data when a single data point lies far from the rest of the distribution.

Collective Anomalies: The collection of data samples is anomalous with respect

to the entire population, but not individual values.

Contextual Anomalies: The data points are anomalous only in specific or some defined context. Basically, it has noise in the data.

Techniques to Detect Anomalies:

Rule-based Anomaly Detection: This is already framed by specific set of rules that describe an anomaly and assigns the thresholds and limits. We typically rely on the experience of industry experts whose findings are ideal to detect known anomalies. These anomalies are familiar anomalies to us and we can easily recognize, whether it is normal or abnormal. One of the major flaws of rule-based systems is that they don’t detect anomalies automatically as patterns change. To learn new patterns, a new model would have to be built each time.

Machine based Anomaly Detection: It requires a labeled data set containing both normal and anomalous samples to construct a predictive model and to classify future data points. The real-life scenarios are quite complex and full of uncertainties, everything may not be happening in a known way. So, machine-based detection is most appropriate. The commonly used algorithms for this purpose are Neural Networks, Support Vector Machine learning, K-Nearest Neighbors Classifier, etc.

Use Cases of Anomaly Detection:

* Health Care - Critical cases, Fraud detection (Insurance).

* Engineering Industry - Fault/Defect Detection.

* Finance - Credit/Debit cards Fraud cases.

* Network Security - Intrusion Detection.

To explore more about the anomalies and its uses. Contact us at info@kratosolutions.com