|
|
| Line 1: |
Line 1: |
| In [[anomaly detection]], the '''local outlier factor''' ('''LOF''') is an algorithm proposed by Markus M. Breunig, [[Hans-Peter Kriegel]], Raymond T. Ng and Jörg Sander in 2000 for finding anomalous data points by measuring the local deviation of a given data point with respect to its neighbours.<ref>{{cite doi|10.1145/335191.335388}}</ref>
| | I'm a 36 years old and working at the high school (Environmental Studies).<br>In my free time I'm trying to teach myself Japanese. I've been there and look forward to go there anytime soon. I love to read, preferably on my ipad. I like to watch Psych and The Vampire Diaries as well as documentaries about nature. I love Rugby league football. |
| | |
| LOF shares some concepts with [[DBSCAN]] and [[OPTICS algorithm|OPTICS]] such as the concepts of "core distance" and "reachability distance", which are used for local density estimation.<ref>{{cite doi | 10.1007/978-3-540-48247-5_28}}</ref>
| |
| | |
| == Basic idea ==
| |
| | |
| [[File:LOF-idea.svg|thumb|right|250px|Basic idea of LOF: comparing the local density of a point with the densities of its neighbors. A has a much lower density than its neighbors.]]
| |
| | |
| As indicated by the title, the local outlier factor is based on a concept of a local density, where locality is given by <math>k</math> nearest neighbors, whose distance is used to estimate the density. By comparing the local density of an object to the local densities of its neighbors, one can identify regions of similar density, and points that have a substantially lower density than their neighbors. These are considered to be [[outlier]]s.
| |
| | |
| The local density is estimated by the typical distance at which a point can be "reached" from its neighbors. The definition of "reachability distance" used in LOF is an additional measure to produce more stable results within clusters.
| |
| | |
| == Formal ==
| |
| | |
| Let <math>\mbox{k-distance}(A)</math> be the distance of the object <math>A</math> to the ''k'' nearest neighbor. Note that the set of the ''k'' nearest neighbors includes all objects at this distance, which can in the case of a "tie" be more than ''k'' objects. We denote the set of ''k'' nearest neighbors as <math>N_k(A)</math>.
| |
| | |
| [[File:Reachability-distance.svg|thumb|right|250px|Illustration of the reachability distance. Objects ''B'' and ''C'' have the same reachability distance (k=3), while ''D'' is not a ''k'' nearest neighbor]]
| |
| | |
| This distance is used to define what is called ''reachability distance'':
| |
| | |
| <math>\mbox{reachability-distance}_k(A,B)=\max\{\mbox{k-distance}(B), d(A,B)\}</math>
| |
| | |
| In words, the ''reachability distance'' of an object <math>A</math> ''from'' <math>B</math> is the true distance of the two objects, but at least the <math>\mbox{k-distance}</math> of <math>B</math>. Objects that belong to the ''k'' nearest neighbors of <math>B</math> (the "core" of <math>B</math>, see [[DBSCAN|DBSCAN cluster analysis]]) are considered to be equally distant. The reason for this distance is to get more stable results. Note that this is not a [[distance]] in the mathematical definition, since it is not symmetric. (While it is a common mistake<ref name="generalized" /> to always use the <math>\mbox{k-distance}</math>, this yields a slightly different method, referred to as Simplified-LOF<ref name="generalized" />) | |
| | |
| The ''local reachability density'' of an object <math>A</math> is defined by
| |
| | |
| <math>\mbox{lrd}(A):=1/\left(\frac{\sum_{B\in N_k(A)}\mbox{reachability-distance}_k(A, B)}{|N_k(A)|}\right)</math>
| |
| | |
| Which is the quotient of the average reachability distance of the object <math>A</math> ''from'' its neighbors. Note that it is not the average reachability of the neighbors from <math>A</math> (which by definition would be the <math>\mbox{k-distance}(A)</math>), but the distance at which it can be "reached" ''from'' its neighbors. With duplicate points, this value can become infinite.
| |
| | |
| The local reachability densities are then compared with those of the neighbors using
| |
| | |
| <math>
| |
| \mbox{LOF}_k(A):=\frac{\sum_{B\in N_k(A)}\frac{\mbox{lrd}(B)}{\mbox{lrd}(A)}}{|N_k(A)|}
| |
| = \frac{\sum_{B\in N_k(A)}\mbox{lrd}(B)}{|N_k(A)|} / \mbox{lrd}(A)
| |
| </math>
| |
| | |
| Which is the ''average local reachability density of the neighbors'' divided by the objects own local reachability density. A value of approximately <math>1</math> indicates that the object is comparable to its neighbors (and thus not an outlier). A value below <math>1</math> indicates a denser region (which would be an inlier), while values significantly larger than <math>1</math> indicate outliers.
| |
| | |
| == Advantages ==
| |
| | |
| [[File:LOF.svg|thumb|right|400px|LOF scores as visualized by [[Environment for DeveLoping KDD-Applications Supported by Index-Structures|ELKI]]. While the upper right cluster has a comparable density to the outliers close to the bottom left cluster, they are detected correctly.]]
| |
| | |
| Due to the local approach, LOF is able to identify outliers in a data set that would not be outliers in another area of the data set. For example, a point at a "small" distance to a very dense cluster is an outlier, while a point within a sparse cluster might exhibit similar distances to its neighbors.
| |
| | |
| While the geometric intuition of LOF is only applicable to low dimensional vector spaces, the algorithm can be applied in any context a dissimilarity function can be defined. It has experimentally been shown to work very well in numerous setups, often outperforming the competitors, for example in [[Network intrusion detection system|network intrusion detection]].<ref>{{cite journal | title=A comparative study of anomaly detection schemes in network intrusion detection | year=2003 | authors=Ar Lazarevic, Aysel Ozgur, Levent Ertoz, Jaideep Srivastava, Vipin Kumar | journal=Proc. 3rd SIAM International Conference on Data Mining | url=http://www.siam.org/proceedings/datamining/2003/dm03_03LazarevicA.pdf | pages=25–36}}</ref>
| |
| | |
| The LOF family of methods can be easily generalized and then applied to various other problems, such as detecting outliers in geographic data, video streams or authorship networks.<ref name="generalized" />
| |
| | |
| == Disadvantages and Extensions ==
| |
| | |
| The resulting values are [[quotient]]-values and hard to interpret. A value of 1 or even less indicates a clear inlier, but there is no clear rule for when a point is an outlier. In one data set, a value of 1.1 may already be an outlier, in another dataset and parameterization (with strong local fluctuations) a value of 2 could still be an inlier. These differences can also occur within a dataset due to the locality of the method. There exist extensions of LOF that try to improve over LOF in these aspects:
| |
| | |
| * ''Feature Bagging for Outlier Detection'' <ref>{{cite doi|10.1145/1081870.1081891}}</ref> runs LOF on multiple projections and combines the results for improved detection qualities in high dimensions.
| |
| * ''Local Outlier Probability'' (LoOP)<ref>{{cite doi|10.1145/1645953.1646195}}</ref> is a method derived from LOF but using inexpensive local statistics to become less sensitive to the choice of the parameter ''k''. In addition, the resulting values are scaled to a value range of <math>[0:1]</math>.
| |
| * ''Interpreting and Unifying Outlier Scores'' <ref>{{cite journal | title=Interpreting and Unifying Outlier Scores | year=2011 | authors=[[Hans-Peter Kriegel]], Peer Kröger, Erich Schubert, Arthur Zimek | journal=Proc. 11th SIAM International Conference on Data Mining | url=http://siam.omnibooksonline.com/2011datamining/data/papers/018.pdf }}</ref> proposes a normalization of the LOF outlier scores to the interval <math>[0:1]</math> using statistical scaling to increase [[usability]] and can be seen a improved version of the LoOP ideas.
| |
| * ''On Evaluation of Outlier Rankings and Outlier Scores'' <ref>{{cite journal | title=On Evaluation of Outlier Rankings and Outlier Scores | year=2012 | authors=Erich Schubert, Remigius Wojdanowski, [[Hans-Peter Kriegel]], Arthur Zimek | journal=Proc. 12 SIAM International Conference on Data Mining | url=http://siam.omnibooksonline.com/2012datamining/data/papers/107.pdf }}</ref> proposes methods for measuring similarity and diversity of methods for building advanced outlier detection [[Ensemble learning|ensembles]] using LOF variants and other algorithms and improving on the Feature Bagging approach discussed above.
| |
| * ''Local outlier detection reconsidered: a generalized view on locality with applications to spatial, video, and network outlier detection''<ref name="generalized">{{cite doi | 10.1007/s10618-012-0300-z}}</ref> discusses the general pattern in various local outlier detection methods (including e.g. LOF, a simplified version of LOF and LoOP) and abstracts from this into a general framework. This framework is then applied e.g. to detecting outliers in geographic data, video streams and authorship networks.
| |
| | |
| == References ==
| |
| | |
| <references />
| |
| | |
| [[Category:Statistical outliers]]
| |
| [[Category:Data mining]]
| |
| [[Category:Machine learning algorithms]]
| |
I'm a 36 years old and working at the high school (Environmental Studies).
In my free time I'm trying to teach myself Japanese. I've been there and look forward to go there anytime soon. I love to read, preferably on my ipad. I like to watch Psych and The Vampire Diaries as well as documentaries about nature. I love Rugby league football.