Minuscule 629: Difference between revisions
Jump to navigation
Jump to search
en>Bucksburg →Text: Unique Readings |
en>Leszek Jańczuk |
||
| Line 1: | Line 1: | ||
A | '''Competitive learning''' is a form of [[unsupervised learning]] in [[artificial neural networks]], in which nodes compete for the right to respond to a subset of the input data.<ref>{{cite book | last = Rumelhart | first = David | authorlink = David Rumelhart | coauthors = David Zipser, James L. McClelland, et al. | title = Parallel Distributed Processing, Vol. 1 | publisher = MIT Press | year = 1986 | location = | pages = 151–193 | url = | doi = | id = | isbn = }}</ref> A variant of [[Hebbian learning]], competitive learning works by increasing the specialization of each node in the network. It is well suited to finding [[cluster analysis|clusters]] within data. | ||
Models and algorithms based on the principle of competitive learning include [[vector quantization]] and [[self-organising map]]s (Kohonen maps). | |||
== Architecture and implementation == | |||
[[File:Competitive neural network architecture.png|thumb|Competitive neural network architecture]] | |||
Competitive Learning is usually implemented with Neural Networks that contain a hidden layer which is commonly known as “competitive layer”.<ref>{{cite web |url=http://jsalatas.ictpro.gr/implementation-of-competitive-learning-networks-for-weka/ |title=Implementation of Competitive Learning Networks for WEKA |author=Salatas, John |date= 24 August 2011|work= |publisher= ICT Research Blog|accessdate=28 January 2012}}</ref> Every competitive neuron i is described by a vector of weights <math>{\mathbf{w}}_i = \left( {w_{i1} ,..,w_{id} } \right)^T ,i = 1,..,M</math> and calculates the similarity measure between the input data <math>{\mathbf{x}}^n = \left( {x_{n1} ,..,x_{nd} } \right)^T \in \mathbb{R}^d</math> and the weight vector <math>{\mathbf{w}}_i</math> . | |||
For every input vector, the competitive neurons “compete” with each other to see which one of them is the most similar to that particular input vector. The winner neuron m sets its output <math>o_i = 1</math> and all the other competitive neurons set their output <math>o_i = 0 , i = 1,..,M, i \ne m</math>. | |||
Usually, in order to measure similarity the inverse of the Euclidean distance is used: <math>\left\| {{\mathbf{x}} - {\mathbf{w}}_i } \right\|</math> between the input vector <math>{\mathbf{x}}^n</math> and the weight vector <math>{\mathbf{w}}_i</math>. | |||
== Example algorithm == | |||
Here is a simple competitive learning algorithm to find three clusters within some input data. | |||
1. (Set-up.) Let a set of sensors all feed into three different nodes, so that every node is connected to every sensor. Let the weights that each node gives to its sensors be set randomly between 0.0 and 1.0. Let the output of each node be the sum of all its sensors, each sensor's signal strength being multiplied by its weight. | |||
2. When the net is shown an input, the node with the highest output is deemed the winner. The input is classified as being within the cluster corresponding to that node. | |||
3. The winner updates each of its weights, moving weight from the connections that gave it weaker signals to the connections that gave it stronger signals. | |||
Thus, as more data are received, each node converges on the centre of the cluster that it has come to represent and activates more strongly for inputs in this cluster and more weakly for inputs in other clusters. | |||
==References== | |||
{{reflist}} | |||
==Further Information and Software== | |||
* [http://www.demogng.de/JavaPaper/t.html Draft Report "Some Competitive Learning Methods"] (contains descriptions of several related algos) | |||
* [http://www.demogng.de DemoGNG - Java simulator for competitive learning methods] | |||
[[Category:Neural networks]] | |||
Revision as of 22:47, 25 May 2013
Competitive learning is a form of unsupervised learning in artificial neural networks, in which nodes compete for the right to respond to a subset of the input data.[1] A variant of Hebbian learning, competitive learning works by increasing the specialization of each node in the network. It is well suited to finding clusters within data.
Models and algorithms based on the principle of competitive learning include vector quantization and self-organising maps (Kohonen maps).
Architecture and implementation
Competitive Learning is usually implemented with Neural Networks that contain a hidden layer which is commonly known as “competitive layer”.[2] Every competitive neuron i is described by a vector of weights and calculates the similarity measure between the input data and the weight vector .
For every input vector, the competitive neurons “compete” with each other to see which one of them is the most similar to that particular input vector. The winner neuron m sets its output and all the other competitive neurons set their output .
Usually, in order to measure similarity the inverse of the Euclidean distance is used: between the input vector and the weight vector .
Example algorithm
Here is a simple competitive learning algorithm to find three clusters within some input data.
1. (Set-up.) Let a set of sensors all feed into three different nodes, so that every node is connected to every sensor. Let the weights that each node gives to its sensors be set randomly between 0.0 and 1.0. Let the output of each node be the sum of all its sensors, each sensor's signal strength being multiplied by its weight.
2. When the net is shown an input, the node with the highest output is deemed the winner. The input is classified as being within the cluster corresponding to that node.
3. The winner updates each of its weights, moving weight from the connections that gave it weaker signals to the connections that gave it stronger signals.
Thus, as more data are received, each node converges on the centre of the cluster that it has come to represent and activates more strongly for inputs in this cluster and more weakly for inputs in other clusters.
References
43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.
Further Information and Software
- Draft Report "Some Competitive Learning Methods" (contains descriptions of several related algos)
- DemoGNG - Java simulator for competitive learning methods
- ↑ 20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.
My blog: http://www.primaboinca.com/view_profile.php?userid=5889534 - ↑ Template:Cite web