Theorem on friends and strangers: Difference between revisions
en>David Eppstein remove redundant category |
simple graphs only have (x, y) x != y type of edge, but text didn't explicitly mention that by "graph" we meant "simple graph" |
||
Line 1: | Line 1: | ||
'''Consistent hashing''' is a special kind of [[hash function|hashing]] such that when a hash table is resized and consistent hashing is used, only <math>K/n</math> keys need to be remapped on average, where <math>K</math> is the number of keys, and <math>n</math> is the number of slots. In contrast, in most traditional [[hash table]]s, a change in the number of array slots causes nearly all keys to be remapped. | |||
Consistent hashing achieves the same goals as '''[[Rendezvous hashing]]''' (also called HRW Hashing). The two techniques use different algorithms, and were devised independently and contemporaneously. | |||
==History== | |||
Originally devised by [[David Karger|Karger]] ''et al.'' at MIT for use in distributed caching, the idea has now been expanded to other areas also. | |||
An academic paper from 1997 introduced the term "consistent hashing" as a way of distributing requests among a changing population of Web servers. Each slot is then represented by a node in a distributed system. The addition (joins) and removal (leaves/failures) of nodes only requires <math>K/n</math> items to be re-shuffled when the number of slots/nodes change.<ref name="KargerEtAl1997">{{cite conference | |||
| url = http://portal.acm.org/citation.cfm?id=258660 | |||
| doi = 10.1145/258533.258660 | |||
| author = Karger, D.; Lehman, E.; [[F. Thomson Leighton|Leighton, T.]]; Panigrahy, R.; Levine, M.; [[Daniel M. Lewin|Lewin, D.]] | |||
| booktitle = Proceedings of the Twenty-ninth Annual ACM [[Symposium on Theory of Computing]] | |||
| pages = 654–663 | |||
| year = 1997 | |||
| publisher = ACM Press New York, NY, USA | |||
| title = Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web | |||
}}</ref> | |||
Consistent hashing has also been used to reduce the impact of partial system failures in large Web applications as to allow for robust caches without incurring the system wide fallout of a failure.<ref name=KargerEtAl1999>{{cite journal | |||
| url = http://www8.org/w8-papers/2a-webserver/caching/paper2.html | |||
| doi = 10.1016/S1389-1286(99)00055-9 | |||
| author = Karger, D.; Sherman, A.; Berkheimer, A.; Bogstad, B.; Dhanidina, R.; Iwamoto, K.; Kim, B.; Matkins, L.; Yerushalmi, Y. | |||
| journal = Computer Networks | |||
| volume = 31 | |||
| issue = 11 | |||
| pages = 1203–1213 | |||
| year = 1999 | |||
| title = Web Caching with Consistent Hashing | |||
}}</ref> | |||
The consistent hashing concept also applies to the design of [[distributed hash table]]s (DHTs). DHTs use consistent hashing to partition a keyspace among a distributed set of nodes, and additionally provide an overlay network that connects nodes such that the node responsible for any key can be efficiently located. | |||
[[Rendezvous hashing]], designed at the same time as consistent hashing, achieves the same goals using the very different Highest Random Weight (HRW) algorithm. | |||
<!-- should describe here why the "consistency" of consistent hashing is essential to DHTs --> | |||
==Need for consistent hashing== | |||
While running collections of caching machines some limitations are experienced. A common way of load balancing <math>n</math> cache machines is to put object <math>o</math> in cache machine number <math>\mbox{hash}(o) \mod n</math>. But this will not work if a cache machine is added or removed because <math>n</math> changes and every object is hashed to a new location. This can be disastrous since the originating content servers are flooded with requests from the cache machines. Hence consistent hashing is needed to avoid swamping of servers. | |||
Consistent hashing maps objects to the same cache machine, as far as possible. It means when a cache machine is added, it takes its share of objects from all the other cache machines and when it is removed, its objects are shared between the remaining machines. | |||
The main idea behind the consistent hashing algorithm is to associate each cache with one or more hash value intervals where the interval boundaries are determined by calculating the hash of each cache identifier. (The hash function used to define the intervals does not have to be the same function used to hash the cached values. Only the range of the two functions need match.) If the cache is removed its interval is taken over by a cache with an adjacent interval. All the remaining caches are unchanged. | |||
==Technique== | |||
Consistent hashing is based on mapping each object to a point on the edge of a circle (or equivalently, mapping each object to a real angle). | |||
The system maps each available machine (or other storage bucket) to many pseudo-randomly distributed points on the edge of the same circle. | |||
To find where an object should be placed, the system finds the location of that object's key on the edge of the circle; | |||
then walks around the circle until falling into the first bucket it encounters (or equivalently, the first available bucket with a higher angle). | |||
The result is that each bucket contains all the resources located between its point and the previous bucket point. | |||
If a bucket becomes unavailable (for example because the computer it resides on is not reachable), then the angles it maps to will be removed. Requests for resources that would have mapped to each of those points now map to the next highest point. Since each bucket is associated with many pseudo-randomly distributed points, the resources that were held by that bucket will now map to many different buckets. The items that mapped to the lost bucket must be redistributed among the remaining ones, but values mapping to other buckets will still do so and do not need to be moved. | |||
A similar process occurs when a bucket is added. By adding a bucket point, we make any resources between that and the next smaller angle map to the new bucket. These resources will no longer be associated with the previous bucket, and any value previously stored there will not be found by the selection method described above. | |||
The portion of the keys associated with each bucket can be altered by altering the number of angles that bucket maps to. | |||
== Monotonic keys == | |||
If it is known that key values will always increase monotonically, [[Hash_table#Monotonic_keys|an alternative approach to consistent hashing]] is possible. | |||
==Properties== | |||
[[David Karger]] et al. list several properties of consistent hashing that make it useful for distributed caching protocols on the Internet:<ref name="KargerEtAl1997" /> | |||
* "spread" | |||
* "load" | |||
* "smoothness" | |||
* "balance" | |||
* "monotonic" | |||
== Examples of use == | |||
Consistent hashing is used in [[Openstack]]'s Object Storage Service Swift.<ref>http://docs.openstack.org/developer/swift/ring.html</ref> | |||
Consistent hashing is used in the partitioning component of Amazon's storage system Dynamo.<ref name="Amazon2007">{{cite journal | |||
| author = DeCandia, G.; Hastorun, D.; Jampani, M.; Kakulapati, G.; Lakshman, A.; Pilchin, A.; Sivasubramanian, S.; Vosshall, P.; Vogels, W. | |||
| journal = Proceedings of the 21st ACM Symposium on Operating Systems Principles | |||
| year = 2007 | |||
| title =Dynamo: Amazon's Highly Available Key-Value Store | |||
}}</ref> | |||
== References == | |||
{{Reflist}} | |||
== External links == | |||
* [http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/ Understanding Consistent hashing] | |||
* Implentations in various languages: | |||
** [http://www.martinbroadhurst.com/Consistent-Hash-Ring.html C++] | |||
** [http://code.google.com/p/consistent-hash/ C#] | |||
** [https://github.com/basho/riak_core/blob/master/src/chash.erl Erlang] | |||
** [https://github.com/stathat/consistent Go] | |||
** [http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html Java] | |||
** [https://github.com/pda/flexihash PHP] | |||
** [http://amix.dk/blog/post/19367 Python] | |||
** [https://github.com/superfeedr/consistent_hashr Ruby] | |||
<!--Interwikies--> | |||
{{DEFAULTSORT:Consistent hashing}} | |||
<!--Categories--> | |||
[[Category:Hashing]] |
Latest revision as of 18:15, 10 September 2013
Consistent hashing is a special kind of hashing such that when a hash table is resized and consistent hashing is used, only keys need to be remapped on average, where is the number of keys, and is the number of slots. In contrast, in most traditional hash tables, a change in the number of array slots causes nearly all keys to be remapped.
Consistent hashing achieves the same goals as Rendezvous hashing (also called HRW Hashing). The two techniques use different algorithms, and were devised independently and contemporaneously.
History
Originally devised by Karger et al. at MIT for use in distributed caching, the idea has now been expanded to other areas also. An academic paper from 1997 introduced the term "consistent hashing" as a way of distributing requests among a changing population of Web servers. Each slot is then represented by a node in a distributed system. The addition (joins) and removal (leaves/failures) of nodes only requires items to be re-shuffled when the number of slots/nodes change.[1]
Consistent hashing has also been used to reduce the impact of partial system failures in large Web applications as to allow for robust caches without incurring the system wide fallout of a failure.[2]
The consistent hashing concept also applies to the design of distributed hash tables (DHTs). DHTs use consistent hashing to partition a keyspace among a distributed set of nodes, and additionally provide an overlay network that connects nodes such that the node responsible for any key can be efficiently located.
Rendezvous hashing, designed at the same time as consistent hashing, achieves the same goals using the very different Highest Random Weight (HRW) algorithm.
Need for consistent hashing
While running collections of caching machines some limitations are experienced. A common way of load balancing cache machines is to put object in cache machine number . But this will not work if a cache machine is added or removed because changes and every object is hashed to a new location. This can be disastrous since the originating content servers are flooded with requests from the cache machines. Hence consistent hashing is needed to avoid swamping of servers.
Consistent hashing maps objects to the same cache machine, as far as possible. It means when a cache machine is added, it takes its share of objects from all the other cache machines and when it is removed, its objects are shared between the remaining machines.
The main idea behind the consistent hashing algorithm is to associate each cache with one or more hash value intervals where the interval boundaries are determined by calculating the hash of each cache identifier. (The hash function used to define the intervals does not have to be the same function used to hash the cached values. Only the range of the two functions need match.) If the cache is removed its interval is taken over by a cache with an adjacent interval. All the remaining caches are unchanged.
Technique
Consistent hashing is based on mapping each object to a point on the edge of a circle (or equivalently, mapping each object to a real angle). The system maps each available machine (or other storage bucket) to many pseudo-randomly distributed points on the edge of the same circle.
To find where an object should be placed, the system finds the location of that object's key on the edge of the circle; then walks around the circle until falling into the first bucket it encounters (or equivalently, the first available bucket with a higher angle). The result is that each bucket contains all the resources located between its point and the previous bucket point.
If a bucket becomes unavailable (for example because the computer it resides on is not reachable), then the angles it maps to will be removed. Requests for resources that would have mapped to each of those points now map to the next highest point. Since each bucket is associated with many pseudo-randomly distributed points, the resources that were held by that bucket will now map to many different buckets. The items that mapped to the lost bucket must be redistributed among the remaining ones, but values mapping to other buckets will still do so and do not need to be moved.
A similar process occurs when a bucket is added. By adding a bucket point, we make any resources between that and the next smaller angle map to the new bucket. These resources will no longer be associated with the previous bucket, and any value previously stored there will not be found by the selection method described above.
The portion of the keys associated with each bucket can be altered by altering the number of angles that bucket maps to.
Monotonic keys
If it is known that key values will always increase monotonically, an alternative approach to consistent hashing is possible.
Properties
David Karger et al. list several properties of consistent hashing that make it useful for distributed caching protocols on the Internet:[1]
- "spread"
- "load"
- "smoothness"
- "balance"
- "monotonic"
Examples of use
Consistent hashing is used in Openstack's Object Storage Service Swift.[3]
Consistent hashing is used in the partitioning component of Amazon's storage system Dynamo.[4]
References
43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.
External links
- ↑ 1.0 1.1 55 years old Systems Administrator Antony from Clarence Creek, really loves learning, PC Software and aerobics. Likes to travel and was inspired after making a journey to Historic Ensemble of the Potala Palace.
You can view that web-site... ccleaner free download - ↑ One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting
In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang
Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules
Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.
A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running
The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more
There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang - ↑ http://docs.openstack.org/developer/swift/ring.html
- ↑ One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting
In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang
Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules
Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.
A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running
The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more
There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang