|
|
| Line 1: |
Line 1: |
| {{Expert-subject|mathematics|date=November 2008}}
| | I would like to introduce myself to you, I am Andrew and my wife doesn't like it at all. He is an purchase clerk and it's something he really enjoy. The favorite hobby for him and his kids is style and he'll be starting some thing else along with it. For many years he's been living in Mississippi and he doesn't plan on changing it.<br><br>Look into my weblog: best [http://isaworld.pe.kr/?document_srl=392088 live psychic reading] [http://www.skullrocker.com/blogs/post/10991 love psychic readings] ([http://hknews.classicmall.com.hk/groups/some-simple-tips-for-personal-development-progress/ mouse click the next webpage]) |
| | |
| In [[computer science]], '''multiply-with-carry''' (MWC) is a method invented by [[George Marsaglia]] for generating sequences of random integers based on an initial set from two to many thousands of randomly chosen seed values. The main advantages of the MWC method are that it invokes simple computer integer arithmetic and leads to very fast generation of sequences of random numbers with immense periods, ranging from around 2<sup>60</sup> to 2<sup>2000000</sup>.
| |
| | |
| As with most [[pseudorandom number generator]]s, the resulting sequences are functions of the randomly chosen seed values.
| |
| | |
| ==General theory==
| |
| A MWC sequence is based on arithmetic modulo a base ''b'',
| |
| usually ''b'' = 2<sup>32</sup>, because arithmetic modulo of that ''b''
| |
| is automatic in most computers. However, sometimes a base such as
| |
| ''b'' = 2<sup>32</sup> − 1 is used, because arithmetic for modulus 2<sup>32</sup> − 1
| |
| requires only a simple adjustment from that
| |
| for 2<sup>32</sup>, and theory for MWC sequences based on
| |
| modulus 2<sup>32</sup> has some nagging difficulties avoided
| |
| by using ''b'' = 2<sup>32</sup> − 1.
| |
| | |
| In its most common form, a '''lag-r''' MWC generator requires a
| |
| '''base''' ''b'', a '''multiplier''' ''a'', and a set of
| |
| ''r+1'' random '''seed''' values, consisting of ''r'' residues of ''b'',
| |
| :''x''<sub>0</sub>, ''x''<sub>1</sub>, ''x''<sub>2 </sub>,..., ''x''<sub>''r''−1</sub>,
| |
| and an initial '''carry''' ''c<sub>''r''−1</sub>'' < ''a''.
| |
| | |
| The lag-''r'' MWC sequence is then a sequence of pairs
| |
| ''x''<sub>''n''</sub>, ''c''<sub>''n''</sub> determined by
| |
| | |
| : <math> x_n=(ax_{n-r}+c_{n-1})\,\bmod\,b,\ c_n=\left\lfloor\frac{ax_{n-r}+c_{n-1}}{b}\right\rfloor,\ n\ge r,</math>
| |
| | |
| and the MWC generator output is the sequence of ''x'''s, | |
| :''x''<sub>''r''</sub> , ''x''<sub>''r''+1</sub> , ''x''<sub>''r''+2</sub>, ...
| |
| | |
| The period of a lag-''r'' MWC generator is the order of ''b'' in the multiplicative group of numbers modulo ''ab<sup>r</sup>'' − 1. | |
| It is customary to choose ''a'''s so that ''p = ab<sup>r</sup>'' − 1 is a prime for which the order of ''b'' can be determined.
| |
| Because 2 is a [[quadratic residue]] of numbers of the form ''8k±1'', ''b'' = 2<sup>32</sup> cannot be a primitive root of ''p = ab<sup>r</sup>'' − 1. Therefore there are no MWC generators for base 2<sup>32</sup> that have the maximum possible period, one of the difficulties that use of ''b'' = 2<sup>32</sup> − 1 overcomes.
| |
| | |
| A theoretical problem with MWC generators, pointed out by Couture and l'Ecuyer (1997) is that the most significant bits are slightly biased; [[#Complementary-multiply-with-carry generators|complementary-multiply-with-carry]] generators do not share this problem: "We shall see that, for the complementary MWC, each bit of the output value is fair, that is, the two binary digits will appear equally often in a full period, a property not shared by MWC generators." They do not appear to elaborate further as to the extent of the bias. Complementary-multiply-with-carry generators also require slightly more computation time per iteration, so there is a tradeoff to evaluate depending on implementation requirements.
| |
| | |
| ==Comparisons with linear congruential generators==
| |
| [[Linear congruential generator]]s are implemented as
| |
| :<math>x_{n+1}=(ax_n+c)\ \bmod\,2^{32},</math>
| |
| because most
| |
| arithmetic processors are able to put the multiplier ''a'' and
| |
| the current ''x'' in 32-bit registers, form the 64-bit
| |
| product in adjoining registers, and take the lower 32 bits as the product, that is,
| |
| form
| |
| :<math>a\times x\ \bmod\,2^{32}</math>.
| |
| Adding the 32-bit ''c'' to that lower half then
| |
| provides (''ax''+''c'') mod 2<sup>32</sup>.
| |
| If ''a'' mod 8 is 1 or 5 and ''c'' is odd, the resulting base
| |
| 2<sup>32</sup> congruential sequence will have period
| |
| 2<sup>32</sup>.
| |
| | |
| A lag-1 multiply-with-carry generator allows us to make the period
| |
| nearly 2<sup>63</sup>
| |
| by using those same computer operations,
| |
| except that this time the top half of the 64-bit product is used rather than ignored after
| |
| the 64 bits are formed. It's used as a '''new''' carry value
| |
| ''c'' rather than the fixed carry value
| |
| of the standard congruential sequence:
| |
| Get ''ax''+''c'' in 64-bits, then form a new ''c'' as the top half of those 64 bits, and the
| |
| new ''x'' as the bottom half.
| |
| | |
| With multiplier ''a'' specified,
| |
| each pair of input values ''x'', ''c'' is converted to a new pair,
| |
| | |
| :<math>x\leftarrow (ax+c)\,\bmod\,2^{32},\ \ c\leftarrow \left\lfloor\frac{ax+c}{2^{32}}\right\rfloor.</math>
| |
| | |
| If ''x'' and ''c'' are not both zero, then the period of the resulting multiply-with-carry sequence will be the order of ''b'' = 2<sup>32</sup>
| |
| in the multiplicative group of residues modulo
| |
| ''ab'' − 1, that is, the smallest ''n'' such that
| |
| ''b''<sup>''n''</sup> = 1 mod (''ab'' − 1).
| |
| If we choose an ''a'' of 28 to 31 bits such that ''ab''−1 is a "[[safe prime]]",
| |
| that is both ''ab'' − 1 and ''ab''/2 − 1 are prime, then the period will be ''ab''/2 − 1, approaching 2<sup>63</sup>, which in practice may be an acceptably large subset of the number of possible 32-bit pairs (''x'', ''c'').
| |
| | |
| Following are some maximal values of ''a'' for computer applications which satisfy the above safe prime condition:
| |
| | |
| {| class="wikitable" border="1"
| |
| |-
| |
| ! Bits in ''a''
| |
| ! ''b''
| |
| ! Maximum ''a'' Such That ''ab''-1 is a Safe Prime
| |
| ! Period
| |
| |-
| |
| | 15
| |
| | 2<sup>16</sup>
| |
| | 32,718
| |
| | 1,072,103,423
| |
| |-
| |
| | 16
| |
| | 2<sup>16</sup>
| |
| | 65,184
| |
| | 2,135,949,311
| |
| |-
| |
| | 31
| |
| | 2<sup>32</sup>
| |
| | 2,147,483,085
| |
| | 4,611,684,809,394,094,079
| |
| |-
| |
| | 32
| |
| | 2<sup>32</sup>
| |
| | 4,294,967,118
| |
| | 9,223,371,654,602,686,463
| |
| |-
| |
| | 64
| |
| | 2<sup>64</sup>
| |
| | 18,446,744,073,709,550,874
| |
| | 170,141,183,460,469,224,887,945,252,369,640,456,191
| |
| |-
| |
| | 128
| |
| | 2<sup>128</sup>
| |
| | 2<sup>128</sup>-10,408
| |
| | 2<sup>127</sup>(2<sup>128</sup>-10,408)-1
| |
| |-
| |
| | 256
| |
| | 2<sup>256</sup>
| |
| | 2<sup>256</sup>-9166
| |
| | 2<sup>255</sup>(2<sup>256</sup>-9166)-1
| |
| |-
| |
| | 512
| |
| | 2<sup>512</sup>
| |
| | 2<sup>512</sup>-150,736
| |
| | 2<sup>511</sup>(2<sup>512</sup>-150,736)-1
| |
| |}
| |
| | |
| However, as being a safe prime does not affect the randomness of the sequence<!--i.e. why does the above table even exist-->, one may instead simply choose ''a'' such that the order of ''b'' is ''ab''/2 − 1. The following are again maximum values of ''a'' of various sizes.
| |
| <!-- Pari/GP script: maxord(bb,ab)=local(b=2^bb,am=2^ab);for(ma=1,1000,a=am-ma;p=a*b-1;if(isprime(p),o=znorder(Mod(b,p));if(p==o*2+1,print(a" "znorder(Mod(b,p)));break))) -->
| |
| | |
| {| class="wikitable" border="1"
| |
| |-
| |
| ! Bits in ''a''
| |
| ! ''b''
| |
| ! Maximum ''a'' Such that ''b'' has order ''ab''/2−1
| |
| ! Period
| |
| |-
| |
| | 15
| |
| | 2<sup>16</sup>
| |
| | 32,739
| |
| | 1,072,791,551
| |
| |-
| |
| | 16
| |
| | 2<sup>16</sup>
| |
| | 65,514
| |
| | 2,146,762,751
| |
| |-
| |
| | 31
| |
| | 2<sup>32</sup>
| |
| | 2,147,483,580
| |
| | 4,611,685,872,398,499,839
| |
| |-
| |
| | 32
| |
| | 2<sup>32</sup>
| |
| | 4,294,967,220
| |
| | 9,223,371,873,646,018,559
| |
| |-
| |
| | 63
| |
| | 2<sup>64</sup>
| |
| | 9,223,372,036,854,775,668
| |
| | 85,070,591,730,234,614,574,571,566,698,273,439,743
| |
| |-
| |
| | 64
| |
| | 2<sup>64</sup>
| |
| | 18,446,744,073,709,551,500
| |
| | 170,141,183,460,469,230,661,776,147,440,730,111,999
| |
| |}
| |
| | |
| Here is a comparison of congruential and MWC sequences for the simple case of arithmetic modulo 10; here the "registers" are a single digit, adjoining registers are two digits:
| |
| | |
| Starting with <math>x_0=1</math>, the congruential sequence
| |
| | |
| :<math>x_n=(7x_{n-1}+3)\,\bmod\,10,</math>
| |
| | |
| has this sequence of adjoining registers:
| |
| | |
| :<math>10,03,24,31,10,03,24,31,10,\ldots,</math>
| |
| | |
| and the output sequence of ''x'''s, (the rightmost register), has period 4:
| |
| | |
| :<math>0,3,4,1,0,3,4,1,0,3,4,1,\ldots</math>
| |
| | |
| Starting with <math>x_0=1,c_0=3</math>,
| |
| the MWC sequence
| |
| | |
| :<math>x_n=(7x_{n-1}+c_{n-1})\,\bmod\,10,\ c_n=\left\lfloor\frac{7x_{n-1}+c_{n-1}}{10}\right\rfloor,</math>
| |
| | |
| has this sequence of adjoining registers
| |
| | |
| :10,01,07,49,67,55,40,04,28,58,61,13,22,16,43,25,37,52,19,64,34,31 10,01,07,...
| |
| | |
| with output sequence of ''x'''s having period 22:
| |
| :0,1,7,9,7,5,0,4,8,8,1,3,2,6,3,5,7,2,9,4,4,1 0,1,7,9,7,5,0,...
| |
| | |
| Notice that if those repeated segments of ''x'' values are put in reverse order starting from a <math>x_{22n+20}</math>,
| |
| :<math>449275\cdots97101\,449275\cdots9710144\cdots</math>
| |
| we get the expansion
| |
| ''j''/(''ab''−1) with ''a''=7, ''b''=10, ''j=31'':
| |
| | |
| :<math>\frac{31}{69}=.4492753623188405797101\,4492753623\ldots</math>
| |
| | |
| This is true in general: The sequence of ''x'''s produced by a lag-''r'' MWC generator:
| |
| | |
| :<math> x_n=(ax_{n-r}+c_{n-1})\bmod\,b\,,\ \ c_n=\left\lfloor\frac{ax_{n-r}+c_{n-1}}{b}\right\rfloor,</math>
| |
| | |
| when put in reverse order, will be the base-''b'' expansion of a rational ''j''/(''ab''<sup>''r''</sup> − 1) for some 0 < ''j'' < ''ab''<sup>''r''</sup>.
| |
| | |
| Also notice that if, starting with <math>x_0=34</math>, we generate the ordinary congruential sequence
| |
| :<math>x_n=7x_{n-1}\,\bmod\,69</math>,
| |
| we get the period 22 sequence
| |
| :31,10,1,7,49,67,55,40,4,28,58,61,13,22,16,43,25,37,52,19,64,34, 31,10,1,7,...
| |
| and that sequence, reduced mod 10, is
| |
| :1,0,1,7,9,7,5,0,4,8,8,1,3,2,6,3,5,7,2,9,4,4, 1,0,1,7,9,7,5,0,...
| |
| the same sequence of ''x'''s resulting from the MWC sequence.
| |
| | |
| This is true in general, (but apparently only for
| |
| lag-1 MWC sequences):
| |
| | |
| Given initial values <math>x_0,c_0</math>, the sequence <math>x_1,x_2,\ldots</math> resulting from the lag-1 MWC sequence
| |
| | |
| :<math> x_n=(ax_{n-1}+c_{n-1})\,\bmod b\,,\ \ c_n=\left\lfloor\frac{ax_{n-1}+c_{n-1}}{b}\right\rfloor</math>
| |
| | |
| is exactly the congruential sequence ''y''<sub>''n''</sub> = ''ay''<sub>''n'' − 1</sub> mod(''ab'' − 1),
| |
| reduced modulo ''b''.
| |
| | |
| Choice of initial value ''y''<sub>0</sub> merely rotates the cycle of ''x'''s.
| |
| | |
| ==Complementary-multiply-with-carry generators==
| |
| Establishing the period of a lag-''r'' MWC generator usually entails choosing multiplier ''a'' so that ''p''=''ab''<sup>''r''</sup> − 1 is prime.
| |
| If ''p'' is a [[safe prime]], then the order of ''b'' will be ''p'' − 1
| |
| or (''p'' − 1)/2. Otherwise, it is likely that ''p'' − 1 will have to be factored in order to find the order of ''b'' mod ''p'',
| |
| and ''p'' = ''ab''<sup>''r''</sup> − 1 may be difficult to factor.
| |
| | |
| But a prime of the form ''p'' = ''ab''<sup>''r''</sup> + 1 will make ''p''−1 easy to factor, so a
| |
| version of multiply-with-carry that involves the order of ''b'' for a prime ''p'' = ''ab''<sup>''r''</sup> + 1 would reduce considerably the
| |
| computational number theory required to establish the period of a MWC sequence.
| |
| | |
| Fortunately, a slight modification of the MWC procedure leads
| |
| to primes of the form ''ab''<sup>''r''</sup> + 1. The new procedure is called complementary-multiply-with-carry (CMWC),
| |
| | |
| and the setup is the same as that for lag-''r'' MWC: '''multiplier''' ''a'', '''base''' ''b'', ''r'' + 1 seeds
| |
| | |
| : ''x''<sub>0</sub>, ''x''<sub>1</sub>, ''x''<sub>2</sub>, ..., ''x''<sub>''r''−1</sub>, and ''c''<sub>''r'' − 1</sub>.
| |
| | |
| There is a slight change in the generation of a new pair (''x'', ''c''):
| |
| <math> x_n=(b-1)-(ax_{n-r}+c_{n-1})\,\bmod\,b,\ c_n=\left\lfloor\frac{ax_{n-r}+c_{n-1}}{b}\right\rfloor.</math>
| |
| | |
| That is, take the complement, (''b''−1)−''x'', when
| |
| forming the new ''x''.
| |
| | |
| The resulting sequence of ''x'''s produced by the CMWC RNG will have period the order of ''b'' in the multiplicative group of residues modulo
| |
| ''ab''<sup>''r''</sup>+1, and the output ''x'''s, in reverse order, will form the base ''b'' expansion of
| |
| ''j''/(''ab''<sup>''r''</sup>+1) for some 0<''j''<''ab''<sup>''r''</sup>.
| |
| | |
| Use of lag-''r'' CMWC makes it much easier to find periods
| |
| for ''r'''s as large as 512, 1024, 2048, etc.
| |
| (Making ''r'' a power of 2 makes it slightly easier (and faster)
| |
| to access elements in the array containing the ''r'' most recent ''x'''s.)
| |
| | |
| Some examples:
| |
| With ''b''=2<sup>32</sup>, the period of the lag-1024 CMWC
| |
| | |
| : <math> x_n=(b-1)-(ax_{n-1024}+c_{n-1})\,\bmod\,b,\ c_n=\left\lfloor\frac{ax_{n-1024}+c_{n-1}}{b}\right\rfloor.</math> | |
| | |
| will be ''a''<math>\cdot</math>2<sup>32762</sup>,
| |
| about 10<sup>9867</sup> for these three ''a''s: 109111 or 108798 or 108517.
| |
| | |
| With ''b'' = 2<sup>32</sup> and ''a'' = 3636507990, ''p'' = ''ab''<sup>1359</sup> − 1 is a safe prime, so the MWC sequence based on that ''a'' has period 3636507990<math>\cdot</math>2<sup>43487</sup> <math>\approx</math>10<sup>13101</sup>.
| |
| | |
| With ''b'' = 2<sup>32</sup>, a CMWC RNG with near record period may be based
| |
| on the prime ''p'' = 15455296''b''<sup>42658</sup> + 1. The order of ''b'' for that prime is 241489*2<sup>1365056</sup>, about 10<sup>410928</sup>.
| |
| | |
| ==Implementation==
| |
| The following is an implementation of the CMWC algorithm in the [[C (programming language)|C programming language]]. Also, included in the program is a sample initialization function. In this implementation the lag ''r''=4096. The period of the resulting generator is about <math>2^{131104 }</math>.
| |
| | |
| <source lang="c">
| |
| #include <stdint.h>
| |
| | |
| #define PHI 0x9e3779b9
| |
| | |
| static uint32_t Q[4096], c = 362436;
| |
| | |
| void init_rand(uint32_t x)
| |
| {
| |
| int i;
| |
| | |
| Q[0] = x;
| |
| Q[1] = x + PHI;
| |
| Q[2] = x + PHI + PHI;
| |
| | |
| for (i = 3; i < 4096; i++)
| |
| Q[i] = Q[i - 3] ^ Q[i - 2] ^ PHI ^ i;
| |
| }
| |
| | |
| uint32_t rand_cmwc(void) {
| |
| static uint32_t i = 4095;
| |
| uint64_t t;
| |
|
| |
| i = (i + 1) & 4095;
| |
| t = (18705ULL * Q[i]) + c;
| |
| c = t >> 32;
| |
| Q[i] = 0xfffffffe - t;
| |
|
| |
| return Q[i];
| |
| }
| |
| </source>
| |
| | |
| ==See also==
| |
| * [[Pseudorandom number generator]]
| |
| | |
| ==References==
| |
| *{{cite journal |first=G. |last=Marsaglia |first2=A. |last2=Zaman |year=1991 |title=A new class of random number generators |journal=Annals of Applied Probability |volume=1 |issue=3 |pages=462–480 |doi=10.1214/aoap/1177005878 }}
| |
| *{{cite journal |first=G. |last=Marsaglia |title=Random number generators |journal=Journal of Modern Applied Statistical Methods |volume=2 |issue= |pages= |year=2003 |url=http://www.jmasm.com/journal/2003_vol2_no1.pdf }}
| |
| *{{cite journal|last=Marsaglia|first=George|title=Xorshift RNGs|journal=Journal of Statistical Software|year=2003|volume=8|issue=14|pages=1-6|url=http://www.jstatsoft.org/v08/i14/paper}}
| |
| *{{cite journal |first=G. |last=Marsaglia |title=On the randomness of Pi and other decimal expansions |journal=Interstat |year=2005 |volume=5 |url=http://interstat.statjournals.net/YEAR/2005/articles/0510005.pdf }}
| |
| *{{cite journal
| |
| | last =Couture
| |
| | first =Raymond
| |
| | last2 =L'Ecuyer
| |
| | first2 =Pierre
| |
| | title =Distribution properties of Multiply-with-carry random number generators
| |
| | journal =Mathematics of Computation
| |
| | volume =66
| |
| | issue =218
| |
| | pages =591–607
| |
| | year =1997
| |
| | doi =10.1090/S0025-5718-97-00827-2}}
| |
| *{{Cite book | last1=Press | first1=WH | last2=Teukolsky | first2=SA | last3=Vetterling | first3=WT | last4=Flannery | first4=BP | year=2007 | title=Numerical Recipes: The Art of Scientific Computing | edition=3rd | publisher=Cambridge University Press | publication-place=New York | isbn=978-0-521-88068-8 | chapter=Section 7.1.2.B Multiply With Carry (MWC) | chapter-url=http://apps.nrbook.com/empanel/index.html#pg=347}}
| |
| | |
| {{Use dmy dates|date=September 2010}}
| |
| | |
| {{DEFAULTSORT:Multiply-With-Carry}}
| |
| [[Category:Pseudorandom number generators]]
| |