GENERATION OF STANDARD NORMAL RANDOM NUMBERS Naveen Kumar Boiroju and M. Krishna Reddy Department of Statistics, Osmania University, Hyderabad- 500 007, INDIA Email: nanibyrozu@gmail.com, reddymk54@gmail.com Abstract: In this paper, we propose an easy method to generate standard normal random variables and this method is compared with the other existing methods. It is observed that the proposed method is quite competitive with most of the existing methods. Randomness of the generated sample is tested using the autocorrelation function and Ljung-Box test. Normality of the samples is checked with the help of relation between testing of hypothesis and confidence intervals. Keywords: Autocorrelation function, Bootstrap confidence intervals, Central limit theorem, Ljung-Box Statistic, Standard normal distribution, Random numbers.. Introduction The normal distribution is also called as the Gaussian distribution and it is the most important distribution in statistics. The distribution is given by f ( x) ( x µ ) = exp ; <, < > 0 x µ and σ σ π σ
where µ is a location parameter, equal to the mean, and σ the standard deviation. For µ = 0 and σ =, we refer to this distribution as the standard normal distribution. Some characteristics of a standard normal distribution include the following:. Symmetric about mean and Mean=Median=Mode.. Mean=0, Standard deviation =, Skewness=0 and Kurtosis relative to normal ( β 3) =0. 3. First and third quartiles are -0.6745 and 0.6745 respectively. 4. Normal random variables X s with mean µ and variance σ are generated by the relationship X = µ + σ Z, where Z is the standard normal random variable. Generation or simulation of random numbers, using deterministic algorithms, is widely used by statisticians for several purposes. These include evaluating performance of estimation and test procedures, and also use of simulation as a part of statistical inference. Generating normal random numbers is an old and very important problem in the statistical literature. Standard normal random variables are frequently used in computer science, computational statistics, and in particular, in applications of the Monte Carlo method. Several algorithms are available in the literature to generate normal random numbers like method of sums of uniforms, Box-Muller method, Polar method, acceptance-rejection method, inversion method, etc. Howell and Rheinfurth (98), Devroye (986), Johnson et.al. (995), Thomas et.al. (007) and Rao et.al. (0) provides an extensive list of different algorithms for the generation of standard normal random numbers. In view of the enormous recent development in statistics that relies on the use of simulation, it is essential for every statistician to have some knowledge of the corresponding techniques. It is also useful to note that because of its extensive use, search for an appropriate simulation technique is itself an area of research. In Section, some of the existing methods to generate standard normal
random numbers discussed and a very simple and convenient method is proposed. Testing of randomness and normality of the generated samples is presented in Section 3. Final conclusion is presented in Section 4.. Methods for Generating Standard Normal Random Numbers Some existing methods for generating standard normal random numbers discussed in this section. A new algorithm to generate standard normal random numbers is also proposed and is named as method-9 in this article. Method-: Sum of Uniform Random Variables The simplest way of generating normal variables is an application of the central limit theorem. The central limit theorem is a weak convergence result that expresses the fact that any sum of many small independent random variables is approximately normally distributed. Use of the central limit theorem on U(0,) random variables provide a simple method for closely approximating normal random variates. The following algorithm is used to generate the standard normal variables.. Generate independent uniform random numbers,. Return Z = U i i= 6. U, U, K, U ( ) ~ iid U 0,. This method requires uniform random variables to generate a single standard normal random number. 3
Method-: Box-Muller Method This method is due to Box and Muller (958) and generates a pair of independent standard normal random variables using a pair of uniform random numbers as follows:. Generate two independent random numbers Uand U from U(0,) distribution.. Return Z = lnu Cos( πu ) and Z lnu Sin( πu ) =. Method-3: Polar Method Another form of the Box-Muller method is called the polar technique. This improves over the previous technique in being quicker as well as numerically more robust. The algorithm can be summarized as follows:. Generate two random numbers Uand U from U(0,) distribution.. Set V = U, V = U and S = V +. Note that V and V are U(-,). V 3. If S>, go to step, otherwise go to step 4. 4. Return two independent standard normal variables Z ln S S ln S =. S = V and Z V This algorithm also requires two uniform variables to generate a single standard normal random number. Method-4: Inversion Method follows: This method is due to Hastings (955) and the algorithm can be summarized as 4
. Generate U ~ U(0,).. Return Z = Φ ( U ) 3.if U 0.5, use Z = -Φ = -w + i= 0 3 j= 0 ( U ). i a w i i b w j ; U 0.5. where w = a a a 0 ln ( U ) =.5557; = 0.80853; = 0.0038; ; b b b b 3 0 = ; =.43788; = 0.8969; = 0.00308. This method requires single uniform random number but involves seven constants. Method-5: Acceptance-Rejection Method. If The following method is borrowed from the paper of Sigman (007).. Generate two independent exponentials at rate; Y ( Y ) /, set 3. Generate U.Set Z = 3 Z = Y ; otherwise go to back to step. Z if U 3 0.5, set Z = - Z if U Y = ln 3 ( U ) and Y = ln( U ). > 0.5. This algorithm is an iterative algorithm and requires at least three uniform random numbers to generate a single standard random number. Method-6: Using Generalized Exponential Distribution The following algorithm is given by Kundu et.al. (006).. Generate an uniform (0,) random variable, U ~. Generate X = -ln - 0.0775 ( U ). 3. Compute the standard normal random variable as Z U(0,). ln = ( X).08. 0.3807 5
This algorithm requires only one uniform random number for the generation of single standard normal random number. Method-7: Bol shev Formula A formula was suggested by Bol shev (959) is explained in the following steps.. Let the five uniform random numbers, U, U, U and U ~ ( 0,) [ U j ] 5. Compute = 3( ) X. 5 j= 3 3. Z = X 0.0( 3X X ) U., 3 4 5 U This formula requires atleast five uniform random numbers to generate a single standard normal random number. Method-8: Inversion Method The following algorithm proposed by Rao et.al. (0) and it is explained in the following steps.. Generate U from the U(0,) distribution.. Return ln U Z =..70 This algorithm requires only a single uniform random variable to generate the standard normal variables and it is very much easier to apply and compute. 6
Method-9: Proposed Method The proposed algorithm is developed using feedforward neural networks and summarized as follows:. Generate uniform random number U from U(0,) distribution.. Set X = Tanh(-3.35694+8.7754U), X =Tanh(-.5736-3.6364U), X 3 =Tanh(3.94963-.66888U) and X 4 =Tanh(.39+.8489U). 3. Return Z=0.4665+90.79X -89.36967X -96.55499X 3 +97.36346X 4. This method requires only one uniform random number to generate the standard normal random numbers. 3. Testing Randomness and Normality A comparative study of the above methods discussed in this section. 000 random numbers generated using each of the methods discussed in section and tested for the randomness and normality of the random numbers generated using the proposed method. Testing Randomness: For a random series, we expect to find that autocorrelation function (ACF) is zero for all non-zero values of lag k and none of the autocorrelations is significantly different from zero at the level 0.05. Hence, the ACF is used as a graphical test procedure to test the 7
randomness of the generated data (Chatfield, 99). The sample ACF of the standard normal random numbers generated from the proposed method (Method-9) is presented in the following figure. Figure : Autocorrelation function of the sample generated from the proposed method As the results indicate, none of these autocorrelations of the standard normal random numbers generated from the proposed method is significantly different from zero at the level 0.05. This proves that the generated sample is a random sample. Randomness of the sample generated from the proposed method is tested using Ljung- Box test (Chatfield, 99; Ljung and Box, 978). The following table presents the values of Ljung-Box statistic and its significant probability values for different lags of the autocorrelations. 8
Table : Ljung-Box test Lag Ljung-Box Statistic Value df P-value 0 6.96 0 0.79 0 5.838 0 0.77 30 6.07 30 0.674 40 43.304 40 0.33 50 55.90 50 0.63 60 70.495 60 0.67 70 84.4 70 0.5 80 95.60 80 0. 90 0.98 90 0.77 00 08.68 00 0.6 From the above table, it is observed that the P-values of the Ljung-Box statistic at different lags is greater than the level 0.05, therefore we accept the null hypothesis of randomness of the sample and we may conclude that the sample generated using the proposed method is a random sample. Testing of Normality: The histogram of the sample generated using the proposed method (Method-9) is presented in the following figure. 9
Figure : Histogram of the sample generated using the proposed method From the above histogram, it is observed that the sample generated using the proposed method is symmetric and normal. The following table presents the descriptive statistics of the samples generated using each of the methods. Table : Descriptive Statistics Descriptive Method Statistics M M M3 M4 M5 M6 M7 M8 M9 N Valid 000 000 000 000 000 000 000 000 000 Mean -0.00-0.03-0.03 0.060 0.048 0.05 0.039-0.04 0.04 Median -0.007-0.00-0.054 0.09 0.049 0.0 0.03-0.0 0.059 Mode -0.06 0.059-0.8-0.033 0.049 0.00-0.04-0.006 0.094 Std. Deviation 0.983.04.005 0.984.00.05 0.960 0.989 0.998 Skewness 0.000-0.003 0.093 0.09-0.39 0.09 0.056 0.006-0.054 Kurtosis -0.78 0.030-0.94 0.47-0.09-0.6-0.63-0.039-0.055 Range 5.930 6.074 6. 6.634 6.530 6.503 5.65 5.8 5.745 Minimum -3.80 -.963 -.909-3.03-3.337-3.76 -.83 -.590 -.895 Maximum.750 3. 3.0 3.6 3.94 3.36.80.68.85 Quartiles Q -0.670-0.74-0.705-0.597-0.636-0.703-0.663-0.650-0.633 Q -0.007-0.00-0.054 0.09 0.049 0.0 0.03-0.0 0.059 Q3 0.673 0.653 0.643 0.77 0.75 0.753 0.694 0.603 0.705 0
From the above table, it is clear that the mean, median and mode are almost equal to zero and same for the samples generated from the each method. Standard deviation, skewness and kurtosis of the random numbers generated using each of the methods is near to one, zero and zero respectively. This indicates that the methods producing standard normal random numbers. The sample quartiles are also agreeing with that of the standard normal distribution. Method-4 generates a wide range of the standard random numbers whereas method-8 generates a low range of the standard normal random numbers. The relation between testing of hypothesis and confidence intervals is used to test the normality of the samples. The characteristics of the standard normal distribution is checked and tested with the help of bootstrap confidence intervals (Becher et.al.; 993 and Efron and Tibishirani; 993). If all the parameters (mean=0, standard deviation=, skewness=0 and kurtosis=0 ) of the standard normal distribution lies in between the lower and upper bounds of bootstrap confidence interval, then the normality of the sample is satisfied. Bootstrap confidence intervals for the mean, standard deviation, skewness and kurtosis of the samples generated from each of the method is presented in the following table.
Table 3: Bootstrap confidence intervals Bootstrap method (0,000 iterations) Method Descriptive Statistics Statistic 95% Confidence Interval Bias Std. Error Lower Upper Mean -0.00 0 0.03-0.063 0.058 Method- Std. Deviation 0.983-0.00 0.0 0.94.04 Skewness 0 0.00 0.068-0.35 0.34 Kurtosis -0.78-0.004 0.5-0.40 0.058 Mean -0.03 0.00 0.03-0.095 0.03 Method- Std. Deviation.04-0.00 0.03 0.978.069 Skewness -0.003-0.00 0.073-0.44 0.36 Kurtosis 0.03-0.004 0.6-0.93 0.64 Mean -0.03 0 0.03-0.086 0.037 Method-3 Std. Deviation.005 0 0.0 0.965.046 Skewness 0.093 0 0.064-0.035 0.9 Kurtosis -0.94 0.00 0. -0.399 0.04 Mean 0.06 0.00 0.03 0.00 0. Method-4 Std. Deviation 0.984-0.00 0.03 0.94.09 Skewness 0.09 0 0.084-0.073 0.55 Kurtosis 0.47-0.009 0.7-0.79 0.483 Mean 0.048 0.00 0.03-0.04 0. Method-5 Std. Deviation.00 0 0.0 0.958.046 Skewness -0.39-0.00 0.078-0.93 0.0 Kurtosis -0.09-0.00 0.53-0.306 0.99 Mean 0.05 0.00 0.03-0.047 0.08 Method-6 Std. Deviation.05-0.00 0.0 0.98.067 Skewness 0.09 0.00 0.069-0.05 0.66 Kurtosis -0.6-0.00 0. -0.39 0.08 Mean 0.039 0 0.03-0.09 0. Method-7 Std. Deviation 0.96 0 0.0 0.9.00 Skewness 0.056 0.00 0.069-0.077 0.94 Kurtosis -0.63-0.004 0.08-0.374 0.045 Mean -0.04 0 0.03-0.073 0.045 Method-8 Std. Deviation 0.989-0.00 0.0 0.948.030 Skewness 0.006 0 0.068-0.6 0.36 Kurtosis -0.039-0.00 0.03-0.38 0.7 Mean 0.04 0 0.03-0.0 0.0 Method-9 Std. Deviation 0.998-0.00 0.0 0.956.04 Skewness -0.054 0 0.07-0.93 0.086 Kurtosis -0.055-0.005 0.4-0.75 0.66
From the above table, it is observed that the parameters of the standard normal distribution belong to the respective bootstrap confidence intervals for all the methods. This implies that all the methods considered in this article can be used to generate the standard normal random numbers. 4. Conclusion From the above study, it is observed that the proposed method possesses the randomness as well as the normality. The proposed method is an easy and convenient method to generate the standard normal random numbers. Hence, the proposed method can be viewed as a competitive method to generate the standard normal random numbers. References: [] Becher H., Hall P., Wilson S.R. (993), Bootstrap hypothesis testing procedures, Biometrics, 49(4): 68-7. [] Bol shev, L.N. (959), On transformations of random variables, Teoriega Veroyatitnosteti i ee primeneniya, 4, 9-4. [3] Box, G. E. P. and Muller, M. E. (958), A note on the generation of random normal deviates, Annals of Mathematical Statistics, 9, 60-6. [4] Chatfield, C. (99), The Analysis of Time Series: An Introduction, 5 th ed., Chapman and Hall, London. [5] Devroye L. (986), Non-uniform random variable distribution, Springer-Verlag: New York. [6] Efron B., Tibshirani R.J. (993), An introduction to the Bootstrap, Chapman & Hall, New York. 3
[7] Hastings, C.(955), Approximations for Digital Computers. Princeton University Press, Princeton, N.J. [8] Howell, L.W. and Rheinfurth, M.H. (98), Generation of Pseudo-Random Numbers, Technical Paper 05, NASA. [9] Johnson, N., Kotz, S. and Balakrishnan, N. (995), Continuous Univariate Distribution, Vol., John Wiley and Sons, New York. [0] Kundu, D., Gupta, R.D. and Manglick, A. (006), A convenient way of generating normal random variables using generalized exponential distribution J. Modern Appl. Statist. Methods, 5, 66 7. (home.iitk.ac.in/~kundu/paper04.pdf.) [] Ljung, G. M. and Box, G. E. P. (978), On a measure of lack of fit in time series models, Biometrika, 65, 97 303. [] Rao, K.R., Boiroju, N.K. and Reddy, M.K. (0), Generation of standard normal random variables, Indian Journal of Scientific Research, (4), 83-85. [3] Sigman, K. (007), Simulating normal (Gaussian) rvs with applications to simulating Brownian motion and geometric Brownian motion in one and two dimensions, www.columbia.edu/~ks0/...sigman/4703-07-notes-bm-gbm-i.pdf. [4] Thomas, D.B., Luk, W., Leong, P.H.W., Villasenor, J.D. (007), Gaussian Random Number Generators, ACM Computing Surveys, Vol. 39, No. 4, Article. 4