using the hardware rng from python

2024/9/16 23:27:07

Are there any ready made libraries so that the intel hardware prng (rdrand) can be used by numpy programs to fill buffers of random numbers?

Failing this can someone point me in the right direction for some C code that I could adapt or use (I use CPython and Cython with numpy so the bare minimum wrapper shd be enough).

The random generators I want are uniform random numbers between [0,1).

Answer

This code will use /dev/urandom (Unix) or CryptGenRandom APIs (Windows). Which RNG is used, hardware or software, is dependent on the operating system.

If you want to control exactly which generator is used, you must query it through its hardware driver or library. When you have the random bits as a string, you proceeed similarly to this code, using np.fromstring.

Normally we can trust the operating system to use the best entropy sources for its cryptographic services, including the random bit generator. If there is a hardware RNG it is likely to be used, usually combination with other entropy sources.

import os
import numpy as npdef cryptorand(n):a = np.fromstring(os.urandom(n*4), dtype=np.uint32) >> 5b = np.fromstring(os.urandom(n*4), dtype=np.uint32) >> 6return (a * 67108864.0 + b) / 9007199254740992.0

Here is the distribution of 1,000,000 random deviates generated with this method on Mac OS X. As you can see it is quite uniform on [0,1):

enter image description here

If you need really strong cryptographic random deviates, you can use /dev/random instead of /dev/urandom. This applies only to Unix-like systems, not Windows:

import numpy as npdef cryptorand(n):with open('/dev/random','rb') as rnd:a = np.fromstring(rnd.read(n*4), dtype=np.uint32) >> 5b = np.fromstring(rnd.read(n*4), dtype=np.uint32) >> 6return (a * 67108864.0 + b) / 9007199254740992.0

Note that this function might block, unlike the version that uses os.urandom as entropy source.

(Edit1: Updated normalization to equate that of NumPy)

Edit 2: The comments indicates the reason for the question was speed, not cryptographic strength. The purpose of hardware RNG is not speed but strength, so it makes the question invalid. However, a fast and good PRNG which can be an alternative to the Mersenne Twister is George Marsaglia's multiply-with-carry generator. Here is a simple implementation in Cython:

import numpy as np
cimport numpy as cnpcdef cnp.uint32_t gw = 152315241 # any number except 0 or 0x464fffff
cdef cnp.uint32_t gz = 273283728 # any number except 0 or 0x9068ffffdef rand(cnp.intp_t n):"""Generate n random numbers using George Marsaglia's multiply-with-carry method."""global gw, gzcdef cnp.uint32_t w, z, a, bcdef cnp.intp_t icdef cnp.ndarray x = cnp.PyArray_SimpleNew(1, &n, cnp.NPY_FLOAT64)cdef cnp.float64_t *r = <cnp.float64_t*> cnp.PyArray_DATA(x)w,z = gw,gzfor i in range(n): z = 36969 * (z & 65535) + (z >> 16)w = 18000 * (w & 65535) + (w >> 16)a = (z << 16) + wz = 36969 * (z & 65535) + (z >> 16)w = 18000 * (w & 65535) + (w >> 16)b = (z << 16) + wr[i] = (a * 67108864.0 + b) / 9007199254740992.0gw,gz = w,zreturn x

Beware that neither Mersenne Twister nor multiply-with-carry have cryptographic strength.

https://en.xdnf.cn/q/72893.html

Related Q&A

How do I revert sys.stdout.close()?

In the interactive console:>>> import sys >>> sys.stdout <open file <stdout>, mode w at 0xb7810078> >>> sys.stdout.close() >>> sys.stdout # confirming th…

Find a value from x axis that correspond to y axis in matplotlib python

I am trying to do simple task such as to read values of x axis that corresponds to value of y axis in matplotlib and I cannot see what is wrong. In this case I am interested for example to find which v…

Django accessing OneToOneField

Made a view that extended User:class Client(models.Model):user = models.OneToOneField(User, related_name=user)def __unicode__(self):return "%s" % (self.user) I am currently trying to access…

Pandas DataFrame: copy the contents of a column if it is empty

I have the following DataFrame with named columns and index:a a* b b* 1 5 NaN 9 NaN 2 NaN 3 3 NaN 3 4 NaN 1 NaN 4 NaN 9 NaN 7The data…

Solving the most profit algorithm [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.Want to improve this question? Update the question so it focuses on one problem only by editing this post.Closed 9…

Get combobox value in python

Im developing an easy program and I need to get the value from a Combobox. It is easy when the Combobox is in the first created window but for example if I have two windows and the Combobox is in the s…

PyAudio (PortAudio issue) Python

I installed pyaudio with anaconda python. Using conda install pyaudio on windows. It said it installed and it also installed PortAudio with it.However, when I create my file and run it now I get the fo…

Python multiprocessing with M1 Mac

I have a mac (Mac Os 11.1, Python Ver 3.8.2) and need to work in multiprocessing, but the procedures doesn’t work. import multiprocessingdef func(index: int):print(index)manager = multiprocessing.Mana…

What is faster in Python, while or for xrange

We can do numeric iteration like:for i in xrange(10):print i,and in C-style:i = 0 while i < 10:print i,i = i + 1Yes, I know, the first one is less error-prone, more pythonic but is it fast enough as…

Concatenate Numpy arrays with least memory

Not I have 50GB dataset saved as h5py, which is a dictionary inside. The dictionary contains keys from 0 to n, and the values are numpy ndarray(3 dimension) which have the same shape. For example:dicti…