Comments (6)
I've known about this for a long time but didn't really want to address it since it was a bit difficult in the structure of the NumPy random system.
I have not added retries
which defaults to 10, but can be any non-negative integer (0 for no retries, something large like 1000 (this will delay for up to 40us which should be equivalent to infinite in practice). This was simple and should have been done.
The exception handling was more fragile. If you just want 64 bits, then it is very robust and will raise RuntimError. If you want to use it with a NumPy Generator, then it relies on a custom Lock to be used around all calls where the lock checks for an error on exit. The lock will raise RuntimError if RDRAND fails.
Finally, if you do end up in a failed state, the RDRAND instance should not be used again (and will in fact error).
from randomgen.
Our problem is that we need approximately 7 x 10^14 bits of randomness for our application. We are running on a cluster with approximately 160 individual CPUs (1920 cores). It will take 2 hours or so to generate the randomness that we need. We are using numpy. We need to have RDRAND check the carry flag.
How does one end up in a 'failed state' ?
It would be nice to have a fallback, such as 'use the 0th RDRAND value as a seed for a CSPRNG`
How should we proceed?
from randomgen.
You end up in the failed state if RDRAND fails to produce a random value after retries
retries.
You can check for RuntimeError and then decide what you would like to do:
Here is an intention fail that uses a trick to make it always fail:
import randomgen as rg
import numpy as np
rd = rg.RDRAND()
st = rd.state
st["retries"] = -1 # always fail!
rd.state = st
gen = np.random.Generator(rd)
try:
gen.integers(0,2**64, dtype=np.uint64, size=10)
except RuntimeError:
print("RuntimeError hit, do something else or wait a bit and retry")
Once it passes CI I will put together a 1.19.2 release with these changes.
from randomgen.
I have also added a property success
of a RDRAND
instance that you can directly access to see if there has ever been a failure.
from randomgen.
We're running this in a batch environment. We basically want retries=∞. It looks like with your code we can do:
st["retries"] = 1E9
Thanks for attending to this.
from randomgen.
The wheels are now available on PyPI.
from randomgen.
Related Issues (20)
- Please document source and meaning of constant 11400714819323198485ULL HOT 6
- randomgen does not build under numpy 1.18.1 HOT 8
- Partially-Sampled Random Numbers HOT 1
- ENH: Allow all values to be set in AesCounter counter range on initialization HOT 1
- SPECK128's period question HOT 1
- DOC: COrrect ChaCha's period HOT 1
- The same period -1 problem exists with Philox, Threefry, and AESCounter HOT 1
- Enquiries about EFIIX64 HOT 4
- In Threefry's docstrings, it says "Philox"? HOT 1
- The period of AESCounter is 2**128-1? HOT 1
- Randomgen declares cython, setuptools, and wheel as runtime requirements HOT 3
- New release HOT 2
- Mistakes in xoroshiro128 jump implementation HOT 3
- wheels for 3.11 HOT 2
- Xoshiro256 jump is not deterministic HOT 3
- invalid bytes? HOT 4
- Feature request/query: Squares, OpenRand
- xorshift2014 in multihtreaded mode
- Shall we add ARS to randomgen?
- Numpy 2.0 compatibility HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from randomgen.