tf.random.Generator

View source on GitHub

Random-number generator.

tf.random.Generator(
    copy_from=None, state=None, alg=None
)

Used in the notebooks

Example:

Creating a generator from a seed:

g = tf.random.Generator.from_seed(1234)
g.normal(shape=(2, 3))
<tf.Tensor: shape=(2, 3), dtype=float32, numpy=
array([[ 0.9356609 ,  1.0854305 , -0.93788373],
       [-0.5061547 ,  1.3169702 ,  0.7137579 ]], dtype=float32)>

Creating a generator from a non-deterministic state:

g = tf.random.Generator.from_non_deterministic_state()
g.normal(shape=(2, 3))
<tf.Tensor: shape=(2, 3), dtype=float32, numpy=...>

All the constructors allow explicitly choosing an Random-Number-Generation (RNG) algorithm. Supported algorithms are "philox" and "threefry". For example:

g = tf.random.Generator.from_seed(123, alg="philox")
g.normal(shape=(2, 3))
<tf.Tensor: shape=(2, 3), dtype=float32, numpy=
array([[ 0.8673864 , -0.29899067, -0.9310337 ],
       [-1.5828488 ,  1.2481191 , -0.6770643 ]], dtype=float32)>

CPU, GPU and TPU with the same algorithm and seed will generate the same integer random numbers. Float-point results (such as the output of normal) may have small numerical discrepancies between different devices.

This class uses a tf.Variable to manage its internal state. Every time random numbers are generated, the state of the generator will change. For example:

g = tf.random.Generator.from_seed(1234)
g.state
<tf.Variable ... numpy=array([1234,    0,    0])>
g.normal(shape=(2, 3))
<...>
g.state
<tf.Variable ... numpy=array([2770,    0,    0])>

The shape of the state is algorithm-specific.

There is also a global generator:

g = tf.random.get_global_generator()
g.normal(shape=(2, 3))
<tf.Tensor: shape=(2, 3), dtype=float32, numpy=...>

When creating a generator inside a tf.distribute.Strategy scope, each replica will get a different stream of random numbers.

For example, in this code:

strat = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"])
with strat.scope():
  g = tf.random.Generator.from_seed(1)
  def f():
    return g.normal([])
  results = strat.run(f).values

results[0] and results[1] will have different values.

If the generator is seeded (e.g. created via Generator.from_seed), the random numbers will be determined by the seed, even though different replicas get different numbers. One can think of a random number generated on a replica as a hash of the replica ID and a "master" random number that may be common to all replicas. Hence, the whole system is still deterministic.

(Note that the random numbers on different replicas are not correlated, even if they are deterministically determined by the same seed. They are not correlated in the sense that no matter what statistics one calculates on them, there won't be any discernable correlation.)

Generators can be freely saved and restored using tf.train.Checkpoint. The checkpoint can be restored in a distribution strategy with a different number of replicas than the original strategy. If a replica ID is present in both the original and the new distribution strategy, its state will be properly restored (i.e. the random-number stream from the restored point will be the same as that from the saving point) unless the replicas have already diverged in their RNG call traces before saving (e.g. one replica has made one RNG call while another has made two RNG calls). We don't have such guarantee if the generator is saved in a strategy scope and restored outside of any strategy scope, or vice versa.

When a generator is created within the scope of tf.distribute.experimental.ParameterServerStrategy, the workers will share the generator's state (placed on one of the parameter servers). In this way the workers will still get different random-number streams, as stated above. (This is similar to replicas in a tf.distribute.MirroredStrategy sequentially accessing a generator created outside the strategy.) Each RNG call on a worker will incur a round-trip to a parameter server, which may have performance impacts. When creating a tf.distribute.experimental.ParameterServerStrategy, please make sure that the variable_partitioner argument won't shard small variables of shape [2] or [3] (because generator states must not be sharded). Ways to avoid sharding small variables include setting variable_partitioner to None or to tf.distribute.experimental.partitioners.MinSizePartitioner with a large enough min_shard_bytes (see tf.distribute.experimental.ParameterServerStrategy's documentation for more details).

Methods

binomial

View source

binomial(
    shape,
    counts,
    probs,
    dtype=tf.dtypes.int32,
    name=None
)

Outputs random values from a binomial distribution.

The generated values follow a binomial distribution with specified count and probability of success parameters.

Example:

counts = [10., 20.]
# Probability of success.
probs = [0.8]

rng = tf.random.Generator.from_seed(seed=234)
binomial_samples = rng.binomial(shape=[2], counts=counts, probs=probs)


counts = ... # Shape [3, 1, 2]
probs = ...  # Shape [1, 4, 2]
shape = [3, 4, 3, 4, 2]
rng = tf.random.Generator.from_seed(seed=1717)
# Sample shape will be [3, 4, 3, 4, 2]
binomial_samples = rng.binomial(shape=shape, counts=counts, probs=probs)

from_key_counter

View source

@classmethod
from_key_counter(
    key, counter, alg
)

Creates a generator from a key and a counter.

This constructor only applies if the algorithm is a counter-based algorithm. See method key for the meaning of "key" and "counter".

from_non_deterministic_state

View source

@classmethod
from_non_deterministic_state(
    alg=None
)

Creates a generator by non-deterministically initializing its state.

The source of the non-determinism will be platform- and time-dependent.

from_seed

View source

@classmethod
from_seed(
    seed, alg=None
)

Creates a generator from a seed.

A seed is a 1024-bit unsigned integer represented either as a Python integer or a vector of integers. Seeds shorter than 1024-bit will be padded. The padding, the internal structure of a seed and the way a seed is converted to a state are all opaque (unspecified). The only semantics specification of seeds is that two different seeds are likely to produce two independent generators (but no guarantee).

from_state

View source

@classmethod
from_state(
    state, alg
)

Creates a generator from a state.

See __init__ for description of state and alg.

make_seeds

View source

make_seeds(
    count=1
)

Generates seeds for stateless random ops.

For example:

seeds = get_global_generator().make_seeds(count=10)
for i in range(10):
  seed = seeds[:, i]
  numbers = stateless_random_normal(shape=[2, 3], seed=seed)
  ...

normal

View source

normal(
    shape,
    mean=0.0,
    stddev=1.0,
    dtype=tf.dtypes.float32,
    name=None
)

Outputs random values from a normal distribution.

reset

View source

reset(
    state
)

Resets the generator by a new state.

See __init__ for the meaning of "state".

reset_from_key_counter

View source

reset_from_key_counter(
    key, counter
)

Resets the generator by a new key-counter pair.

See from_key_counter for the meaning of "key" and "counter".

reset_from_seed

View source

reset_from_seed(
    seed
)

Resets the generator by a new seed.

See from_seed for the meaning of "seed".

skip

View source

skip(
    delta
)

Advance the counter of a counter-based RNG.

split

View source

split(
    count=1
)

Returns a list of independent Generator objects.

Two generators are independent of each other in the sense that the random-number streams they generate don't have statistically detectable correlations. The new generators are also independent of the old one. The old generator's state will be changed (like other random-number generating methods), so two calls of split will return different new generators.

For example:

gens = get_global_generator().split(count=10)
for gen in gens:
  numbers = gen.normal(shape=[2, 3])
  # ...
gens2 = get_global_generator().split(count=10)
# gens2 will be different from gens

The new generators will be put on the current device (possible different from the old generator's), for example:

with tf.device("/device:CPU:0"):
  gen = Generator(seed=1234)  # gen is on CPU
with tf.device("/device:GPU:0"):
  gens = gen.split(count=10)  # gens are on GPU

truncated_normal

View source

truncated_normal(
    shape,
    mean=0.0,
    stddev=1.0,
    dtype=tf.dtypes.float32,
    name=None
)

Outputs random values from a truncated normal distribution.

The generated values follow a normal distribution with specified mean and standard deviation, except that values whose magnitude is more than 2 standard deviations from the mean are dropped and re-picked.

uniform

View source

uniform(
    shape,
    minval=0,
    maxval=None,
    dtype=tf.dtypes.float32,
    name=None
)

Outputs random values from a uniform distribution.

The generated values follow a uniform distribution in the range [minval, maxval). The lower bound minval is included in the range, while the upper bound maxval is excluded. (For float numbers especially low-precision types like bfloat16, because of rounding, the result may sometimes include maxval.)

For floats, the default range is [0, 1). For ints, at least maxval must be specified explicitly.

In the integer case, the random integers are slightly biased unless maxval - minval is an exact power of two. The bias is small for values of maxval - minval significantly smaller than the range of the output (either 2**32 or 2**64).

For full-range random integers, pass minval=None and maxval=None with an integer dtype (for integer dtypes, minval and maxval must be both None or both not None).

uniform_full_int

View source

uniform_full_int(
    shape,
    dtype=tf.dtypes.uint64,
    name=None
)

Uniform distribution on an integer type's entire range.

This method is the same as setting minval and maxval to None in the uniform method.