View source on GitHub |
Initializer capable of adapting its scale to the shape of weights tensors.
Inherits From: Initializer
tf.keras.initializers.VarianceScaling(
scale=1.0, mode='fan_in', distribution='truncated_normal', seed=None
)
Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized.
With distribution="truncated_normal" or "untruncated_normal"
, samples are
drawn from a truncated/untruncated normal distribution with a mean of zero and
a standard deviation (after truncation, if used) stddev = sqrt(scale / n)
where n is:
- number of input units in the weight tensor, if mode = "fan_in"
- number of output units, if mode = "fan_out"
- average of the numbers of input and output units, if mode = "fan_avg"
With distribution="uniform"
, samples are drawn from a uniform distribution
within [-limit, limit], with limit = sqrt(3 * scale / n)
.
Examples:
def make_variables(k, initializer):
return (tf.Variable(initializer(shape=[k], dtype=tf.float32)),
tf.Variable(initializer(shape=[k, k], dtype=tf.float32)))
v1, v2 = make_variables(3, tf.initializers.VarianceScaling(scale=1.))
v1
<tf.Variable ... shape=(3,) ... numpy=array([...], dtype=float32)>
v2
<tf.Variable ... shape=(3, 3) ... numpy=
make_variables(4, tf.initializers.VarianceScaling(distribution='uniform'))
(<tf.Variable...shape=(4,) dtype=float32...>, <tf.Variable...shape=(4, 4) ...
Args | |
---|---|
scale
|
Scaling factor (positive float). |
mode
|
One of "fan_in", "fan_out", "fan_avg". |
distribution
|
Random distribution to use. One of "truncated_normal", "untruncated_normal" and "uniform". |
seed
|
A Python integer. Used to create random seeds. See
tf.random.set_seed for behavior.
|
Raises | |
---|---|
ValueError
|
In case of an invalid value for the "scale", mode" or "distribution" arguments. |
Methods
from_config
@classmethod
from_config( config )
Instantiates an initializer from a configuration dictionary.
Example:
initializer = RandomUniform(-1, 1)
config = initializer.get_config()
initializer = RandomUniform.from_config(config)
Args | |
---|---|
config
|
A Python dictionary.
It will typically be the output of get_config .
|
Returns | |
---|---|
An Initializer instance. |
get_config
get_config()
Returns the configuration of the initializer as a JSON-serializable dict.
Returns | |
---|---|
A JSON-serializable Python dict. |
__call__
__call__(
shape, dtype=tf.dtypes.float32
)
Returns a tensor object initialized as specified by the initializer.
Args | |
---|---|
shape
|
Shape of the tensor. |
dtype
|
Optional dtype of the tensor. Only floating point types are supported. |
Raises | |
---|---|
ValueError
|
If the dtype is not floating point |