tf.contrib.layers.variance_scaling_initializer

Returns an initializer that generates tensors without scaling variance.

tf.contrib.layers.variance_scaling_initializer(
    factor=2.0, mode='FAN_IN', uniform=False, seed=None, dtype=tf.dtypes.float32
)

When initializing a deep network, it is in principle advantageous to keep the scale of the input variance constant, so it does not explode or diminish by reaching the final layer. This initializer use the following formula:

  if mode='FAN_IN': # Count only number of input connections.
    n = fan_in
  elif mode='FAN_OUT': # Count only number of output connections.
    n = fan_out
  elif mode='FAN_AVG': # Average number of inputs and output connections.
    n = (fan_in + fan_out)/2.0

    truncated_normal(shape, 0.0, stddev=sqrt(factor / n))

To get Delving Deep into Rectifiers (also know as the "MSRA initialization"), use (Default):
factor=2.0 mode='FAN_IN' uniform=False
To get Convolutional Architecture for Fast Feature Embedding, use:
factor=1.0 mode='FAN_IN' uniform=True
To get Understanding the difficulty of training deep feedforward neural networks, use:
factor=1.0 mode='FAN_AVG' uniform=True.
To get xavier_initializer use either:
factor=1.0 mode='FAN_AVG' uniform=True, or
factor=1.0 mode='FAN_AVG' uniform=False.

Args
`factor`	Float. A multiplicative factor.
`mode`	String. 'FAN_IN', 'FAN_OUT', 'FAN_AVG'.
`uniform`	Whether to use uniform or normal distributed random initialization.
`seed`	A Python integer. Used to create random seeds. See `tf.compat.v1.set_random_seed` for behavior.
`dtype`	The data type. Only floating point types are supported.

Returns
An initializer that generates tensors with unit variance.

Raises
`ValueError`	if `dtype` is not a floating point type.
`TypeError`	if `mode` is not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG'].

tf.contrib.layers.variance_scaling_initializer

Args

Returns

Raises