tf.keras.layers.Layer

This is the class from which all layers inherit.

Inherits From: Module

tf.keras.layers.Layer(
    trainable=True, name=None, dtype=None, dynamic=False, **kwargs
)

A layer is a callable object that takes as input one or more tensors and that outputs one or more tensors. It involves computation, defined in the call() method, and a state (weight variables). State can be created in various places, at the convenience of the subclass implementer:

in __init__();
in the optional build() method, which is invoked by the first __call__() to the layer, and supplies the shape(s) of the input(s), which may not have been known at initialization time;
in the first invocation of call(), with some caveats discussed below.

Layers are recursively composable: If you assign a Layer instance as an attribute of another Layer, the outer layer will start tracking the weights created by the inner layer. Nested layers should be instantiated in the __init__() method.

Users will just instantiate a layer and then treat it as a callable.

Args
`trainable`	Boolean, whether the layer's variables should be trainable.
`name`	String name of the layer.
`dtype`	The dtype of the layer's computations and weights. Can also be a `tf.keras.mixed_precision.Policy`, which allows the computation and weight dtype to differ. Default of `None` means to use `tf.keras.mixed_precision.global_policy()`, which is a float32 policy unless set to different value.
`dynamic`	Set this to `True` if your layer should only be run eagerly, and should not be used to generate a static computation graph. This would be the case for a Tree-RNN or a recursive network, for example, or generally for any layer that manipulates tensors using Python control flow. If `False`, we assume that the layer can safely be used to generate a static computation graph.

We recommend that descendants of Layer implement the following methods:

__init__(): Defines custom layer attributes, and creates layer weights that do not depend on input shapes, using add_weight(), or other state.
build(self, input_shape): This method can be used to create weights that depend on the shape(s) of the input(s), using add_weight(), or other state. __call__() will automatically build the layer (if it has not been built yet) by calling build().
call(self, inputs, *args, **kwargs): Called in __call__ after making sure build() has been called. call() performs the logic of applying the layer to the inputs. The first invocation may additionally create state that could not be conveniently created in build(); see its docstring for details. Two reserved keyword arguments you can optionally use in call() are:
- training (boolean, whether the call is in inference mode or training mode). See more details in the layer/model subclassing guide
- mask (boolean tensor encoding masked timesteps in the input, used in RNN layers). See more details in the layer/model subclassing guide A typical signature for this method is call(self, inputs), and user could optionally add training and mask if the layer need them. *args and **kwargs is only useful for future extension when more input parameters are planned to be added.
get_config(self): Returns a dictionary containing the configuration used to initialize this layer. If the keys differ from the arguments in __init__, then override from_config(self) as well. This method is used when saving the layer or a model that contains this layer.

Examples:

Here's a basic example: a layer with two variables, w and b, that returns y = w . x + b. It shows how to implement build() and call(). Variables set as attributes of a layer are tracked as weights of the layers (in layer.weights).

class SimpleDense(Layer):

  def __init__(self, units=32):
      super(SimpleDense, self).__init__()
      self.units = units

  def build(self, input_shape):  # Create the state of the layer (weights)
    w_init = tf.random_normal_initializer()
    self.w = tf.Variable(
        initial_value=w_init(shape=(input_shape[-1], self.units),
                             dtype='float32'),
        trainable=True)
    b_init = tf.zeros_initializer()
    self.b = tf.Variable(
        initial_value=b_init(shape=(self.units,), dtype='float32'),
        trainable=True)

  def call(self, inputs):  # Defines the computation from inputs to outputs
      return tf.matmul(inputs, self.w) + self.b

# Instantiates the layer.
linear_layer = SimpleDense(4)

# This will also call `build(input_shape)` and create the weights.
y = linear_layer(tf.ones((2, 2)))
assert len(linear_layer.weights) == 2

# These weights are trainable, so they're listed in `trainable_weights`:
assert len(linear_layer.trainable_weights) == 2

Note that the method add_weight() offers a shortcut to create weights:

class SimpleDense(Layer):

  def __init__(self, units=32):
      super(SimpleDense, self).__init__()
      self.units = units

  def build(self, input_shape):
      self.w = self.add_weight(shape=(input_shape[-1], self.units),
                               initializer='random_normal',
                               trainable=True)
      self.b = self.add_weight(shape=(self.units,),
                               initializer='random_normal',
                               trainable=True)

  def call(self, inputs):
      return tf.matmul(inputs, self.w) + self.b

Besides trainable weights, updated via backpropagation during training, layers can also have non-trainable weights. These weights are meant to be updated manually during call(). Here's a example layer that computes the running sum of its inputs:

class ComputeSum(Layer):

  def __init__(self, input_dim):
      super(ComputeSum, self).__init__()
      # Create a non-trainable weight.
      self.total = tf.Variable(initial_value=tf.zeros((input_dim,)),
                               trainable=False)

  def call(self, inputs):
      self.total.assign_add(tf.reduce_sum(inputs, axis=0))
      return self.total

my_sum = ComputeSum(2)
x = tf.ones((2, 2))

y = my_sum(x)
print(y.numpy())  # [2. 2.]

y = my_sum(x)
print(y.numpy())  # [4. 4.]

assert my_sum.weights == [my_sum.total]
assert my_sum.non_trainable_weights == [my_sum.total]
assert my_sum.trainable_weights == []

For more information about creating layers, see the guide Making new Layers and Models via subclassing

Attributes
`name`	The name of the layer (string).
`dtype`	The dtype of the layer's weights.
`variable_dtype`	Alias of `dtype`.
`compute_dtype`	The dtype of the layer's computations. Layers automatically cast inputs to this dtype which causes the computations and output to also be in this dtype. When mixed precision is used with a `tf.keras.mixed_precision.Policy`, this will be different than `variable_dtype`.
`dtype_policy`	The layer's dtype policy. See the `tf.keras.mixed_precision.Policy` documentation for details.
`trainable_weights`	List of variables to be included in backprop.
`non_trainable_weights`	List of variables that should not be included in backprop.
`weights`	The concatenation of the lists trainable_weights and non_trainable_weights (in this order).
`trainable`	Whether the layer should be trained (boolean), i.e. whether its potentially-trainable weights should be returned as part of `layer.trainable_weights`.
`input_spec`	Optional (list of) `InputSpec` object(s) specifying the constraints on inputs that can be accepted by the layer.
`activity_regularizer`	Optional regularizer function for the output of this layer.
`dynamic`	Whether the layer is dynamic (eager-only); set in the constructor.
`input`	Retrieves the input tensor(s) of a layer. Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.
`losses`	List of losses added using the `add_loss()` API. Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing `losses` under a `tf.GradientTape` will propagate gradients back to the corresponding variables. `class MyLayer(tf.keras.layers.Layer):` `def call(self, inputs):` `self.add_loss(tf.abs(tf.reduce_mean(inputs)))` `return inputs` `l = MyLayer()` `l(np.ones((10, 1)))` `l.losses` `[1.0]` `inputs = tf.keras.Input(shape=(10,))` `x = tf.keras.layers.Dense(10)(inputs)` `outputs = tf.keras.layers.Dense(1)(x)` `model = tf.keras.Model(inputs, outputs)` `# Activity regularization.` `len(model.losses)` `0` `model.add_loss(tf.abs(tf.reduce_mean(x)))` `len(model.losses)` `1` `inputs = tf.keras.Input(shape=(10,))` `d = tf.keras.layers.Dense(10, kernel_initializer='ones')` `x = d(inputs)` `outputs = tf.keras.layers.Dense(1)(x)` `model = tf.keras.Model(inputs, outputs)` `# Weight regularization.` `model.add_loss(lambda: tf.reduce_mean(d.kernel))` `model.losses` `[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]`
`metrics`	List of metrics attached to the layer.
`output`	Retrieves the output tensor(s) of a layer. Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.
`supports_masking`	Whether this layer supports computing a mask using `compute_mask`.

Args
`losses`	Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.
`**kwargs`	Used for backwards compatibility only.

Args
`name`	Variable name.
`shape`	Variable shape. Defaults to scalar if unspecified.
`dtype`	The type of the variable. Defaults to `self.dtype`.
`initializer`	Initializer instance (callable).
`regularizer`	Regularizer instance (callable).
`trainable`	Boolean, whether the variable should be part of the layer's "trainable_variables" (e.g. variables, biases) or "non_trainable_variables" (e.g. BatchNorm mean and variance). Note that `trainable` cannot be `True` if `synchronization` is set to `ON_READ`.
`constraint`	Constraint instance (callable).
`use_resource`	Whether to use a `ResourceVariable` or not. See this guide for more information.
`synchronization`	Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. If `synchronization` is set to `ON_READ`, `trainable` must not be set to `True`.
`aggregation`	Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`.
`**kwargs`	Additional keyword arguments. Accepted values are `getter`, `collections`, `experimental_autocast` and `caching_device`.

Args
`inputs`	Tensor or list of tensors.
`mask`	Tensor or list of tensors.

Args
`*args`	Positional arguments to be passed to `self.call`.
`**kwargs`	Keyword arguments to be passed to `self.call`.

Raises
`ValueError`	if the layer's `call` method returns None (an invalid value).
`RuntimeError`	if `super().__init__()` was not called in the constructor.

tf.keras.layers.Layer

Args

Examples:

Attributes

Methods

add_loss

Example:

Example:

Example:

add_weight

build

build_from_config

call

compute_mask

compute_output_shape

compute_output_signature

count_params

from_config

get_build_config

get_config

get_weights

load_own_variables

save_own_variables

set_weights

__call__

`add_loss`

`add_weight`

`build`

`build_from_config`

`call`

`compute_mask`

`compute_output_shape`

`compute_output_signature`

`count_params`

`from_config`

`get_build_config`

`get_config`

`get_weights`

`load_own_variables`

`save_own_variables`

`set_weights`

`call`