View source on GitHub

Gradient Boosted Trees learning algorithm.

Inherits From: CoreModel, InferenceCoreModel

tfdf.keras.wrappers.GradientBoostedTreesModel(
    task: Optional[TaskType] = core.Task.CLASSIFICATION,
    features: Optional[List[core.FeatureUsage]] = None,
    exclude_non_specified_features: Optional[bool] = False,
    preprocessing: Optional['tf_keras.models.Functional'] = None,
    postprocessing: Optional['tf_keras.models.Functional'] = None,
    training_preprocessing: Optional['tf_keras.models.Functional'] = None,
    ranking_group: Optional[str] = None,
    uplift_treatment: Optional[str] = None,
    temp_directory: Optional[str] = None,
    verbose: int = 1,
    hyperparameter_template: Optional[str] = None,
    advanced_arguments: Optional[tfdf.keras.AdvancedArguments] = None,
    num_threads: Optional[int] = None,
    name: Optional[str] = None,
    max_vocab_count: Optional[int] = 2000,
    try_resume_training: Optional[bool] = True,
    check_dataset: Optional[bool] = True,
    tuner: Optional[tfdf.tuner.Tuner] = None,
    discretize_numerical_features: bool = False,
    num_discretized_numerical_bins: int = 255,
    multitask: Optional[List[MultiTaskItem]] = None,
    adapt_subsample_for_maximum_training_duration: Optional[bool] = False,
    allow_na_conditions: Optional[bool] = False,
    apply_link_function: Optional[bool] = True,
    categorical_algorithm: Optional[str] = 'CART',
    categorical_set_split_greedy_sampling: Optional[float] = 0.1,
    categorical_set_split_max_num_items: Optional[int] = -1,
    categorical_set_split_min_item_frequency: Optional[int] = 1,
    compute_permutation_variable_importance: Optional[bool] = False,
    cross_entropy_ndcg_truncation: Optional[int] = 5,
    dart_dropout: Optional[float] = None,
    early_stopping: Optional[str] = 'LOSS_INCREASE',
    early_stopping_initial_iteration: Optional[int] = 10,
    early_stopping_num_trees_look_ahead: Optional[int] = 30,
    focal_loss_alpha: Optional[float] = 0.5,
    focal_loss_gamma: Optional[float] = 2.0,
    forest_extraction: Optional[str] = 'MART',
    goss_alpha: Optional[float] = 0.2,
    goss_beta: Optional[float] = 0.1,
    growing_strategy: Optional[str] = 'LOCAL',
    honest: Optional[bool] = False,
    honest_fixed_separation: Optional[bool] = False,
    honest_ratio_leaf_examples: Optional[float] = 0.5,
    in_split_min_examples_check: Optional[bool] = True,
    keep_non_leaf_label_distribution: Optional[bool] = True,
    l1_regularization: Optional[float] = 0.0,
    l2_categorical_regularization: Optional[float] = 1.0,
    l2_regularization: Optional[float] = 0.0,
    lambda_loss: Optional[float] = 1.0,
    loss: Optional[str] = 'DEFAULT',
    max_depth: Optional[int] = 6,
    max_num_nodes: Optional[int] = None,
    maximum_model_size_in_memory_in_bytes: Optional[float] = -1.0,
    maximum_training_duration_seconds: Optional[float] = -1.0,
    mhld_oblique_max_num_attributes: Optional[int] = None,
    mhld_oblique_sample_attributes: Optional[bool] = None,
    min_examples: Optional[int] = 5,
    missing_value_policy: Optional[str] = 'GLOBAL_IMPUTATION',
    ndcg_truncation: Optional[int] = 5,
    num_candidate_attributes: Optional[int] = -1,
    num_candidate_attributes_ratio: Optional[float] = -1.0,
    num_trees: Optional[int] = 300,
    pure_serving_model: Optional[bool] = False,
    random_seed: Optional[int] = 123456,
    sampling_method: Optional[str] = 'RANDOM',
    selective_gradient_boosting_ratio: Optional[float] = 0.01,
    shrinkage: Optional[float] = 0.1,
    sorting_strategy: Optional[str] = 'PRESORT',
    sparse_oblique_max_num_projections: Optional[int] = None,
    sparse_oblique_normalization: Optional[str] = None,
    sparse_oblique_num_projections_exponent: Optional[float] = None,
    sparse_oblique_projection_density_factor: Optional[float] = None,
    sparse_oblique_weights: Optional[str] = None,
    split_axis: Optional[str] = 'AXIS_ALIGNED',
    subsample: Optional[float] = 1.0,
    uplift_min_examples_in_treatment: Optional[int] = 5,
    uplift_split_score: Optional[str] = 'KULLBACK_LEIBLER',
    use_hessian_gain: Optional[bool] = False,
    validation_interval_in_trees: Optional[int] = 1,
    validation_ratio: Optional[float] = 0.1,
    explicit_args: Optional[Set[str]] = None
)

Used in the notebooks

A Gradient Boosted Trees (GBT), also known as Gradient Boosted Decision Trees (GBDT) or Gradient Boosted Machines (GBM), is a set of shallow decision trees trained sequentially. Each tree is trained to predict and then "correct" for the errors of the previously trained trees (more precisely each tree predict the gradient of the loss relative to the model output).

Usage example:

import tensorflow_decision_forests as tfdf
import pandas as pd

dataset = pd.read_csv("project/dataset.csv")
tf_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(dataset, label="my_label")

model = tfdf.keras.GradientBoostedTreesModel()
model.fit(tf_dataset)

print(model.summary())

Hyper-parameter tuning:

import tensorflow_decision_forests as tfdf
import pandas as pd

dataset = pd.read_csv("project/dataset.csv")
tf_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(dataset, label="my_label")

tuner = tfdf.tuner.RandomSearch(num_trials=20)

# Hyper-parameters to optimize.
tuner.discret("max_depth", [4, 5, 6, 7])

model = tfdf.keras.GradientBoostedTreesModel(tuner=tuner)
model.fit(tf_dataset)

print(model.summary())

self.input_spec = tf.keras.layers.InputSpec(ndim=4)

ValueError: Input 0 of layer conv2d is incompatible with the layer:
expected ndim=4, found ndim=1. Full shape received: [2]

class MyLayer(tf.keras.layers.Layer):
  def call(self, inputs):
    self.add_loss(tf.abs(tf.reduce_mean(inputs)))
    return inputs
l = MyLayer()
l(np.ones((10, 1)))
l.losses
[1.0]

inputs = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(10)(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Activity regularization.
len(model.losses)
0
model.add_loss(tf.abs(tf.reduce_mean(x)))
len(model.losses)
1

inputs = tf.keras.Input(shape=(10,))
d = tf.keras.layers.Dense(10, kernel_initializer='ones')
x = d(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Weight regularization.
model.add_loss(lambda: tf.reduce_mean(d.kernel))
model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

inputs = tf.keras.layers.Input(shape=(3,))
outputs = tf.keras.layers.Dense(2)(inputs)
model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer="Adam", loss="mse", metrics=["mae"])
[m.name for m in model.metrics]
[]

x = np.random.random((2, 3))
y = np.random.randint(0, 2, (2, 2))
model.fit(x, y)
[m.name for m in model.metrics]
['loss', 'mae']

inputs = tf.keras.layers.Input(shape=(3,))
d = tf.keras.layers.Dense(2, name='out')
output_1 = d(inputs)
output_2 = d(inputs)
model = tf.keras.models.Model(
   inputs=inputs, outputs=[output_1, output_2])
model.add_metric(
   tf.reduce_sum(output_2), name='mean', aggregation='mean')
model.compile(optimizer="Adam", loss="mse", metrics=["mae", "acc"])
model.fit(x, (y, y))
[m.name for m in model.metrics]
['loss', 'out_loss', 'out_1_loss', 'out_mae', 'out_acc', 'out_1_mae',
'out_1_acc', 'mean']

inputs = tf.keras.layers.Input(shape=(3,))
outputs = tf.keras.layers.Dense(2)(inputs)
model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer="Adam", loss="mse", metrics=["mae"])
model.metrics_names
[]

x = np.random.random((2, 3))
y = np.random.randint(0, 2, (2, 2))
model.fit(x, y)
model.metrics_names
['loss', 'mae']

inputs = tf.keras.layers.Input(shape=(3,))
d = tf.keras.layers.Dense(2, name='out')
output_1 = d(inputs)
output_2 = d(inputs)
model = tf.keras.models.Model(
   inputs=inputs, outputs=[output_1, output_2])
model.compile(optimizer="Adam", loss="mse", metrics=["mae", "acc"])
model.fit(x, (y, y))
model.metrics_names
['loss', 'out_loss', 'out_1_loss', 'out_mae', 'out_acc', 'out_1_mae',
'out_1_acc']

a = tf.Module()
b = tf.Module()
c = tf.Module()
a.b = b
b.c = c
list(a.submodules) == [b, c]
True
list(b.submodules) == [c]
True
list(c.submodules) == []
True

Methods

add_loss

add_loss(
    losses, **kwargs
)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies.

This method can be used inside a subclassed layer or model's call function, in which case losses should be a Tensor or list of Tensors.

Example:

class MyLayer(tf.keras.layers.Layer):
  def call(self, inputs):
    self.add_loss(tf.abs(tf.reduce_mean(inputs)))
    return inputs

The same code works in distributed training: the input to add_loss() is treated like a regularization loss and averaged across replicas by the training loop (both built-in Model.fit() and compliant custom training loops).

The add_loss method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model's Inputs. These losses become part of the model's topology and are tracked in get_config.

Example:

inputs = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(10)(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Activity regularization.
model.add_loss(tf.abs(tf.reduce_mean(x)))

If this is not the case for your loss (if, for example, your loss references a Variable of one of the model's layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model's topology since they can't be serialized.

Example:

inputs = tf.keras.Input(shape=(10,))
d = tf.keras.layers.Dense(10)
x = d(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Weight regularization.
model.add_loss(lambda: tf.reduce_mean(d.kernel))

build

build(
    input_shape
)

Builds the model based on input shapes received.

This is to be used for subclassed models, which do not know at instantiation time what their inputs look like.

This method only exists for users who want to call model.build() in a standalone way (as a substitute for calling the model on real data to build it). It will never be called by the framework (and thus it will never throw unexpected errors in an unrelated workflow).

build_from_config

build_from_config(
    config
)

Builds the layer's states with the supplied config dict.

By default, this method calls the build(config["input_shape"]) method, which creates weights based on the layer's input shape in the supplied config. If your config contains other information needed to load the layer's state, you should override this method.

call

View source

call(
    inputs, training=False
)

Inference of the model.

This method is used for prediction and evaluation of a trained model.

call_get_leaves

View source

call_get_leaves(
    inputs
)

Computes the index of the active leaf in each tree.

The active leaf is the leave that that receive the example during inference.

The returned value "leaves[i,j]" is the index of the active leave for the i-th example and the j-th tree. Leaves are indexed by depth first exploration with the negative child visited before the positive one (similarly as "iterate_on_nodes()" iteration). Leaf indices are also available with LeafNode.leaf_idx.

capabilities

View source

@staticmethod
capabilities() -> abstract_learner_pb2.LearnerCapabilities

Lists the capabilities of the learning algorithm.

collect_data_step

View source

collect_data_step(
    data, is_training_example
)

Collect examples e.g. training or validation.

compile

View source

compile(
    metrics=None, weighted_metrics=None, **kwargs
)

Configure the model for training.

Unlike for most Keras model, calling "compile" is optional before calling "fit".

compile_from_config

compile_from_config(
    config
)

Compiles the model with the information given in config.

This method uses the information in the config (optimizer, loss, metrics, etc.) to compile the model.

compute_loss

compute_loss(
    x=None, y=None, y_pred=None, sample_weight=None
)

Compute the total loss, validate it, and return it.

Subclasses can optionally override this method to provide custom loss computation logic.

Example:

class MyModel(tf.keras.Model):

  def __init__(self, *args, **kwargs):
    super(MyModel, self).__init__(*args, **kwargs)
    self.loss_tracker = tf.keras.metrics.Mean(name='loss')

  def compute_loss(self, x, y, y_pred, sample_weight):
    loss = tf.reduce_mean(tf.math.squared_difference(y_pred, y))
    loss += tf.add_n(self.losses)
    self.loss_tracker.update_state(loss)
    return loss

  def reset_metrics(self):
    self.loss_tracker.reset_states()

  @property
  def metrics(self):
    return [self.loss_tracker]

tensors = tf.random.uniform((10, 10)), tf.random.uniform((10,))
dataset = tf.data.Dataset.from_tensor_slices(tensors).repeat().batch(1)

inputs = tf.keras.layers.Input(shape=(10,), name='my_input')
outputs = tf.keras.layers.Dense(10)(inputs)
model = MyModel(inputs, outputs)
model.add_loss(tf.reduce_sum(outputs))

optimizer = tf.keras.optimizers.SGD()
model.compile(optimizer, loss='mse', steps_per_execution=10)
model.fit(dataset, epochs=2, steps_per_epoch=10)
print('My custom loss: ', model.loss_tracker.result().numpy())

compute_mask

compute_mask(
    inputs, mask=None
)

Computes an output mask tensor.

compute_metrics

compute_metrics(
    x, y, y_pred, sample_weight
)

Update metric states and collect all metrics to be returned.

Subclasses can optionally override this method to provide custom metric updating and collection logic.

Example:

class MyModel(tf.keras.Sequential):

  def compute_metrics(self, x, y, y_pred, sample_weight):

    # This super call updates `self.compiled_metrics` and returns
    # results for all metrics listed in `self.metrics`.
    metric_results = super(MyModel, self).compute_metrics(
        x, y, y_pred, sample_weight)

    # Note that `self.custom_metric` is not listed in `self.metrics`.
    self.custom_metric.update_state(x, y, y_pred, sample_weight)
    metric_results['custom_metric_name'] = self.custom_metric.result()
    return metric_results

compute_output_shape

compute_output_shape(
    input_shape
)

Computes the output shape of the layer.

This method will cause the layer's state to be built, if that has not happened before. This requires that the layer will later be used with inputs that match the input shape provided here.

count_params

count_params()

Count the total number of scalars composing the weights.

evaluate

evaluate(
    x=None,
    y=None,
    batch_size=None,
    verbose='auto',
    sample_weight=None,
    steps=None,
    callbacks=None,
    max_queue_size=10,
    workers=1,
    use_multiprocessing=False,
    return_dict=False,
    **kwargs
)

Returns the loss value & metrics values for the model in test mode.

Computation is done in batches (see the batch_size arg.)

See the discussion of Unpacking behavior for iterator-like inputs for Model.fit.

export

export(
    filepath
)

Create a SavedModel artifact for inference (e.g. via TF-Serving).

This method lets you export a model to a lightweight SavedModel artifact that contains the model's forward pass only (its call() method) and can be served via e.g. TF-Serving. The forward pass is registered under the name serve() (see example below).

The original code of the model (including any custom layers you may have used) is no longer necessary to reload the artifact -- it is entirely standalone.

Example:

# Create the artifact
model.export("path/to/location")

# Later, in a different process / environment...
reloaded_artifact = tf.saved_model.load("path/to/location")
predictions = reloaded_artifact.serve(input_data)

If you would like to customize your serving endpoints, you can use the lower-level keras.export.ExportArchive class. The export() method relies on ExportArchive internally.

fit

View source

fit(
    x=None,
    y=None,
    callbacks=None,
    verbose: Optional[Any] = None,
    validation_steps: Optional[int] = None,
    validation_data: Optional[Any] = None,
    sample_weight: Optional[Any] = None,
    steps_per_epoch: Optional[Any] = None,
    class_weight: Optional[Any] = None,
    **kwargs
) -> tf_keras.callbacks.History

Trains the model.

Local training

It is recommended to use a Pandas Dataframe dataset and to convert it to a TensorFlow dataset with pd_dataframe_to_tf_dataset():

  pd_dataset = pandas.Dataframe(...)
  tf_dataset = pd_dataframe_to_tf_dataset(dataset, label="my_label")
  model.fit(pd_dataset)

The following dataset formats are supported:

1. "x" is a tf.data.Dataset containing a tuple "(features, labels)". "features" can be a dictionary a tensor, a list of tensors or a dictionary of tensors (recommended). "labels" is a tensor.

2. "x" is a tensor, list of tensors or dictionary of tensors containing the input features. "y" is a tensor.

3. "x" is a numpy-array, list of numpy-arrays or dictionary of numpy-arrays containing the input features. "y" is a numpy-array.

1. The dataset need to be read exactly once. If you use a TensorFlow dataset, make sure NOT to add a "repeat" operation.
2. The algorithm does not benefit from shuffling the dataset. If you use a TensorFlow dataset, make sure NOT to add a "shuffle" operation.
3. The dataset needs to be batched (i.e. with a "batch" operation). However, the number of elements per batch has not impact on the model. Generally, it is recommended to use batches as large as possible as its speeds-up reading the dataset in TensorFlow.

Input features do not need to be normalized (e.g. dividing numerical values by the variance) or indexed (e.g. replacing categorical string values by an integer). Additionally, missing values can be consumed natively.

Distributed training

Some of the learning algorithms will support distributed training with the ParameterServerStrategy.

In this case, the dataset is read asynchronously in between the workers. The distribution of the training depends on the learning algorithm.

Like for non-distributed training, the dataset should be read exactly once. The simplest solution is to divide the dataset into different files (i.e. shards) and have each of the worker read a non overlapping subset of shards.

Currently (to be changed), the validation dataset (if provided) is simply feed to the model.evaluate() method. Therefore, it should satisfy Keras' evaluate API. Notably, for distributed training, the validation dataset should be infinite (i.e. have a repeat operation).

See https://www.tensorflow.org/decision_forests/distributed_training for more details and examples.

Here is a single example of distributed training using PSS for both dataset reading and training distribution.

  def dataset_fn(context, paths, training=True):
    ds_path = tf.data.Dataset.from_tensor_slices(paths)


    if context is not None:
      # Train on at least 2 workers.
      current_worker = tfdf.keras.get_worker_idx_and_num_workers(context)
      assert current_worker.num_workers > 2

      # Split the dataset's examples among the workers.
      ds_path = ds_path.shard(
          num_shards=current_worker.num_workers,
          index=current_worker.worker_idx)

    def read_csv_file(path):
      numerical = tf.constant([math.nan], dtype=tf.float32)
      categorical_string = tf.constant([""], dtype=tf.string)
      csv_columns = [
          numerical,  # age
          categorical_string,  # workclass
          numerical,  # fnlwgt
          ...
      ]
      column_names = [
        "age", "workclass", "fnlwgt", ...
      ]
      label_name = "label"
      return tf.data.experimental.CsvDataset(path, csv_columns, header=True)

    ds_columns = ds_path.interleave(read_csv_file)

    def map_features(*columns):
      assert len(column_names) == len(columns)
      features = {column_names[i]: col for i, col in enumerate(columns)}
      label = label_table.lookup(features.pop(label_name))
      return features, label

    ds_dataset = ds_columns.map(map_features)
    if not training:
      dataset = dataset.repeat(None)
    ds_dataset = ds_dataset.batch(batch_size)
    return ds_dataset

  strategy = tf.distribute.experimental.ParameterServerStrategy(...)
  sharded_train_paths = [list of dataset files]
  with strategy.scope():
    model = DistributedGradientBoostedTreesModel()
    train_dataset = strategy.distribute_datasets_from_function(
      lambda context: dataset_fn(context, sharded_train_paths))

    test_dataset = strategy.distribute_datasets_from_function(
      lambda context: dataset_fn(context, sharded_test_paths))

  model.fit(sharded_train_paths)
  evaluation = model.evaluate(test_dataset, steps=num_test_examples //
    batch_size)

fit_on_dataset_path

View source

fit_on_dataset_path(
    train_path: str,
    label_key: Optional[str] = None,
    weight_key: Optional[str] = None,
    valid_path: Optional[str] = None,
    dataset_format: Optional[str] = 'csv',
    max_num_scanned_rows_to_accumulate_statistics: Optional[int] = 100000,
    try_resume_training: Optional[bool] = True,
    input_model_signature_fn: Optional[tf_core.InputModelSignatureFn] = tfdf.keras.build_default_input_model_signature,
    num_io_threads: int = 10
)

Trains the model on a dataset stored on disk.

This solution is generally more efficient and easier than loading the dataset with a tf.Dataset both for local and distributed training.

model = keras.GradientBoostedTreesModel()
model.fit_on_dataset_path(
  train_path="/path/to/dataset.csv",
  label_key="label",
  dataset_format="csv")
model.save("/model/path")

with tf.distribute.experimental.ParameterServerStrategy(...).scope():
  model = model = keras.DistributedGradientBoostedTreesModel()
model.fit_on_dataset_path(
  train_path="/path/to/dataset@10",
  label_key="label",
  dataset_format="tfrecord+tfe")
model.save("/model/path")

from_config

@classmethod
from_config(
    config, custom_objects=None
)

Creates a layer from its config.

This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

get_build_config

get_build_config()

Returns a dictionary with the layer's input shape.

This method returns a config dict that can be used by build_from_config(config) to create all states (e.g. Variables and Lookup tables) needed by the layer.

By default, the config only contains the input shape that the layer was built with. If you're writing a custom layer that creates state in an unusual way, you should override this method to make sure this state is already created when TF-Keras attempts to load its value upon model loading.

get_compile_config

get_compile_config()

Returns a serialized config with information for compiling the model.

This method returns a config dictionary containing all the information (optimizer, loss, metrics, etc.) with which the model was compiled.

get_config

View source

get_config()

Not supported by TF-DF, returning empty directory to avoid warnings.

get_layer

get_layer(
    name=None, index=None
)

Retrieves a layer based on either its name (unique) or index.

If name and index are both provided, index will take precedence. Indices are based on order of horizontal graph traversal (bottom-up).

get_metrics_result

get_metrics_result()

Returns the model's metrics values as a dict.

If any of the metric result is a dict (containing multiple metrics), each of them gets added to the top level returned dict of this method.

get_weight_paths

get_weight_paths()

Retrieve all the variables and their paths for the model.

The variable path (string) is a stable key to identify a tf.Variable instance owned by the model. It can be used to specify variable-specific configurations (e.g. DTensor, quantization) from a global view.

This method returns a dict with weight object paths as keys and the corresponding tf.Variable instances as values.

Note that if the model is a subclassed model and the weights haven't been initialized, an empty dict will be returned.

Example:

class SubclassModel(tf.keras.Model):

  def __init__(self, name=None):
    super().__init__(name=name)
    self.d1 = tf.keras.layers.Dense(10)
    self.d2 = tf.keras.layers.Dense(20)

  def call(self, inputs):
    x = self.d1(inputs)
    return self.d2(x)

model = SubclassModel()
model(tf.zeros((10, 10)))
weight_paths = model.get_weight_paths()
# weight_paths:
# {
#    'd1.kernel': model.d1.kernel,
#    'd1.bias': model.d1.bias,
#    'd2.kernel': model.d2.kernel,
#    'd2.bias': model.d2.bias,
# }

# Functional model
inputs = tf.keras.Input((10,), batch_size=10)
x = tf.keras.layers.Dense(20, name='d1')(inputs)
output = tf.keras.layers.Dense(30, name='d2')(x)
model = tf.keras.Model(inputs, output)
d1 = model.layers[1]
d2 = model.layers[2]
weight_paths = model.get_weight_paths()
# weight_paths:
# {
#    'd1.kernel': d1.kernel,
#    'd1.bias': d1.bias,
#    'd2.kernel': d2.kernel,
#    'd2.bias': d2.bias,
# }

get_weights

get_weights()

Retrieves the weights of the model.

load_own_variables

load_own_variables(
    store
)

Loads the state of the layer.

You can override this method to take full control of how the state of the layer is loaded upon calling keras.models.load_model().

load_weights

View source

load_weights(
    *args, **kwargs
)

No-op for TensorFlow Decision Forests models.

load_weights is not supported by TensorFlow Decision Forests models. To save and restore a model, use the SavedModel API i.e. model.save(...) and tf_keras.models.load_model(...). To resume the training of an existing model, create the model with try_resume_training=True (default value) and with a similar temp_directory argument. See documentation of try_resume_training for more details.

make_inspector

View source

make_inspector(
    index: int = 0
) -> tfdf.inspector.AbstractInspector

Creates an inspector to access the internal model structure.

Usage example:

inspector = model.make_inspector()
print(inspector.num_trees())
print(inspector.variable_importances())

make_predict_function

View source

make_predict_function()

Prediction of the model (!= evaluation).

make_test_function

View source

make_test_function()

Predictions for evaluation.

make_train_function

make_train_function(
    force=False
)

Creates a function that executes one step of training.

This method can be overridden to support custom training logic. This method is called by Model.fit and Model.train_on_batch.

Typically, this method directly controls tf.function and tf.distribute.Strategy settings, and delegates the actual training logic to Model.train_step.

This function is cached the first time Model.fit or Model.train_on_batch is called. The cache is cleared whenever Model.compile is called. You can skip the cache and generate again the function with force=True.

predefined_hyperparameters

View source

@staticmethod
predefined_hyperparameters() -> List[tfdf.keras.core.HyperParameterTemplate]

Returns a better than default set of hyper-parameters.

They can be used directly with the hyperparameter_template argument of the model constructor.

These hyper-parameters outperform the default hyper-parameters (either generally or in specific scenarios). Like default hyper-parameters, existing pre-defined hyper-parameters cannot change.

predict

predict(
    x,
    batch_size=None,
    verbose='auto',
    steps=None,
    callbacks=None,
    max_queue_size=10,
    workers=1,
    use_multiprocessing=False
)

Generates output predictions for the input samples.

Computation is done in batches. This method is designed for batch processing of large numbers of inputs. It is not intended for use inside of loops that iterate over your data and process small numbers of inputs at a time.

For small numbers of inputs that fit in one batch, directly use __call__() for faster execution, e.g., model(x), or model(x, training=False) if you have layers such as tf.keras.layers.BatchNormalization that behave differently during inference. You may pair the individual model call with a tf.function for additional performance inside your inner loop. If you need access to numpy array values instead of tensors after your model call, you can use tensor.numpy() to get the numpy array value of an eager tensor.

Also, note the fact that test loss is not affected by regularization layers like noise and dropout.

See the discussion of Unpacking behavior for iterator-like inputs for Model.fit. Note that Model.predict uses the same interpretation rules as Model.fit and Model.evaluate, so inputs must be unambiguous for all three methods.

predict_get_leaves

View source

predict_get_leaves(
    x
)

Gets the index of the active leaf of each tree.

The active leaf is the leave that that receive the example during inference.

The returned value "leaves[i,j]" is the index of the active leave for the i-th example and the j-th tree. Leaves are indexed by depth first exploration with the negative child visited before the positive one (similarly as "iterate_on_nodes()" iteration). Leaf indices are also available with LeafNode.leaf_idx.

predict_on_batch

predict_on_batch(
    x
)

Returns predictions for a single batch of samples.

predict_step

predict_step(
    data
)

The logic for one inference step.

This method can be overridden to support custom inference logic. This method is called by Model.make_predict_function.

This method should contain the mathematical logic for one step of inference. This typically includes the forward pass.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_predict_function, which can also be overridden.

ranking_group

View source

ranking_group() -> Optional[str]

reset_metrics

reset_metrics()

Resets the state of all the metrics in the model.

Examples:

inputs = tf.keras.layers.Input(shape=(3,))
outputs = tf.keras.layers.Dense(2)(inputs)
model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer="Adam", loss="mse", metrics=["mae"])

x = np.random.random((2, 3))
y = np.random.randint(0, 2, (2, 2))
_ = model.fit(x, y, verbose=0)
assert all(float(m.result()) for m in model.metrics)

model.reset_metrics()
assert all(float(m.result()) == 0 for m in model.metrics)

reset_states

reset_states()

save

View source

save(
    filepath: str, overwrite: Optional[bool] = True, **kwargs
)

Saves the model as a TensorFlow SavedModel.

The exported SavedModel contains a standalone Yggdrasil Decision Forests model in the "assets" sub-directory. The Yggdrasil model can be used directly using the Yggdrasil API. However, this model does not contain the "preprocessing" layer (if any).

save_own_variables

save_own_variables(
    store
)

Saves the state of the layer.

You can override this method to take full control of how the state of the layer is saved upon calling model.save().

save_spec

save_spec(
    dynamic_batch=True
)

Returns the tf.TensorSpec of call args as a tuple (args, kwargs).

This value is automatically defined after calling the model for the first time. Afterwards, you can use it when exporting the model for serving:

model = tf.keras.Model(...)

@tf.function
def serve(*args, **kwargs):
  outputs = model(*args, **kwargs)
  # Apply postprocessing steps, or add additional outputs.
  ...
  return outputs

# arg_specs is `[tf.TensorSpec(...), ...]`. kwarg_specs, in this
# example, is an empty dict since functional models do not use keyword
# arguments.
arg_specs, kwarg_specs = model.save_spec()

model.save(path, signatures={
  'serving_default': serve.get_concrete_function(*arg_specs,
                                                 **kwarg_specs)
})

save_weights

save_weights(
    filepath, overwrite=True, save_format=None, options=None
)

Saves all layer weights.

Either saves in HDF5 or in TensorFlow format based on the save_format argument.

When saving in HDF5 format, the weight file has:

• layer_names (attribute), a list of strings (ordered names of model layers).
• For every layer, a group named layer.name
  • For every such layer group, a group attribute weight_names, a list of strings (ordered names of weights tensor of the layer).
  • For every weight in the layer, a dataset storing the weight value, named after the weight tensor.

When saving in TensorFlow format, all objects referenced by the network are saved in the same format as tf.train.Checkpoint, including any Layer instances or Optimizer instances assigned to object attributes. For networks constructed from inputs and outputs using tf.keras.Model(inputs, outputs), Layer instances used by the network are tracked/saved automatically. For user-defined classes which inherit from tf.keras.Model, Layer instances must be assigned to object attributes, typically in the constructor. See the documentation of tf.train.Checkpoint and tf.keras.Model for details.

While the formats are the same, do not mix save_weights and tf.train.Checkpoint. Checkpoints saved by Model.save_weights should be loaded using Model.load_weights. Checkpoints saved using tf.train.Checkpoint.save should be restored using the corresponding tf.train.Checkpoint.restore. Prefer tf.train.Checkpoint over save_weights for training checkpoints.

The TensorFlow format matches objects and variables by starting at a root object, self for save_weights, and greedily matching attribute names. For Model.save this is the Model, and for Checkpoint.save this is the Checkpoint even if the Checkpoint has a model attached. This means saving a tf.keras.Model using save_weights and loading into a tf.train.Checkpoint with a Model attached (or vice versa) will not match the Model's variables. See the guide to training checkpoints for details on the TensorFlow format.

set_weights

set_weights(
    weights
)

Sets the weights of the layer, from NumPy arrays.

The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer's weights must be instantiated before calling this function, by calling the layer.

For example, a Dense layer returns a list of two values: the kernel matrix and the bias vector. These can be used to set the weights of another Dense layer:

layer_a = tf.keras.layers.Dense(1,
  kernel_initializer=tf.constant_initializer(1.))
a_out = layer_a(tf.convert_to_tensor([[1., 2., 3.]]))
layer_a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
layer_b = tf.keras.layers.Dense(1,
  kernel_initializer=tf.constant_initializer(2.))
b_out = layer_b(tf.convert_to_tensor([[10., 20., 30.]]))
layer_b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
layer_b.set_weights(layer_a.get_weights())
layer_b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

summary

View source

summary(
    line_length=None, positions=None, print_fn=None
)

Shows information about the model.

support_distributed_training

View source

support_distributed_training()

test_on_batch

test_on_batch(
    x, y=None, sample_weight=None, reset_metrics=True, return_dict=False
)

Test the model on a single batch of samples.

test_step

test_step(
    data
)

The logic for one evaluation step.

This method can be overridden to support custom evaluation logic. This method is called by Model.make_test_function.

This function should contain the mathematical logic for one step of evaluation. This typically includes the forward pass, loss calculation, and metrics updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_test_function, which can also be overridden.

to_json

to_json(
    **kwargs
)

Returns a JSON string containing the network configuration.

To load a network from a JSON save file, use keras.models.model_from_json(json_string, custom_objects={}).

to_yaml

to_yaml(
    **kwargs
)

Returns a yaml string containing the network configuration.

To load a network from a yaml save file, use keras.models.model_from_yaml(yaml_string, custom_objects={}).

custom_objects should be a dictionary mapping the names of custom losses / layers / etc to the corresponding functions / classes.

train_on_batch

View source

train_on_batch(
    *args, **kwargs
)

No supported for Tensorflow Decision Forests models.

Decision forests are not trained in batches the same way neural networks are. To avoid confusion, train_on_batch is disabled.

train_step

View source

train_step(
    data
)

Collects training examples.

uplift_treatment

View source

uplift_treatment() -> Optional[str]

valid_step

View source

valid_step(
    data
)

Collects validation examples.

with_name_scope

@classmethod
with_name_scope(
    method
)

Decorator to automatically enter the module name scope.

class MyModule(tf.Module):
  @tf.Module.with_name_scope
  def __call__(self, x):
    if not hasattr(self, 'w'):
      self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
    return tf.matmul(x, self.w)

Using the above module would produce tf.Variables and tf.Tensors whose names included the module name:

mod = MyModule()
mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>

yggdrasil_model_path_tensor

View source

yggdrasil_model_path_tensor(
    multitask_model_index: int = 0
) -> Optional[tf.Tensor]

Gets the path to yggdrasil model, if available.

The effective path can be obtained with:

yggdrasil_model_path_tensor().numpy().decode("utf-8")

yggdrasil_model_prefix

View source

yggdrasil_model_prefix(
    index: int = 0
) -> str

Gets the prefix of the internal yggdrasil model.

__call__

__call__(
    *args, **kwargs
)