דוגמאות להגירה: אומדנים משומרים

הצג באתר TensorFlow.org הפעל בגוגל קולאב צפה במקור ב-GitHub הורד מחברת

אומדנים משומרים (או מוכנים מראש) שימשו באופן מסורתי ב-TensorFlow 1 כדרכים מהירות וקלות לאמן דגמים למגוון מקרי שימוש טיפוסיים. TensorFlow 2 מספק תחליפים משוערים ופשוטים למספר מהם באמצעות דגמי Keras. עבור אותם אומדנים משומרים שאין להם תחליפים מובנים של TensorFlow 2, אתה עדיין יכול לבנות תחליף משלך די בקלות.

מדריך זה עובר על מספר דוגמאות של מקבילות ישירות והחלפות מותאמות אישית כדי להדגים כיצד ניתן להעביר את המודלים שמקורם ב-tf.estimator של tf.estimator 1 ל-TF2 עם Keras.

כלומר, מדריך זה כולל דוגמאות להגירה:

מבשר נפוץ לאימון של מודל הוא עיבוד מקדים של תכונה, הנעשה עבור מודלים של TensorFlow 1 Estimator עם tf.feature_column . למידע נוסף על עיבוד מוקדם של תכונות ב-TensorFlow 2, עיין במדריך זה על העברת עמודות תכונות .

להכין

התחל עם כמה יבואים הכרחיים של TensorFlow,

pip install tensorflow_decision_forests

import keras
import pandas as pd
import tensorflow as tf
import tensorflow.compat.v1 as tf1
import tensorflow_decision_forests as tfdf

WARNING:root:TF Parameter Server distributed training not available (this is expected for the pre-build release).

הכינו כמה נתונים פשוטים להדגמה ממערך הנתונים הסטנדרטי של Titanic,

x_train = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/train.csv')
x_eval = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/eval.csv')
x_train['sex'].replace(('male', 'female'), (0, 1), inplace=True)
x_eval['sex'].replace(('male', 'female'), (0, 1), inplace=True)

x_train['alone'].replace(('n', 'y'), (0, 1), inplace=True)
x_eval['alone'].replace(('n', 'y'), (0, 1), inplace=True)

x_train['class'].replace(('First', 'Second', 'Third'), (1, 2, 3), inplace=True)
x_eval['class'].replace(('First', 'Second', 'Third'), (1, 2, 3), inplace=True)

x_train.drop(['embark_town', 'deck'], axis=1, inplace=True)
x_eval.drop(['embark_town', 'deck'], axis=1, inplace=True)

y_train = x_train.pop('survived')
y_eval = x_eval.pop('survived')

# Data setup for TensorFlow 1 with `tf.estimator`
def _input_fn():
  return tf1.data.Dataset.from_tensor_slices((dict(x_train), y_train)).batch(32)


def _eval_input_fn():
  return tf1.data.Dataset.from_tensor_slices((dict(x_eval), y_eval)).batch(32)


FEATURE_NAMES = [
    'age', 'fare', 'sex', 'n_siblings_spouses', 'parch', 'class', 'alone'
]

feature_columns = []
for fn in FEATURE_NAMES:
  feat_col = tf1.feature_column.numeric_column(fn, dtype=tf.float32)
  feature_columns.append(feat_col)

וליצור שיטה ליצירת אופטימיזציה לדוגמה פשטנית לשימוש עם דגמי TensorFlow 1 Estimator ו- TensorFlow 2 Keras השונים.

def create_sample_optimizer(tf_version):
  if tf_version == 'tf1':
    optimizer = lambda: tf.keras.optimizers.Ftrl(
        l1_regularization_strength=0.001,
        learning_rate=tf1.train.exponential_decay(
            learning_rate=0.1,
            global_step=tf1.train.get_global_step(),
            decay_steps=10000,
            decay_rate=0.9))
  elif tf_version == 'tf2':
    optimizer = tf.keras.optimizers.Ftrl(
        l1_regularization_strength=0.001,
        learning_rate=tf.keras.optimizers.schedules.ExponentialDecay(
            initial_learning_rate=0.1, decay_steps=10000, decay_rate=0.9))
  return optimizer

דוגמה 1: הגירה מ-LinearEstimator

TF1: שימוש ב-LinearEstimator

ב-TensorFlow 1, אתה יכול להשתמש ב- tf.estimator.LinearEstimator כדי ליצור מודל ליניארי בסיסי עבור בעיות רגרסיה וסיווג.

linear_estimator = tf.estimator.LinearEstimator(
    head=tf.estimator.BinaryClassHead(),
    feature_columns=feature_columns,
    optimizer=create_sample_optimizer('tf1'))

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpvoycvffz
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpvoycvffz
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpvoycvffz', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpvoycvffz', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

linear_estimator.train(input_fn=_input_fn, steps=100)
linear_estimator.evaluate(input_fn=_eval_input_fn, steps=10)

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:401: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:401: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/canned/linear.py:1478: UserWarning: `layer.add_variable` is deprecated and will be removed in a future version. Please use `layer.add_weight` method instead.
  getter=tf.compat.v1.get_variable)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/ftrl.py:149: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/ftrl.py:149: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.6931472, step = 0
INFO:tensorflow:loss = 0.6931472, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpvoycvffz/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Loss for final step: 0.55268794.
INFO:tensorflow:Loss for final step: 0.55268794.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:45
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:45
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpvoycvffz/model.ckpt-20
INFO:tensorflow:Restoring parameters from /tmp/tmpvoycvffz/model.ckpt-20
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Inference Time : 0.50224s
INFO:tensorflow:Inference Time : 0.50224s
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:45
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:45
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.70075756, accuracy_baseline = 0.625, auc = 0.75472915, auc_precision_recall = 0.65362054, average_loss = 0.5759378, global_step = 20, label/mean = 0.375, loss = 0.5704812, precision = 0.6388889, prediction/mean = 0.41331062, recall = 0.46464646
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.70075756, accuracy_baseline = 0.625, auc = 0.75472915, auc_precision_recall = 0.65362054, average_loss = 0.5759378, global_step = 20, label/mean = 0.375, loss = 0.5704812, precision = 0.6388889, prediction/mean = 0.41331062, recall = 0.46464646
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpvoycvffz/model.ckpt-20
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpvoycvffz/model.ckpt-20
{'accuracy': 0.70075756,
 'accuracy_baseline': 0.625,
 'auc': 0.75472915,
 'auc_precision_recall': 0.65362054,
 'average_loss': 0.5759378,
 'label/mean': 0.375,
 'loss': 0.5704812,
 'precision': 0.6388889,
 'prediction/mean': 0.41331062,
 'recall': 0.46464646,
 'global_step': 20}

TF2: שימוש ב-Keras LinearModel

ב-TensorFlow 2, אתה יכול ליצור מופע של Keras tf.compat.v1.keras.models.LinearModel שהוא התחליף ל- tf.estimator.LinearEstimator . הנתיב tf.compat.v1.keras משמש כדי לציין שהמודל המוכן מראש קיים לצורך תאימות.

linear_model = tf.compat.v1.keras.experimental.LinearModel()
linear_model.compile(loss='mse', optimizer=create_sample_optimizer('tf2'), metrics=['accuracy'])
linear_model.fit(x_train, y_train, epochs=10)
linear_model.evaluate(x_eval, y_eval, return_dict=True)

Epoch 1/10
20/20 [==============================] - 0s 2ms/step - loss: 2.8157 - accuracy: 0.6300
Epoch 2/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2758 - accuracy: 0.6427
Epoch 3/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2470 - accuracy: 0.6699
Epoch 4/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1954 - accuracy: 0.7177
Epoch 5/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1931 - accuracy: 0.7145
Epoch 6/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1816 - accuracy: 0.7496
Epoch 7/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1766 - accuracy: 0.7751
Epoch 8/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2198 - accuracy: 0.7560
Epoch 9/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1657 - accuracy: 0.7959
Epoch 10/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1738 - accuracy: 0.7959
9/9 [==============================] - 0s 2ms/step - loss: 0.2278 - accuracy: 0.6780
{'loss': 0.22778697311878204, 'accuracy': 0.6780303120613098}

דוגמה 2: הגירה מ-DNNEstimator

TF1: שימוש ב-DNNEstimator

ב-TensorFlow 1, אתה יכול להשתמש ב- tf.estimator.DNNEstimator כדי ליצור מודל DNN בסיסי עבור בעיות רגרסיה וסיווג.

dnn_estimator = tf.estimator.DNNEstimator(
    head=tf.estimator.BinaryClassHead(),
    feature_columns=feature_columns,
    hidden_units=[128],
    activation_fn=tf.nn.relu,
    optimizer=create_sample_optimizer('tf1'))

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmphckb8f81
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmphckb8f81
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmphckb8f81', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmphckb8f81', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

dnn_estimator.train(input_fn=_input_fn, steps=100)
dnn_estimator.evaluate(input_fn=_eval_input_fn, steps=10)

INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 2.1811047, step = 0
INFO:tensorflow:loss = 2.1811047, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmphckb8f81/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Loss for final step: 0.5881681.
INFO:tensorflow:Loss for final step: 0.5881681.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:48
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:48
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmphckb8f81/model.ckpt-20
INFO:tensorflow:Restoring parameters from /tmp/tmphckb8f81/model.ckpt-20
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Inference Time : 0.47075s
INFO:tensorflow:Inference Time : 0.47075s
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:49
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:49
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.7083333, accuracy_baseline = 0.625, auc = 0.70716256, auc_precision_recall = 0.6146256, average_loss = 0.60399944, global_step = 20, label/mean = 0.375, loss = 0.5986442, precision = 0.6486486, prediction/mean = 0.41256863, recall = 0.4848485
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.7083333, accuracy_baseline = 0.625, auc = 0.70716256, auc_precision_recall = 0.6146256, average_loss = 0.60399944, global_step = 20, label/mean = 0.375, loss = 0.5986442, precision = 0.6486486, prediction/mean = 0.41256863, recall = 0.4848485
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmphckb8f81/model.ckpt-20
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmphckb8f81/model.ckpt-20
{'accuracy': 0.7083333,
 'accuracy_baseline': 0.625,
 'auc': 0.70716256,
 'auc_precision_recall': 0.6146256,
 'average_loss': 0.60399944,
 'label/mean': 0.375,
 'loss': 0.5986442,
 'precision': 0.6486486,
 'prediction/mean': 0.41256863,
 'recall': 0.4848485,
 'global_step': 20}

TF2: שימוש ב-Keras ליצירת מודל DNN מותאם אישית

ב-TensorFlow 2, אתה יכול ליצור מודל DNN מותאם אישית במקום אחד שנוצר על ידי tf.estimator.DNNEstimator , עם רמות דומות של התאמה אישית שצוינה על ידי המשתמש (לדוגמה, כמו בדוגמה הקודמת, היכולת להתאים אישית אופטימיזציה של מודל נבחר) .

ניתן להשתמש בזרימת עבודה דומה כדי להחליף את tf.estimator.experimental.RNNEstimator במודל RNN של Keras. Keras מספקת מספר אפשרויות מובנות הניתנות להתאמה אישית באמצעות tf.keras.layers.RNN , tf.keras.layers.LSTM ו- tf.keras.layers.GRU - ראה כאן לפרטים נוספים.

dnn_model = tf.keras.models.Sequential(
    [tf.keras.layers.Dense(128, activation='relu'),
     tf.keras.layers.Dense(1)])

dnn_model.compile(loss='mse', optimizer=create_sample_optimizer('tf2'), metrics=['accuracy'])

dnn_model.fit(x_train, y_train, epochs=10)
dnn_model.evaluate(x_eval, y_eval, return_dict=True)

Epoch 1/10
20/20 [==============================] - 0s 2ms/step - loss: 551.2993 - accuracy: 0.5997
Epoch 2/10
20/20 [==============================] - 0s 2ms/step - loss: 16.8562 - accuracy: 0.6427
Epoch 3/10
20/20 [==============================] - 0s 2ms/step - loss: 0.3048 - accuracy: 0.7161
Epoch 4/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2475 - accuracy: 0.7416
Epoch 5/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2334 - accuracy: 0.7512
Epoch 6/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2200 - accuracy: 0.7416
Epoch 7/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2012 - accuracy: 0.7656
Epoch 8/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2025 - accuracy: 0.7624
Epoch 9/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2185 - accuracy: 0.7703
Epoch 10/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2046 - accuracy: 0.7687
9/9 [==============================] - 0s 2ms/step - loss: 0.2227 - accuracy: 0.6856
{'loss': 0.2227054387331009, 'accuracy': 0.685606062412262}

דוגמה 3: הגירה מ-DNNLinearCombinedEstimator

TF1: שימוש ב-DNNLinearCombinedEstimator

ב-TensorFlow 1, אתה יכול להשתמש ב- tf.estimator.DNNLinearCombinedEstimator כדי ליצור מודל משולב בסיסי עבור בעיות רגרסיה וסיווג עם יכולת התאמה אישית עבור הרכיבים הליניאריים וה-DNN שלו.

optimizer = create_sample_optimizer('tf1')

combined_estimator = tf.estimator.DNNLinearCombinedEstimator(
    head=tf.estimator.BinaryClassHead(),
    # Wide settings
    linear_feature_columns=feature_columns,
    linear_optimizer=optimizer,
    # Deep settings
    dnn_feature_columns=feature_columns,
    dnn_hidden_units=[128],
    dnn_optimizer=optimizer)

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpwl5e5eaq
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpwl5e5eaq
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpwl5e5eaq', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpwl5e5eaq', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

combined_estimator.train(input_fn=_input_fn, steps=100)
combined_estimator.evaluate(input_fn=_eval_input_fn, steps=10)

INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/canned/linear.py:1478: UserWarning: `layer.add_variable` is deprecated and will be removed in a future version. Please use `layer.add_weight` method instead.
  getter=tf.compat.v1.get_variable)
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 2.5475807, step = 0
INFO:tensorflow:loss = 2.5475807, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 20...
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Saving checkpoints for 20 into /tmp/tmpwl5e5eaq/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 20...
INFO:tensorflow:Loss for final step: 0.58060575.
INFO:tensorflow:Loss for final step: 0.58060575.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:53
INFO:tensorflow:Starting evaluation at 2022-01-29T02:21:53
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpwl5e5eaq/model.ckpt-20
INFO:tensorflow:Restoring parameters from /tmp/tmpwl5e5eaq/model.ckpt-20
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [1/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [2/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [3/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [4/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [5/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [6/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [7/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [8/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Evaluation [9/10]
INFO:tensorflow:Inference Time : 0.54029s
INFO:tensorflow:Inference Time : 0.54029s
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:53
INFO:tensorflow:Finished evaluation at 2022-01-29-02:21:53
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.6931818, accuracy_baseline = 0.625, auc = 0.73532283, auc_precision_recall = 0.630229, average_loss = 0.65179086, global_step = 20, label/mean = 0.375, loss = 0.63768697, precision = 0.60714287, prediction/mean = 0.4162652, recall = 0.5151515
INFO:tensorflow:Saving dict for global step 20: accuracy = 0.6931818, accuracy_baseline = 0.625, auc = 0.73532283, auc_precision_recall = 0.630229, average_loss = 0.65179086, global_step = 20, label/mean = 0.375, loss = 0.63768697, precision = 0.60714287, prediction/mean = 0.4162652, recall = 0.5151515
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpwl5e5eaq/model.ckpt-20
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 20: /tmp/tmpwl5e5eaq/model.ckpt-20
{'accuracy': 0.6931818,
 'accuracy_baseline': 0.625,
 'auc': 0.73532283,
 'auc_precision_recall': 0.630229,
 'average_loss': 0.65179086,
 'label/mean': 0.375,
 'loss': 0.63768697,
 'precision': 0.60714287,
 'prediction/mean': 0.4162652,
 'recall': 0.5151515,
 'global_step': 20}

TF2: שימוש ב-Keras WideDeepModel

ב-TensorFlow 2, אתה יכול ליצור מופע של Keras tf.compat.v1.keras.models.WideDeepModel אחד שנוצר על ידי tf.estimator.DNNLinearCombinedEstimator , עם רמות דומות של התאמה אישית שצוינה על ידי המשתמש (לדוגמה, כמו ב- דוגמה קודמת, היכולת להתאים אישית מייעל מודל נבחר).

WideDeepModel זה בנוי על בסיס מודל LinearModel מכונן ומודל DNN מותאם אישית, שניהם נדונים בשתי הדוגמאות הקודמות. ניתן להשתמש גם בדגם ליניארי מותאם אישית במקום ה-Keras LinearModel אם תרצה בכך.

אם תרצה לבנות דגם משלך במקום אומדן משומר, בדוק כיצד לבנות מודל keras.Sequential . למידע נוסף על הדרכה ואופטימיזציה מותאמים אישית, תוכל גם לבדוק את המדריך הזה .

# Create LinearModel and DNN Model as in Examples 1 and 2
optimizer = create_sample_optimizer('tf2')

linear_model = tf.compat.v1.keras.experimental.LinearModel()
linear_model.compile(loss='mse', optimizer=optimizer, metrics=['accuracy'])
linear_model.fit(x_train, y_train, epochs=10, verbose=0)

dnn_model = tf.keras.models.Sequential(
    [tf.keras.layers.Dense(128, activation='relu'),
     tf.keras.layers.Dense(1)])
dnn_model.compile(loss='mse', optimizer=optimizer, metrics=['accuracy'])

combined_model = tf.compat.v1.keras.experimental.WideDeepModel(linear_model,
                                                               dnn_model)
combined_model.compile(
    optimizer=[optimizer, optimizer], loss='mse', metrics=['accuracy'])
combined_model.fit([x_train, x_train], y_train, epochs=10)
combined_model.evaluate(x_eval, y_eval, return_dict=True)

Epoch 1/10
20/20 [==============================] - 0s 2ms/step - loss: 1118.0448 - accuracy: 0.6715
Epoch 2/10
20/20 [==============================] - 0s 2ms/step - loss: 0.5682 - accuracy: 0.7305
Epoch 3/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2719 - accuracy: 0.7671
Epoch 4/10
20/20 [==============================] - 0s 2ms/step - loss: 0.2032 - accuracy: 0.7831
Epoch 5/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1911 - accuracy: 0.7783
Epoch 6/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1895 - accuracy: 0.7863
Epoch 7/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1882 - accuracy: 0.7863
Epoch 8/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1717 - accuracy: 0.7974
Epoch 9/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1701 - accuracy: 0.7927
Epoch 10/10
20/20 [==============================] - 0s 2ms/step - loss: 0.1684 - accuracy: 0.7990
9/9 [==============================] - 0s 2ms/step - loss: 0.1930 - accuracy: 0.7424
{'loss': 0.19299836456775665, 'accuracy': 0.7424242496490479}

דוגמה 4: הגירה מ-BostedTreesEstimator

TF1: שימוש ב-BostedTreesEstimator

ב-TensorFlow 1, אתה יכול להשתמש ב- tf.estimator.BoostedTreesEstimator כדי ליצור קו בסיס ליצירת מודל להגברת שיפוע בסיסי באמצעות מכלול של עצי החלטה עבור בעיות רגרסיה וסיווג. פונקציונליות זו אינה כלולה עוד ב-TensorFlow 2.

bt_estimator = tf1.estimator.BoostedTreesEstimator(
    head=tf.estimator.BinaryClassHead(),
    n_batches_per_layer=1,
    max_depth=10,
    n_trees=1000,
    feature_columns=feature_columns)

bt_estimator.train(input_fn=_input_fn, steps=1000)
bt_estimator.evaluate(input_fn=_eval_input_fn, steps=100)

TF2: שימוש ביערות החלטה של ​​TensorFlow

ב-TensorFlow 2, התחליף הארוז מראש הקרוב ביותר למודל שנוצר על ידי tf.estimator.BoostedTreesEstimator הוא כזה שנוצר באמצעות tfdf.keras.GradientBoostedTreesModel , אשר יוצר רצף מאומן ברצף של עצי החלטה רדודים, שכל אחד מהם נועד "ללמוד" משגיאות שנעשו על ידי קודמיו ברצף.

GradientBoostedTreesModel מספק אפשרויות נוספות להתאמה אישית, ומאפשרים מפרט של כל דבר, החל מאילוצי עומק בסיסיים ועד לתנאי עצירה מוקדמים. ראה כאן לפרטים נוספים של תכונת GradientBoostedTreesModel .

gbt_model = tfdf.keras.GradientBoostedTreesModel(
    task=tfdf.keras.Task.CLASSIFICATION)
gbt_model.compile(metrics=['mse', 'accuracy'])

Use /tmp/tmpbr1acn2_ as temporary training directory

train_df, eval_df = x_train.copy(), x_eval.copy()
train_df['survived'], eval_df['survived'] = y_train, y_eval

train_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(train_df, label='survived')
eval_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(eval_df, label='survived')

gbt_model.fit(train_dataset)
gbt_model.evaluate(eval_dataset, return_dict=True)

Starting reading the dataset
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_decision_forests/keras/core.py:2036: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  features_dataframe = dataframe.drop(label, 1)
1/1 [==============================] - ETA: 0s
Dataset read in 0:00:03.161776
Training model
Model trained in 0:00:00.102649
Compiling model
1/1 [==============================] - 3s 3s/step
[INFO kernel.cc:1153] Loading model from path
[INFO abstract_model.cc:1063] Engine "GradientBoostedTreesQuickScorerExtended" built
[INFO kernel.cc:1001] Use fast generic engine
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f95e9db4e60> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f95e9db4e60> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f95e9db4e60> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
1/1 [==============================] - 0s 388ms/step - loss: 0.0000e+00 - mse: 0.1308 - accuracy: 0.8144
{'loss': 0.0, 'mse': 0.13076548278331757, 'accuracy': 0.814393937587738}

ב-TensorFlow 2, קיים גם תחליף TFDF זמין נוסף למודל שנוצר על ידי tf.estimator.BoostedTreesEstimator - tfdf.keras.RandomForestModel . RandomForestModel יוצר לומד חזק, עמיד בפני התאמה יתר, המורכב מאוכלוסיית הצבעה של עצי החלטה עמוקים, שכל אחד מהם מאומן על קבוצות משנה אקראיות של מערך ההכשרה של הקלט.

RandomForestModel ו- GradientBoostedTreesModel מספקים רמות נרחבות דומות של התאמה אישית. הבחירה ביניהם היא ספציפית לבעיה ותלויה במשימה או ביישום שלך.

בדוק את מסמכי ה-API לקבלת מידע נוסף על RandomForestModel ו- GradientBoostedTreesModel .

rf_model = tfdf.keras.RandomForestModel(
    task=tfdf.keras.Task.CLASSIFICATION)
rf_model.compile(metrics=['mse', 'accuracy'])

Use /tmp/tmpluh2ebcj as temporary training directory

rf_model.fit(train_dataset)
rf_model.evaluate(eval_dataset, return_dict=True)

Starting reading the dataset
1/1 [==============================] - ETA: 0s
Dataset read in 0:00:00.094262
Training model
Model trained in 0:00:00.083656
Compiling model
1/1 [==============================] - 0s 260ms/step
[INFO kernel.cc:1153] Loading model from path
[INFO kernel.cc:1001] Use fast generic engine
1/1 [==============================] - 0s 123ms/step - loss: 0.0000e+00 - mse: 0.1270 - accuracy: 0.8636
{'loss': 0.0, 'mse': 0.12698587775230408, 'accuracy': 0.8636363744735718}