tfm.core.base_task.RuntimeConfig

High-level configurations for Runtime.

Inherits From: Config, ParamsDict

tfm.core.base_task.RuntimeConfig(
    default_params: dataclasses.InitVar[Optional[Mapping[str, Any]]] = None,
    restrictions: dataclasses.InitVar[Optional[List[str]]] = None,
    distribution_strategy: str = 'mirrored',
    enable_xla: bool = False,
    gpu_thread_mode: Optional[str] = None,
    dataset_num_private_threads: Optional[int] = None,
    per_gpu_thread_count: int = 0,
    tpu: Optional[str] = None,
    num_gpus: int = 0,
    worker_hosts: Optional[str] = None,
    task_index: int = -1,
    all_reduce_alg: Optional[str] = None,
    num_packs: int = 1,
    mixed_precision_dtype: Optional[str] = None,
    loss_scale: Optional[Union[str, float]] = None,
    run_eagerly: bool = False,
    batchnorm_spatial_persistent: bool = False,
    tpu_enable_xla_dynamic_padder: Optional[bool] = None,
    num_cores_per_replica: int = 1,
    default_shard_dim: int = -1,
    use_tpu_mp_strategy: bool = False
)

These include parameters that are not directly related to the experiment, e.g. directories, accelerator type, etc.

distribution_strategy e.g. 'mirrored', 'tpu', etc.
enable_xla Whether or not to enable XLA.
per_gpu_thread_count thread count per GPU.
gpu_thread_mode Whether and how the GPU device uses its own threadpool.
dataset_num_private_threads Number of threads for a private threadpool created for all datasets computation.
tpu The address of the TPU to use, if any.
num_gpus The number of GPUs to use, if any.
worker_hosts comma-separated list of worker ip:port pairs for running multi-worker models with DistributionStrategy.
task_index If multi-worker training, the task index of this worker.
all_reduce_alg Defines the algorithm for performing all-reduce.
num_packs Sets num_packs in the cross device ops used in MirroredStrategy. For details, see tf.distribute.NcclAllReduce.
mixed_precision_dtype dtype of mixed precision policy. It can be 'float32', 'float16', or 'bfloat16'.
loss_scale The type of loss scale, or 'float' value. This is used when setting the mixed precision policy.
run_eagerly Whether or not to run the experiment eagerly.
batchnorm_spatial_persistent Whether or not to enable the spatial persistent mode for CuDNN batch norm kernel for improved GPU performance.
BUILDER

default_params Dataclass field
restrictions Dataclass field
tpu_enable_xla_dynamic_padder Dataclass field
num_cores_per_replica Dataclass field
default_shard_dim Dataclass field
use_tpu_mp_strategy Dataclass field

Methods

as_dict

View source

as_dict()

Returns a dict representation of params_dict.ParamsDict.

For the nested params_dict.ParamsDict, a nested dict will be returned.

from_args

View source

@classmethod
from_args(
    *args, **kwargs
)

Builds a config from the given list of arguments.

from_json

View source

@classmethod
from_json(
    file_path: str
)

Wrapper for from_yaml.

from_yaml

View source

@classmethod
from_yaml(
    file_path: str
)

get

View source

get(
    key, value=None
)

Accesses through built-in dictionary get method.

lock

View source

lock()

Makes the ParamsDict immutable.

model_parallelism

View source

model_parallelism()

override

View source

override(
    override_params, is_strict=True
)

Override the ParamsDict with a set of given params.

Args
override_params a dict or a ParamsDict specifying the parameters to be overridden.
is_strict a boolean specifying whether override is strict or not. If True, keys in override_params must be present in the ParamsDict. If False, keys in override_params can be different from what is currently defined in the ParamsDict. In this case, the ParamsDict will be extended to include the new keys.

replace

View source

replace(
    **kwargs
)

Overrides/returns a unlocked copy with the current config unchanged.

validate

View source

validate()

Validate the parameters consistency based on the restrictions.

This method validates the internal consistency using the pre-defined list of restrictions. A restriction is defined as a string which specifies a binary operation. The supported binary operations are {'==', '!=', '<', '<=', '>', '>='}. Note that the meaning of these operators are consistent with the underlying Python immplementation. Users should make sure the define restrictions on their type make sense.

For example, for a ParamsDict like the following

a:
  a1: 1
  a2: 2
b:
  bb:
    bb1: 10
    bb2: 20
  ccc:
    a1: 1
    a3: 3

one can define two restrictions like this ['a.a1 == b.ccc.a1', 'a.a2 <= b.bb.bb2']

What it enforces are

  • a.a1 = 1 == b.ccc.a1 = 1
  • a.a2 = 2 <= b.bb.bb2 = 20

Raises
KeyError if any of the following happens (1) any of parameters in any of restrictions is not defined in ParamsDict, (2) any inconsistency violating the restriction is found.
ValueError if the restriction defined in the string is not supported.

__contains__

View source

__contains__(
    key
)

Implements the membership test operator.

__eq__

__eq__(
    other
)

IMMUTABLE_TYPES (<class 'str'>, <class 'int'>, <class 'float'>, <class 'bool'>, <class 'NoneType'>)
RESERVED_ATTR ['_locked', '_restrictions']
SEQUENCE_TYPES (<class 'list'>, <class 'tuple'>)
all_reduce_alg None
batchnorm_spatial_persistent False
dataset_num_private_threads None
default_params None
default_shard_dim -1
distribution_strategy 'mirrored'
enable_xla False
gpu_thread_mode None
loss_scale None
mixed_precision_dtype None
num_cores_per_replica 1
num_gpus 0
num_packs 1
per_gpu_thread_count 0
restrictions None
run_eagerly False
task_index -1
tpu None
tpu_enable_xla_dynamic_padder None
use_tpu_mp_strategy False
worker_hosts None