tf.data.experimental.AutoShardPolicy
bookmark_borderbookmark
Stay organized with collections
Save and categorize content based on your preferences.
Represents the type of auto-sharding to use.
View aliases
Compat aliases for migration
See
Migration guide for
more details.
tf.compat.v1.data.experimental.AutoShardPolicy
OFF: No sharding will be performed.
AUTO: Attempts FILE-based sharding, falling back to DATA-based sharding.
FILE: Shards by input files (i.e. each worker will get a set of files to
process). When this option is selected, make sure that there is at least as
many files as workers. If there are fewer input files than workers, a runtime
error will be raised.
DATA: Shards by elements produced by the dataset. Each worker will process the
whole dataset and discard the portion that is not for itself. Note that for
this mode to correctly partitions the dataset elements, the dataset needs to
produce elements in a deterministic order.
HINT: Looks for the presence of shard(SHARD_HINT, ...) which is treated as a
placeholder to replace with shard(num_workers, worker_index).
Class Variables |
|---|
|
AUTO
|
<AutoShardPolicy.AUTO: 0>
|
|
DATA
|
<AutoShardPolicy.DATA: 2>
|
|
FILE
|
<AutoShardPolicy.FILE: 1>
|
|
HINT
|
<AutoShardPolicy.HINT: 3>
|
|
OFF
|
<AutoShardPolicy.OFF: -1>
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates. Some content is licensed under the numpy license.
Last updated 2022-11-04 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2022-11-04 UTC."],[],[]]
View source on GitHub