Creates a dataset that shards the input dataset.
Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).
This dataset will throw a NotFound error if we cannot shard the dataset automatically.
Nested Classes
| class | AutoShardDataset.Options | Optional attributes for AutoShardDataset
|
|
Constants
| String | OP_NAME | The name of this op, as known by TensorFlow core engine |
Public Methods
| Output<TType> |
asOutput()
Returns the symbolic handle of the tensor.
|
| static AutoShardDataset.Options |
autoShardPolicy(Long autoShardPolicy)
|
| static AutoShardDataset | |
| Output<?> |
handle()
|
Inherited Methods
Constants
public static final String OP_NAME
The name of this op, as known by TensorFlow core engine
Public Methods
public Output<TType> asOutput ()
Returns the symbolic handle of the tensor.
Inputs to TensorFlow operations are outputs of another TensorFlow operation. This method is used to obtain a symbolic handle that represents the computation of the input.
public static AutoShardDataset create (Scope scope, Operand<?> inputDataset, Operand<TInt64> numWorkers, Operand<TInt64> index, List<Class<? extends TType>> outputTypes, List<Shape> outputShapes, Options... options)
Factory method to create a class wrapping a new AutoShardDataset operation.
Parameters
| scope | current scope |
|---|---|
| inputDataset | A variant tensor representing the input dataset. |
| numWorkers | A scalar representing the number of workers to distribute this dataset across. |
| index | A scalar representing the index of the current worker out of num_workers. |
| options | carries optional attributes values |
Returns
- a new instance of AutoShardDataset