View source on GitHub |
Holds a list of enqueue operations for a queue, each to be run in a thread.
tf.compat.v1.train.QueueRunner(
queue=None, enqueue_ops=None, close_op=None, cancel_op=None,
queue_closed_exception_types=None, queue_runner_def=None, import_scope=None
)
Queues are a convenient TensorFlow mechanism to compute tensors asynchronously using multiple threads. For example in the canonical 'Input Reader' setup one set of threads generates filenames in a queue; a second set of threads read records from the files, processes them, and enqueues tensors on a second queue; a third set of threads dequeues these input records to construct batches and runs them through training operations.
There are several delicate issues when running multiple threads that way: closing the queues in sequence as the input is exhausted, correctly catching and reporting exceptions, etc.
The QueueRunner
, combined with the Coordinator
, helps handle these issues.
Args | |
---|---|
queue
|
A Queue .
|
enqueue_ops
|
List of enqueue ops to run in threads later. |
close_op
|
Op to close the queue. Pending enqueue ops are preserved. |
cancel_op
|
Op to close the queue and cancel pending enqueue ops. |
queue_closed_exception_types
|
Optional tuple of Exception types that
indicate that the queue has been closed when raised during an enqueue
operation. Defaults to (tf.errors.OutOfRangeError,) . Another common
case includes (tf.errors.OutOfRangeError, tf.errors.CancelledError) ,
when some of the enqueue ops may dequeue from other Queues.
|
queue_runner_def
|
Optional QueueRunnerDef protocol buffer. If specified,
recreates the QueueRunner from its contents. queue_runner_def and the
other arguments are mutually exclusive.
|
import_scope
|
Optional string . Name scope to add. Only used when
initializing from protocol buffer.
|
Raises | |
---|---|
ValueError
|
If both queue_runner_def and queue are both specified.
|
ValueError
|
If queue or enqueue_ops are not provided when not
restoring from queue_runner_def .
|
RuntimeError
|
If eager execution is enabled. |
Eager Compatibility
QueueRunners are not compatible with eager execution. Instead, please
use tf.data
to get data into your model.
Attributes | |
---|---|
cancel_op
|
|
close_op
|
|
enqueue_ops
|
|
exceptions_raised
|
Exceptions raised but not handled by the QueueRunner threads.
Exceptions raised in queue runner threads are handled in one of two ways
depending on whether or not a
|
name
|
The string name of the underlying Queue. |
queue
|
|
queue_closed_exception_types
|
Methods
create_threads
create_threads(
sess, coord=None, daemon=False, start=False
)
Create threads to run the enqueue ops for the given session.
This method requires a session in which the graph was launched. It creates
a list of threads, optionally starting them. There is one thread for each
op passed in enqueue_ops
.
The coord
argument is an optional coordinator that the threads will use
to terminate together and report exceptions. If a coordinator is given,
this method starts an additional thread to close the queue when the
coordinator requests a stop.
If previously created threads for the given session are still running, no new threads will be created.
Args | |
---|---|
sess
|
A Session .
|
coord
|
Optional Coordinator object for reporting errors and checking
stop conditions.
|
daemon
|
Boolean. If True make the threads daemon threads.
|
start
|
Boolean. If True starts the threads. If False the
caller must call the start() method of the returned threads.
|
Returns | |
---|---|
A list of threads. |
from_proto
@staticmethod
from_proto( queue_runner_def, import_scope=None )
Returns a QueueRunner
object created from queue_runner_def
.
to_proto
to_proto(
export_scope=None
)
Converts this QueueRunner
to a QueueRunnerDef
protocol buffer.
Args | |
---|---|
export_scope
|
Optional string . Name scope to remove.
|
Returns | |
---|---|
A QueueRunnerDef protocol buffer, or None if the Variable is not in
the specified name scope.
|