tf.experimental.RowPartition

Partitioning of a sequence of values into contiguous subsequences ("rows").

tf.experimental.RowPartition(
    row_splits,
    row_lengths=None,
    value_rowids=None,
    nrows=None,
    uniform_row_length=None,
    nvals=None,
    internal=False
)

A RowPartition describes how a sequence with nvals items should be divided into nrows contiguous subsequences ("rows"). For example, a RowPartition could be used to partition the vector [1, 2, 3, 4, 5] into subsequences [[1, 2], [3], [], [4, 5]]. Note that RowPartition stores information about how values are partitioned, but does not include the partitioned values themselves. tf.RaggedTensor is used to pair a values tensor with one or more RowPartitions, providing a complete encoding for a ragged tensor (i.e. a tensor with variable-length dimensions).

RowPartitions may be defined using several different schemes:

  • row_lengths: an integer vector with shape [nrows], which specifies the length of each row.

  • row_splits: an integer vector with shape [nrows+1], specifying the "split points" between each row.

  • row_starts: an integer vector with shape [nrows], which specifies the start offset for each row. Equivalent to row_splits[:-1].

  • row_limits: an integer vector with shape [nrows], which specifies the stop offset for each row. Equivalent to row_splits[1:].

  • value_rowids is an integer vector with shape [nvals], corresponding one-to-one with sequence values, which specifies the row that each value belongs to. If the partition has empty trailing rows, then nrows must also be specified.

  • uniform_row_length is an integer scalar, specifying the length of every row. This scheme may only be used if all rows have the same length.

For example, the following RowPartitions all represent the partitioning of 8 values into 5 sublists as follows: [[*, *, *, *], [], [*, *, *], [*], []].

p1 = RowPartition.from_row_lengths([4, 0, 3, 1, 0])
p2 = RowPartition.from_row_splits([0, 4, 4, 7, 8, 8])
p3 = RowPartition.from_row_starts([0, 4, 4, 7, 8], nvals=8)
p4 = RowPartition.from_row_limits([4, 4, 7, 8, 8])
p5 = RowPartition.from_value_rowids([0, 0, 0, 0, 2, 2, 2, 3], nrows=5)

For more information about each scheme, see the documentation for the its factory method. For additional examples, see the documentation on tf.RaggedTensor.

Precomputed Encodings

RowPartition always stores at least one encoding of the partitioning, but it can be configured to cache additional encodings as well. This can avoid unnecessary recomputation in eager mode. (In graph mode, optimizations such as common subexpression elimination will typically prevent these unnecessary recomputations.) To check which encodings are precomputed, use RowPartition.has_precomputed_<encoding>. To cache an additional encoding, use RowPartition.with_precomputed_<encoding>.

row_splits A 1-D integer tensor with shape [nrows+1].
row_lengths A 1-D integer tensor with shape [nrows]
value_rowids A 1-D integer tensor with shape [nvals].
nrows A 1-D integer scalar tensor.
uniform_row_length A scalar tensor.
nvals A scalar tensor.
internal Private key value, required to ensure that this private constructor is only called from the factory methods.

TypeError If a row partitioning tensor has an inappropriate dtype.
TypeError If exactly one row partitioning argument was not specified.
ValueError If a row partitioning tensor has an inappropriate shape.
ValueError If multiple partitioning arguments are specified.
ValueError If nrows is specified but value_rowids is not None.

dtype The DType used to encode the row partition (either int32 or int64).
static_nrows The number of rows in this partition, if statically known.

self.row_lengths().shape == [self.static_nrows]
self.row_starts().shape == [self.static_nrows]
self.row_limits().shape == [self.static_nrows]
self.row_splits().shape == [self.static_nrows + 1]

static_nvals The number of values in this partition, if statically known.

self.value_rowids().shape == [self.static_vals]

static_uniform_row_length The number of values in each row of this partition, if statically known.

Methods

from_row_lengths

View source

@classmethod
from_row_lengths(
    row_lengths, validate=True, dtype=None, dtype_hint=None
)

Creates a RowPartition with rows partitioned by row_lengths.

This RowPartition divides a sequence values into rows by indicating the length of each row:

partitioned_rows = [[values.pop(0) for _ in range(length)]
                    for length in row_lengths]

Args
row_lengths A 1-D integer tensor with shape [nrows]. Must be nonnegative.
validate If true, then use assertions to check that the arguments form a valid RowPartition.
dtype Optional dtype for the RowPartition. If missing, the type is inferred from the type of row_lengths, dtype_hint, or tf.int64.
dtype_hint Optional dtype for the RowPartition, used when dtype is None. In some cases, a caller may not have a dtype in mind when converting to a tensor, so dtype_hint can be used as a soft preference. If the conversion to dtype_hint is not possible, this argument has no effect.

Returns
A RowPartition.

from_row_limits

View source

@classmethod
from_row_limits(
    row_limits, validate=True, dtype=None, dtype_hint=None
)

Creates a RowPartition with rows partitioned by row_limits.

Equivalent to: from_row_splits(values, concat([0, row_limits], axis=0)).

Args
row_limits A 1-D integer tensor with shape [nrows]. Must be sorted in ascending order.
validate If true, then use assertions to check that the arguments form a valid RowPartition.
dtype Optional dtype for the RowPartition. If missing, the type is inferred from the type of row_limits, dtype_hint, or tf.int64.
dtype_hint Optional dtype for the RowPartition, used when dtype is None. In some cases, a caller may not have a dtype in mind when converting to a tensor, so dtype_hint can be used as a soft preference. If the conversion to dtype_hint is not possible, this argument has no effect.

Returns
A RowPartition.

from_row_splits

View source

@classmethod
from_row_splits(
    row_splits, validate=True, dtype=None, dtype_hint=None
)

Creates a RowPartition with rows partitioned by row_splits.

This RowPartition divides a sequence values into rows by indicating where each row begins and ends:

partitioned_rows = []
for i in range(len(row_splits) - 1):
  row_start = row_splits[i]
  row_end = row_splits[i + 1]
  partitioned_rows.append(values[row_start:row_end])

Args
row_splits A 1-D integer tensor with shape [nrows+1]. Must not be empty, and must be sorted in ascending order. row_splits[0] must be zero.
validate If true, then use assertions to check that the arguments form a valid RowPartition.
dtype Optional dtype for the RowPartition. If missing, the type is inferred from the type of row_splits, dtype_hint, or tf.int64.
dtype_hint Optional dtype for the RowPartition, used when dtype is None. In some cases, a caller may not have a dtype in mind when converting to a tensor, so dtype_hint can be used as a soft preference. If the conversion to dtype_hint is not possible, this argument has no effect.

Returns
A RowPartition.

Raises
ValueError If row_splits is an empty list.

from_row_starts

View source

@classmethod
from_row_starts(
    row_starts, nvals, validate=True, dtype=None, dtype_hint=None
)

Creates a RowPartition with rows partitioned by row_starts.

Equivalent to: from_row_splits(concat([row_starts, nvals], axis=0)).

Args
row_starts A 1-D integer tensor with shape [nrows]. Must be nonnegative and sorted in ascending order. If nrows>0, then row_starts[0] must be zero.
nvals A scalar tensor indicating the number of values.
validate If true, then use assertions to check that the arguments form a valid RowPartition.
dtype Optional dtype for the RowPartition. If missing, the type is inferred from the type of row_starts, dtype_hint, or tf.int64.
dtype_hint Optional dtype for the RowPartition, used when dtype is None. In some cases, a caller may not have a dtype in mind when converting to a tensor, so dtype_hint can be used as a soft preference. If the conversion to dtype_hint is not possible, this argument has no effect.

Returns
A RowPartition.

from_uniform_row_length

View source

@classmethod
from_uniform_row_length(
    uniform_row_length,
    nvals=None,
    nrows=None,
    validate=True,
    dtype=None,
    dtype_hint=None
)

Creates a RowPartition with rows partitioned by uniform_row_length.

This RowPartition divides a sequence values into rows that all have the same length:

partitioned_rows = [[values.pop(0) for _ in range(uniform_row_length)]
         for _ in range(nrows)]

Note that either or both of nvals and nrows must be specified.

Args
uniform_row_length A scalar integer tensor. Must be nonnegative. The size of the outer axis of values must be evenly divisible by uniform_row_length.
nvals a non-negative scalar integer tensor for the number of values. Must be specified if nrows is not specified. If not specified, defaults to uniform_row_length*nrows
nrows The number of rows in the constructed RowPartition. If not specified, then it defaults to nvals/uniform_row_length (or 0 if uniform_row_length==0). nrows only needs to be specified if uniform_row_length might be zero. uniform_row_length*nrows must be nvals.
validate If true, then use assertions to check that the arguments form a valid RowPartition.
dtype Optional dtype for the RowPartition. If missing, the type is inferred from the type of uniform_row_length, dtype_hint, or tf.int64.
dtype_hint Optional dtype for the RowPartition, used when dtype is None. In some cases, a caller may not have a dtype in mind when converting to a tensor, so dtype_hint can be used as a soft preference. If the conversion to dtype_hint is not possible, this argument has no effect.

Returns
A RowPartition.

from_value_rowids

View source

@classmethod
from_value_rowids(
    value_rowids, nrows=None, validate=True, dtype=None, dtype_hint=None
)

Creates a RowPartition with rows partitioned by value_rowids.

This RowPartition divides a sequence values into rows by specifying which row each value should be added to:

partitioned_rows = [[] for _ in nrows]
for (value, rowid) in zip(values, value_rowids):
  partitioned_rows[rowid].append(value)
``

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Args</th></tr>

<tr>
<td>
`value_rowids`
</td>
<td>
A 1-D integer tensor with shape `[nvals]`, which corresponds
one-to-one with `values`, and specifies each value's row index.  Must be
nonnegative, and must be sorted in ascending order.
</td>
</tr><tr>
<td>
`nrows`
</td>
<td>
An integer scalar specifying the number of rows.  This should be
specified if the `RowPartition` may containing empty training rows. Must
be greater than `value_rowids[-1]` (or greater than or equal to zero if
`value_rowids` is empty). Defaults to `value_rowids[-1] + 1` (or zero if
`value_rowids` is empty).
</td>
</tr><tr>
<td>
`validate`
</td>
<td>
If true, then use assertions to check that the arguments form a
valid `RowPartition`.
</td>
</tr><tr>
<td>
`dtype`
</td>
<td>
Optional dtype for the RowPartition. If missing, the type
is inferred from the type of `value_rowids`, dtype_hint, or tf.int64.
</td>
</tr><tr>
<td>
`dtype_hint`
</td>
<td>
Optional dtype for the RowPartition, used when dtype
is None. In some cases, a caller may not have a dtype in mind when
converting to a tensor, so dtype_hint can be used as a soft preference.
If the conversion to `dtype_hint` is not possible, this argument has no
effect.
</td>
</tr>
</table>



<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A `RowPartition`.
</td>
</tr>

</table>



<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Raises</th></tr>

<tr>
<td>
`ValueError`
</td>
<td>
If `nrows` is incompatible with `value_rowids`.
</td>
</tr>
</table>


#### Example:

<pre class="devsite-click-to-copy prettyprint lang-py">
<code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">print(RowPartition.from_value_rowids(</code>
<code class="devsite-terminal" data-terminal-prefix="...">    value_rowids=[0, 0, 0, 0, 2, 2, 2, 3],</code>
<code class="devsite-terminal" data-terminal-prefix="...">    nrows=4))</code>
<code class="no-select nocode">tf.RowPartition(row_splits=[0 4 4 7 8])</code>
</pre>


<h3 id="is_uniform"><code>is_uniform</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L921-L932">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>is_uniform()
</code></pre>

Returns true if the partition is known to be uniform statically.

This is based upon the existence of self._uniform_row_length. For example:
RowPartition.from_row_lengths([3,3,3]).is_uniform()==false
RowPartition.from_uniform_row_length(5, nvals=20).is_uniform()==true
RowPartition.from_row_lengths([2,0,2]).is_uniform()==false

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
Whether a RowPartition is known to be uniform statically.
</td>
</tr>

</table>



<h3 id="nrows"><code>nrows</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L777-L789">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>nrows()
</code></pre>

Returns the number of rows created by this `RowPartition`.


<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
scalar integer Tensor
</td>
</tr>

</table>



<h3 id="nvals"><code>nvals</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L762-L775">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>nvals()
</code></pre>

Returns the number of values partitioned by this `RowPartition`.

If the sequence partitioned by this `RowPartition` is a tensor, then
`nvals` is the size of that tensor's outermost dimension -- i.e.,
`nvals == values.shape[0]`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
scalar integer Tensor
</td>
</tr>

</table>



<h3 id="offsets_in_rows"><code>offsets_in_rows</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L903-L919">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>offsets_in_rows()
</code></pre>

Return the offset of each value.

RowPartition takes an array x and converts it into sublists.
offsets[i] is the index of x[i] in its sublist.
Given a shape, such as:
[*,*,*],[*,*],[],[*,*]
This returns:
0,1,2,0,1,0,1

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
an offset for every value.
</td>
</tr>

</table>



<h3 id="row_lengths"><code>row_lengths</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L829-L840">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>row_lengths()
</code></pre>

Returns the lengths of rows in this `RowPartition`.


<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A 1-D integer Tensor with shape `[self.nrows]`.
The returned tensor is nonnegative.
`tf.reduce_sum(self.row_lengths) == self.nvals()`.
</td>
</tr>

</table>



<h3 id="row_limits"><code>row_limits</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L816-L827">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>row_limits()
</code></pre>

Returns the limit indices for rows in this row partition.

These indices specify where the values for each row end.
`partition.row_limits()` is equal to `partition.row_splits()[:-1]`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A 1-D integer Tensor with shape `[self.nrows]`.
The returned tensor is nonnegative, and is sorted in ascending order.
`self.row_limits()[-1] == self.nvals()`.
</td>
</tr>

</table>



<h3 id="row_splits"><code>row_splits</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L733-L746">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>row_splits()
</code></pre>

Returns the row-split indices for this row partition.

`row_splits` specifies where the values for each row begin and end.
In particular, the values for row `i` are stored in the slice
`values[row_splits[i]:row_splits[i+1]]`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A 1-D integer `Tensor` with shape `[self.nrows+1]`.
The returned tensor is non-empty, and is sorted in ascending order.
`self.row_splits()[0] == 0`.
`self.row_splits()[-1] == self.nvals()`.
</td>
</tr>

</table>



<h3 id="row_starts"><code>row_starts</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L802-L814">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>row_starts()
</code></pre>

Returns the start indices for rows in this row partition.

These indices specify where the values for each row begin.
`partition.row_starts()` is equal to `partition.row_splits()[:-1]`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A 1-D integer Tensor with shape `[self.nrows()]`.
The returned tensor is nonnegative, and is sorted in ascending order.
`self.row_starts()[0] == 0`.
`self.row_starts()[-1] <= self.nvals()`.
</td>
</tr>

</table>



<h3 id="uniform_row_length"><code>uniform_row_length</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L791-L800">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>uniform_row_length()
</code></pre>

Returns the length of each row in this partition, if rows are uniform.

If all rows in this `RowPartition` have the same length, then this returns
that length as a scalar integer `Tensor`.  Otherwise, it returns `None`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
scalar Tensor with `type=self.dtype`, or `None`.
</td>
</tr>

</table>



<h3 id="value_rowids"><code>value_rowids</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L748-L760">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>value_rowids()
</code></pre>

Returns the row indices for this row partition.

`value_rowids` specifies the row index fo reach value.  In particular,
`value_rowids[i]` is the row index for `values[i]`.

<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A 1-D integer `Tensor` with shape `[self.nvals()]`.
The returned tensor is nonnegative, and is sorted in ascending order.
</td>
</tr>

</table>



<h3 id="with_dtype"><code>with_dtype</code></h3>

<a target="_blank" class="external" href="https://github.com/tensorflow/tensorflow/blob/v2.9.3/tensorflow/python/ops/ragged/row_partition.py#L963-L986">View source</a>

<pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link">
<code>with_dtype(
    dtype
)
</code></pre>

Returns a copy of this RowPartition with the given encoding dtype.


<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Args</th></tr>

<tr>
<td>
`dtype`
</td>
<td>
The dtype for encoding tensors, such as `row_splits` and `nrows`.
One of <a href="../../tf#int32"><code>tf.int32</code></a> or <a href="../../tf#int64"><code>tf.int64</code></a>.
</td>
</tr>
</table>



<!-- Tabular view -->
 <table class="responsive fixed orange">
<colgroup><col width="214px"><col></colgroup>
<tr><th colspan="2">Returns</th></tr>
<tr class="alt">
<td colspan="2">
A copy of this RowPartition, with the encoding tensors cast to the given
type.
</td>
</tr>

</table>