tf.keras.preprocessing.text.one_hot

One-hot encodes a text into a list of word indexes of size n.

tf.keras.preprocessing.text.one_hot(
    input_text,
    n,
    filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
    lower=True,
    split=' ',
    analyzer=None
)

This function receives as input a string of text and returns a list of encoded integers each corresponding to a word (or token) in the given input string.

input_text Input text (string).
n int. Size of vocabulary.
filters list (or concatenation) of characters to filter out, such as punctuation. Default: '!"#$%&()*+,-./:;<=>?@[\]^_`{|}~\t\n, includes basic punctuation, tabs, and newlines.
lower boolean. Whether to set the text to lowercase.
split str. Separator for word splitting.
analyzer function. Custom analyzer to split the text

List of integers in [1, n]. Each integer encodes a word (unicity non-guaranteed).