tflite_support.metadata_writers.metadata_info.RegexTokenizerMd

A container for the Regex tokenizer 1 metadata information.

Main aliases

tflite_support.metadata_writers.audio_classifier.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.audio_classifier.metadata_writer.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.bert_nl_classifier.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.bert_nl_classifier.metadata_writer.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.image_classifier.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.image_classifier.metadata_writer.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.image_segmenter.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.image_segmenter.metadata_writer.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.nl_classifier.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.nl_classifier.metadata_writer.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.object_detector.metadata_info.RegexTokenizerMd, tflite_support.metadata_writers.object_detector.metadata_writer.metadata_info.RegexTokenizerMd

delim_regex_pattern the regular expression to segment strings and create tokens.
vocab_file_path path to the vocabulary file.

Methods

create_metadata

View source

Creates the Bert tokenizer metadata based on the information.

Returns
A Flatbuffers Python object of the Bert tokenizer metadata.