Performs classification on images.
The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..
The API supports models with one image input tensor and one classification output tensor. To be more specific, here are the requirements.
- Input image tensor (
kTfLiteUInt8
/kTfLiteFloat32
)- image input of size
[batch x height x width x channels]
. - batch inference is not supported (
batch
is required to be 1). - only RGB inputs are supported (
channels
is required to be 3). - if type is
kTfLiteFloat32
, NormalizationOptions are required to be attached to the metadata for input normalization.
- image input of size
kTfLiteUInt8
/kTfLiteFloat32
)
- with
N
classes of either 2 or 4 dimensions, such as[1 x N]
or[1 x 1 x 1 x N]
- the label file is required to be packed to the metadata. See the example of creating metadata for an image classifier. If no label files are packed, it will use index as label in the result.
An example of such model can be found on TensorFlow Hub..
Nested Classes
class | ImageClassifier.ImageClassifierOptions | Options for setting up an ImageClassifier. |
Public Methods
Inherited Methods
Public Methods
public List<Classifications> classify (TensorImage image)
Performs actual classification on the provided TensorImage
.
ImageClassifier
supports the following TensorImage
color space types:
Parameters
image | a UINT8 TensorImage object that represents an RGB or YUV image |
---|
Throws
IllegalArgumentException | if the color space type of image is unsupported |
---|
public List<Classifications> classify (TensorImage image, ImageProcessingOptions options)
Performs actual classification on the provided TensorImage
with ImageProcessingOptions
.
ImageClassifier
supports the following options:
- Region of interest (ROI) (through
ImageProcessingOptions.Builder.setRoi(Rect)
). It defaults to the entire image. - image rotation (through
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
). It defaults toImageProcessingOptions.Orientation.TOP_LEFT
.
ImageClassifier
supports the following TensorImage
color space types:
Parameters
image | a UINT8 TensorImage object that represents an RGB or YUV image |
---|---|
options |
Throws
IllegalArgumentException | if the color space type of image is unsupported |
---|
public List<Classifications> classify (MlImage image)
Performs actual classification on the provided MlImage
.
Parameters
image | an MlImage object that represents an image |
---|
Throws
IllegalArgumentException | if the storage type or format of the image is unsupported |
---|
public List<Classifications> classify (MlImage image, ImageProcessingOptions options)
Performs actual classification on the provided MlImage
with ImageProcessingOptions
.
ImageClassifier
supports the following options:
- Region of interest (ROI) (through
ImageProcessingOptions.Builder.setRoi(Rect)
). It defaults to the entire image. - image rotation (through
ImageProcessingOptions.Builder.setOrientation(ImageProcessingOptions.Orientation)
). It defaults toImageProcessingOptions.Orientation.TOP_LEFT
.MlImage.getRotation()
is not effective.
Parameters
image | a MlImage object that represents an image |
---|---|
options | configures options including ROI and rotation |
Throws
IllegalArgumentException | if the storage type or format of the image is unsupported |
---|
public static ImageClassifier createFromBuffer (ByteBuffer modelBuffer)
Creates an ImageClassifier
instance with a model buffer and the default ImageClassifier.ImageClassifierOptions
.
Parameters
modelBuffer | a direct ByteBuffer or a MappedByteBuffer of the
classification model |
---|
Throws
IllegalArgumentException | if the model buffer is not a direct ByteBuffer or a
MappedByteBuffer |
---|---|
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageClassifier createFromBufferAndOptions (ByteBuffer modelBuffer, ImageClassifier.ImageClassifierOptions options)
Creates an ImageClassifier
instance with a model buffer and ImageClassifier.ImageClassifierOptions
.
Parameters
modelBuffer | a direct ByteBuffer or a MappedByteBuffer of the
classification model |
---|---|
options |
Throws
IllegalArgumentException | if the model buffer is not a direct ByteBuffer or a
MappedByteBuffer |
---|---|
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageClassifier createFromFile (Context context, String modelPath)
Creates an ImageClassifier
instance from the default ImageClassifier.ImageClassifierOptions
.
Parameters
context | |
---|---|
modelPath | path of the classification model with metadata in the assets |
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageClassifier createFromFile (File modelFile)
Creates an ImageClassifier
instance from the default ImageClassifier.ImageClassifierOptions
.
Parameters
modelFile | the classification model File instance |
---|
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageClassifier createFromFileAndOptions (Context context, String modelPath, ImageClassifier.ImageClassifierOptions options)
Creates an ImageClassifier
instance from ImageClassifier.ImageClassifierOptions
.
Parameters
context | |
---|---|
modelPath | path of the classification model with metadata in the assets |
options |
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |
public static ImageClassifier createFromFileAndOptions (File modelFile, ImageClassifier.ImageClassifierOptions options)
Creates an ImageClassifier
instance.
Parameters
modelFile | the classification model File instance |
---|---|
options |
Throws
IOException | if an I/O error occurs when loading the tflite model |
---|---|
IllegalArgumentException | if an argument is invalid |
IllegalStateException | if there is an internal error |
RuntimeException | if there is an otherwise unspecified error |