In this tutorial you'll install and run a React Native example app that uses a TensorFlow pose detection model (MoveNet.SinglePose.Lightning) to do real-time pose detection. Built on the TensorFlow.js platform adapter for React Native, the app supports both portrait and landscape modes with the front and back cameras.
Prerequisites
To complete this tutorial, you need the following installed in your development environment:
The TensorFlow.js React Native platform adapter depends on expo-gl and expo-gl-cpp, so you must use a version of React Native that's supported by Expo.
Install and run the example app
- Clone or download the
tfjs-examples
repository. Change into the
react-native/pose-detection
directory:cd tfjs-examples/react-native/pose-detection
Install dependencies:
yarn
Run the example app locally:
yarn start
When you run the app, the terminal displays a QR code.
Starting Metro Bundler
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
█ ▄▄▄▄▄ █▄▀ ▀ ███ ▄▄▄▄▄ █
█ █ █ █ █▀ █ ▀█ █ █ █
█ █▄▄▄█ █▄█▀ ▄▀█▄▀█ █▄▄▄█ █
█▄▄▄▄▄▄▄█▄█ █ █▄▀ █▄▄▄▄▄▄▄█
█▄ ▄▀█▄█ █ ▄ ▀▀▄▄▄██▄ ▄▀▄█
██▄▀▀█▀▄█▀ ▄▄▀ █▀█ ▀██▀███
█▄▄▀ ▀▀▄▄▄ ▄▀ ▄█ ▄█ ▄ █ █▀█
█▀▄▄ ▄▄▄▀ ▄ █▄██ ▀▀█▀▀█ ▀█
███▄▄██▄█▀▄██▄ ▄ ▄▄▄ ▀▄█▀█
█ ▄▄▄▄▄ ██ ▀██▀█ █▄█ █▄▄ █
█ █ █ █▀█ ███▀▀▄▄ █▀ ▀█
█ █▄▄▄█ █ ▀▀▀▀▀▄▀▄▀▄█▄▄ ▄██
█▄▄▄▄▄▄▄█▄██▄▄██▄██████▄▄▄█
› Metro waiting on exp://192.168.0.6:19000
Scan the QR code with a phone or other test device that has Expo Go installed. The example app should open in Expo Go.
The screenshots below show the app detecting and rendering keypoints of the user's body.
To understand how the TensorFlow.js libraries are used in the example, look at App.tsx. The comments in that file explain how to configure tensor size, load the model, run pose detection, and more.
To learn more about pose detection using TensorFlow.js, see this blog post.
More about the Reactive Native platform adapter
The TensorFlow.js platform adapter for React Native provides GPU-accelerated execution of TensorFlow.js and supports all major modes of TensorFlow.js usage:
- Support for both model inference and training
- GPU support with WebGL via expo-gl
- Support for loading pretrained models from the web
IOHandler
s to support loading models from async storage and models that are compiled into the app bundle
For full installation instructions, see the React Native platform adapter README.
Troubleshoot TensorFlow.js for React Native
If your app crashes on startup, you might need to alter your setup. To learn more about setting up the platform adapter for React Native, see the README.