README.md

OpenPose-Plus: Fast and Flexible OpenPose Framework based on TensorFlow and TensorLayer

Motivation

OpenPose is the state-of-the-art real-time 2D pose estimation algorithm. In the official Caffe-based codebase, data processing, training, and neural network blocks are heavily interleaved and mostly hard-coded. This makes it difficult to be customised for achieving the best performance in our custom pose estimation applications. Hence, we develop OpenPose-Plus, a fast and flexible pose estimation framework that offers the following powerful features:

• Flexible combination of standard training dataset with your own custom labelled data.
• Customisable data augmentation pipeline without compromising performance
• Deployment on embedded platforms using TensorRT
• Switchable neural networks (e.g., changing VGG to MobileNet for minimal memory consumption)
• Integrated training on a single GPU and multiple GPUs

Work in progress

This project is still under active development, some of the TODOs are as follows:

• Distributed training
• Pose Proposal Networks, ECCV 2018

Key project files

• config.py : config of the training details.
• train.py: trains the model.

Preparation

Build C++ library for post processing. See instruction

cd inference/pafprocess
make

# ** before recompiling **
rm -rf build
rm *.so

Training your model

Training the model is implemented using TensorFlow. To run train.py, you would need to install packages, shown in requirements.txt, in your virtual environment (Python <=3.6):

pip install -r requirements.txt
pip install pycocotools

train.py will automatically download MSCOCO 2017 dataset into dataset/coco17. The default model in models.py is based on VGG19, which is the same with the original paper. If you want to customize the model, simply change it in models.py. And then train.py will train the model to the end.

In config.py, config.DATA.train_data can be:

• coco: training data is COCO dataset only (default)
• custom: training data is your dataset specified by config.DATA.your_xxx
• coco_and_custom: training data is COCO and your dataset

config.MODEL.name can be:

• vgg: VGG19 version (default), slow
• vggtiny: VGG tiny version, faster
• mobilenet: MobileNet version, faster

config.TRAIN.train_mode can be:

• local: single GPU training
• distributed: multiple GPU training (on-going work)

5. Inference

Currently we provide two C++ APIs for inference, both defined in include/openpose-plus.hpp. They are for running the tensorflow model with tensorRT and post-processing respectively.

You can look the examples in the examples folder to see how to use the APIs. Running ./scripts/live-camera.sh will give you a quick review of how it works.

You can build the APIs into a standard C++ library by just running make pack, provided that you have the following dependencies installed

• tensorRT
• opencv

Discussion

• TensorLayer Slack
• TensorLayer WeChat
• TensorLayer Issues 434
• TensorLayer Issues 416