README.md

Fast and Accurate Online Video Object Segmentation via Tracking Parts (FAVOS)

Contact: Jingchun Cheng (chengjingchun14 at 163 dot com)

Paper

Fast and Accurate Online Video Object Segmentation via Tracking Parts
Jingchun Cheng, Yi-Hsuan Tsai, Wei-Chih Hung, Shengjin Wang and Ming-Hsuan Yang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018 (Spotlight)

This is the authors' demo (single-GPU-version) code for the DAVIS 2016 dataset as described in the above paper. Please cite our paper if you find it useful for your research.

@article{Cheng_favos_2018,
  author = {J. Cheng and Y.-H. Tsai and W.-C. Hung and S. Wang and M.-H. Yang},
  journal = {arXiv preprint arXiv:1806.02323},
  title = {Fast and Accurate Online Video Object Segmentation via Tracking Parts},
  year = {2018}
}

FAVOS results

Segmentation Comparisons with Fast Online Methods

Example Video of Part Tracking

Requirements

• caffe (pycaffe)
• opencv
• matlab
• A GPU with at least 12GB memory

Download DAVIS 2016 dataset, trained models, tracked parts and pre-computed results

sh download_all.sh

Test our model

We provide an example testing script test_davis16.sh.

# Please run download_all.sh first
# Usage: sh test_davis16.sh <GPU-id> <sequence-name>

sh test_davis16.sh 0 blackswan

The results would be saved in results-demo/res_favos/<sequence-name>. You can replace the sequence name with others in the DAVIS 2016 validation set to obatin results for other videos.

Train your own ROISegNet

Download ResNet-101 model and save it in the folder "models" as "init.caffemodel"

cd ROISegNet
python solve.py ../models/init.caffemodel solver_davis16.prototxt 0

Tracker

We use the SiaFC tracker in Fully-Convolutional Siamese Networks for Object Tracking.
The pre-computed parts and tracking results on DAVIS 2016 can be downloaded here.

Note that, we are currently working on a stable version to combine part tracking and ROISegNet for practical usage on any videos. We will update the code in a near future.

Download our segmentation results on the DAVIS datasets

• FAVOS on DAVIS2016 link
• FAVOS on DAVIS2017 link

Note

The models and code are available for non-commercial research purposes only.

06/2018: demo code released