

GitHub - ShichenLiu/CondenseNet: CondenseNet: An Efficient DenseNet using Learne...
source link: https://github.com/ShichenLiu/CondenseNet
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
CondenseNets
This repository contains the code (in PyTorch) for "CondenseNet: An Efficient DenseNet using Learned Group Convolutions" paper by Gao Huang*, Shichen Liu*, Laurens van der Maaten and Kilian Weinberger (* Authors contributed equally).
Citation
If you find our project useful in your research, please consider citing:
@inproceedings{huang2018condensenet,
title={Condensenet: An efficient densenet using learned group convolutions},
author={Huang, Gao and Liu, Shichen and Van der Maaten, Laurens and Weinberger, Kilian Q},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={2752--2761},
year={2018}
}
Contents
Introduction
CondenseNet is a novel, computationally efficient convolutional network architecture. It combines dense connectivity between layers with a mechanism to remove unused connections. The dense connectivity facilitates feature re-use in the network, whereas learned group convolutions remove connections between layers for which this feature re-use is superfluous. At test time, our model can be implemented using standard grouped convolutions —- allowing for efficient computation in practice. Our experiments demonstrate that CondenseNets are much more efficient than other compact convolutional networks such as MobileNets and ShuffleNets.
Figure 1: Learned Group Convolution with G=C=3.
Figure 2: CondenseNets with Fully Dense Connectivity and Increasing Growth Rate.
Usage
Dependencies
Train
As an example, use the following command to train a CondenseNet on ImageNet
python main.py --model condensenet -b 256 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0,1,2,3,4,5,6,7 --resume
As another example, use the following command to train a CondenseNet on CIFAR-10
python main.py --model condensenet -b 64 -j 12 cifar10 \
--stages 14-14-14 --growth 8-16-32 --gpu 0 --resume
Evaluation
We take the ImageNet model trained above as an example.
To evaluate the trained model, use evaluate
to evaluate from the default checkpoint directory:
python main.py --model condensenet -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate
or use evaluate-from
to evaluate from an arbitrary path:
python main.py --model condensenet -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate-from /PATH/TO/BEST/MODEL
Note that these models are still the large models. To convert the model to group-convolution version as described in the paper, use the convert-from
function:
python main.py --model condensenet -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--convert-from /PATH/TO/BEST/MODEL
Finally, to directly load from a converted model (that is, a CondenseNet), use a converted model file in combination with the evaluate-from
option:
python main.py --model condensenet_converted -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate-from /PATH/TO/CONVERTED/MODEL
Other Options
We also include DenseNet implementation in this repository.
For more examples of usage, please refer to script.sh
For detailed options, please python main.py --help
Results
Results on ImageNet
Model | FLOPs | Params | Top-1 Err. | Top-5 Err. | Pytorch Model |
---|---|---|---|---|---|
CondenseNet-74 (C=G=4) | 529M | 4.8M | 26.2 | 8.3 | Download (18.69M) |
CondenseNet-74 (C=G=8) | 274M | 2.9M | 29.0 | 10.0 | Download (11.68M) |
Results on CIFAR
Model | FLOPs | Params | CIFAR-10 | CIFAR-100 |
---|---|---|---|---|
CondenseNet-50 | 28.6M | 0.22M | 6.22 | - |
CondenseNet-74 | 51.9M | 0.41M | 5.28 | - |
CondenseNet-86 | 65.8M | 0.52M | 5.06 | 23.64 |
CondenseNet-98 | 81.3M | 0.65M | 4.83 | - |
CondenseNet-110 | 98.2M | 0.79M | 4.63 | - |
CondenseNet-122 | 116.7M | 0.95M | 4.48 | - |
CondenseNet-182* | 513M | 4.2M | 3.76 | 18.47 |
(* trained 600 epochs)
Inference time on ARM platform
Model | FLOPs | Top-1 | Time(s) |
---|---|---|---|
VGG-16 | 15,300M | 28.5 | 354 |
ResNet-18 | 1,818M | 30.2 | 8.14 |
1.0 MobileNet-224 | 569M | 29.4 | 1.96 |
CondenseNet-74 (C=G=4) | 529M | 26.2 | 1.89 |
CondenseNet-74 (C=G=8) | 274M | 29.0 | 0.99 |
Contact
[email protected]
[email protected]
We are working on the implementation on other frameworks.
Any discussions or concerns are welcomed!
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK