Convolutional Neural Networks with Recurrent Neural Filters

Author: Yi Yang

Contact: [email protected]

Basic description

This is the Python implementation of the recurrent neural filters for convolutional neural networks, described in

Yi Yang
"Convolutional Neural Networks with Recurrent Neural Filters"
EMNLP 2018

[pdf]

BibTeX

@inproceedings{yang2018convolutional,
  title={Convolutional Neural Networks with Recurrent Neural Filters},
  author={Yang, Yi},
  booktitle={Proceedings of Empirical Methods in Natural Language Processing},
  year={2018}
}

Dependencies

1. TensorFlow
2. Keras
3. Optional: CUDA Toolkit for GPU programming.

We use the Stanford Sentiment Treebank (SST) datasets processed by Lei et al. (2015). Please put all the files of this directory into the data/sst_text_convnet folder.

Please download the pre-trained GloVe vectors and unzip it into the data folder.

Results

Running the code requires two steps:

1. Prepare the data and generate the required data files

   # binary sentiment classification
   python proc_data.py data/stsa.binary.pkl
   
   # fine-grained sentiment classification
   python proc_data.py --train-path data/sst_text_convnet/stsa.fine.phrases.train \
                       --dev-path   data/sst_text_convnet/stsa.fine.dev \
                       --test-path  data/sst_text_convnet/stsa.fine.test \
                       data/stsa.fine.pkl
   

2. CNNs for sentiment classification with linear filters and recurrent neural filters (RNFs)

   # binary sentiment classification
   python cnn_keras.py --filter-type linear data/stsa.binary.pkl
   python cnn_keras.py --filter-type rnf data/stsa.binary.pkl
   
   # fine-grained sentiment classification
   python cnn_keras.py --filter-type linear data/stsa.fine.pkl
   python cnn_keras.py --filter-type rnf data/stsa.fine.pkl
   

Hyperparameter tunning may be needed to achive the best results reported in the paper.

Unfortunately, I failed to find out how to entirely eliminate randomness for training Keras-based models. However, you should be easily able to achieve 89%+ and 52%+ accuracies with RNFs after a few runs.

Recurrent neural filters consistently outperform linear filters across different filter widths, by 3-4% accuracy.