知否？知否？一文看懂深度文本分类之DPCNN原理与代码

DPCNN结构

究竟是多么牛逼的网络呢？我们下面来窥探一下模型的芳容。

DPCNN结构细节

上述介绍了DPCNN的整体架构，可见DPCNN的架构之精美。本文是在原始论文以及知乎上的一篇文章的基础上进行整理。本文可能也会有很多错误，如果有错误，欢迎大家指出来！建议大家为了更好的理解DPCNN ，看一下原始论文和参考里面的知乎。

用Keras实现DPCNN网络

这里参考了一下kaggle的代码，模型一共用了七层，模型的参数与论文不太相同。这里滤波器通道个数为64(论文中为256)，具体的参数可以参考下面的代码，部分我写了注释。

def CNN(x):
    block = Conv1D(filter_nr, kernel_size=filter_size, padding=same, activation=linear,
                kernel_regularizer=conv_kern_reg, bias_regularizer=conv_bias_reg)(x)
    block = BatchNormalization()(block)
    block = PReLU()(block)
    block = Conv1D(filter_nr, kernel_size=filter_size, padding=same, activation=linear,
                kernel_regularizer=conv_kern_reg, bias_regularizer=conv_bias_reg)(block)
    block = BatchNormalization()(block)
    block = PReLU()(block)
    return block

def DPCNN():
    filter_nr = 64 #滤波器通道个数
    filter_size = 3 #卷积核
    max_pool_size = 3 #池化层的pooling_size
    max_pool_strides = 2 #池化层的步长
    dense_nr = 256 #全连接层
    spatial_dropout = 0.2
    dense_dropout = 0.5
    train_embed = False
    conv_kern_reg = regularizers.l2(0.00001)
    conv_bias_reg = regularizers.l2(0.00001)

    comment = Input(shape=(maxlen,))
    emb_comment = Embedding(max_features, embed_size, weights=[embedding_matrix], trainable=train_embed)(comment)
    emb_comment = SpatialDropout1D(spatial_dropout)(emb_comment)

    #region embedding层
    resize_emb = Conv1D(filter_nr, kernel_size=1, padding=same, activation=linear,
                kernel_regularizer=conv_kern_reg, bias_regularizer=conv_bias_reg)(emb_comment)
    resize_emb = PReLU()(resize_emb)
    #第一层
    block1 = CNN(emb_comment)
    block1_output = add([block1, resize_emb])
    block1_output = MaxPooling1D(pool_size=max_pool_size, strides=max_pool_strides)(block1_output)
    #第二层
    block2 = CNN(block1_output)
    block2_output = add([block2, block1_output])
    block2_output = MaxPooling1D(pool_size=max_pool_size, strides=max_pool_strides)(block2_output)
    #第三层
    block3 = CNN(block2_output)
    block3_output = add([block3, block2_output])
    block3_output = MaxPooling1D(pool_size=max_pool_size, strides=max_pool_strides)(block3_output)  
    #第四层
    block4 = CNN(block3_output)
    block4_output = add([block4, block3_output])
    block4_output = MaxPooling1D(pool_size=max_pool_size, strides=max_pool_strides)(block4_output)
    #第五层
    block5 = CNN(block4_output)
    block5_output = add([block5, block4_output])
    block5_output = MaxPooling1D(pool_size=max_pool_size, strides=max_pool_strides)(block5_output)
    #第六层
    block6 = CNN(block5_output)
    block6_output = add([block6, block5_output])
    block6_output = MaxPooling1D(pool_size=max_pool_size, strides=max_pool_strides)(block6_output)
    #第七层
    block7 = CNN(block6_output)
    block7_output = add([block7, block6_output])
    output = GlobalMaxPooling1D()(block7_output)
    #全连接层
    output = Dense(dense_nr, activation=linear)(output)
    output = BatchNormalization()(output)
    output = PReLU()(output)
    output = Dropout(dense_dropout)(output)
    output = Dense(6, activation=sigmoid)(output)

    model = Model(comment, output)
    model.summary()
    model.compile(loss=binary_crossentropy,
                optimizer=optimizers.Adam(),
                metrics=[accuracy])
    return model

DPCNN实战

上面我们用keras实现了我们的DPCNN网络，这里我们借助kaggle的有毒评论文本分类竞赛来实战下我们的DPCNN网络。

具体地代码，大家可以去我的GitHub上面找到源码 ： 

https://github.com/hecongqing/TextClassification/blob/master/DPCNN.ipynb

参考：

https://ai.tencent.com/ailab/media/publications/ACL3-Brady.pdf

https://zhuanlan.zhihu.com/p/35457093

https://www.kaggle.com/michaelsnell/conv1d-dpcnn-in-keras