4

GitHub - wilile26811249/Fastformer-PyTorch: Unofficial PyTorch implementation of...

 2 years ago
source link: https://github.com/wilile26811249/Fastformer-PyTorch
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Fastformer-PyTorch

Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need.

Usage :

import torch
import Fastformer

model = Fastformer(dim = 3, decode_dim = 8)
x = torch.randn(4, 6, 3)
mask = torch.ones(1, 8).bool()
result = model(x, mask)
print(result.size())

Citation :

@misc{wu2021fastformer,
    title={Fastformer: Additive Attention Can Be All You Need},
    author={Chuhan Wu, Fangzhao Wu, Tao Qi and Yongfeng Huang},
    year={2021},
    eprint={2108.09084v2},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

If this implement have any problem please let me know, thank you.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK