OTAS: An Elastic Transformer Serving System via Token Adaptation

Jinyu Chen, Wenchao Xu, Zicong Hong, Song Guo, Haozhao Wang, Jie Zhang, Deze Zeng
May 2024
PDF
Type
Conference paper
Publication
IEEE International Conference on Computer Communications (INFOCOM) (CCF-A)