ML Anthology

MoDE: A Mixture-of-Experts Model with Mutual Distillation Among the Experts

Zhitian Xie, Yinger Zhang, Chenyi Zhuang, Qitao Shi, Zhining Liu, Jinjie Gu, Guannan Zhang
AAAI 2024 pp. 16067-16075
doi:10.1609/AAAI.V38I14.29539 /aaai/2024/xie2024aaai-mode/
AAAI Semantic Scholar

Cite

Text

Xie et al. "MoDE: A Mixture-of-Experts Model with Mutual Distillation Among the Experts." AAAI Conference on Artificial Intelligence, 2024. doi:10.1609/AAAI.V38I14.29539

Markdown

[Xie et al. "MoDE: A Mixture-of-Experts Model with Mutual Distillation Among the Experts." AAAI Conference on Artificial Intelligence, 2024.](https://mlanthology.org/aaai/2024/xie2024aaai-mode/) doi:10.1609/AAAI.V38I14.29539

BibTeX

@inproceedings{xie2024aaai-mode,
  title     = {{MoDE: A Mixture-of-Experts Model with Mutual Distillation Among the Experts}},
  author    = {Xie, Zhitian and Zhang, Yinger and Zhuang, Chenyi and Shi, Qitao and Liu, Zhining and Gu, Jinjie and Zhang, Guannan},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2024},
  pages     = {16067-16075},
  doi       = {10.1609/AAAI.V38I14.29539},
  url       = {https://mlanthology.org/aaai/2024/xie2024aaai-mode/}
}