Incremental Sense Weight Training for In-Depth Interpretation of Contextualized Word Embeddings (Student Abstract)

Xinyi Jiang, Zhengzhe Yang, Jinho D. Choi

AAAI 2020 pp. 13823-13824

doi:10.1609/AAAI.V34I10.7183 /aaai/2020/jiang2020aaai-incremental/

Abstract

We present a novel online algorithm that learns the essence of each dimension in word embeddings. We first mask dimensions determined unessential by our algorithm, apply the masked word embeddings to a word sense disambiguation task (WSD), and compare its performance against the one achieved by the original embeddings. Our results show that the masked word embeddings do not hurt the performance and can improve it by 3%.

PDF AAAI Semantic Scholar

Cite

Text

Jiang et al. "Incremental Sense Weight Training for In-Depth Interpretation of Contextualized Word Embeddings (Student Abstract)." AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/AAAI.V34I10.7183

Markdown

[Jiang et al. "Incremental Sense Weight Training for In-Depth Interpretation of Contextualized Word Embeddings (Student Abstract)." AAAI Conference on Artificial Intelligence, 2020.](https://mlanthology.org/aaai/2020/jiang2020aaai-incremental/) doi:10.1609/AAAI.V34I10.7183

BibTeX

@inproceedings{jiang2020aaai-incremental,
  title     = {{Incremental Sense Weight Training for In-Depth Interpretation of Contextualized Word Embeddings (Student Abstract)}},
  author    = {Jiang, Xinyi and Yang, Zhengzhe and Choi, Jinho D.},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2020},
  pages     = {13823-13824},
  doi       = {10.1609/AAAI.V34I10.7183},
  url       = {https://mlanthology.org/aaai/2020/jiang2020aaai-incremental/}
}