ML Anthology

On Analyzing the Role of Image for Visual-Enhanced Relation Extraction (Student Abstract)

Lei Li, Xiang Chen, Shuofei Qiao, Feiyu Xiong, Huajun Chen, Ningyu Zhang
AAAI 2023 pp. 16254-16255
doi:10.1609/AAAI.V37I13.26987 /aaai/2023/li2023aaai-analyzing/

Abstract

Multimodal relation extraction is an essential task for knowledge graph construction. In this paper, we take an in-depth empirical analysis that indicates the inaccurate information in the visual scene graph leads to poor modal alignment weights, further degrading performance. Moreover, the visual shuffle experiments illustrate that the current approaches may not take full advantage of visual information. Based on the above observation, we further propose a strong baseline with an implicit fine-grained multimodal alignment based on Transformer for multimodal relation extraction. Experimental results demonstrate the better performance of our method. Codes are available at https://github.com/zjunlp/DeepKE/tree/main/example/re/multimodal.

PDF AAAI Semantic Scholar

Cite

Text

Li et al. "On Analyzing the Role of Image for Visual-Enhanced Relation Extraction (Student Abstract)." AAAI Conference on Artificial Intelligence, 2023. doi:10.1609/AAAI.V37I13.26987

Markdown

[Li et al. "On Analyzing the Role of Image for Visual-Enhanced Relation Extraction (Student Abstract)." AAAI Conference on Artificial Intelligence, 2023.](https://mlanthology.org/aaai/2023/li2023aaai-analyzing/) doi:10.1609/AAAI.V37I13.26987

BibTeX

@inproceedings{li2023aaai-analyzing,
  title     = {{On Analyzing the Role of Image for Visual-Enhanced Relation Extraction (Student Abstract)}},
  author    = {Li, Lei and Chen, Xiang and Qiao, Shuofei and Xiong, Feiyu and Chen, Huajun and Zhang, Ningyu},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2023},
  pages     = {16254-16255},
  doi       = {10.1609/AAAI.V37I13.26987},
  url       = {https://mlanthology.org/aaai/2023/li2023aaai-analyzing/}
}