Unmasking Deception: A Topic-Oriented Multimodal Approach to Uncover False Information on Social Media

Abstract

In the digital landscape, social media has emerged as a prevalent channel for global communication, connecting like-minded individuals worldwide. However, while facilitating information exchange, it is also susceptible to the dissemination of false information, posing a constant challenge to the reliability of online content. To address this issue, this paper introduces a novel methodology called TM-FID (Topic-oriented Multimodal False Information Detection), which combines false information detection and neural topic modeling within a semi-supervised multimodal approach. By jointly leveraging textual and visual information contained in online news, our approach provides insights into how false information influences specific discussion topics, thus enabling a comprehensive and fine-grained understanding of its spread and impact on social media conversation. Experimental evaluation carried out on a set of multimodal gossip-related news demonstrates the quality of the identified topics, assessed through a novel centroid-based metric, as well as the efficacy of the cross-attention mechanism used within TM-FID to accurately identify false information in multimodal news. Overall, the proposed methodology can enable effective strategies to counter the spread of false information, thereby fostering trust and confidence in the information shared on social media platforms.