ML Anthology

Community-Aware Graph Transformer: Preserving Community Semantics for Effective Global Aggregation

Yutai Duan, Jie Liu, Jianhua Wu, Jialin Liu
ECML-PKDD 2025 pp. 108-124
doi:10.1007/978-3-032-06066-2_7 /ecmlpkdd/2025/duan2025ecmlpkdd-communityaware/

Abstract

Graph Transformers (GTs) address the locality limitation of traditional GNNs, which aggregate only local neighbor information, by leveraging global attention. However, they suffer from two significant issues: neglecting community structures and information over-squeezing. In this paper, we first identify these two problems and propose a Community-Aware Graph Transformer (CoGT) to solve them. CoGT introduces a novel node-community-global hierarchical aggregation framework. This design preserves community-level semantics while reducing the volume of aggregated information, alleviating the over-squeezing problem. CoGT first employs a two-stage positional encoding to identify latent communities and enhance semantic consistency. Then, a hierarchical and parallel transformer computation method based on community representations facilitates global information interaction. Furthermore, we enable community-wise parallel attention computation, improving computational efficiency. Experimental results demonstrate that CoGT outperforms existing methods across multiple real-world datasets.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Duan et al. "Community-Aware Graph Transformer: Preserving Community Semantics for Effective Global Aggregation." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2025. doi:10.1007/978-3-032-06066-2_7

Markdown

[Duan et al. "Community-Aware Graph Transformer: Preserving Community Semantics for Effective Global Aggregation." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2025.](https://mlanthology.org/ecmlpkdd/2025/duan2025ecmlpkdd-communityaware/) doi:10.1007/978-3-032-06066-2_7

BibTeX

@inproceedings{duan2025ecmlpkdd-communityaware,
  title     = {{Community-Aware Graph Transformer: Preserving Community Semantics for Effective Global Aggregation}},
  author    = {Duan, Yutai and Liu, Jie and Wu, Jianhua and Liu, Jialin},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2025},
  pages     = {108-124},
  doi       = {10.1007/978-3-032-06066-2_7},
  url       = {https://mlanthology.org/ecmlpkdd/2025/duan2025ecmlpkdd-communityaware/}
}