Multi-Modal Foundation Model for Material Design
Abstract
We propose a multi-modal foundation model for small molecules, a shift from traditional AI models that are tailored for individual tasks and modalities. This model uses a late fusion strategy to align and fuse three distinct modalities: SELFIES, DFT properties, and optical spectrum. The model is pre-trained with over 6 billion samples to provide two primary functions, generating fused feature representations across the three modalities, and cross-modal predictions and genrations. As preliminary experiments, we demonstrate that the fused representation successfully improves the performance of property predictions for chromophore molecules, and showcase 6 distinct cross-modal inferences.
Cite
Text
Takeda et al. "Multi-Modal Foundation Model for Material Design." NeurIPS 2023 Workshops: AI4Mat, 2023.Markdown
[Takeda et al. "Multi-Modal Foundation Model for Material Design." NeurIPS 2023 Workshops: AI4Mat, 2023.](https://mlanthology.org/neuripsw/2023/takeda2023neuripsw-multimodal/)BibTeX
@inproceedings{takeda2023neuripsw-multimodal,
title = {{Multi-Modal Foundation Model for Material Design}},
author = {Takeda, Seiji and Priyadarsini, Indra and Kishimoto, Akihiro and Shinohara, Hajime and Hamada, Lisa and Masataka, Hirose and Fuchiwaki, Junta and Nakano, Daiju},
booktitle = {NeurIPS 2023 Workshops: AI4Mat},
year = {2023},
url = {https://mlanthology.org/neuripsw/2023/takeda2023neuripsw-multimodal/}
}