Incorporating LLM Priors into Tabular Learners
Abstract
We present a method to integrate Large Language Models (LLMs) and traditional tabular data classification techniques, addressing LLMs’ challenges like data serialization sensitivity and biases. We introduce two strategies utilizing LLMs for ranking categorical variables and generating priors on correlations between continuous variables and targets, enhancing performance in few-shot scenarios. We focus on Logistic Regression, introducing MonotonicLR that employs a non-linear monotonic function for mapping ordinals to cardinals while preserving LLM-determined orders. Validation against baseline models reveals the superior performance of our approach, especially in low-data scenarios, while remaining interpretable.
Cite
Text
Zhu et al. "Incorporating LLM Priors into Tabular Learners." NeurIPS 2023 Workshops: TRL, 2023.Markdown
[Zhu et al. "Incorporating LLM Priors into Tabular Learners." NeurIPS 2023 Workshops: TRL, 2023.](https://mlanthology.org/neuripsw/2023/zhu2023neuripsw-incorporating/)BibTeX
@inproceedings{zhu2023neuripsw-incorporating,
title = {{Incorporating LLM Priors into Tabular Learners}},
author = {Zhu, Max and Stanivuk, Siniša and Petrovic, Andrija and Nikolic, Mladen and Lio, Pietro},
booktitle = {NeurIPS 2023 Workshops: TRL},
year = {2023},
url = {https://mlanthology.org/neuripsw/2023/zhu2023neuripsw-incorporating/}
}