The Metric Is the Message: Benchmarking Challenges for Neural Symbolic Regression
Abstract
The neural symbolic regression (NSR) literature has thus far been hindered by an over-reliance on individual, ad hoc evaluation metrics, producing seemingly favorable performance for the method using it but making comparison between methods difficult. Here we compare the performance of several NSR methods using diverse metrics reported in the literature, and some of our own devising. We show that reliance on a single metric can hide an NSR method’s shortcomings, causing performance rankings between methods to change as the evaluation metric changes. We further show that metrics which consider the structure of equations generated after training can help reveal these shortcomings, and suggest ways to correct for them. Given our results, we suggest best practices on what metrics to use to best advance this new field.
Cite
Text
Bertschinger et al. "The Metric Is the Message: Benchmarking Challenges for Neural Symbolic Regression." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023. doi:10.1007/978-3-031-43421-1_10Markdown
[Bertschinger et al. "The Metric Is the Message: Benchmarking Challenges for Neural Symbolic Regression." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023.](https://mlanthology.org/ecmlpkdd/2023/bertschinger2023ecmlpkdd-metric/) doi:10.1007/978-3-031-43421-1_10BibTeX
@inproceedings{bertschinger2023ecmlpkdd-metric,
title = {{The Metric Is the Message: Benchmarking Challenges for Neural Symbolic Regression}},
author = {Bertschinger, Amanda and Davis, Q. Tyrell and Bagrow, James P. and Bongard, Josh C.},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2023},
pages = {161-177},
doi = {10.1007/978-3-031-43421-1_10},
url = {https://mlanthology.org/ecmlpkdd/2023/bertschinger2023ecmlpkdd-metric/}
}