Disagreement Evaluation of Solutions for Math Word Problem

Abstract

It has been shown that the generation method works well for modeling the math word problem. To enhance models’ performance on math word problems, some studies employ multi-task learning, such as auxiliary tasks trained with the objective function of generation tasks. Previous work has contributed outstandingly to improving the model’s ability to discriminate solution errors by utilizing ranking tasks. However, these approaches only use the solution’s overall representation as the foundation for evaluation, preventing the model from distinguishing between right and wrong solutions. To address this deficiency, we propose a method called disagreement evaluation of solutions that involves training ranking tasks with disagreement points that are located using the longest prefix matching between correct and incorrect solutions. Next, this work employs a multi-stage approach to fine-tune the model incrementally to avoid the instability of joint optimization. Moreover, we explain the cooperation between ranking and generation tasks. Our experiments on the widely used Math23k and MAWPS datasets show that our method can achieve competitive results under low trainable parameters^1(Codes are available at https://github.com/vincent-hyx/DEoS/ .)