Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model (Student Abstract)
Abstract
Consider a natural language sentence describing a specific step in a food recipe. In such instructions, recognizing actions (such as press, bake, etc.) and the resulting changes in the state of the ingredients (shape molded, custard cooked, temperature hot, etc.) is a challenging task. One way to cope with this challenge is to explicitly model a simulator module that applies actions to entities and predicts the resulting outcome (Bosselut et al. 2018). However, such a model can be unnecessarily complex. In this paper, we propose a simplified neural network model that separates action recognition and state change prediction, while coupling the two through a novel loss function. This allows learning to indirectly influence each other. Our model, although simpler, achieves higher state change prediction performance (67% average accuracy for ours vs. 55% in (Bosselut et al. 2018)) and takes fewer samples to train (10K ours vs. 65K+ by (Bosselut et al. 2018)).
Cite
Text
Wan and Choe. "Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model (Student Abstract)." AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/AAAI.V34I10.7245Markdown
[Wan and Choe. "Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model (Student Abstract)." AAAI Conference on Artificial Intelligence, 2020.](https://mlanthology.org/aaai/2020/wan2020aaai-action/) doi:10.1609/AAAI.V34I10.7245BibTeX
@inproceedings{wan2020aaai-action,
title = {{Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model (Student Abstract)}},
author = {Wan, Qing and Choe, Yoonsuck},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2020},
pages = {13945-13946},
doi = {10.1609/AAAI.V34I10.7245},
url = {https://mlanthology.org/aaai/2020/wan2020aaai-action/}
}