A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science
Abstract
This paper explores the use of large language models (LLMs) to score and explain short-answer assessments in K-12 science. While existing methods can score more structured math and computer science assessments, they often do not provide explanations for the scores. Our study focuses on employing GPT-4 for automated assessment in middle school Earth Science, combining few-shot and active learning with chain-of-thought reasoning. Using a human-in-the-loop approach, we successfully score and provide meaningful explanations for formative assessment responses. A systematic analysis of our method's pros and cons sheds light on the potential for human-in-the-loop techniques to enhance automated grading for open-ended science assessments.
Cite
Text
Cohn et al. "A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science." AAAI Conference on Artificial Intelligence, 2024. doi:10.1609/AAAI.V38I21.30364Markdown
[Cohn et al. "A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science." AAAI Conference on Artificial Intelligence, 2024.](https://mlanthology.org/aaai/2024/cohn2024aaai-chain/) doi:10.1609/AAAI.V38I21.30364BibTeX
@inproceedings{cohn2024aaai-chain,
title = {{A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science}},
author = {Cohn, Clayton and Hutchins, Nicole and Le, Tuan and Biswas, Gautam},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2024},
pages = {23182-23190},
doi = {10.1609/AAAI.V38I21.30364},
url = {https://mlanthology.org/aaai/2024/cohn2024aaai-chain/}
}