ML Anthology

Using Next Sentence Prediction to Test ChatGPT's Text Comprehension (Student Abstract)

Ojas M. Agarwal, Madelein Villegas, Jack Mostow
AAAI 2025 pp. 29322-29324
doi:10.1609/AAAI.V39I28.35238 /aaai/2025/agarwal2025aaai-using/

Abstract

We propose the Next Sentence Prediction (NSP) task as a simple, objective, scalable, automated way to test ChatGPT’s text comprehension. Given a context excerpted from a children’s story, the task is to distinguish the next story sentence from a later sentence in the story. We analyze how ChatGPT’s performance on this task is related to various features of the text, using data from English and Swahili children’s stories.

AAAI Semantic Scholar

Cite

Text

Agarwal et al. "Using Next Sentence Prediction to Test ChatGPT's Text Comprehension (Student Abstract)." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I28.35238

Markdown

[Agarwal et al. "Using Next Sentence Prediction to Test ChatGPT's Text Comprehension (Student Abstract)." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/agarwal2025aaai-using/) doi:10.1609/AAAI.V39I28.35238

BibTeX

@inproceedings{agarwal2025aaai-using,
  title     = {{Using Next Sentence Prediction to Test ChatGPT's Text Comprehension (Student Abstract)}},
  author    = {Agarwal, Ojas M. and Villegas, Madelein and Mostow, Jack},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2025},
  pages     = {29322-29324},
  doi       = {10.1609/AAAI.V39I28.35238},
  url       = {https://mlanthology.org/aaai/2025/agarwal2025aaai-using/}
}