From Crayons to Code: AI-Driven Insights into a Child's Mental Health Through Drawings

Abstract

Children's mental health is crucial for their development, but it's often overlooked, leading to psychological issues. Many children struggle to express their thoughts and feelings effectively. To address this issue, we have proposed a novel approach to analyze children's drawings for psychological screening using artificial intelligence. Specifically, we're focusing on the `draw a person' (DAP) test, where a child's drawing is used to identify potential indicators of their mental and emotional state. Thus, we are introducing an AI-powered technique to automate the psychological screening process for children using the DAP test, which a human professional would traditionally conduct. The screening tool would suggest whether the child needs or doesn’t need further psychological referral. We have collected a dataset consisting of children's drawings and labeled them by experts as either `need' or `no need', indicating whether the child needs or does not need a referral. We have proposed two alternative approaches for the screening process. The first approach consists of extracting features from the drawings following expert guidelines and training a classification model using the features to classify the drawing as either `need' or `no need'. We also propose an out-of-the-box technique applying prompt engineering on state-of-the-art LLMs to automatically extract features from the images. The second approach involves training an image classification model using the drawings. Both approaches are challenged by the issue of class imbalance, as most of the drawings correspond to the `no-need' class. To address this challenge, we introduce Siamese++, a novel Siamese network for image classification, which uses feature embedding and an adaptive distance threshold for classification, instead of the nearest neighbor classification employed by traditional Siamese. Our proposed method achieves a high F1 score (up to 88%) even with a large class imbalance and without the need for any image augmentation. Thus, we have proposed an innovative interdisciplinary integration of AI with psychology and developed novel techniques to solve the real-world problem of psychological screening.