TSOSVNet: Teacher-Student Collaborative Knowledge Distillation for Online Signature Verification

Abstract

Online signature verification (OSV) is a standardized personal authentication scheme with wide social acceptance in critical real-time applications include access control, m-commerce, etc. Even though the current advances in Deep learning (DL) technologies catalysed state-of-the-art frameworks for challenging domains like computer vision, speech recognition, etc., the DL-based frameworks are voluminous with huge trainable parameters and are hard to deploy in real-time systems demanding faster inference. To adopt DL into OSV for improved performance, we propose an OSV framework made up of teacher-student collaborative knowledge distillation (TSKD) technique. A heavy Transformer based teacher is trained first and the teacher knowledge is distilled into a very lightweight Convolutional Neural Network (CNN) based student. A well trained teacher network results in an efficient deep representative feature learning by the student and results in a performance improvement. In a thorough set of experiments with three popular and standard datasets, i.e., the MCYT-100, SUSIG, and SVC, TSOSVNet framework, with a CNN based student model requiring only 3266 trainable parameters results in an EER of 12.42% compared to the recent SOTA 13.38% by a model with 206277 parameters in skilled_01 category of MCYT-100 dataset. In comparison to cutting-edge CNN-based OSV models, the proposed TSOSVNet produced a state-of-the-art EER in the most of the test categories with an average of 90% lesser trainable parameters.