Spatial-Slice Feature Learning Using Visual Transformer and Essential Slices Selection Module for COVID-19 Detection of CT Scans in the Wild

Abstract

Computed tomography (CT) imaging could be convenient for diagnosing various diseases. However, the CT images could be diverse since their resolution and number of slices are determined by the machine and its settings. Conventional deep learning models are hard to tickle such diverse data since the essential requirement of the deep neural network is the consistent shape of the input data in each dimension. A way to overcome this issue is based on the slice-level classifier and aggregating the predictions for each slice to make the final result. However, it lacks slice-wise feature learning, leading to suppressed performance. This paper proposes an effective spatial-slice feature learning (SSFL) to tickle this issue for COVID-19 symptom classification. First, the semantic feature embedding of each slice for a CT scan is extracted by a conventional 2D convolutional neural network (CNN) and followed by using the visual Transformer-based sub-network to deal with feature learning between slices, leading to joint feature representation. Then, an essential slices set algorithm is proposed to automatically select a subset of the CT scan, which could effectively remove the uncertain slices as well as improve the performance of our SSFL. Comprehensive experiments reveal that the proposed SSFL method shows not only excellent performance but also achieves stable detection results.