MADia: A Multimodal Machine Learning Model for ASD Diagnosis
MADia is a multimodal diagnostic model for the binary classification of ASD, integrating both Magnetic Resonance Imaging (MRI) geometric structure and phenotypic data from patients for a more accurate diagnosis. I was responsible for the machine learning aspect of this project, and my contributions are explained below.
Data Preprocessing
A total of 58 participants’ data collected by the Barrow Neurological Institute was used to train our model. Data included 3D anatomical MRI scans and phenotypic data. Participants were aged 18-64 and samples contained typical controls as well as those diagnosed with ASD. Scales used in the measurement of phenotypic data include the Kaufman Brief IntelligenceTest-2 (KBIT-2) for IQ, Social Responsiveness Scale-2nd Edition (SRS-2), and the Beck Depression Inventory-II (BDS-II). This data was made publicly available through the Autism Brain Imaging Data Exchange II (ABIDE II) initiative and is anonymous. All MRI images underwent skull stripping, a process that isolates the brain by removing non-brain tissue from the MRI in order to reduce noise during training. This was achieved using SynthStrip, a tool that performs this process by extracting brain voxels from the image. An executable was created and run over all MRI data to produce the resulting preprocessed MRIs. The MRI dataset was then converted to Pytorch tensors and stored in a single .pt file as the suitable input format for fusilli (more on this later).
All columns that did not contain sufficient data for either group (typical controls and ASD) were removed; 20 columns were used in the final model. Rows with blank values were dropped, and the corresponding MRI scan was removed from the training set. A total of 42 participant data remained at the end of this process. Fusilli, a Python library used to fuse multimodal data together, was used. All fusion models in the library capable of taking both image and tabular data as inputs were considered. A total of eight models were trained, excluding two unimodal models trained to provide a baseline for the assessment of multimodal model performance. The dataset had an 80-20 train-test split. K-fold cross-validation was used, with the number of folds set to k=3.
Results
Of the eight multimodal models trained on our image and tabular data, the crossmodal multi-head attention achieved the highest performance, with an auroc score of 85.79% and an accuracy of 83.33%.
The crossmodal multi-head attention model performed the best out of the eight models trained with the data. In this model, the tabular data is passed through a fully-connected neural network layer and the MRI data is passed through a Convolutional Neural Network (CNN). After, each modality undergoes self-attention followed by crossmodal attention. This is then concatenated and put through another fully connected neural network layer in order to produce the final prediction on a patient’s ASD diagnosis.
Our project was presented at the Inter-University Big Data Challenge in 2024. Our poster is attached below.