Needle in a Haystack
For this project, I co-authored a peer-reviewed research paper that looked at whether we can use machine learning to identify existing drugs that could be useful for treating rare diseases that don’t currently have a treatment. My contributions focused on building the machine learning models and writing the code behind our project. I’m going to summarize the project below, but if you want to read the full paper you can find it here:
Introduction
Rare diseases are defined as conditions that affect a small percentage of the population, where specific definitions vary by geography, from 1 in 2000 in Europe to 1 in 200,000 individuals in the US [1]. Due to the lack of disease profiling, 95% of rare diseases do not have curative treatments, and instead, existing therapeutics often target symptoms or the overall quality of life of patients [2]. This study will investigate the use of a computational model to predict the mechanisms of action (MoA) for existing therapeutics, which in tandem with disease genetic information, is capable of generating candidates for drug repurposing. While many studies have already explored in silico drug repurposing for a variety of conditions, few have investigated rare diseases, largely in part to their limited profiling [3]. Correspondingly, this study will take a drug-structure lens to predict MoAs, defined as a compound’s specific biological activity [4]. While a method contrary to existing literature, most of which employing target-based strategies, MoA-centric approaches have several advantages in discerning repurposing candidates, including:
- Higher precision in candidate selection, where the greater depth of complexity allows for a reduction in false positive rate [5].
- Early identification of MoA within the R&D pipeline can facilitate efficient progression, enabling early exploration of possible drug combinations, side effects, and pathways [6].
The objectives of this study are:
- Build ML models where compound structural and chemical relations facilitate the identification of MoA-compound interactions.
- Analyze model performance to identify top- performing algorithms capable of accurately predicting drug repurposing candidates.
- Prepare the top models for healthcare deployment with interpretability features.
Methods
The study utilized datasets from PubChem and Clue.io Drug Repurposing Hub, which after being extracted as a CSV file, were merged using pandas with the PubChem Compound Identification, a unique identifying number, as the key column. Domain knowledge was used to select 9 features from the merged dataset: molar weight, polar area, complexity, xlogp, total bond stereocenter count, total atom stereocenter count, heavy atom count, H bond donor count, H bond acceptors count and MoA. The target variable, MoA, was the only categorical variable in the dataset which was encoded into integer values using sklearn’s LabelEncoder. Post data-processing, a total of 2151 unique drugs and 692 MoAs remained in the dataset.
Drug-target predictive models can be distinguished as multilevel classification problems, where the computational framework predicts an MoA from a list of labels [7]. As such, four classification models were used in this study, all of which were supervised: random forest (RF), naive bayes (NB), multi-layer perceptron neural network (MLPNN), and k-nearest neighbours (KNN). Hyperparameter tuning was also performed on the KNN, NB, and MLPNN models using sklearn’s GridSearchCV and RandomizedSearchCV.
Case Study
To demonstrate the suggested use case of the model and workflow, a case study was conducted on a randomly selected target gene within the clue database, SLC6A3. The gene was screened in Orpha.net in disease etiology, where it was linked to the rare disease Infantile dystonia-parkinsonism (IDP). Using the highest-performing computational model, a model inference case was performed, using PubChem data containing 105,552 compounds. After cleaning and preprocessing, the Random Forest model was used to predict the MoA of all the compounds and their probability scores. The dataset was then screened for all of the MoA that targets the selected gene, SLC6A3, and ranked by their predicted probabilities. The top 5 drugs for each associated MoA were further analyzed to find the best repurposing candidates for IDP.
Interpretability
Local Interpretable Model-agnostic Explanations (LIME) and OpenAI were implemented into the pipeline for model interpretability. LIME is an algorithm used to provide information on each specific prediction, often used to provide insight into the predictions of complex models. In this study, the lime_tabular package was imported and fed information from the Random Forest model. The resulting explanation was then inputted into a prompt to OpenAI. The gpt-3.5-turbo model was called using the OpenAI API and prompt engineered to generate a customized explanation for every drug repurposing recommendation based on LIME data.
Results
The evaluation metrics were applied to the models both prior to and after hyperparameter tuning, where results are listed in Table 1 and 2, respectively. The highest accuracy model, RF, was applied for the case study of IDP. The top drug recommendations made by the RF are listed in Table 3.
Discussion
With the usage of compound structural data to predict compound MoAs, this study was able to generate repurposing candidates using only limited disease signatures of rare diseases. Similar to Yu et al., who applied various computational models in predicting drug-target interactions based on molecular structure, RF obtained the highest metrics of the models studied [8]. Furthermore, Yu et al.’s study obtained an accuracy of 0.8133, comparable to the accuracy obtained with the RF model in this study [8].
Case Study
IDP is a rare neurological disease, often due to a mutation in the SLC6A3 gene which causes dopamine transporter deficiency, commonly treated with dopamine agonists [9]. Despite the limited data available for IDP due to its rarity, the RF model was still able to screen and recommend candidate drugs with MoAs correlated to the target gene. Due to the strong correlation of pramipexole, bromocriptine, and ropinirole with SLC6A3, this study proposes these as candidates for treating IDP.
Interpretability
Black box models are algorithms that produce a final prediction without providing much information on the internal processes that occurred to reach it. The top-performing algorithms identified in this study (RF, KNN, and MLPNN) were all black box models. While the strong ability for these models to identify and predict potential drug repurposing candidates is promising, using a black box model in the healthcare field comes with many challenges, including information sensitivity. LIME (Local Interpretable Model-agnostic Explanations) is a technique that can be used to provide interpretability for these models.
From a mathematical standpoint, LIME evaluates the local context of a specific instance x, and takes into account the reliability of the model g, where f is a function representing the probability of instance x to be associated with a certain set of features. Πx denotes the similarity between instance x and another instance, which can help LIME understand how x is related to other examples. The complexity of the given ML model is described by (g), where lower complexity can yield clearer explanations. The gpt-3.5-turbo model was accessed through the OpenAI API and used to explain data from LIME in a way clinicians and patients can easily understand. It should be noted that OpenAI was used to serve as a proof of concept for the ability for LLMs to describe data in healthcare, and a specialized LLM designed for the purpose of drug repurposing would be expected in a true clinical environment.
Conclusion
The main contributions of this study include:
- The development of a model that expands the scope of drug repurposing beyond the limited subset of specialized drug compounds by predicting the MoA of lesser known investigational compounds and using their predicted probabilities to make drug recommendations.
- Identifying the Random Forest model as the top-performing algorithm for MoA prediction, which is in alignment with previous literature [8].
- Demonstrating the use of LIME to integrate interpretability into the model, which holds significant implications for the model’s deployment in the healthcare field.
With further refinement, the use of machine learning for drug repurposing can lead to widespread reform, where intelligent recommendations can expand the treatment options for those living with rare diseases.
The references used can be found using this QR code:
