The Potential of Artificial Intelligence in Pharmacovigilance

Pharmacovigilance (PV) is “the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects and other drug-related safety problems”.¹  Sometimes equated with spontaneous reporting, contemporary PV is much more expansive, seeking convergent lines of evidence from many sources spanning the drug development continuum. PV’s broad scope, high-throughput processing to achieve regulatory, scientific and clinical objectives in a low signal-noise environment, plus a growing treasure trove of data and technology, makes it a high-grade ore for mining by artificial intelligence (AI).

AI is a broad term, including replicating human cognition by machines (i.e. symbolic logic), and machine learning (ML), for which providing accurate predictions, classifications, clusters, or otherwise  useful patterns, is paramount, regardless of whether it is achieved in the same way as the human neural network.

AI can tackle large, complex, high dimensional (e.g. the number of predictors >> number of samples) data containing complex non-linear relationships, unlike standard statistical analysis which may result in an infinite number of possible, unstable and potentially overfitted solutions. This is amplified when some of the huge number of possible combinations of variables are predictive.

ML includes traditional ML (e.g. logistic regression , decision trees and support vector machines (SVM)) and deep learning, such as artificial neural networks (ANNs). A 2017 review reported that SVM and ANN are the most commonly used forms of AI in medicine.²

For SD&E, ML from bioinformatics-based data can assess structure-activity relationships, and via systems biology/network graph analysis, how drug effects ripple through complex biological networks, including off-target effects, helping toxicity prediction prior to, or during drug development, but also substantiating post-marketing safety signals^10-14 via biological plausibility arguments, in line with the Bradford Hill criteria published in 1965 by the English epidemiologist and statistician Sir Austin Bradford Hill. For example, activation of specific subnetworks (“modules”) within complex biological networks may be signatures of drug-induced adverse events, such as myocardial infarction.¹² One exercise was even able to tease apart within-drug class safety differences.¹³ The real-world potential of these types of analyses will depend in part on the development of readily and widely deployable big data analysis environments for seamless workflow.

Much work in applying AI to SD&E aims for pipelining analysis of multiple data sources, such as spontaneous reports and EHRs, to condense the usually serial processes of SD and SE.¹⁵ There is keen interest in NLP for SD&E from these data, since an estimated 80% clinical data is unstructured in EHRs.¹⁷ Large institutional EHR systems, as well as distributed data networks such as the Sentinel system provide comprehensive environments for ML. ^16-18

The vast amount of social media data begs the question of whether AI can mine it for real-time SD? However social media posts are heavily laden with limitations including very low-signal-to-noise ratio, colloquial terminology, emoticons, duplicate reporting (e.g. retweets), and lack of key demographics and identifiers. While NLP is able to extract posts containing product-safety pairs (“proto-Adverse Drug Events”), to date there is no cogent evidence that mining social media discovers signals that are otherwise undetectable by, or detects credible signals earlier than, legacy processes and there are currently no regulatory mandates to screen social media. However, it may add value in selected areas such as addictovigilance.¹⁹

Like drugs and devices, there are side effects, warnings and precautions for AI in PV use!  The distribution of research across peer-reviewed, grey (e.g. abstracts, presentations, pre-print servers), and commercial literature is sometimes skewed. Prospective real-world deployment is somewhat sparse. One example of a ML-based SD system routinely used in PV is vigiRank, a predictive model using twelve predictor variables related to quantity, quality and clinical content of individual reports, as well as trends in time and geographic spread.²⁰

Technology “hype cycles” can bias judgement, compromising methodological quality and clinical judgement. A published deconstruction of pharmacovigilance hype cycle effects provides real-world examples of hype cycle effects from the published literature²¹ which can lead to potentially invalid conclusions e.g. about drug-drug interactions.

Most research is retrospective, and prospective performance often trails due to factors such as-distributional drift, in which training and test data do not represent current data.^22-24  Therefore, prospective performance monitoring is important to detect downward performance drifts requiring model retraining, which will also have to be factored into regulation of ML-based medical devices used for diagnosing adverse drug reactions. An illustration of such distributional shifts occurred during development of a device for detection of hyperkalemia from EKGs using a smartwatch. Initially trained on 1.3 million ECGs with corresponding K+ levels from outpatient clinics, initial performance was poor with clinically serious prediction errors. The outpatient data was imbalanced, under-representing sicker patients with higher potassium levels and with ECGs taken closer to the time of serum potassium measurement.²⁴ Using a less filtered data set (and analyzing the entire waveform and not just the T-waves) performance dramatically improved allowing submission for FDA approval. ²⁵

Since AI, especially deep learning, emphasizes predictive accuracy over model interpretability, machines may use unreliable confounders for prediction during development. e.g. using the overrepresentation of rulers in image sets of malignant versus benign images of skin lesions, to accurately predict malignancy. ²⁵

Quality standards for reporting machine learning studies are thus important; the Transparent Reporting of Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement (2015), and the ensuing TRIPOD-ML should help promote such quality standardization.²⁶

Particularly in a highly regulated environment, concerns exist that machines will miss something important.  In key areas PV professionals achieve nearly 100% accuracy/compliance, a level difficult for a machine to match, underscoring the need to precisely define operational objectives of using AI in PV e.g. reduce the median time for human-in-the-loop during ICSR case processing.

PV professionals may wonder about their professional prospects with increasing AI in PV. A small survey found a generally positive attitude about AI in PV among PV professionals at a pharmaceutical company, e.g., that it would allow shifting human work from high volume, highly routinized work to more meaningful high-value work.²⁷ With the rise of data science and biomedical informatics in drug safety there are also increasing educational opportunities at all levels, even up to distance graduate study programs at world class universities, enabling PV professionals to newly acquire powerful skill sets in drug safety bioinformatics that can massively enrich their career development.