Computer programs can now learn on their own and perform medical tasks at a level comparable to human expert doctors. This has sparked a lot of interest in deep learning within the medical industry. Artificial intelligence known as machine learning has been successful because its algorithms can pick up new information from data without explicit instructions. A particular sort of machine learning called deep learning uses neural networks with several layers to process and interpret data.
Machine learning, including deep learning, has achieved impressive feats in medicine. It has accurately diagnosed sepsis, a severe reaction to infection, helped determine the best diabetes treatment, and interpreted complex electrocardiograms. Deep learning shows great potential in medical imaging. It has been successful in identifying important findings in head CT scans, detecting diabetic retinopathy in the eyes, and even predicting breast cancer.
Machine learning attempts to mimic human thinking by spotting patterns in data and generalizing knowledge, machine learning seeks to imitate human reasoning. Common techniques for extracting generalizable models that reflect the fundamental relationships between variables include logistic regression and linear regression. Regression models, however, do not employ raw data and call for explicit parameterization.
It takes domain expertise and close attention to the interrelationships between variables to feature engineer, which is the process of converting raw data into a representation that the model can use. The complexity of the model and workload for feature engineering is exponentially increased by covariance, confounding, and interactions.
Using a general-purpose learning process that automatically extracts pertinent features via exposure to instances of each category, deep learning is entirely automated feature engineering. Deep neural networks are mathematical operations that convert previous input simply and pass their result into the subsequent layer. A deep learning architecture can be thought of as a hierarchical stack of straightforward models that gradually learn increasingly abstract representations of the data, such as suitable transformations and intricate relationships between variables.
As neural networks automatically learn from raw data, deep learning has two key benefits:
Especially when working with several variables that have complicated interactions, humans tend to build models that reflect relationships between variables in simple linear terms, which may not fully capture the richness of the data. Regardless of the mathematical complexity of the model, neural networks optimize it for prediction.
Predictive analytics in medicine uses large databases to improve patient care. For instance, algorithms have been developed to predict conditions like acute myocardial ischemia, leading to better decision-making in emergency departments. Simple biomarker-based models work for many conditions, but complex cases like sepsis require more advanced machine-learning models that analyze multiple factors over time. These models can detect sepsis earlier and improve outcomes. Machine learning outperforms traditional models when dealing with vast and complex healthcare data. It can predict the onset of diseases like diabetes type II more accurately, using available data without disrupting clinical workflows.
Electronic health records offer valuable insights, including the timing of tests, which can be more predictive of readmission and mortality than the test results themselves. As medical data grows, advanced models with many variables will become increasingly relevant for guiding clinical decisions.
Choosing a test or therapy based on which has the highest likelihood of benefiting the patient is the guiding principle doctors use when making decisions in medicine. Yet, clinical decisions are rarely supported by a clear justification; yet, machine learning and deep learning can shed light on these decisions by extracting pertinent probabilities from data. Early versions of medical decision support systems relied on rule-based judgments obtained from a database of clinical knowledge. However, the process needed to be clearer, and the models needed to be more secure for practical usage.
Even though individuals’ clinical results and diseases differ slightly, modern decision-support models seek to infer clinically significant links that can be generalized to other patients. For instance, in the electronic health records of 10,806 individuals with diabetes mellitus, the K-nearest neighbor algorithm was used to forecast the drug or treatment combination that would optimize glycemic control. In 68% of visits, this algorithm correctly predicted the course of therapy, and in those instances where it did not, it was expected that the predicted course of treatment would result in better glycemic control than the course of treatment that was provided.
Understanding the relationships between medical concepts is crucial for decision support systems; however, in electronic health records, most relationships and useful information are found in unstructured clinical notes and need more formatting. Extraction of statistical connections between important categories, such as diseases and symptoms, is an easy way to tap the massive amount of data in unstructured medical information. The probabilities of the co-occurrence of different medical disorders were computed and made available to the public using 14 million clinical notes from 160,000 patients.
The major drawbacks of these methods are confounding relationships in observational data, restrictions on the environment where the associations arise, and privacy and confidentiality restrictions that prevent sharing the data from whence these correlations were obtained. Clinical decision assistance systems will eventually probably rely on abstracting the correlations between the variables in electronic medical data, including free-text clinical notes.
Even though natural language processing is still unsuitable for broad clinical practice, recent developments give optimism for its potential future application in clinically significant ways. In 2017, Google’s research team became aware of the drawbacks of recurrent neural networks, including their sequential structure and the difficulty in learning correlations between words that are far apart. They suggested a different approach to natural language analysis that uses multi-head attention mechanisms (transformers) that can learn dependencies regardless of the spacing between words in the text and enable parallelization, improving computing speed. The context in language comes from the text before and after any word; therefore, a team at Google AI realized that language representation is constrained if they learn from the text up to that point. They added to transformers by randomly masking 15% of the words and predicting a word based on the words that came before and after the masking. Additionally, they attempted to anticipate the text’s following sentence by either using the actual sentence that follows or a randomly chosen sentence from another part of the text.
A significant advancement in natural language processing, the bidirectional encoder representations from transformers (BERT) model outperformed humans in 11 separate natural language processing challenges. However, because the BERT model was developed for general-purpose material, it might not function as well in texts including context and terms specialized to the biomedical field.
Clinically pertinent EKG(To check for various cardiac diseases, an electrocardiogram (ECG or EKG) measures the electrical signal from the heart.) findings can be interpreted using electrophysiologic tracings like EKG and EEG. A neural network with an area under the curve (AUC) > 0.9 accurately categorized 10 clinically significant EKG abnormalities from 91,232 single-lead EKGs from 53,549 patients. The performance of this model was on par with or better than the average cardiologist despite not undergoing any Fourier or wavelet transforms. Deep learning is being gradually incorporated into EKG software and is anticipated to automate significant aspects of EKG interpretation shortly. EEG interpretation by humans is costly since it takes time and resources. Universities and private businesses are making effective attempts to automate EEG interpretation.
However, due to EEG’s inherent properties (low signal-to-noise ratio and nonstationary signal) and logistical difficulties, the field of automatic EEG interpretation continues to be constrained. The quantity and caliber of EEG repositories have grown recently, producing better outcomes. A convolutional neural network could identify epileptiform activity better than the industry standard. Using the same method, 13,959 epileptiform discharges from 46 epilepsy patients with intracranial mesial temporal lobe monitoring EEG data were classified.
Machine learning is still employed to lighten the workload of the human reviewer by choosing segments that have a high likelihood of containing interictal epileptiform discharges because a completely automated system is not yet suitable for practical use. EEG needs to integrate a varied collection of features instead of EKG, which can achieve acceptable performance by identifying a few typical features and recording predetermined time intervals. Deep learning is particularly well-suited for extracting generalizable features from various patients.
CNNs automatically find the most important features in images, making image classification easier and reducing the workload for humans. They represent data in a way that suits specific tasks and can be adjusted for other purposes. These models can also find complex connections that might be challenging for humans to notice. For instance, a deep learning model predicted cardiovascular risk factors just by analyzing photographs of the retina. This shows that deep learning can reveal important insights that humans might miss due to the complexity of the data.
Fully convolutional neural networks reconstruct the original image, segmenting each object, using a deep learning model to learn abstract representations of data in lower dimensions. These networks learn pertinent information using a downsampling approach and then combine those features to forecast pixels using an upsampling method. Fully convolutional networks started with the same amount of layers for both upsampling and downsampling, but as time went on, they improved with fewer layers. These networks are helpful in medicine because they help identify and define the borders of key image elements.
Deep learning has the potential to revolutionize clinical workflows by improving the accuracy of clinical data. It can be used for various tasks in healthcare, such as classifying head CT scans, identifying acute abnormalities, and predicting lesion development. This technology can help prioritize urgent cases, leading to faster clinical decisions for critically ill patients. Deep learning algorithms can also act as a safety net, aiding in detecting cerebral aneurysms that human radiologists might miss. They can be cost-effective alternatives to human segmentation in fMRI scans, distinguishing between abnormal brain lesions and normal structures. However, the use of deep learning in neuroimaging is limited by the need for more data, particularly for rare neurological conditions. Making the most of the available data is possible with the use of strategies like transfer learning and data augmentation. Data imbalance is challenging in deep understanding, but methods like attention gates and asymmetric loss functions can address this issue.
Interpreting deep learning models can be challenging due to complex interactions in hidden layers. These models are sometimes called “black boxes” because their decision-making process is not fully transparent. Nevertheless, they can provide valuable insights into data that traditional models might overlook. One concern with deep learning is the potential to learn inaccurate correlations or biases from the training data. These issues can be addressed in interpretable models but may need to be noticed in non-interpretable ones like neural networks. Clinicians prefer open models that can provide explanations for their decisions. Techniques like gradient-weighted class activation maps and saliency maps offer insights into how convolutional neural networks reach their classifications.
Deep learning and machine learning are increasingly employed in nonmedical sectors like web search, language translation, and content recommendation. These models can significantly advance medicine by identifying bankruptcies, forecasting therapeutic outcomes, and recommending goods or services based on shared interests across various dimensions. Recommendation systems use large databases to find the most effective patient treatments based on their medical characteristics.
Deep learning has advanced natural language processing, allowing for seamless language translation and accurate linguistic structure and element representations. These developments can extract useful representations and abstractions of data from free-text clinical notes, the most comprehensive data source in electronic health records. Because they have so many applications outside of health, machine learning and deep learning are useful tools for advancing medicine.
Artificial intelligence can potentially improve clinical medicine, but there are some concerns. Machine learning approaches often focus on predicting outcomes, which can be effective for simple problems but may only partially capture the complexity of clinical decision-making. One reinforcement learning strategy allows AI agents to learn the best actions by experimenting and receiving rewards. This method is useful when the best course of action is unclear and can learn from less-than-optimal examples. An example of reinforcement learning in medicine is creating personalized treatment plans for sepsis patients using data from large intensive care unit databases. Implementing reinforcement learning in clinical practice could lead to significant advancements in AI use in medicine.
AI is a branch of data science that includes rule-based systems with explicit instructions and algorithms that learn from data without specific rules. Machine learning is a part of data science that, without explicit programming, enables computers to learn from examples and make predictions. Deep learning, a recent advancement, is based on neural networks and has shown great promise. Radiomics uses imaging data as biomarkers to study various conditions in medical research. In contrast, AI-based neuro-oncologic imaging research focuses on understanding complex brain and nervous system tumors to improve patient outcomes.
Neuro-oncology research mainly focuses on diffuse gliomas, classified into lower-grade gliomas and glioblastomas by the World Health Organization (WHO). Glioblastoma, a grade IV tumor, is the most dangerous primary brain tumor with a poor prognosis. Other types of brain tumors include WHO grade I tumors, pediatric CNS tumors, primary CNS lymphomas, and brain metastases. MRI is the primary tool for detecting and characterizing brain tumors due to its excellent soft-tissue contrast. However, conventional MRI sequences are not specific enough to detect tumor infiltration accurately. Advanced MRI techniques are used to grade gliomas and assess tumor infiltration, including diffusion-weighted imaging, diffusion tensor imaging, dynamic susceptibility-weighted contrast material enhancement, dynamic contrast enhancement, and MR spectroscopy. However, the diversity in equipment, imaging methods, and analysis techniques make it challenging to quantify the results accurately.
Researchers have used a systems-level radiogenomic approach to understand how radiomic indicators (characteristics obtained from medical imaging) relate to global gene expression patterns in different tumors. One study explored the connection between gene expression modules and MRI patterns defined by neuroradiologists. They found that certain imaging features could indicate the activation of specific gene expression programs.
Another study compared genomic pathway analyses with volumetric tumor characteristics. They observed that tumor bulk and edema were associated with specific cellular pathways related to homeostasis and cell cycling. In contrast, necrosis pathways were linked to immune response and apoptosis cellular pathways, and vice versa.
In another examination involving 29 patients, researchers looked at the correlation between radiomic characteristics of gliomas and specific gene mutations (TP53, PTEN, and EGFR). They discovered that texture analysis of the radiomic features provided distinct gene expression signatures associated with these mutations and different radiomic parts.
Clinical prognostication in tumors relies on histologic tumor grade and patient factors like age, sex, and functional status. However, imaging features and radiomic metrics, which can capture essential tumor biology and outcomes, have yet to be widely used in clinical prognostic models.
Previous radiomic studies have shown that basic imaging metrics, such as tumor size and enhancing volume, can predict patient outcomes better than clinical models alone. Some studies have combined clinical, imaging, and genetic variables to create rule-based models with the highest predictive accuracy in patients with TCIA.
Machine learning methods using multiparametric MR images have also been used to predict patient survival. These methods have identified specific features that indicate a poor prognosis, like volume, shape, texture, and wavelet features. Another model based on support vector machines achieved up to 80% accuracy in predicting survival groups using features related to tumor volume, angiogenesis, peritumoral infiltration, cell density, and distance to the ventricles. These advanced imaging techniques promise to improve our ability to predict patient outcomes in tumor cases.
The biology of gliomas has been taken into account when developing novel radiomic feature groups using unsupervised machine-learning approaches. For example, in a study of 121 solitary glioblastomas, researchers found multifocal, spherical, and rim-enhancing clusters. These clusters were further confirmed in 144 patients from different institutions, and their survival outcomes showed significant differences.
Another study involving 208 patients with glioblastoma used a comprehensive set of features and unsupervised high-dimensional clustering to identify related subtypes. These subgroups had different survival rates, with the rim-enhancing subgroup showing the best survival. However, more extensive sample sizes and validation are needed to ensure the reliability and robustness of these initial subgroups.
Evaluating treatment response in brain tumors can be complicated by a phenomenon called pseudoprogression. This occurs when there is a development of abnormal signals in T2/FLAIR imaging, along with additional areas of enhancement, following a combination of radiation and chemotherapy. Pseudoprogression is more common in tumors with IDH mutations and MGMT methylation. Additionally, antiangiogenic medications like bevacizumab can lead to a pseudoresponse, where the enhancement lessens, but the invading components of the tumor continue to progress.
Initially, the Macdonald criteria only considered the size of enhancing features for response evaluation. However, the Response Assessment for Neuro-Oncology (RANO) criteria have been updated to include T2/FLAIR non-enhancing signal intensity abnormalities and changes in enhancing tissue to improve accuracy.
Nevertheless, RANO has some limitations, as it relies on arbitrary two-dimensional measurements and does not incorporate advanced imaging technologies like MRI spectroscopy, diffusion tensor imaging, and perfusion imaging. Additionally, the diverse inflammatory reactions caused by newer immunotherapy drugs can further complicate response evaluation. The RANO criteria aim to account for immunological inflammatory-related pseudoprogression.
Identifying infiltrative tissue margins on preoperative MR images by machine learning techniques can inform the planning of radiation therapy, localized biopsies, and extended surgical resections. FLAIR and apparent diffusion coefficient maps can be used to forecast the locations of future tumor recurrence using a voxel-wise logistic regression model. After recording areas of glioblastoma recurrences to preoperative MR scans, Akbari et al. developed a multivariate support vector machine technique that combines information from both traditional and cutting-edge MRI modalities. With around 90% cross-validated accuracy, this method creates predictive geographic maps of invaded peritumoral tissue. Chang made a fully automated approach to register biopsy sites in 36 patients utilizing a CNN and neuronavigational crosshairs.
This method produced noninvasive maps of cell density that helped locate glioma infiltrative margins. These techniques promise to create noninvasive tools for patient stratification in clinical trials and direct more aggressive therapies. For patients who have just undergone an initial resection, the method by Akbari et al. has already inspired a clinical study with enhanced radiation to areas of the infiltrative tumor.
Machine learning approaches have been used to tackle difficult diagnostic situations and improve accuracy and efficiency in dealing with various CNS tumors, such as brain metastases and CNS lymphoma. Three-dimensional CNNs have been effective in identifying and localizing brain metastases, aiding in the planning of stereotactic radiation therapy. Distinguishing between brain metastases, primary CNS lymphoma, and glioblastoma can be challenging using conventional clinical and imaging methods. Researchers have developed decision trees and logistic regression models using advanced imaging techniques like diffusion tensor imaging and dynamic susceptibility-weighted contrast-enhanced MRI to differentiate between these entities.
In some cases, patients are diagnosed with brain metastases without knowing the primary location of the cancer. Machine learning algorithms have been applied in these scenarios, utilizing molecular variations and their effects on the surrounding tissue to identify different radiomic properties. For example, a random forest model was used to distinguish brain metastases resulting from lung, melanoma, and breast cancer based on texture analysis of MRI sequences.
In radiology, AI faces challenges such as high-quality ground truth data, generalizable and interpretable methods, and user-centric workflow integration. However, concerns about the “black box” nature of AI algorithms have diminished with the development of methods like saliency maps and principal component analysis. A better understanding of feature patterns and underlying biology will be helpful for clinical acceptance and for improving the biological and treatment relevance of the patterns revealed by these methods. One of the primary challenges in AI research is the availability of large, well-annotated data sets.
However, most available data must still be siloed within individual institutions and hospital systems. Larger and more heterogeneous datasets may be required to improve algorithms, with data sharing among institutions being an essential component. Other ways to enhance data sets include statistical techniques to harmonize data sets and adopt standardized neuro-oncology imaging protocols across institutions. Interestingly, novel deep learning methods, such as generative adversarial networks, have shown promise in improving performance by generating synthetic data.
Another barrier to developing more robust neuro-oncologic imaging and radiomics algorithms is the lack of clear, targeted “use cases” or specific tasks against which their performance can be benchmarked. The measured performance of individual algorithms is highly task-dependent, data set-dependent, and strongly influenced by the particular scientific question, which limits the comparison of different algorithms developed by other groups. The American College of Radiology Data Science Institute is helping define standard use cases, annotation tools, and data sets, which should help with standardization and benchmarking relevant to academic pursuits and commercial ventures.
AI algorithms are becoming more popular in research, but there are challenges to using them effectively in clinical settings. These tools need to be user-friendly and easily integrated into radiologist workflows. Some processes still require manual work, causing delays in results. To address this, efforts are being made to develop tools that can share and translate these methods easily, using open-source platforms.
The ultimate goal is to have a fully automated system that can analyze images in real-time and provide accurate diagnostic reports. This system could determine the likelihood of a lesion being a specific tumor and suggest appropriate imaging procedures and personalized treatment options. With advanced deep learning, it could also track treatment progress in real-time, making healthcare more precise. For now, a “centaur” radiologist will combine data from images, AI tools, and health records to improve accuracy until AI becomes a routine part of medical practice.
Get updates and learn from the best
How can AI improve drug optimization with symptom scoring Intro Increasing medication optimization using symptom rating presents several modern difficulties for the pharmaceutical and healthcare
How can AI help with therapy in healthcare? The application of embodied artificial intelligence (AI) in psychiatry, psychology, and other mental health services like psychotherapy
Main Menu
