Delving into Machine Learning's Influence on Disease Diagnosis and Prediction

1.1. Purpose of the Study

This paper's overarching goal is to talk about the use of machine learning methods and algorithms for clinical diagnosis and prognosis. The authors of this publication provide important insights for those working in the field of disease diagnostics using machine learning. They can use this data to pick the most applicable and cutting-edge machine learning methods. As a result, the diagnostic procedure is more likely to discover and classify diseases efficiently and accurately. This research summarises the state of the art of machine learning algorithms for disease detection and prognosis. This study did a thorough analysis of existing literature about the use of machine learning algorithms in the medical field in the context of disease diagnosis. In this study, we examine the similarities and differences between various algorithms, evaluation techniques, and final results [9].

1.2. Milestones of AI

The advent of artificial intelligence (AI) has ushered in a new era of innovation, significantly impacting various sectors, including healthcare. Within this realm, machine learning (ML), a subset of AI, has become a pivotal tool in revolutionizing disease diagnosis and prediction. This introduction delves into the milestones of AI, acquaints us with its distribution, and outlines the advances and challenges within the context of healthcare. Machine learning's journey in healthcare began as a part of the broader evolution of AI, which traces back to the mid-20th century. The conceptual foundation laid by pioneers like Alan Turing set the stage for AI's development, with the first neural network for computers being created in the 1950s. However, it was not until the advent of more powerful computing resources and the availability of large datasets that machine learning began to significantly influence healthcare [7, 8].

1.3. ML's Applications in Healthcare

Today, ML's applications in healthcare are extensive and growing. From predictive analytics that forecast disease outbreaks to algorithms that detect patterns in medical images for early disease diagnosis, machine learning is reshaping how we approach health and medicine. For instance, ML algorithms have been instrumental in diagnosing diseases from various medical imaging techniques, such as MRI and CT scans, by identifying patterns imperceptible to the human eye. The distribution of machine learning in healthcare is global, with both developed and developing countries harnessing its potential. Advanced healthcare systems use machine learning to enhance precision medicine, while in resource-limited settings, it offers tools for improving access to diagnostics and treatment planning [12].

1.4. Significant Advances in ML

Significant advances in ML, such as deep learning, have propelled the field forward. Deep learning algorithms, which mimic the human brain's neural networks, have achieved remarkable success in diagnosing conditions such as diabetic retinopathy and various cancers. These algorithms continually learn from vast amounts of data, improving their diagnostic accuracy over time. However, the integration of machine learning in healthcare is not without challenges. Ethical concerns, such as data privacy and the potential for algorithmic bias, pose significant hurdles. Moreover, the “black box” nature of some ML algorithms, where decision-making processes are not transparent, raises questions about accountability in clinical settings. As we delve into machine learning's influence on disease diagnosis and prediction, it is crucial to recognize its roots in the broader AI landscape, appreciate its global distribution and advances, and address the challenges it faces. By doing so, we can harness the full potential of machine learning to transform healthcare, making it more predictive, personalized, and accessible [13, 14].

2.1. Utilizing Machine Learning for Disease Diagnosis in Clinical Settings

The decision tree approach is often regarded as the most popular data modelling technique [14] due to its simplicity and intuitiveness. Better techniques like support vector machines (SVM) and artificial neural networks (ANN) have become widely used in recent years [15]. Predicting protein function and gene expression using machine learning algorithms is only one of several areas where this technique is being studied [16]. In contrast to methods that rely on the protein's sequence or structure, those that employ machine learning to predict protein function don't require any prior knowledge of homology or parameters derived from homology. Therefore, there has been a rise in studies targeted at creating effective machine-learning strategies for predicting protein function in the context of disease diagnosis.

Fig. (1) illustrates the diverse methodologies within machine learning, showcasing a range of approaches from supervised and unsupervised learning to reinforcement learning, each with distinct mechanisms and applications in data analysis and prediction. This figure provides a visual taxonomy of machine learning techniques, highlighting their unique characteristics and the contexts in which they are optimally employed. The process of medical diagnosis, often considered empirical yet not fully understood as a cognitive task [17], undergoes segmentation when applying computer systems, parti- cularly Machine Learning (ML). It encompasses several stages to streamline the complexity. Commencing with data collection, which spans various formats—clinical data from interviews, demographics, imaging, speech, patient history, and even physiological sounds [18] though this list is not exhaustive. Subsequent stages involve data processing, entailing data preparation tasks like handling missing values, reducing data dimensions, and managing noisy data [19]. Following this, the process moves to identifying the target variable and predictors. Then, models are trained using this data, providing them with the necessary information for subsequent diagnostic applications.

2.2. Advantages of Using Machine Learning to Diagnose Disease

The presence of several overlapping structures and instances, as well as distractions, fatigue, and limits in the human visual system, might create challenges. In such situations, the availability of a “second opinion” can be advantageous [20]. The use of computer-aided design (CAD) systems has been fostered in the context of diagnostic procedures. Computer-aided diagnosis (CAD) is a theoretical framework that assigns equal importance to both doctors and computers, with the aim of supporting clinicians in making optimal clinical choices and implementing best practises [21]. Furthermore, the use of electronic health record (EHR) systems has been prevalent in the healthcare industry as a means to aid in clinical decision-making. This is mostly attributed to the rising complexity seen among patients, the occurrence of high diagnostic mistakes, and the abundance of data accessible for analysis [22].

ML approaches play a crucial role in unravelling intriguing linkages within data, aided by the accessibility of intelligent tools for data analysis [23]. It can be used to either bolster or cast doubt on the conclusions reached by clinicians [24]. Clinical decision-making and the automatic tuning of bedside devices have both benefited from the introduction of machine learning (ML) based systems that constantly examine the expanding influx of data streams for patterns. As a result of these developments, patient treatment outcomes have improved, and overall treatment costs have decreased significantly [25]. According to the cited articles [26], the potential of machine learning (ML) to provide optimal therapeutic support has not been realised so far. This may be attributed to the lack of transparency in ML algorithms and analytics. Additionally, the issues of data quality and generalizability of machine learning models are significant concerns [27].

2.3. Primary Weaknesses in these Regions

Some of the weaknesses are listed as follows:

• One of the primary weaknesses in these regions is the lack of comprehensive digital infrastructure to support AI and ML technologies. Limited access to quality data is another significant challenge, as machine learning's efficacy heavily depends on the availability of large, annotated, and high-quality datasets. In many developing areas, the data that exists is often fragmented, non-digitized, or of poor quality, hampering the development and deployment of effective ML-based diagnostic tools.
• Moreover, the shortage of local expertise in AI and machine learning poses a substantial barrier. Developing regions often lack the educational programs and research institutions necessary to cultivate talent in these cutting-edge fields, leading to a dependency on external expertise that may not fully grasp the local healthcare context and needs.
• Another critical concern is the ethical considerations surrounding the deployment of AI in healthcare, including issues of privacy, consent, and bias. These issues are magnified in regions without stringent regulations or frameworks governing the use of AI, potentially leading to mistrust and reluctance among healthcare providers and patients to adopt these technologies.

In summary, while machine learning holds immense potential to revolutionize disease diagnosis and prediction, its global impact is uneven due to significant weaknesses in certain regions. Addressing these challenges requires a concerted effort involving infrastructure development, education, data management, and ethical governance to ensure that the benefits of AI in healthcare are accessible to all, irrespective of geographic location.

3.1. Support Vector Machine

Use a support vector machine (SVM) for more help. SVM has been proven to be useful for many classification problems. It searches for the optimal hyperplane by enumerating the number of nodes lying on the border between two classes. Margin refers to the value disparity between two groups. The success rate of a categorization system rises when there is a larger margin of safety. Boundary information is represented via support vectors. SVM can be used to tackle both regression and classification problems [28]. Both linear and nonlinear data sets benefit from this approach. The SVM method utilizes several different kernel types for use in a prediction model. These include linear radial basis function (RBF), polynomial, and sigmoid kernels. SVM utilizes a high-dimensional space for attributes and selects the optimum hyperplane to categories data points into two groups. It is effective on large and small datasets that are normally challenging to analyses. Support vector machine (SVM) analysis of hyperplane data for diabetes detection is depicted in a relatively basic manner in Fig. (2) [29]. Fig. (2) depicts the process of classification using a Support Vector Machine (SVM), illustrating how this machine learning model constructs a hyperplane or set of hyperplanes in a high-dimensional space to categorize data points into distinct classes. The figure visually demonstrates the concept of maximizing the margin between different classes, which is fundamental to the SVM's effectiveness in classifying complex datasets.

3.2. Naive Bayes Algorithm

Naive Bayes (NB) is a classification algorithm used in statistics and probability, see Fig. (3). It is a common technique in machine learning applications because of its ease of use and the fact that all factors can be considered equally when making a final call. The NB method is intriguing because of its simplicity and its versatility due to its computing efficiency. The NB classification relies on three key concepts: prior, posterior, and class conditional probability [30]. In addition to its many other merits, this method is particularly well-suited to and easy to apply to huge datasets. It could be applied to problems of binary and multi-class classification. Less data is required for training purposes, and both discrete and continuous data can be used. The application of this technique could be used to classify papers and eliminate junk mail [31]. Fig. (3) illustrates the Naïve Bayes Classifier approach, a probabilistic machine learning model that applies Bayes' theorem with the assumption of independence among predictors to predict the category of a given sample. The figure visually represents how this classifier calculates the probability of each class and the conditional probability of each feature within those classes to make predictions.

3.3. Decision Tree Algorithm

In order to tackle regression and classification issues, the Decision Tree (DT) uses periodic splitting of data based on a given variable. Nodes represent data structures, while the tree's leaf represents evaluations, (Fig. 4). The purpose of the decision tree is to construct a model for forecasting the dependent variable from training data by learning simple decision rules [32]. The tree is learned by constructing it from the training data. The name of the class is stored in a decision node, which is a non-leaf node. Both numerical and categorical information can be processed using the decision tree. The effectiveness of the tree remains unaffected by the nonlinear connection between arguments [33, 34]. No preliminary data preparation is required. When the tree is built again and over again, overfitting might occur.

The decision tree shown in Fig. (4) consists of a single leaf node, one child node, and the tree's parent node. Decision trees have found use in the medical field. If we use DT to diagnose breast cancer, for instance, we might assign a benign or malignant label to each of the tree's leaf nodes. Clump thickness (CT) will be used to establish whether or not the tumour is malignant by developing rules among the necessary data set parts. Fig. (4) shows the DT algorithm in action in a breast cancer diagnostic setting [35]. Fig. (4) showcases the classification process using a simple decision tree approach, highlighting how this method splits data into subsets based on the value of input features, creating a tree-like model of decisions. The figure demonstrates the hierarchical structure of decision nodes and leaf nodes, which represent the questions and outcomes, respectively, guiding the user through the logical steps of data classification.

3.4. Random Forest Classification

The random forest ensemble model can also be used to develop a predictor based on the nearest neighbors. An essential principle of ensemble methods is that when multiple models are combined, the resulting model is more robust than any of the individual models. In addition to the random forest, the ensemble also includes a traditional ML technique decision tree. This procedure takes an initial input and branches out into finer and finer subsets of the data. The random forest expands on this concept by merging trees with the idea of an ensemble. A random forest classifier's strengths lie in its flexibility in dealing with missing data, its speedy execution, and its inherent inequity [36]. In a random forest, the newly generated subtrees travel with the testing data or the new dataset. Any decision subtree in the forest can be used to choose the dataset's classification. After a vote is taken, the model will select the most popular option. The fundamental idea behind the random forest algorithm for cardiac illness detection is depicted in Fig. (5) [37]. Fig. (5) illustrates the use of Random Forest for categorizing datasets, depicting how this ensemble learning method constructs multiple decision trees during training and outputs the class that is the mode of the classes of individual trees. This figure visually represents the aggregation of predictions from various decision trees to enhance classification accuracy and mitigate overfitting, characteristic of the Random Forest approach.

3.5. Logistic Regression

Logistic relapse is a regulated learning procedure used to resolve issues of double categorization. The logistic capability and logistic relapse, which have various more perplexing expansions, are utilized to show a twofold order in science. In its most essential structure, logistic relapse is a sort of relapse model that predicts the probability that a given data of interest or section has a place with a particular class [38]. Logistic regression employs the sigmoid function (Fig. 6) to model the data. Logistic regression's regularization simplicity, computational efficiency, and application ease are three of its most important features. Scaling input features is not required. However, overfitting and the ability to solve nonlinear problems [35].

3.6. Deep Learning

As a subfield of machine learning, deep learning is distinguished by its use of multi-level layered nonlinear transformations. The Recurrent Neural Network (RNN) can use the input slices, which can be seen as a sort of sequential data, to infer relationships between slices [39]. Convolutional neural networks (CNNs) are frequently employed in deep learning applications for data analysis and complicated classification tasks. It has four layers total: a convolutional layer, a max-pooling layer, a fully linked layer, and an output layer. A 32x32 pixel section of the original medical image is used as input for the CNN model. In deep neural networks, feature maps are created by convolutional layers and passed on to the following layers, where their size is reduced by pooling layers. In the end, the completely connected layer generates a prediction for the correct class, as described in another study [37].

4.1. Parkinson’s Disease

One medical problem that has been extensively discussed in the ML literature is Parkinson's disease. It is a chronic neurological condition with a sluggish progression rate. Damage to or loss of dopamine-producing neurons in specific brain regions impairs basic cognitive and motor functions. Several other methods based on ML have been suggested. In order to create accurate Parkinson's disease diagnostic systems, researchers like [41] employed algorithms like KNN, SVM, NB, and RF. In their computational results, RF is shown to have the highest performance (90.26% accuracy), whereas NB is shown to have the lowest performance (69.23% accuracy). Near-perfect accuracy was reached on both the training and testing sets when a deep CNN-based model was suggested [42]. However, no issues with overfitting were mentioned throughout the experiment. Finally, the ultimate classification and regression, which is now generally anticipated, especially in CDSS, is not well interpreted by the experimental data. For Parkinson's telemonitoring speech data also turned to DL-based methods at UCI [43]. In their trial, DNN was able to correctly identify Parkinson's illness in 81.67% of cases.

4.2. COVID-19

Another pestilence, SARS-CoV-2 (or Coronavirus), has emerged as quite possibly of humankind's most noteworthy danger. Regardless of the worldwide race to get an immunization to showcase in light of the pestilence, it remained exorbitant to by far most individuals for its whole span. As a result of its high transmission rates and immunization-related opposition, the clever Coronavirus Omicron strain is causing more concern. As of now, Ongoing Reverse Transcription-Polymerase Chain Reaction (RT-PCR) is the highest quality level for deciding if a patient has contracted Coronavirus. The analyst supported the utilization of Machine Learning and Artificial Intelligence in conjunction with other demonstrative strategies like chest X-beams and Computed Tomography (CT) during the plague. Chen et al. (2020) made a UNet++ model with CT information from 51 Coronavirus and 82 non-coronavirus patients, and it accomplished an exactness of 98.5%. The authors surveyed ten different DL models utilizing just 108 Coronavirus patients and 86 non-COVID-19 patients, and they had the option to accomplish a general exactness of almost 100%. Utilizing a tremendous dataset of 453 CT examination pictures, Wang et al. (2020) [44] made a commencement-based model that accomplished 73.1% precision. Nonetheless, data about the degree and nature of the model's organizational activity was restricted. Authors [42] proposed the COVNet model and accomplished 96% exactness utilizing an enormous dataset involving 4356 chest CT pictures of Pneumonia patients, of which 1296 were confirmed Coronavirus occurrences. The summary of existing studies is shown in Table 2.

4.3. Alzheimer’s Disease

Sixty percent to seventy percent of persons who are diagnosed with dementia [43] have Alzheimer's disease, a brain condition that usually starts slowly but develops over time. Alzheimer's disease causes cognitive decline, which manifests in a variety of ways. The steady decline in body functioning is reflected in a prognosis of three to nine years from the time of diagnosis. On the other hand, getting a diagnosis early may help you take preventative measures and start appropriate treatment as soon as possible, both of which increase your chances of survival. Over the years, the use of machine learning and deep learning to identify people with Alzheimer's disease has shown encouraging results. A CNN model for early detection and categorization of Alzheimer's disease was suggested by Ahmed et al. (2020) [44]. The suggested model obtained 99% accuracy within the dataset consisting of 6628 MRI pictures. The deep feature-based models provided by Nawaz et al. (2020) [45] have an accuracy of 99.12%.

4.4. Other Diseases

ML and DL have been used for the diagnosis of a wide range of additional disorders in addition to the one that was discussed before. Two primary factors for this rise in utilisation are the proliferation of large amounts of data and the rise in the processing capability of computers. For instance, Mao et al. (2020) [41] classified diseases based on eye movement by using Decision Tree (DT) and Random Forest (RF). When Nosseir and Shawky (2019) [39] developed automated skin disease classification systems using KNN and SVM, they found that KNN achieved the greatest performance by attaining an accuracy of 98.22%. This was the best performance that was seen utilising KNN. Khan et al. (2020) [37] were able to classify multimodal brain tumours using CNN-based methods such as VGG16 and VGG19. The experiment was conducted using three picture datasets that were readily accessible to the public: BraTs2015, BraTs2017, and BraTs2018. The results showed an accuracy of 97.8%, 96.9%, and 92.5%, respectively.

8.1. Data Quality and Availability

AI systems require vast amounts of high-quality, diverse, and well-annotated data to learn effectively. However, there is a significant gap in the availability of such data, especially from underrepresented groups or regions, leading to potential biases in AI-driven diagnoses.

8.2. Complexity of Medical Conditions

Some medical conditions are incredibly complex and nuanced, requiring a holistic understanding of the patient's history, symptoms, and other factors that AI might not fully integrate or interpret accurately.

8.3. Interdisciplinary Knowledge

Medicine often requires interdisciplinary knowledge, combining insights from various fields (e.g., psychology, sociology, genetics). AI systems, primarily trained on specific datasets, may lack this comprehensive approach.

8.4. Adaptability and Continuous Learning

The medical field is continuously evolving with new research findings and treatment methods. AI systems need constant updates and training to stay current, which can be a significant challenge.

9.1. Responsibility for Incorrect Diagnoses

The risk of incorrect diagnoses and the associated liability is a major concern. Determining the responsibility whether it falls on the AI developers, the healthcare providers, or the technology itself is a complex issue that has yet to be fully resolved.

9.2. Ethical Use of Data

The ethical implications of using patient data to train AI systems are immense. Issues of consent, privacy, and data security are paramount, especially when dealing with sensitive health information.

9.3. Bias and Inequality

AI systems can perpetuate and even exacerbate existing biases and inequalities if they're trained on biased data sets or if their deployment overlooks the diverse needs of different populations. Ensuring fairness and equity in AI-driven medical diagnoses is a significant challenge that needs to be addressed.

In summary, while AI has the potential to augment and, in some cases, transform medical diagnosis, it is not yet in a position to replace human oversight entirely. The technology must overcome substantial gaps in data quality, complexity understanding, adaptability, and ethical governance to ensure that AI's integration into healthcare enhances patient outcomes without compromising on safety, fairness, and privacy.