The Applications of Artificial Intelligence in Chest Imaging of COVID-19 Patients: A Literature Review

Abstract

Diagnostic imaging is regarded as fundamental in the clinical work-up of patients with a suspected or confirmed COVID-19 infection. Recent progress has been made in diagnostic imaging with the integration of artificial intelligence (AI) and machine learning (ML) algorisms leading to an increase in the accuracy of exam interpretation and to the extraction of prognostic information useful in the decision-making process. Considering the ever expanding imaging data generated amid this pandemic, COVID-19 has catalyzed the rapid expansion in the application of AI to combat disease. In this context, many recent studies have explored the role of AI in each of the presumed applications for COVID-19 infection chest imaging, suggesting that implementing AI applications for chest imaging can be a great asset for fast and precise disease screening, identification and characterization. However, various biases should be overcome in the development of further ML-based algorithms to give them sufficient robustness and reproducibility for their integration into clinical practice. As a result, in this literature review, we will focus on the application of AI in chest imaging, in particular, deep learning, radiomics and advanced imaging as quantitative CT.

1. Introduction

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection, named COVID-19 (coronavirus disease 2019), caused a global healthcare and economic crisis. The first cases were observed in Wuhan, China, in December 2019, and it rapidly spread across the world so that in early March 2020, the WHO decided to classify COVID-19 a pandemic.

Diagnostic imaging has a fundamental role in the clinical work-up of patients with suspected or confirmed COVID-19 infection, granting disease identification, screening and stratification based on the severity of lung involvement as well as in predicting the risk of complications and the need of intensive care unit (ICU) admission. Imaging helps, nonetheless, in the differential diagnosis of COVID-19 from other kinds of lung infections and diseases. However, due to the rapid diffusion of COVID-19 pandemic, a lot of hospitals and primary and secondary care structures found themselves unprepared, having trouble getting personal protective equipment (PPE) [1], thus making diagnostic imaging procedures difficult and risky to perform, [2] also considering the difficultly to fully and promptly clean the CT scanners between each examination.

In fact, imaging should be reserved to the following precise cases, as suggested in the advice guide for the diagnosis and management of COVID-19 by the WHO [3]:

• For the diagnostic workup of COVID-19 when RT-PCR testing is not available; when RT-PCR testing is available, but results are delayed; and when initial RT-PCR testing is negative, but with high clinical suspicion of COVID-19. In addition to clinical and laboratory data for patients with suspected or confirmed COVID-19, not currently hospitalized and with mild symptoms in order to decide on hospital admission/home discharge or on regular ward admission/intensive care unit admission.
• In addition to clinical and laboratory data for therapeutic management of patients with suspected or confirmed COVID-19, currently hospitalized and with moderate to severe symptoms.

Due to its high availability, portability and cost-effectiveness, chest X-ray (CXR) is the most widely used diagnostic imaging modality against COVID-19, contributing to the first assessment of patients with respiratory symptoms. Patients affected by COVID-19 can present with a pattern varying from normal lung to bilateral interstitial involvement, to opacification, based on the stage of the disease and the clinical presentation [4].

Chest computed tomography (CT) is usually performed in critically ill patients, in which there could also be the need to rule out pulmonary thromboembolism which can be a fatal complication of COVID-19 infection. CT imaging is more accurate than CXR, and is also used in cases of dubious finding at the radiographs: CT patterns are represented by peribronchial and peripheral ground-glass opacities (GGO), mostly basal and bilateral, with involvement of two or more lung lobes, with an increase in severity and consolidation and/or crazy paving pattern as the disease advances in the middle and late stages. However is important to outline that CT patterns of COVID-10 pneumonia are not specific, and superimposable to many other infectious and non-infectious pneumonia [5,6,7].

Lung ultrasound (US) does not have a clear role in the diagnostic approach to a suspected or confirmed COVID-19 case. Due to its great availability and mobility, it can be of great use for bedside evaluation of subpleural consolidations, pneumothorax and alveolar damage, even though its diagnostic accuracy greatly depends on the operator experience [8,9,10]. Recent progress has been made in diagnostic imaging with the integration of artificial intelligence (AI) with computer-aided design (CAD) softwares [11], leading to an increase in the accuracy of exams’ interpretation and to the extraction of prognostic information useful in the decision-making process [12,13,14,15].

Specifically, COVID-19 has catalyzed the rapid expansion in the application of AI to combat disease. As a result, previous authors made a summary of the work performed and the discriminatory ability of AI in its various diagnostic imaging applications.

Ghaderzadeh et al. in their systematic review analyzed papers published between 1 November 2019, and 20 July 2020 regarding the application of deep learning (DL) in chest X-ray and CT. In this review, they suggested that DL-based models share high accuracy in the detection and diagnosis of COVID-19 and that the application of DL reduces false-positive and negative errors compared to radiological examination performed by a radiologist [16].

Another review article by Shi et al. focused on the role of AI in chest CT and CXR in COVID-19 affected patients. They gave an overview of the whole pipeline regarding the implementation of DL in chest imaging, from image acquisition, segmentation to diagnosis, giving also insights regarding the follow-up and the public datasets available [17].

In this review, we explore the role of AI/ML in the diagnostic imaging of patients with COVID-19, including deep learning integration, radiomics features and quantitative CT imaging algorithms. We discuss its wide-range applications on the following domains:

Identification and screening of COVID-19 pneumonia,

For setting the differential diagnosis between COVID-19 pneumonia and other types of infectious pneumonia.

In the stratification and definition of severity and complications of COVID-19 pneumonia.

2. Search Strategy

Before setting up our search strategy we aimed at answering the following questions:

(1)

What are the main indications for COVID-19 imaging?

(2)

What is the workflow followed in image elaboration for AI solutions?

(3)

Does DL improve the diagnostic abilities of radiologists in COVID-19 patients?

(4)

What are the other applications of AI in COVID-19 patients (apart from the identification of the lesions?

(5)

Are there any limitations for AI in this field?

After defining the aforementioned research question, we searched using the PubMed database by inserting the following keywords: “COVID-19,” “diagnosis,” “artificial intelligence,” “detection,” “chest x-ray,” “chest CT,” “deep learning,” “stratification,” “prognosis,” “differential diagnosis,” eventually, the related published studies were extracted and reviewed. We set inclusion criteria to refine the selection of manuscripts based on our subjective assessment of their relevance, novelty and being in English language.

3. Workflow of Images Segmentation, Annotation and Elaboration

Development of AI-based COVID-19 classification/segmentation models starts from their training with various images sources, usually represented by normal and abnormal (COVID-19, non-COVID-19) chest images. Data collection is, therefore, considered mandatory.

The whole workflow of image annotation, segmentation, and elaboration is shown in Figure 1.

Patients’ data must be downloaded, queried, correctly de-identified and safely stored after ethical consent. The best approach to de-identification is pseudonymitazion; when the DICOM images are pseudonymized, the information that can point to the identity of a subject is replaced by “pseudonyms” or identifiers [18].

Manual selection of similar images according to basic criteria (age, technique, imaging findings) is always performed by expert radiologists to have the best training dataset. Image segmentation is a fundamental part of image processing and analysis for assessment of pathologic examinations. Segmentation is based on delineation of regions of interest (ROIs), as lung lobes, airways, focal or diffuse pathologies in the images [19,20,21,22,23]. A robust training model needs sufficient labeled images, which usually lack in case of COVID-19, mostly due to the time-consuming nature of this task in a pandemic setting; in these cases, the radiologist can be asked to interact with the segmentation network to supervise the machine learning methods [24]. An appropriate segmentation may help in monitoring the progression of COVID-19 pneumonia and the assessment of severity. AI models can be trained using available datasets or with the “transfer learning” method, making the most of already available models which also avoid mixing training and test data [25]. Features obtained from different convolutional neural network models can be classified with a support vector machine (SVM) classifier using images [26]. After training and testing, one or more other sets of images can be used for external validation of the model.

4. Artificial Intelligence in Chest X-ray

Several studies focused on the automatic classification of COVID-19 from CXR images [27,28,29,30,31,32,33,34,35], considering how useful it could be in emergency departments, urgent care, and resource-limited settings. Moreover, by matching CXR findings to clinical data prognostic models can be developed, to predict disease gravity, and stratify patients on the basis of their risk of develo** severe disease and or complications.

4.1. AI in the Identification of COVID-19 Pneumonia at Chest X-ray

CXR can help in identify signs of pneumonia, also in case of negative RT-PCR test: sensitivity of CXR greatly depends on the stage of the lung infection and on the extent of the disease, as well as on the technical quality of the exam (usually performed bedside in critically ill patients), ranging from 50% to 84% [36,37,38]. Specificity is low, attested at 33% [36]. However, the COVID-19 pandemic kickstarted the development of AI-based models worldwide, for the automatic detection of pneumonia signgs on CXR images, which yielded great results: using automated machine learning algorithms and deep convolutional neural networks (DCNN), as well as deep transfer learning techniques, various Authors presented results in COVID-19 detection in which obtained a sensitivity ranging from 97.9% to 100%, a specificity between 95% and 98.8%, an accuracy ranging from 83.5% to 98%, and precision of up to 97.95% [27,35,39,40,41,42].

Accuracy can be improved by up to 99.41% when using support vector machines (SVM), which are supervised learning methods based on statistical learning theory [43] that work by dividing the dataset in training and test subsets [44,45], and up to 100% when using twice transfer learning (also known as transfer learning in three steps), and output neuron kee** (kee** output neurons that classify similar classes between the second and third step of the twice transfer learning), which improves training speed or performances particularly in the first phases of the training process [46]. Other approaches in COVID-19 pneumonia identification were performed using several convolutional layers and applying filters to each layer [33], as well as introducing stochastic pooling in DCNN [47], or using multiresolution approaches with improved results when compared to deep learning methods [48,49].

Moreover, Sahlol et al. used an efficient hybrid classification which adopted a combination of CNN and an improved swarm-based feature selection algorithm. This combination should achieve two main targets; high performance and resource consumption, storage capacity. In addition, they also proposed a novel robust optimizer called Fractional-order Marine Predators Algorithm (FO-MPA) to efficiently select the huge feature vector produced from the CNN. Then, they tested and evaluated the proposed approach by performing extensive comparisons to several state-of-art feature selection algorithms, most recent CNN architectures and most recent relevant works and existing classification methods of COVID-19 images [50].

Table 1. AI in the identification of COVID-19 pneumonia at Chest X-ray.

Authors	Year	Population (No. of Patients)	ML Model	Results
Apostolopoulos et al.	2020	First dataset: 224 Covid+, 1204 covid-. plus a second dataset with 224 Covid+, 1218 Covid-	different CNNs (VGG19, MobileNet v2, Inception, Xception, Inception ResNet v2)	acc 96.78%, sen 98.66%, spe 96.46% (for binary class), acc 93.48% (for multi-class)
Ozturk et al.	2020	127 Covid+	DarkNet	acc 98.08%, sen 95.13%, spe 95.3%, (for binary class) acc 87.02%, sen 85.35%, spe 92.18%, (for multi-class)
Wang et al.	2020	358 Covid+, 13,604 Covid-	covid-net	acc 95%, sen 93%, spe 96% (for multi-class)
Borkowski et al.	2020	training: 484 Covid+, 1000 Covid-; validation: 10 Covid+, 20 Covid-	Microsoft custom vision	acc 97%, sen 100%, spe 95% (for binary)
Chowdhury et al.	2020	219 Covid+, 2659 Covid-	PDCOVID-net	acc 96.58%, pre 96.58%, rec 96.59%, F1 96.58% (for multi-class: covid, normal, viral pneumonia)
Toraman et al.	2020	231 Covid+ (1050 with data augmentation), 2100 Covid-	CapsNet	acc 89.48%, sen 84.22%, spe 92.11% (for multi-class: covid, normal, pneumonia)
Ouchicha et al.	2020	219 Covid+, 2686 Covid-	CVDNet	acc 97.79%, sen 96.83%, spe 98.02% (for multi-class: covid, normal, pneumonia)
Togacar et al.	2020	295 Covid+, 163 Covid-	MobileNet+squeezenet+SVM	acc 98.83%, sen 97.04%, spe 99.15% (for multi-class: covid, normal, pneumonia)
Hassantabar et al.	2020	315 Covid+, 367 Covid-	CNN and DNN	CNN: accuracy 93.2, sensitivity 96.1, DNN: accuracy 83.4, sensitivity 86
Mukherjee et al.	2021	Various datasets	CNN	Accuracy: 96.13

provides a summary of the papers included in the review, focused on AI in the identification of COVID-19 pneumonia signs at CXR. Figure 2 shows the distribution of subjects included considering those studies where it was clearly stated.

4.2. AI in the First Assessment of COVID-19 Pneumonia at Chest X-ray

As CXR is often the first-line diagnostic imaging modality when facing a patient suspected of COVID-19 infection, even if less sensitive than lung CT, it plays a great role in the first assessment of patient. Even though the confirmation of COVID-19 infection should always come from RT-PCR tests performed on naso-pharyngeal swabs [51], these tests could not be readily available and may take time to give the result; therefore, a rapid CXR assessment of patients with respiratory symptoms should be performed, and AI can play an important role, especially when dealing with a large number of requests in the emergency settings [52]. Most literature studies use AI in CXR to distinguish between COVID-19 and other pneumonia and healthy patients [53,54,55]. ** monitoring disease evolution and course, and identifying patients at risk of ICU admission [57,58]. However, there is no standardized method in reporting CXR findings in terms of disease severity. Li et al. used the pulmonary x-ray severity (PXS) score, a DL-based algorithm providing quantitative measures of COVID-19 severity on CXR, as an adjuvant tool to radiologists’ work—which, however, always decided on the severity grading and definitive radiological report-, and noticed an improvement in the assessment of the severity on a 4-point scale (normal/minimal, mild, moderate, severe) and in the inter-reader agreement, with no need for radiologists’ training on the use of the score [59,60]. Li et al. also found that the severity scores were significantly associated with intubation/death within 3 days from the admission, in CXR rated moderate or severe [59]. Mushtaq et al. reported in their retrospective study that an AI-powered severity score based on the percentage of pixels involved by opacity or consolidation for each lung at the CXR, adjusted at the multivariate analysis for demographics and comorbidities, showed that a value ≥30 at the hospital admission CXR was an independent predictor for mortality and ICU admission for COVID19 (p < 0.001), and found a significant link with admission pO2/FiO2 levels [61]. Zhu et al. compared the evaluation of an AI algorithm to the one performed by independent expert radiologists on the results of CXR in patients suspected for COVID19 in terms of disease severity using criteria based on the degree of lung opacity and geographical extent of the opacity, finding a strong correlation between the two severity scores [62].

Table 3. AI in the stratification and definition of severity and complications of COVID-19 pneumonia at CXR.

Authors	Year	Population (No. of Patients)	ML Model	Results
Li et al.	2020	various datasets	convolutional siamese NN	AUC 0.80
Mushtaq et al.	2021	697 covid+	qXR	Achieving a statistical significance in predicting negative outcome in ED patients.
Zhu et al.	2020	131 covid+	VGG16	AI-predicted scores were highly correlated with radiologist scores

provides a summary of the papers included in our review focused on AI in the stratification and definition of severity and complications of COVID-19 pneumonia at CXR. Figure 4 shows the distribution of subjects included considering those studies where it was clearly stated.

4.4. AI in the Differential Diagnosis of COVID-19 Pneumonia from Other Pneumonia at Chest X-ray

Various authors also investigated the effectiveness of supervised AI learning models in aiding medical professionals in the differential diagnosis between COVID-19 pneumonia and other lung diseases, in particular the non-COVID-19 viral pneumonia, with a reported accuracy of up to 87% [33,39,41,42,63,64]. ** et al. proposed a three-step hybrid model, incorporating a feature extractor, feature selector, and an SVM classifier, reporting an overall accuracy rate of 98.6%, with a remarkable reduction of training time and of the training sets size [65].

However, the differential diagnosis is impaired by the aspecific picture of COVID-19 pneumonia, similar to other viral and non-viral interstitial diseases. AI models should be adequately trained to achieve state-of-the-art diagnostic efficacy in the external validation process and in the real-life radiological workflow: CXR obtained in different views (postero-anterior (PA), latero-lateral, as well as bedside ones) must be differentiated, and the same goes for age groups, distinguishing pediatric patients from adults. Some authors chose to train models only on PA views, as it is usually the most common view used in the emergency department, even though bedside CXR are getting more and more important in the first diagnosis and in monitoring critically ill patients [66,67].

AI evolution could aim to help the diagnostic radiology in screening, diagnosing and grading CXRs, even though there are serious concerns on the potential risk of this situation happening [68].

Table 4. AI in the differential diagnosis of COVID-19 pneumonia from other pneumonia at Chest X-ray.

Authors	Year	Population (No. of Patients)	ML Model	Results
Varela-Santos et al.	2021	various datasets (Cohen, Kermany)	FFNN, CNN	Various AUC values depending on the dataset/population/network considered
** et al.	2021	various datasets (NIH chext x ray database and others): 543 covid+, 600 covid-, 600 normal	hybrid ensemble model (AlexNet with ReliefF algorithm and SVM classifier)	accuracy 98.642, specificity 98.644, sensitivity 98.643, AUC 0.9997
Sharma et al.	2020	various datasets	CovidPred	accuracy 93.8
Tsiknakis et al.	2020	various datasets (Cohen, QUIBIM imagingcovid19): 137 covid+, 150 covid-, 150 normal	Inception-V3	sensibility 99, specificity 100, accuracy 100, AUC 1 for binary class (covid vs. other pneumonia)

provides a summary of the papers included in the review focused on AI in the differential diagnosis of COVID-19 pneumonia from other pneumonia at Chest X-ray.

5. Artificial Intelligence in Chest CT

Machine learning approaches applied to CT images in COVID-19 pneumonia show great potential for improving diagnostic accuracy as well as for the prediction of patient outcomes and many studies have been focused on this topic.

Indeed, AI takes advantage of the large quantity of imaging data that can be used to train algorithms, and if effective, it could bring to a revolution in the identification and triage of patients with suspected COVID-19.

5.1. AI in the Identification of COVID-19 Pneumonia and Its Complications at Chest CT

From the beginning of the COVID-19 pandemic, the use of AI for detection of the radiological signs of pneumonia on CT imaging has been investigated, also in cases of false-negative results at RT-PCR [69], and augmented radiologists workload [70].

Considering the central role of imaging in the management of infected patients, multiple deep-learning algorithms have been developed to face the increased needs, also within just 10 days [71]. A pilot study by Yang et al., performed in the first two months of 2020, evaluated the performance of a DenseNet algorithm model—an improved CCN—for COVID-19 detection on HRCT. It yielded an AUC of 0.98 and a sensitivity of 97%, but an accuracy of 92% and specificity of 87% resulted slightly lower than those of an experienced radiologist. The authors concluded that their DL model had a human-level performance and allowed to save time due to a rapid diagnosis in about 30 s versus 5–10 min needed by a radiologist. A limitation of this study was a restricted number of included patients (146 with COVID-19 and 149 controls), further divided into training, validation and test sets [72].

To overcome this limit, multiple studies utilized datasets composed of thousands of patients derived from public sources or as occurred in multicenter trials. Therefore, Harmon et al. analyzed a heterogeneous multinational CT dataset composed of 2617 patients, overcoming a limited applicability to different populations, demographics or geographies, and maximizing the potential for generalizability. The 922 included cases of COVID-19 were from China, Italy and Japan, while the balanced control population was identified either from 2 US institutions or from a publicly available dataset (LIDC). Their image classification model used both hybrid 3D and full 3D models based on a Densnet-121 architecture, and they achieved a 0.949 AUC, resulting in 90.8% accuracy for COVID-19 identification on chest CT [73].

In addition to public datasets, previously validated AI algorithms are available for further confirmation of their performance or as assistant tools to clinicians and radiologists [74]. In this regard, Chen et al. created a cloud-based open access AI platform to improve the diagnosis of COVID-19 pneumonia. They developed a UNet++-based model with an accuracy of 96% for COVID-19 detection on HRCT in multiple testing datasets, either internal (retrospective and prospective) and external ones. Furthermore, the use of a similar deep-learning based model has the potential to reduce the number of missed diagnosis, especially in early phases, because the lung infection foci could be mild and need observation under 0.625-mm layer scanning [75].

Other authors focused not only on the pneumonia detection on a CT scan, but also on a quantitative assessment [74]. In fact, Zhang et al. analyzed images from 2460 patients using the uAI Intelligent Assistant Analysis System (a modified 3D CNN and a combined V-Net with bottle-neck structures) to segment anatomical lung structures and to accurately localize infected regions, according to the specific lobes and segments. Their findings were consistent with those of previous studies [76] that demonstrate a typical bilateral involvement, mainly in the dorsal segments, with GGOs as the most common CT feature [77].

These results have been confirmed also in other studies about the role of quantitative CT [78]. Du et al. evaluated pre-discharge CT scans in asymptomatic patients with negative RT-CR with an AI-assisted system (InferRead CT pneumonia software). Their quantitative image analysis resulted in a prevalence of fibrosis as the second common manifestation after GGOs, characterized by heterogeneous density and rigid reticulation [79].

To ease the evaluation of COVID-19 patients according to the findings on chest CT scan, the standardized score CO-RADS has been introduced to grade the level of suspicion from very low (1) up to very high (5), providing a higher performance in patients with moderate and severe symptoms (average AUC 0.91 for predicting RT-PCR outcome and 0.95 for clinical diagnosis) and a higher interobserver agreement for categories 1 and 5 [80]. Lessmann et al. aimed to develop a CO-RADS AI system to obtain an automated assessment of the suspicion value. CO-RADS AI included three deep-learning algorithms based on a U-Net architecture that automatically performed lobe and lesion segmentation, prediction of a CT severity score according to the percentage of affected parenchymal tissue per lobe and, at last, the assignment of the CO-RADS value. The key result of this study was a high diagnostic performance in the identification of COVID-19 patients with an AUC curve of 0.95 in the internal test set and of 0.88 in the external cohort [81]. However, its use is controversial because it does not take into consideration clinical and laboratory findings to build a diagnosis of COVID-19, also AI-assisted.

In fact, a study by Liu et al. demonstrated that a combined clinical-radiological model outperformed the CO-RADS and a clinical model in the COVID-19 diagnosis. Their preliminary study investigated the performance of a combined radiomics model that included 5 clinical features and a radiomic signature, after multivariate logistic regression analysis: age, lesion distribution (central or peripheral), neutrophil ratio, lymphocyte count, CT score and mean Radscore. The latter was calculated by 8 radiomic features, selected after the application of a mRMR algorithm and LASSO logistic regression algorithm. The result was an open-source constructed radiomics model with an AUC of 0.98, sensitivity of 0.94 and specificity of 0.93 [82]. Similar results have been achieved in another study that confirmed a mixed model—presented as nomogram—as the highest predictor of COVID-19 with an AUC of 0.955 (versus an AUC of 0.626 of the clinical model). It included either CT characteristics of the lesions (distribution, maximum lesion range, involvement of lymph nodes and pleural effusions) and a RadScore based on a signature of 3 features selected by LASSO regression [83].

Another use of radiomic models has been described in the non-invasive monitoring of ARDS, a life-threatening COVID-19 complication. Indeed, Chen et al. compared the performance of traditional quantitative and radiomics analysis of CT images. While the former quantified the infected regions through the calculation of volume and percentage of infection, the latter included 30 radiomic features selected by regression analysis and combined into a risk score. Results showed that the radiomics model was the most promising one because of the highest accuracy and specificity, despite a similar AUC of 0.94. According to the authors, sensitivity is more important than specificity in an ARDS screening due to the high risk related to delayed oxygen treatment in false-positivity results [84].

Voulodimos et al. adopted a semantic segmentation approach, which can be implemented in a two-step process: (i) feature extraction over an image patch and (ii) a training process, using annotated datasets. Using this method, each pixel is described by feature values, extracted locally, over a, typically, small area, denoted as “patch”. Deep learning approaches do both steps for a given set of data [85].

The possibility of segmentation transferability in COVID-19 CT has been investigated by Wang et al. They presented a set of experiments to better understand how different non-COVID19 lung lesions influence the performance of COVID-19 infection segmentation and their different transfer ability under different transfer learning strategies. They concluded clear benefits of pre-training on non-COVID19 lung lesion datasets when public labeled COVID-19 datasets are inadequate to train a robust deep learning model [86].

Saood et al. proposed a new fully automated deep learning framework for rapid quantification and differentiation between lung lesions in COVID-19 pneumonia on both contrast and non-contrast CT images using convolutional Long Short-Term Memory (ConvLSTM) networks. They showed a strong agreement between expert manual and automatic segmentation for lung lesions; describing excellent correlations of 0.978 and 0.981 for ground-glass opacity and high opacity volumes [87].

Akram et al. presented a novel entropy-based fitness optimizer function implementation, which selects the chromosomes with maximum information. The only chromosome with maximum fitness value is selected to get the sub-optimal solution in the minimum number of iterations. To conserve maximum information and to obliterate the redundant features at the initial level, a preliminary selection process is initiated on each feature set using the entropy-controlled fitness optimizer. To exploit the complementary strength of all features, a feature fusion approach is utilized which combines all the competing features to generate a resultant feature vector. The previously adopted methods of machine learning utilize either sole or hybrid approaches for feature extraction. Though both methods have their advantages and drawbacks, but the fused feature space has more capacity to retain the dexterous features. Due to this flexibility, the hybrid approaches have gained much popularity among the researchers. However, selection of the most appropriate feature extraction technique is quite a sensitive task, which needs to be handled carefully, otherwise, it may result in feature redundancy and, therefore, increased correlation. In this work, they utilized four different techniques—belongs to two different categories, statistical and texture. Two feature families were not considered, color and shape, because of their limited impact and significance in this application. Using the proposed framework, the achieved accuracy using the Naive Bayes classifier is 92.6%, 92.6%, whereas other classifiers (EBT, L-SVM and F-KNN) behave significantly better to achieve an average accuracy of 92.2%, 92.1%, 92.2%, 92.1% and 92.0%, 92.0%, respectively. From the sensitivity and specificity values, the proposed framework was successfully managed to achieve high true positive and negative rates [88].

Mukherjee et al. developed a CNN-tailored DNN for COVID-19 diagnosis, integrating either CT and CXR images. Their proposed DNN based on a mixed database of integrated modalities reached an AUC of 0.9808, higher than those of other existing DNN (Inception, MobileNet and ResNet). Moreover, the performances score using separate dataset appeared to be higher for CXRs with an AUC of 0.9908 vs. 0.9731 for CT scan [89].

Table 5. AI in the identification of COVID-19 pneumonia and its complications at Chest CT.

Authors	Year	ML Model	Population (No. of Patients)	Results
Anastasopoulos et al.	2020	U-Net	197 COVID+, 141 COVID-	Dice coefficient: 0.97
Yang et al.	2020	DenseNet	146 COVID+, 149 COVID-	AUC: 0.98
Harmon et al.	2020	AH-Net(segmentation) Densenet3D/2D+1 (classification)	922 COVID+, 1695 COVID-	AUC: 0.949—original design, 0.941—independent population
Ni et al.	2020	MVP-Net, 3D U-Net	14,435 (training): 2154 COVID+, 12,281 COVID- + 96 COVID+ (testing)	Accuracy: 82—per-lobe lung level, 0.94—per-patient level
Chen et al.	2020	U-Net++ with a ResNet50 backbone	106 (training and retrospective testing): 51 COVID+, 55 COVID- +27 (internal prospective testing): 16 COVID+, 11 COVID- +100 (external prospective testing): 50 COVID+, 50 COVID- 27 (internal prospec-tive testing): 16 COVID+, 11 COVID- + 100 (external pro-spective testing): 50 COVID+, 50 COVID-	Accuracy: 95.24—retrospective testing, 92.59—internal prospective testing, 96—external prospective testing
Zhang et al.	2020	QCT	2460 COVID+	Identification of lesions
Ma et al.	2020	QCT	18 COVID+	Identification of lesions and dynamic changes
Du et al.	2020	QCT	125 COVID+	Identification of lesions and dynamic changes
Lessmann et al.	2020	Two-stage U-Net (lobe segmentation and labeling), 3D U-net with nnU-Net framework (CT severity score prediction), 3D-inflated Inception (CO-RADS score prediction)	476 (training) 105 (internal test): 58 COVID+, 47 COVID- 262 (external test): 179 COVID+, 83 COVID-	AUC: 0.95—internal testing, 0.88—external testing
Liu et al.	2021	Radiomics	115 COVID+, 435 COVID-	AUC: 0.93
Fang et al.	2020	Radiomics	239 (training): 136 COVID+, 103 COVID- 90 (validation): 56 COVID+, 34 COVID-	AUC: 0.955
Chen et al.	2020	Radiomics	84 COVID+	AUC: 0.94
Voulodimos et al.	2020	FCN, U-net	10 COVID+	Unclear data: FCN Accuracy: ~0.9 (validation); Accuracy U-net: >0.9 (validation)
Sahood et al.	2021	U-net, SegNet	100—one slice CT scans	Accuracy: SegNet: 0.954; U-Net: 0.949
Mukherjee et al.	2021	CNN	336 COVID+, 336 COVID—(CXR + CT)	AUC CXR+CT: 0.9808 (AUC CT: 0.9731)

provides a summary of the papers included in the review focused on AI in the diagnosis of COVID-19 pneumonia at Chest CT. Figure 5 shows the distribution of subjects included considering those studies where it was clearly stated.

5.2. AI in the Screening of COVID-19 Pneumonia at Chest CT

The application of AI to CT images for the immediate triage of COVID-19 patients may be of assistance due to delayed results of RT-PCR as definitive viral testing.

Javor et al. used an open-source data of 6868 CT images to train their CCN model ResNet50 that achieved high accuracy with an AUC of 0.956, higher than those of radiologists. They described the importance of the ML model in the patient triage for the possibility to identify rule-in and rule-out thresholds for COVID-19 diagnosis, compared to a dichotomous decision of radiologists. In case of high level of suspicion, the patient should be isolated until the confirmation of rejection by an RT-PCR test [90]. However, CT scan may have a low negative predictive value, especially in early phases of the disease. A joint AI algorithm that integrated chest CT findings and clinical history enabled a rapid diagnosis of COVID-19 with an AUC of 0.98 that might have a fundamental role in the triaging, allowing rapid isolation of infected people and avoiding delayed treatments. The evaluated model was first developed on a CNN to learn imaging characteristics on initial CT scans and then on a MLP to classify patients according to the clinical information (sex, age, exposure history, clinical symptoms—fever and cough—and laboratory findings—WBCs). Finally, a neural network model combined radiological and clinical data to predict COVID-19 status [91].

Another study performed in an emergency department confirmed the positive performance of a mixed predictive ML model in the triage. It was based on the CO-RADS score from chest CT and additional data—laboratory findings (ferritin, leukocytes, CK), diarrhea and number of days from onset of the disease. The added value of the prediction model compared with CT alone was increased AUC (0.953 vs. 0.930) and accuracy (93.1% vs. 90.4%), probably due to specific laboratory anomalies. Nevertheless, authors concluded that 9% of the included patients with positive RT-PCR were false negative according to the prediction model and the nasopharyngeal swab should be the primary standard test [92].

In

Table 6. AI in the screening of COVID-19 pneumonia at Chest CT.

Authors	Year	ML Model	Population (No. of Patients)	Results
Javor et al.	2020	ResNet50	209 COVID+, 209 COVID-	AUC: 0.956
Mei et al.	2020	LeNet, YOLO, DenseNet (pipeline developed in previous work)	419 COVID+, 486 COVID-	AUC: 0.92
Hermans et al.	2020	Logistic regression (no DL)	133 COVID+, 16 COVID-	AUC: 0.953

, we provided a summary of the papers included in our review focused on AI in the screening of COVID-19 pneumonia at Chest CT. Figure 6 shows the distribution of subjects included considering those studies where it was clearly stated.

5.3. AI in the Stratification and Definition of Severity and Complications of COVID-19 Pneumonia at Chest CT

Different studies have already demonstrated the correlation between conventional CT scores and prognosis of COVID-19 patients, using semi-quantitative methods based on visual scores [93,94,95]. As an attempt to avoid subjective and time-consuming evaluations, multiple AI models have been developed and tested to accurately stratify patients into severity stages and to improve the clinical decision-making process. According to the ATS, the major criteria for the definition of severe pneumonia are respiratory failure in need for mechanical ventilation (MV) or septic shock treated with vasopressors; other minor criteria include increased respiratory rate (>30/min), P/F ratio < 250 or hypotension requiring fluid resuscitation [96]. Therefore, these are the most common endpoints used to find potential high-risk patients.

According to Chatzitofis et al., a VoI aware DNN could assess patients’ conditions and prognosis even without results of laboratory tests, as occurred shortly after the ED admission. They introduced a two-stage data-driven approach to classify patients into three classes—moderate, severe and extreme, considering their risk to be discharged, hospitalized or admitted to ICU, respectively. The proposed algorithm was trained with a COVID-19_CHDSET Dataset, composed by CT images from Milan, an extensively involved area during the first months of the COVID-19 pandemic. The DenseNet201-VoI model reaches an AUC of 0.97, 0.92 and 1.00 for the three groups, respectively, and accuracy of 88.88%, specificity of 94.73% and sensitivity of 89.77% [93]. ** (Grad-CAM) simplified the interpretability of the proposed model: it was an automatically generated heatmap that applied the red color to the suspected regions associated with the predicted class [133]. Other studies aimed to evaluate not only the performance of a proposed AI model in the differential diagnosis, but also the radiologist’s performance with and without AI assistance [131].

A retrospective study employed an EfficientNet architecture for the pneumonia classification task and a heatmap generated through a Grad-CAM for the visualization of the important image regions. The proposed model achieved an AUC of 0.95 and a higher accuracy, sensitivity and specificity than those of experienced radiologists (96% vs. 85%, 95% vs. 79%, 96% vs. 88%). Authors deduced that the performance of radiologists with AI assistance improved compared to manual interpretation, yielding higher accuracy (90%), sensitivity (88%), and specificity (91%) [133].

Another observation study by Zeng et al. tested a ML algorithm based on a radiomic texture analysis of CT imaging to distinguish pneumonia due to COVID-19 (NCP) and Influenza A (IAP). Their nomogram included 8 radiomic features as independent diagnosticators of NCP after application of LASSO regression model that were subsequently included into a radiomics score (higher values suggested COVID-related pneumonia). Their data suggested an excellent performance of the nomogram with an AUC of 0.87, hel** clinicians in the choice of the right management [135].

Table 8. AI in the differential diagnosis of COVID-19 pneumonia from other pneumonia at Chest CT.

Authors	Year	ML Model	Population (No. of Patients)	Results
Song et al.	2020	BigBiGAN	98 COVID+, 103 COVID-	AUC: 0.972—internal test, 0.850—external validation
Yan et al.	2020	EfficientNetB0	206 COVID+, 412 COVID-	AUC: 0.962—per-slice, 0.934—per-scan
Liu et al.	2020	Radiomics	61 COVID+, 27 COVID-	AUC: 0.99
Yang et al.	2020	ResUNet	118 COVID+, 576 COVID-	AUC: 0.903
Bai et al.	2020	EfficientNet-B4	521 COVID+, 665 COVID-	AUC: 0.95—internal testing, 0.90—independent testing
Li et al.	2020	COVNet (novel)	468 COVID+, 2854 COVID-	AUC: 0.96
Abbasian Ardakani et al.	2021	COVIDiag	306 COVID+, 306 COVID-	AUC: 0.965
Zeng et al.	2020	Radiomics	41 COVID+, 37 COVID-	AUC: 0.87

provides a summary of the papers included in the review focused on AI in the differential diagnosis of COVID-19 pneumonia from other pneumonia at Chest CT. Figure 8 shows the distribution of subjects included considering those studies where it was clearly stated.

6. Computational Cost

A brief introduction to the concept of the computational cost is due. Computational cost is a generic name that refers to the computational power in (usually in terms of number of operations and memory) required to run an algorithm. Even the most demanding algorithms can be executed in reasonable time when more computational resources are provided. Generally speaking pipelines not based on deep learning have a rather low computational cost, both during training and inference. Indeed, studies based on radiomics and quantitative CT do not require expensive or very performant hardware to reach very low run times. Deep learning models, on the other hand, require modern, dedicated hardware (GPUs) to train in reasonable time but may still require multiple days to train.

This does not hinder their effectiveness or their use in production as the inference time is usually significantly lower. Among deep learning architectures some are designed specifically for a lower computational cost [136] while others focus on performance disregarding computational efficiency [137]. In particular, studies employing 3D convolutions [74] or studies that leverage multiple large models [81] are very computationally intensive and probably would require an amount of resources that few hospitals can provide. Nonetheless, for pipelines dedicated to a single disease, the required throughput is not too high and larger models can still provide value.

7. Discussion

In this literature review, we presented a structured review on the applications that AI can have in the clinical setting with regards to chest imaging in COVID-19 patients, describing the performances that the several DL/radiomics models have both in the identification, screening, stratification of patients as well as the differential diagnosis with other pneumonia.

Some of the previously described models showed very high performances, suggesting that the implementation of AI techniques would aid radiologists in their clinical practice, leading to a significant increase in accuracy values and leveraging their daily workflow performance.

However, the potential utility of the machine learning-based models using CXR and CT images for diagnostic and prognostic purposes in COVID-19 has been analyzed in a systematic review that included some of the previously discussed studies [21,84,91,98,99,121,132].

According to Roberts et al. [138], none of the included studies in their systematic review showed a sufficient robustness and reproducibility to be integrated into the regular clinical practice, due to biases in datasets, either too small or too heterogeneous, poor data integration or insufficient validation. In addition, some machine learning models may show over and under-fitting bias.

Specifically, as concerns the quality of the training data of the analyzed studies [138] the authors suggested the following key issues:

• a warning about using online repositories because of (1) the potential bias attributable to source issues and the inability to match demographics through populations (2) the possible overfitting on the shared dataset (3) the eventual low-resolution unbalanced across classes of the images of the shared dataset.
• to pay attention to CXRs projections (anteroposterior vs. posteroanterior) since models can wrongly correlate more severe disease to the view of the radiogram and not to the actual radiographic findings
• most studies did not report the timing between imaging and RT–PCR tests, since a negative RT–PCR test is a definitive exclusion criteria COVID-19 infection.

The authors recommended also major attention in the development of further ML-based algorithms; suggesting external validation, assessment with established frameworks (e.g., QUADAS, CLAIM, RQS) and checklists to identify these weaknesses [138].

Furthermore, other authors advised the sampling of large datasets to reduce predictive uncertainty, even though most works used small image samples, due to the lack of large open COVID-19 datasets (particularly for CXR) [139,140,141,142].

This is why further studies are needed to implement AI capacities in the above discussed settings (identification, screening, patients’ stratification and differential diagnosis), in order to guide the development of AI-empowered tools to reduce human error and assist radiologists in their decision-making process.

Limitations of the study:

Firstly, we would like to cite some limitations of the reviewed studies which include inadequate verification of datasets [138], limited time available considering the on-going pandemic, lack of large datasets for some authors. It’s worth mentioning that the first published work that reviews the usability of X-ray images to detect COVID-19 was of a very limited dataset [143]. In some investigations, the number of positive images used in the training was less than 100, which greatly limits the generalization power of the models, under the CNN paradigm [144]. The rapidly evolving and emerging applications of AL/ ML in COVID-19 can also represent another hurdle for reviewing the previous work. Some authors have managed to release newer versions of their early pandemic studies; enforcing their algorisms with larger datasets, including clinical information, overcoming some of the technical issues that was raised earlier such as over-fitting. Additionally, to avoid the limitations regarding the selection bias, we set a structured criteria for inclusion and exclusion of the selected studies.

8. Conclusions

The combination of chest imaging and artificial intelligence can help for a fast, accurate and precise disease extent quantification as well as for the identification of patients with severe short-term outcomes. AI/ ML as well as radiomics have feasible applications and optimistic potential to help leverage the radiologists’ workflow in the current pandemic. In other words, there are multiple domains that can benefit from AI applications in chest imaging, including identification, screening and risk stratification of COVID-19 cases. As aforementioned, the basic stages to tackle that pandemic include early and accurate identification of COVID-19, and ML can play a crucial role in this setting.

The integration of ML techniques will help in diagnosing this condition faster, cheaper, and safer in the upcoming years. However, various biases should be overcome in the development of further ML-based algorithms to guarantee sufficient robustness and reproducibility for their integration into clinical practice.

Though, as previously stated by Roberts et al. [138], many of those ML models developed could not be proved to be ready for the translation in clinical practice.

Datasets of higher quality, articles with enough documentation to be repeatable as well as external validation are required to give the currently developed ML models a sufficient robustness and reproducibility to integrate them into clinical practice.