1.

Image Acquisition Systems for COVID-19

In a review by Khaksari et al., “ Review of the efficacy of infrared thermography for screening infectious diseases with applications to COVID-19,” authors evaluated the efficacy of infrared thermography (IRT) in COVID-19 screening. They noted the need to assess IRT ability to detect true core body temperature and to identify asymptomatic carriers. The review gives insight in to an imaging technique that is often not encountered by those in radiological image developments. In the authors’ conclusion, they note that thermal cameras alone are not sufficient for COVID-19 screening, and that other vital physiological characteristics need to be monitored, such as change in tissue oxygenation, and cardiovascular and respiratory functions.

Barrett and Caucci, in “ Stochastic models for objects and images in oncology and virology: application to PI3K-Akt-mTOR signaling and COVID-19 disease,” present potential new methods of image acquisition through the mathematics of the physiological objects and their molecular images. The authors discuss the application of patient-specific and disease-specific characteristic functionals to COVID-19, to model the physiological random processes that can potentially be observed through in vivo molecular imaging, such as in single-photon emission computerized tomography (SPECT).

2.

Detection/Diagnosis of COVID-19 with Machine Learning

In the detection/diagnosis of COVID-19 through thoracic imaging (either chest radiographs or CT images), the question addressed by the AI algorithms varies across papers. Reading across these papers brings up many questions, such as what is the definition of “normal?” Many employed transfer learning methods with initial weights from pretrained networks using natural scenes and/or thoracic images with other disease states such as pneumonia.

Manokaran et al., in “ Detection of COVID-19 from chest x-ray images using transfer learning,” developed an automated deep learning system for the detection of COVID-19 from chest radiographs through the assessment of eight CNN approaches. They addressed the question of distinguishing between normal and pneumonia cases and COVID-19 positive-tested cases.

Rao et al., in “ COVID-19 detection method based on SVRNet and SVDNet in lung x-rays,” developed a detection method for distinguishing between COVID-19 positive-tested cases and COVID-19 negative-tested cases. Through the comparison of multiple CNNs, they concluded that COVID-19 positive cases tended to have different radiographic presentations. For example, positive cases tended to show more ground-glass opacities as compared to negative cases.

Chaddad et al., in “ Deep CNN models for predicting COVID-19 in CT and x-rays,” also used transfer learning, multiple public datasets, and multiple deep networks while working with regions within chest radiographs and CT scans. The use of regions allowed the number of inputs to increase substantially and boost performance.

Kusakunniran et al., in “ COVID-19 detection and heatmap generation in chest x-ray images,” used ResNet-101 as the backbone CNN architecture, along with heatmaps based on weights from the final convolutional layer. Similar to others, the authors noted that while high performance could be achieved when distinguishing between COVID-19 cases and normal chest x-ray images, lower performance resulted when distinguishing between COVID-19 and other lung diseases such as fibrosis, pneumonia, and pulmonary edema.

Hu et al., in “ Role of standard and soft tissue chest radiography images in deep-learning-based early diagnosis of COVID-19,” included novel analyses of both conventional and soft tissue images in the deep transfer learning of chest radiographs. Many thoracic radiographic exams are obtained using dual-energy subtraction or virtual bone suppression software. The authors’ dataset consisted of the first CXR exams of 9860 patients acquired within two days after initial reverse transcription polymerase chain reaction tests for the SARS-CoV-2 virus. A sequential transfer learning strategy was employed to fine tune a convolutional neural network in phases on increasingly specific and complex medical interpretation tasks. The authors also incorporated cropping of the lung region via a U-Net variant, in order to yield improved heatmaps of influential regions and avoid irrelevant regions outside of the lung that might erroneously contribute to deep learning prediction scores.

With thoracic CTs, one is able to analyze the 3D nature of the COVID-19 presentation. Li et al., in “ 3D CNN classification model for accurate diagnosis of coronavirus 2019 using computed tomography images,” compared the performance of various 3D ResNet models to distinguish COVID-19 positive images from those of other common pneumonia and of normal controls. A novel aspect of their work was to superimpose CT slices into volumes of different depths. The authors confirmed that CT slices in the test set were selected from individuals who were not included in the training and validation stages. They also produced saliency maps to indicate influential regions.

3.

Prognosis of COVID-19 with Machine Learning to Assess Severity

While many AI developments have involved methods for detection and diagnosis of COVID-19 of patients, there is also a need for AI methods to help assess the severity of the COVID-19 involvement (i.e., prognosis) and in patient management during treatment.

Fuhrman et al., in “ Cascaded deep transfer learning on thoracic CT in COVID-19 patients treated with steroids,” used transfer learning in the analysis of thoracic slices using a fine-tuned deep network followed by feature extraction in the task of distinguishing between patients administered steroids and those who were not. Here, the use of steroids served as the surrogate prognostic marker of severity. The authors also conducted a preliminary temporal trend analysis on the prognostic prediction score on multiple time-point CT images obtained during hospitalization, demonstrating the improved progression of those patients on steroids.

4.

Digital Pathology of COVID-19

Machine learning in imaging spans spatial scales enabling automatic interpretation of pathology images. The role of AI allows for the automatic identification of cells, such as lymphocytes—a task that would be manually overwhelming.

Corredor et al., in “ Computational pathology reveals unique spatial patterns of immune response in H&E images from COVID-19 autopsies: preliminary findings,” used computerized image analysis and machine learning approaches to characterize the spatial morphology of features of the immune response from digitized autopsied H&E tissue lung images from COVID-19 patients. The authors found that in COVID-19 patients, the immune response consistently included small-size lymphocyte clusters. Such analyses may potentially aid in the future in the development of therapeutics that work on modulating immune response in COVID-19 patients.

5.

Need for Data and a Role in Virtual Clinical Trials

The role of public data sets played a major role in many of the methods presented in this COVID-19 JMI special issue. Use of public datasets enabled training with large numbers of imaging studies such that transfer learning to the specific COVID-19 question was possible. However, the existence of multiple public datasets also yields concern on the potential of leakage between datasets due to duplicated cases, and thus potentially compromising training and testing paradigms.

Perez et al., in “ Methodology to create 3D models of COVID-19 pathologies for virtual clinical trials,” described their computational models of COVID-19 lung pathologies that were subsequently used to create simulated chest radiographs. The authors modeled nine COVID-19 disease models at different disease stages from actual CT scans, in order to obtain their virtual 3D anatomical phantoms. Simulated chest radiographs from the virtual 3D phantoms showed varying COVID-19 involvement in the lungs, and the radiographs were assessed for realistic presentation by both radiologists and AI algorithms. The research is intended for virtual clinical trials, and the resulting simulated images could be used to augment machine learning methods or to study the effect of image acquisition system parameters (such as exposure) on the ability to detect COVID-19.

The COVID-19 pandemic led to the convergence of experts from multiple disciplines including clinicians, medical physicists, imaging scientists, computer scientists, and informatics experts. However, such a rapid convergence over a very brief period of time created its own challenges. As noted earlier, as part of the MIDRC initiative, El Naqa et al., in “ Lessons learned in transitioning to AI in the medical imaging of COVID-19,” discuss the lessons learned from various AI career transitions and give recommendations relative to three types of AI investigators: (1) AI of nonimaging data to AI of medical imaging data, (2) medical imaging clinician to AI of medical imaging, and (3) AI of medical imaging to AI of COVID-19 imaging. The cautions are consistent with the motivations that led to MIDRC.

MIDRC is a multi-institutional initiative driven by the medical imaging community and aimed at accelerating the transfer of knowledge and innovation in the COVID-19 pandemic. MIDRC is funded by the National Institute of Biomedical Imaging and Bioengineering (NIBIB), hosted at the University of Chicago, and co-led by the American College of Radiology^® (ACR), the Radiological Society of North America (RSNA), and the American Association of Physicists in Medicine (AAPM). The aim of MIDRC is to foster machine learning innovation through data sharing for rapid and flexible collection, analysis, and dissemination of imaging and associated clinical data. Currently, MIDRC includes 24 collaborating institutions along with various supporting societies, such as SPIE.

MIDRC is different from other repositories by design. It is focused on ethically sourced, trustworthy data collected within a collaborative community having multiple expertise. MIDRC collects COVID-19 imaging studies from various medical centers, hospitals, and private practices through two input portals (RSNA RICORD and ACR CIRR). Subsequently, the imaging studies, along with some clinical and demographic data, undergo deidentification, quality assessment, and harmonization. Then, the data are released through a Gen3 portal yielding an open data commons with representative data, thus lowering the barrier of access for all AI researchers. Having all imaging studies within one commons solves additional problems, such as having studies duplicated across multiple commons resulting in leakage between training and testing. Currently over 60,000 imaging studies have been ingested by MIDRC. In addition to the open commons, MIDRC has a sequestered commons that will only be used to test AI algorithms in order to expedite the pathway through regulatory and translation to the public. In the future, AI in medical imaging publications will be able to handle some of today’s limitations through the use of MIDRC imaging studies.

In summary, the overwhelming number of research groups developing imaging systems and AI for medical images has greatly increased over the timeline of the COVID-19 pandemic. While there are many promising methods, more research is needed for them to reach clinical use.

Many thanks to SPIE for making this special issue on COVID-19 imaging open access.