In computational pathology, training and inference of conventional deep convolutional neural networks (CNN) are usually limited to patches of small sizes (e.g., 256 × 256) sampled from whole slide images. In practice, however, diagnostic and prognostic information could lie within the context of tumor microenvironment across multiple regions, far beyond the scope of individual patches. For instance, the spatial relationship of tumor-infiltrating lymphocytes (TIL) across regions of interest might be prognostic for non-small cell lung cancer (NSCLC). This poses a multi-instance learning (MIL) problem, and a single-patch-driven CNN typically fails to learn spatial information and context between multiple patches, especially their spatial relationship. In this work, we present a cell graph-based MIL framework to predict the risk of death for early-stage NSCLC by aggregating feature representation of TIL-enclosing patches according to their spatial relationship. Inspired by PATCHY-SAN, a graph-embedding framework for CNNs, we use graph kernel-based approaches to embed a bag of patches into a sequence with their spatial information encoded into the sequence order. A transformer model was then trained to aggregate patch-level features based on spatial information. We demonstrate the capability of this framework to predict the likelihood of the patient with NSCLC in two cohorts (n=240) to survive for more than 5 years. The training cohort (n=195) comprised hematoxylin and eosin (H&E)-stained whole slide images (WSI), while the testing cohort (n=45) comprised H&E-stained tumor microarrays (TMA). We show that, with the spatial context of multiple patches encoded as an ordered patch sequence, the performance in the testing cohort of our approach achieves an area under the receiver operating characteristic curve (AUC) of 0.836 (p=0.009; HR=5.62), as opposed to a baseline conventional CNN with an AUC of 0.542 (p=0.105; HR=1.66). The results suggest that the Transformer is a generic spatial information aware MIL framework that can learn the spatial relationship of multiple TIL-enclosing patches from the graph representation of immune cells.
A number of papers have established that a high density of tumor-infiltrating lymphocytes (TILs) is highly correlated with a better prognosis for many different cancer types. More recently, some studies have shown that the spatial interplay between different subtypes of TILs (e.g. CD3, CD4, CD8) is more prognostic of disease outcome compared to just metrics related to TIL density. A challenge with TIL subtyping is that it relies on quantitative immunofluoresence or immunohistochemistry, complex and tissue-destructive technologies. In this paper we present a new approach called PhenoTIL to identify TIL sub-populations and quantify the interplay between these sub-populations and show the association of these interplay features with recurrence in early stage lung cancer. The approach comprises a Dirichlet Process Gaussian Mixture Model that clusters lymphocytes on H&E images. The approach was evaluated on a cohort of N=178 early stage non-small cell lung cancer patients, N=100 being used for model training and N=78 being used for independent validation. A Linear Discriminant Analysis classifier was trained in conjunction with 186 PhenoTIL features to predict the likelihood of recurrence in the test set. The PhenoTIL features yielded an AUC=0.84 compared to an approach involving just TIL density alone (AUC=0.58). In addition, a Kaplan-Meier analysis showed that the PhenoTIL features were able to statistically significantly distinguish early from late recurrence (p = 4 ∗ 10 −5 ).
Automatic detection and quantification of glands in gastric cancer may contribute to objectively measure the lesion severity, to develop strategies for early diagnosis, and most importantly to improve the patient categorization. This article presents an entire framework for automatic detection of glands in gastric cancer images. This approach starts by selecting gland candidates from a binarized version of the hematoxylin channel. Next, the gland’s shape and nuclei are characterized using local features which feed a Monte Carlo Cross validation method classifier trained previously with manually labeled images. Validation was carried out using a dataset with 1330 annotated structures (2372 glands) from seven fields of view extracted from gastric cancer whole slide images. Results showed an accuracy of 93% using a simple linear classifier. The presented strategy is quite simple, flexible and easily adapted to an actual pathology laboratory.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.