A proper cancer diagnosis is imperative for determining the medical treatment for a patient. It necessitates a good staging and classification of the tumor alongside with additional factors to predict response to treatment. Mitotic count-based tumor proliferation grade provides the most reproducible and independent prognostic value. In practice, pathologists examine H&E-stained, giga-pixel-sized digital whole-slide images of a tissue specimen for counting the mitotic index. Considering the enormity of the images, focus for analysis is centered on specific, so-called, high-power- fields (HPFs) on the periphery of the invasive parts of the tumor. Selection of the HPFs is very subjective. Additionally, tumor heterogeneity impacts both the region selection and the quality of the area analyzed. Several efforts have been made to automate the tumor proliferation score estimation by counting the mitotic figures in certain regions-of-interest. But the region selection algorithms are inconspicuous and do not ensure to encompass the crucial regions interesting for pathological analysis, thereby, making the grading sub-optimal. In this work, we aim at addressing this problem by proposing to visualize a distance weighted mitotic distribution in the entire invasive tumor region. Our approach provides a holistic view of the mitotic activity and localizes active proliferating regions in the tumor with tissue architecture context, enabling the pathologists to more objectively select the HPFs. We propose a deep learning-based framework to generate the mitotic activity heat-maps. Additionally, in the framework, we develop a number of significant tools for digital pathology; a semi-supervised tumor region delineation tool, a fast nuclei segmentation and detection tool, and a mitotic figure localization tool.
Rapid digitization of whole-slide images (WSIs) with slide scanners, along with the advancements in deep learning strategies has empowered the development of computerized image analysis algorithms for automated diagnosis, prognosis, and prediction of various types of cancers in digital pathology. These analyses can be enhanced and expedited by confining them to relevant tumor region on the large-sized and multi-resolution WSIs. The detection of tumor-region-of-interest (TRoI) on WSIs can facilitate to automatically measure the tumor size as well as to compute the distance to the resection margin. It can also ease the process of identifying high-power-fields (HPFs), which are essential towards the grading of tumor proliferation scores. In practice, pathologists select these regions by visual inspection of WSIs, which is a cumbersome, time-consuming process and affected by inter- and intra- pathologist variability. State-of-the-art deep learning-based methods perform well on the TRoI detection task by using supervised algorithms, however, they require accurate TRoI and non-TRoI annotations to train the algorithms. Acquiring such annotations is a tedious task and incurs observational variability. In this work, we propose a positive and unlabeled learning approach that uses a few examples of HPF regions (positive annotations) to localize the invasive TRoIs on breast cancer WSIs. We use unsupervised deep autoencoders with Gaussian Mixture Model-based clustering to identify the TRoI in a patch-wise manner. The algorithm is developed using 90 HPF-annotated WSIs and is validated on 30 fully-annotated WSIs. It yielded a Dice coefficient of 75.21%, a true positive rate of 78.62% and a true negative rate of 97.48% in terms of pixel-bypixel evaluation compared to the pathologists annotations. Significant correspondence between the results of the proposed algorithm and the state-of-the-art supervised ConvNet indicates the efficacy of the proposed algorithm.
The focus of this paper is to illustrate how computational image processing and machine learning can help address
two of the challenges of histological image analysis, namely, the cellular heterogeneity, and the imprecise labeling.
We propose an unsupervised method of generating representative image signatures based on an autoencoder
architecture which reduces the dependency on labels that tend to be imprecise and tedious to get. We have
modified and enhanced the architecture to simultaneously produce representative image features as well as
perform dictionary learning on these features to enable robust characterization of the cellular phenotypes. We
integrate the extracted features in a disease grading framework, test it in prostate tissues immunostained for
different protein visualization and show significant improvement in terms of grading accuracy compared to
alternative supervised feature-extraction methods.
Images of tissue specimens enable evidence-based study of disease susceptibility and stratification. Moreover, staining technologies empower the evidencing of molecular expression patterns by multicolor visualization, thus enabling personalized disease treatment and prevention. However, translating molecular expression imaging into direct health benefits has been slow. Two major factors contribute to that. On the one hand, disease susceptibility and progression is a complex, multifactorial molecular process. Diseases, such as cancer, exhibit cellular heterogeneity, impeding the differentiation between diverse grades or types of cell formations. On the other hand, the relative quantification of the stained tissue selected features is ambiguous, tedious and time consuming, prone to clerical error, leading to intra- and inter-observer variability and low throughput. Image analysis of digital histopathology images is a fast-developing and exciting area of disease research that aims to address the above limitations. We have developed a computational framework that extracts unique signatures using color, morphological and topological information and allows the combination thereof. The integration of the above information enables diagnosis of disease with AUC as high as 0.97. Multiple staining show significant improvement with respect to most proteins, and an AUC as high as 0.99.
Early identification of problematic patterns in very large scale integration (VLSI) designs is of great value as the lithographic simulation tools face significant timing challenges. To reduce the processing time, such a tool selects only a fraction of possible patterns which have a probable area of failure, with the risk of missing some problematic patterns. We introduce a fast method to automatically extract patterns based on their structure and context, using the Voronoi diagram of line-segments as derived from the edges of VLSI design shapes. Designers put line segments around the problematic locations in patterns called “gauges,” along which the critical distance is measured. The gauge center is the midpoint of a gauge. We first use the Voronoi diagram of VLSI shapes to identify possible problematic locations, represented as gauge centers. Then we use the derived locations to extract windows containing the problematic patterns from the design layout. The problematic locations are prioritized by the shape and proximity information of the design polygons. We perform experiments for pattern selection in a portion of a 22-nm random logic design layout. The design layout had 38,584 design polygons (consisting of 199,946 line segments) on layer Mx, and 7079 markers generated by an optical rule checker (ORC) tool. The optical rules specify requirements for printing circuits with minimum dimension. Markers are the locations of some optical rule violations in the layout. We verify our approach by comparing the coverage of our extracted patterns to the ORC-generated markers. We further derive a similarity measure between patterns and between layouts. The similarity measure helps to identify a set of representative gauges that reduces the number of patterns for analysis.
OPC models have become critical in the manufacturing of integrated circuits (ICs) by allowing correction of complex designs, as we approach the physical limits of scaling in IC chip design. The accuracy of these models depends upon the ability of the calibration set to sufficiently cover the design space, and be manageable enough to address metrology constraints. We show that the proposed method provides results of at least similar quality, in some cases superior quality compared to both the traditional method and sample plan sets of higher size. The main advantage of our method over the existing ones is that it generates a calibration set much faster, considering a large initial set and even more importantly, by automatically selecting its minimum optimal size.
Early identification of problematic patterns in real designs is of great value as the lithographic simulation tools face significant timing challenges. To reduce the processing time such a tool selects only a fraction of possible patterns,which have a probable area of failure, with the risk of missing some problematic patterns. In this paper, we introduce a fast method to automatically extract patterns based on their structure and context, using the Voronoi diagram of VLSI design shapes. We first identify possible problematic locations, represented as gauge centers, and then use the derived locations to extract windows and problematic patterns from the design layout. The problematic locations are prioritized by the shape and proximity information of design polygons. We performed experiments for pattern selection in a portion of a 22nm random logic design layout. The design layout had 38584 design polygons (consisting of 199946 line-segments) on layer Mx, and 7079 markers generated by an Optical Rule Checker (ORC) tool. We verified our approach by comparing the coverage of our extracted patterns to the ORC generated markers.
Evaluating pattern sensitivity to variability of the process parameters is of increasing importance to improve resolution enhancement techniques. In this paper, we propose an efficient algorithm to extract printed shapes from SEM images, a novel quality metric which analyzes the topology of the extracted printed shapes with respect to the target mask shape and a unique set of descriptors that define the sensitivity of a pattern. Compared to traditional CD methods, the proposed method has better accuracy, increased robustness and ability to spot global changes. Compared to contours distance methods, it is designed to expose most critical regions and capture context effects.
Assessing pattern printability in new large layouts faces important challenges of runtime and false detection. Lithographic simulation tools and classification techniques do not scale well. We propose a fast pattern detection method that builds jointly a structured overcomplete basis, representing each reference pattern, and a linear predictor of their lithographic difficulty. A pattern from a new design is detected “novel” if its reconstruction error, when coded in the learned basis, is large. This allows a fast detection of unseen clips and a fast prediction of their lithographic difficulty. We show high speedup (1000×) compared to nearest neighbor search, and very high correlation between predicted and calculated lithographic estimate values.
It is desired to reduce the time required to produce metrology data for calibration of Optical Proximity Correction (OPC) models and also maintain or improve the quality of the data collected with regard to how well that data represents the types of patterns that occur in real circuit designs. Previous work based on clustering in geometry and/or image parameter space has shown some benefit over strictly manual or intuitive selection, but leads to arbitrary pattern exclusion or selection which may not be the best representation of the product. Forming the pattern selection as an optimization problem, which co-optimizes a number of objective functions reflecting modelers’ insight and expertise, has shown to produce models with equivalent quality to the traditional plan of record (POR) set but in a less time.
Many chip design and manufacturing applications including design rules development, optical proximity correction
tuning, and source optimization can benefit from rapid estimation of relative difficulty or printability. Simultaneous
source optimization of thousands of clips has been demonstrated recently, but presents performance challenges. We
describe a fast, source independent method to identify patterns which are likely to dominate the solution. In the context
of source optimization the estimator may be used as a filter after clustering, or to influence the selection of representative
cluster elements. A weighted heuristic formula identifies spectral signatures of several factors contributing to difficulty.
Validation methods are described showing improved process window and reduced error counts on 22 nm layout
compared with programmable illuminator sources derived from hand picked patterns, when the formula is used to
influence training clip selection in source optimization. We also show good correlation with fail prediction on a source
produced with hand picked training clips with some level of optical proximity correction tuning.
Joint optimization (JO) of source and mask together is known to produce better SMO solutions than sequential
optimization of the source and the mask. However, large scale JO problems are very difficult to solve because the global
impact of the source variables causes an enormous number of mask variables to be coupled together. This work presents
innovation that minimize this runtime bottleneck. The proposed SMO parallelization algorithm allows separate mask
regions to be processed efficiently across multiple CPUs in a high performance computing (HPC) environment, despite
the fact that a truly joint optimization is being carried out with source variables that interact across the entire mask.
Building on this engine a progressive deletion (PD) method was developed that can directly compute "binding
constructs" for the optimization, i.e. our method can essentially determine the particular feature content which limits the
process window attainable by the optimum source. This method allows us to minimize the uncertainty inherent to
different clustering/ranking methods in seeking an overall optimum source that results from the use of heuristic metrics.
An objective benchmarking of the effectiveness of different pattern sampling methods was performed during postoptimization
analysis. The PD serves as a golden standard for us to develop optimum pattern clustering/ranking
algorithms. With this work, it is shown that it is not necessary to exhaustively optimize the entire mask together with the
source in order to identify these binding clips. If the number of clips to be optimized exceeds the practical limit of the
parallel SMO engine one can starts with a pattern selection step to achieve high clip count compression before SMO.
With this LSSO capability one can address the challenging problem of layout-specific design, or improve the technology
source as cell layouts and sample layouts replace lithography test structures in the development cycle.
A method for scalable motion compensated up-conversion of video sequences is proposed. By computing suitable objective quality estimates,(which show a high correlation with subjective quality scores), the method allows to predict the resources needed to up-convert a given input video sequence achieving a certain visual quality of the output video sequence. This method permits to dynamically change the resource utilization in a way which is optimal for the system, e.g. for a programmable platform for media processing.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.