Using Generative Adversarial Networks for H&E-to-HER2 Stain Translation in Digital Pathology Images

Sammanfattning: In digital pathology, hematoxylin & eosin (H&E) is a routine stain which is performed on most clinical cases and it often provides clinicians with sufficient information for diagnosis. However, when making decisions on how to guide breast cancer treatment, immunohistochemical staining of human epidermal growth factor 2 (HER2 staining) is also needed. Over-expression of the HER2 protein plays a significant role in the progression of breast cancer and is therefore important to consider during treatment planning. However, the downside of HER2 staining is that it is both time consuming and rather expensive. This thesis explores the possibility for H&E-to-HER2 stain translation using generative adversarial networks (GANs). If effective, this has the potential to reduce the costs and time spent on tissue processing while still providing clinicians with the images necessary to make a complete diagnosis. To explore this area two supervised (Pix2Pix, PyramidPix2Pix) and one unsupervised (cycleGAN) GAN structure was implemented and trained on digital pathology images from the MIST dataset. These models were trained two times, with 256x256 and 512x512 patches, to see which effect patch size has on stain translation performance as well. In addition, a methodology for evaluating the quality of the generated HER2 patches was also presented and utilized. This methodology consists of structural similarity index (SSIM) and peak signal to noise ratio (PSNR) comparison to the ground truth, and a HER2 status classification protocol. In the latter a classification tool provided by Sectra was used to assign each patch with a HER2 status of No tumor, 1+, 2+ or 3+ and the statuses of the generated patches were then compared to the statuses of the ground truths. The results show that the supervised Pyramid Pix2Pix model trained on 512x512 patches performs the best according to the SSIM and PSNR metrics. However, the unsupervised cycleGAN model shows more promising results when it comes to both visual assessment and the HER2 status classification protocol. Especially when trained on 256x256 patches for 200 epochs which gave an accuracy of 0.655, F1-score of 0.674 and MCC of 0.490. In conclusion the HER2 status classification protocol is deemed as a suitable way to evaluate H&E-to-HER2 stain translation and thereby the unsupervised method is considered to be better than the supervised. Moreover, it is also concluded that a smaller patch size result in worse translation of cellular structure for the supervised methods. Further studies should focus on incorporating HER2 status classification in the cycleGAN loss function and more extensive training runs to further improve the quality of H&E-to-HER2 stain translation.