Semantic segmentation of reflectance confocal microscopy mosaics of pigmented lesions using weak labels.
Cast your vote
You can rate an item by clicking the amount of stars they wish to award to this item.
When enough users have cast their vote on this item, the average rating will also be shown.
Your vote was cast
Thank you for your feedback
Thank you for your feedback
Brooks, Dana H
Dy, Jennifer G
Journal titleScientific reports
Publication Begin page3679
MetadataShow full item record
AbstractReflectance confocal microscopy (RCM) is a non-invasive imaging tool that reduces the need for invasive histopathology for skin cancer diagnoses by providing high-resolution mosaics showing the architectural patterns of skin, which are used to identify malignancies in-vivo. RCM mosaics are similar to dermatopathology sections, both requiring extensive training to interpret. However, these modalities differ in orientation, as RCM mosaics are horizontal (parallel to the skin surface) while histopathology sections are vertical, and contrast mechanism, RCM with a single (reflectance) mechanism resulting in grayscale images and histopathology with multi-factor color-stained contrast. Image analysis and machine learning methods can potentially provide a diagnostic aid to clinicians to interpret RCM mosaics, eventually helping to ease the adoption and more efficiently utilizing RCM in routine clinical practice. However standard supervised machine learning may require a prohibitive volume of hand-labeled training data. In this paper, we present a weakly supervised machine learning model to perform semantic segmentation of architectural patterns encountered in RCM mosaics. Unlike more widely used fully supervised segmentation models that require pixel-level annotations, which are very labor-demanding and error-prone to obtain, here we focus on training models using only patch-level labels (e.g. a single field of view within an entire mosaic). We segment RCM mosaics into "benign" and "aspecific (nonspecific)" regions, where aspecific regions represent the loss of regular architecture due to injury and/or inflammation, pre-malignancy, or malignancy. We adopt Efficientnet, a deep neural network (DNN) proven to accurately accomplish classification tasks, to generate class activation maps, and use a Gaussian weighting kernel to stitch smaller images back into larger fields of view. The trained DNN achieved an average area under the curve of 0.969, and Dice coefficient of 0.778 showing the feasibility of spatial localization of aspecific regions in RCM images, and making the diagnostics decision model more interpretable to the clinicians.
CitationD'Alonzo M, Bozkurt A, Alessi-Fox C, Gill M, Brooks DH, Rajadhyaksha M, Kose K, Dy JG. Semantic segmentation of reflectance confocal microscopy mosaics of pigmented lesions using weak labels. Sci Rep. 2021 Feb 11;11(1):3679. doi: 10.1038/s41598-021-82969-9. PMID: 33574486; PMCID: PMC7878861.
The following license files are associated with this item:
- Creative Commons
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International
- Role of In Vivo Reflectance Confocal Microscopy in the Analysis of Melanocytic Lesions.
- Authors: Serban ED, Farnetani F, Pellacani G, Constantin MM
- Issue date: 2018 Apr
- Skin strata delineation in reflectance confocal microscopy images using recurrent convolutional networks with attention.
- Authors: Bozkurt A, Kose K, Coll-Font J, Alessi-Fox C, Brooks DH, Dy JG, Rajadhyaksha M
- Issue date: 2021 Jun 15
- Utilizing Machine Learning for Image Quality Assessment for Reflectance Confocal Microscopy.
- Authors: Kose K, Bozkurt A, Alessi-Fox C, Brooks DH, Dy JG, Rajadhyaksha M, Gill M
- Issue date: 2020 Jun
- Segmentation of cellular patterns in confocal images of melanocytic lesions in vivo via a multiscale encoder-decoder network (MED-Net).
- Authors: Kose K, Bozkurt A, Alessi-Fox C, Gill M, Longo C, Pellacani G, Dy JG, Brooks DH, Rajadhyaksha M
- Issue date: 2021 Jan
- Morphological evaluation of melanocytic lesions with three-dimensional line-field confocal optical coherence tomography: correlation with histopathology and reflectance confocal microscopy. A pilot study.
- Authors: Perez-Anker J, Puig S, Alos L, García A, Alejo B, Cinotti E, Orte Cano C, Tognetti L, Lenoir C, Monnier J, Machuca N, Castillo P, Gibert PR, Rubegni P, Suppa M, Perrot JL, Del Marmol V, Malvehy J
- Issue date: 2022 Dec