DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images

Published 18 May 2023 in eess.IV, cs.CV, and cs.LG | (2305.10655v1)

Abstract: Automatic segmentation of medical images is a key step for diagnostic and interventional tasks. However, achieving this requires large amounts of annotated volumes, which can be tedious and time-consuming task for expert annotators. In this paper, we introduce DeepEdit, a deep learning-based method for volumetric medical image annotation, that allows automatic and semi-automatic segmentation, and click-based refinement. DeepEdit combines the power of two methods: a non-interactive (i.e. automatic segmentation using nnU-Net, UNET or UNETR) and an interactive segmentation method (i.e. DeepGrow), into a single deep learning model. It allows easy integration of uncertainty-based ranking strategies (i.e. aleatoric and epistemic uncertainty computation) and active learning. We propose and implement a method for training DeepEdit by using standard training combined with user interaction simulation. Once trained, DeepEdit allows clinicians to quickly segment their datasets by using the algorithm in auto segmentation mode or by providing clicks via a user interface (i.e. 3D Slicer, OHIF). We show the value of DeepEdit through evaluation on the PROSTATEx dataset for prostate/prostatic lesions and the Multi-Atlas Labeling Beyond the Cranial Vault (BTCV) dataset for abdominal CT segmentation, using state-of-the-art network architectures as baseline for comparison. DeepEdit could reduce the time and effort annotating 3D medical images compared to DeepGrow alone. Source code is available at https://github.com/Project-MONAI/MONAILabel

Abstract PDF Upgrade to Chat

Authors (20)

First 10 authors:

Citations (17)

View on Semantic Scholar

Summary

The paper introduces DeepEdit, a dual-mode segmentation framework that combines automated deep learning with clinician-guided interactive refinement.
It leverages simulated-click training and uncertainty metrics, achieving a mean Dice score of 0.93 for prostate segmentation with 10 clicks.
The integrated approach significantly reduces annotation effort while enhancing segmentation accuracy across diverse 3D medical imaging tasks.

DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images

The paper "DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images" presents a novel approach to ease the process of medical image segmentation leveraging deep learning methodologies. Segmentation is a crucial task in medical imaging, pivotal for diagnosis and intervention, yet it demands extensive annotated datasets which are cumbersome to produce.

Overview of DeepEdit

DeepEdit integrates automatic and interactive segmentation into one deep learning framework, utilizing network architectures like nnU-Net, UNET, or UNETR for initial segmentation and DeepGrow for refinement through user interactions. This dual-mode approach enhances segmentation accuracy while facilitating clinician interaction.

Methodology

DeepEdit encompasses three operational modes: a fully automatic segmentation mode, a semi-automatic mode initialized and guided by user clicks, and a refinement mode for existing segmentations. This flexibility is achieved through a novel training paradigm which includes user interaction simulation and integrates click-free and simulated-click training iterations. In contrast to DeepGrow, DeepEdit supports multi-label segmentation and can manage tasks of varying complexity without minimum click restrictions.

Training and Interaction

The training process is a blend of automatic and interactive iterations. This approach not only promotes robustness in segmentation tasks but also allows automatic calculation of uncertainty metrics—aleatoric and epistemic—which aid in active learning by prioritizing challenging cases for annotation.

Experimental Evaluation

The paper demonstrates DeepEdit's performance using datasets from the PROSTATEx challenge for prostate and prostatic lesion segmentation, and the BTCV dataset for abdominal CT scans. It shows improved results over solely automatic methods like nnU-Net and solely interactive methods like DeepGrow, with 0.93 mean Dice score on prostate segmentation (10 clicks) versus nnU-Net’s 0.91 (0 clicks).

Implications and Future Directions

The implications of DeepEdit are significant, offering a reduced annotation effort and improved model efficiency, crucial for scaling deep learning in medical imaging. The integration of uncertainty-based active learning further augments its applicability, ensuring the model’s ability to learn effectively from limited annotated data. Future directions could explore its extension to other imaging modalities and incorporation of even more sophisticated user interaction.

By facilitating an efficient workflow that marries automation with interactive editing, DeepEdit sets a foundation for more adaptive and precise segmentation tools in medical contexts.

Markdown Report Issue