VigilSAM3: Reinforcement-Optimized Memory Gating for Ultra-Long Surgical Video Segmentation
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI, CCF-B), 2026. Conference
Laparoscopic hepatectomy demands real-time identification of critical vascular structures, including Glisson's pedicle, hepatic veins, and transient bleeding points, to enable pre-emptive vessel clamping and prevent life-threatening haemorrhage. These targets are frequently obscured and deformable, and in ultra-long procedures spanning thousands of frames they further suffer extreme scale imbalance and intermittent visibility, making memory-based propagation prone to temporal drift where a single corrupted entry cascades forward irreversibly. We present VigilSAM3, built on SAM3's dual-system architecture, which introduces a hierarchical memory firewall that guards memory quality at pixel, class, and frame levels. A causal-Transformer scheduling agent selects among Trust, Correct, and Reset actions, with its policy refined via Group Relative Policy Optimization (GRPO) using class-aware rewards. We further introduce Hepa-Vein, the first multi-class large-scale hepatectomy surgical video semantic segmentation dataset. Experiments on Hepa-Vein and DSAD demonstrate state-of-the-art performance, with pronounced gains on small, transient vascular structures.
@inproceedings{yao2026vigilsam3,
title = {VigilSAM3: Reinforcement-Optimized Memory Gating for Ultra-Long Surgical Video Segmentation},
author = {Yao, Xincheng and Yang, Yijun and Song, Zhuojin and Guo, Kangwei and Tao, Haisu and Yang, Jian and Zhu, Lei},
booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)},
year = {2026}
}