The Segment Anything Model (SAM) has demonstrated outstanding zero-shot performance in image segmentation through efficient point and box prompts. In this paper, we propose a SAM-based Semantic-enhanced Point-Box joint prompting (SAM-SPB) framework for Video Object Segmentation (VOS). SAM-SPB leverages the local structure information and the global semantic cues of interest objects, leading to strong and robust segmentation. To be specific, the local structure information of the objects is maintained by a point tracking branch, and the semantic consistency of the objects across frames are propagated through our proposed semantic-aware memory-based box tracking branch. Compared with previous SAM-based point-centric video segmentation method, we highlight the importance of point-box joint prompting for video object segmentation. The state-of-the-art experimental results on popular VOS …
本文提出基于SAM的语义增强点框联合提示框架SAM-SPB，用于视频对象分割。该框架通过点跟踪分支维持对象局部结构信息，并利用语义感知的基于记忆的框跟踪分支跨帧传播对象语义一致性，从而结合局部与全局线索实现鲁棒分割。在主流VOS基准上取得了领先性能，验证了点框联合提示相比仅用点提示的优势。