论文标题
购物车：因果关系驱动的机器人工具从视觉和运动学数据分割
CaRTS: Causality-driven Robot Tool Segmentation from Vision and Kinematics Data
论文作者
论文摘要
机器人辅助手术期间机器人工具的基于视觉的分割可以实现下游应用，例如增强现实反馈，同时允许机器人运动学的不准确性。随着深度学习的引入，提出了许多直接和仅从图像中求解仪器分割的方法。尽管这些方法在基准数据集上取得了显着的进展，但与其鲁棒性有关的基本挑战仍然存在。我们提出了CARTS，这是一种因果关系驱动的机器人工具分割算法，它是基于机器人工具分割任务的互补因果模型而设计的。 CARTs通过向前的运动学和可区分的渲染来更新最初的不正确的机器人运动学参数，而不是从观察到的图像中直接从观察到的图像中推断分段掩码，而是将工具模型与图像观测值对齐。我们基准在精确控制的场景中生成的dvrk的合成和真实数据基准了竞争技术，以允许反事实综合。在训练域测试数据上，在对反事实更改的测试数据进行测试时，卡车的骰子得分为93.4（骰子得分为91.8），表现出低亮度，烟雾，血液和背景模式改变。这比基于SOTA图像的方法的骰子得分分别与95.0和86.7的骰子得分进行了比较。未来的工作将涉及加速推车以实现视频帧速率，并估计闭塞在实践中的影响。尽管存在这些局限性，但我们的结果还是很有希望的：除了实现高分子的精度外，购物车还提供了真正的机器人运动学的估计，这可能会受益于诸如力估计的应用。代码可用：https：//github.com/hding2455/carts
Vision-based segmentation of the robotic tool during robot-assisted surgery enables downstream applications, such as augmented reality feedback, while allowing for inaccuracies in robot kinematics. With the introduction of deep learning, many methods were presented to solve instrument segmentation directly and solely from images. While these approaches made remarkable progress on benchmark datasets, fundamental challenges pertaining to their robustness remain. We present CaRTS, a causality-driven robot tool segmentation algorithm, that is designed based on a complementary causal model of the robot tool segmentation task. Rather than directly inferring segmentation masks from observed images, CaRTS iteratively aligns tool models with image observations by updating the initially incorrect robot kinematic parameters through forward kinematics and differentiable rendering to optimize image feature similarity end-to-end. We benchmark CaRTS with competing techniques on both synthetic as well as real data from the dVRK, generated in precisely controlled scenarios to allow for counterfactual synthesis. On training-domain test data, CaRTS achieves a Dice score of 93.4 that is preserved well (Dice score of 91.8) when tested on counterfactually altered test data, exhibiting low brightness, smoke, blood, and altered background patterns. This compares favorably to Dice scores of 95.0 and 86.7, respectively, of the SOTA image-based method. Future work will involve accelerating CaRTS to achieve video framerate and estimating the impact occlusion has in practice. Despite these limitations, our results are promising: In addition to achieving high segmentation accuracy, CaRTS provides estimates of the true robot kinematics, which may benefit applications such as force estimation. Code is available at: https://github.com/hding2455/CaRTS
