Robot performs autonomous cholecystectomy using video training, voice feedback

Editor's Note 

A surgical robot independently performed a lengthy phase of gallbladder removal with expert-level precision, according to a July 9 announcement from Johns Hopkins university. "This advancement moves us from robots that can execute specific surgical tasks to robots that truly understand surgical procedures," said medical roboticist Axel Krieger, one of the authors of findings published in Science Robotics. "This is a critical distinction that brings us significantly closer to clinically viable autonomous surgical systems that can work in the messy, unpredictable reality of actual patient care." 

The robot, called the Surgical Robot Transformer-Hierarchical (SRT-H), operated on a lifelike model and responded to real-time voice commands, adjusting movements in response to instructions such as “grab the gallbladder head” or “move the left arm a bit to the left.” Researchers said it functioned like a novice surgeon learning from experienced mentors.

According to the university, the robot demonstrated consistent performance across variable anatomical conditions and unexpected scenarios, including altered starting positions and changes in tissue appearance using blood-like dyes. These capabilities mark a significant advance from previous systems limited to predefined tasks.

As detailed in the article, SRT-H is powered by imitation learning and the same large language model architecture behind tools like ChatGPT. It can interpret complex surgical scenes, respond dynamically to intraoperative changes, and self-correct as needed. The team emphasizes that this moves surgical robots from merely executing tasks to truly understanding surgical workflows.

Training was conducted using videos of human surgeons performing gallbladder removals on pig cadavers, with captions describing each of 17 surgical steps. After training, the robot executed all steps—including identification of ducts and arteries, placement of clips, and tissue cutting—with 100% task accuracy. Though slower than a human, the robot’s results were comparable to those of an expert surgeon, the university reports.

According to the article, the team previously developed the Smart Tissue Autonomous Robot (STAR), which in 2022 performed laparoscopic surgery on a pig. Unlike STAR, which relied on highly controlled conditions, SRT-H adapts in real time, without requiring marked tissue or a fixed plan.The team now plans to train the system on additional procedures with the long-term goal of fully autonomous surgery.