As part of a special series on artificial intelligence (AI), OR Manager is taking a deep dive into the many facets of this new technology and its impact on patient care. Part 1 of this introduction to the series (OR Manager, May 2019, 1, 7-11) defined several different types of AI and made some projections about future applications of the technology. Part 2 will describe the many current and potential surgical applications of AI.
Artificial intelligence is already making an impact in surgery, and its use is expected to grow rapidly. “I think AI will increase efficiency, reduce costs, and really help people,” says John Beard, MD, MBA, medical director at ICU Medical, Inc, in San Clemente, California. He notes that there are specific uses for AI at every stage of the surgical continuum, as described in this article.
John Beard, MD, MBA
AI may be able to help with better tools to assess patients for risk of intraoperative and postoperative complications. “[These tools] are going to be used at the patient level, at the surgeon’s office, and in the preoperative clinic,” says Dr Beard. “They’ll reduce delays by ensuring that all test results are available, all risk factors are optimally mitigated, and that the anesthesiologist, surgeon, and team have all the information they need to optimally administer the anesthetic and perform the surgery.”
Frederick Memorial Hospital in Frederick, Maryland, has a scheduling system that uses AI to help forecast how long a case will take based on a computation of the surgeon’s recent case activity. “The software reviews the last 10 bookings of a specific surgeon performing a specific surgery and drops the high and low case times to determine the average of the remaining eight cases,” says Cynthia Russell, MSN, RN-BC, perioperative services information systems liaison. The recommended case length is based on that average.
“The caveat is that the prediction is based on the surgery that is booked,” Russell says, noting that a surgeon may not perform the exact booked case because of a clinical situation encountered during the procedure. Using ICD-10 or CPT codes helps improve forecasting accuracy by providing a structured language for comparison purposes.
Research has already identified several areas in which AI could enhance anesthesia management, including fluid administration. Dr Beard adds that AI will help identify those at highest risk for postoperative pain, nausea, and respiratory depression, and can help develop a prevention plan that is implemented intraoperatively. A 2018 report from Kendale and colleagues, for example, noted that machine-learning models could predict postinduction hypotension, although more refinement is needed.
AI also is already helping to identify areas to target for quality improvement. One example is the OR Black Box® platform developed by Teodor Grantcharov, MD, PhD, FACS, professor of surgery at the University of Toronto in Ontario, Canada. The system records and analyzes everything that occurs in surgery, which can reveal potential problems.
For example, one hospital using the OR Black Box platform learned that the OR doors were being opened too often during surgery. Subsequent discussion with OR leadership revealed that the suture had been relocated to outside the room, so it was returned to its original location within the OR. In addition, the platform has identified errors that had gone undetected by the care providers. (To learn more about the OR Black Box, see “Artificial intelligence makes surgical ‘black box’ smarter” on p 13).
Nicolas Padoy, PhD, associate professor on a chair of excellence research program in medical robotics at the University of Strasbourg in Strasbourg, France, has extensively studied AI’s role in surgical workflow with his clinical partners at IHU Strasbourg, IRCAD, and University Hospital of Strasbourg.
“Operating rooms are very complex environments,” he says. “We believe that with the digitalization of the OR, the digital information coming from the different information systems, electronic equipment, and sensors can be used to develop an AI system that can understand the surgical processes taking place in the room, namely recognize in real time the current status of the surgery and of the OR.” The current status would include, for example, the surgical step performed by the surgeon and the actions performed by the circulating nurse (sidebar above).
Nicolas Padoy, PhD
Such a system allows the development of tools that can support the surgical team with routine tasks, safety monitoring, and teaching. “When AI techniques are sufficiently mature, I envision that user interfaces in the OR will provide much more contextual support, for example, by showing the right information, instruction, and buttons at the right location, at the right time, and to the right person,” Padoy says.
The system could also broadcast information to those outside the OR to reduce distractions and to ensure case efficiency. “The system could ensure that safety checkpoints specific to each procedure have been performed and reported,” he adds.
Padoy and his colleagues used AI and human pose estimation, which consists of computing the location of the persons and their body parts using video data as input, to reduce exposure to radiation during procedures. “The positions of the persons are used to compute the x-ray exposure risk for each person present in the room,” he says. “The radiation exposure risk can then also be displayed on the person, per body part.”
Other intraoperative uses of AI include:
Estimating remaining surgical time.“Standard procedures all follow an underlying workflow,” Padoy says. “By looking at a large number of procedure examples, an algorithm can learn automatically this underlying workflow, namely the sequence of common steps, and how they relate to the visual characteristics of the images in the input.”
A 2018 study by Padoy and his colleagues showed that AI outperformed traditional methods for estimating remaining surgical time for laparoscopic cholecystectomy (based on 120 videos) and gastric bypass (based on 170 videos) procedures. Daniel Hashimoto, MD, MS, surgical artificial intelligence and innovation fellow at Massachusetts General Hospital in Boston, believes that AI will be able to predict case time relatively soon, but its ability to augment surgical decisions is farther in the future.
Daniel Hashimoto, MD, MS
Anticipating complications. Padoy says AI data from the past hundred or thousand procedures could be compared to current procedural data to detect a possible anomaly so that the surgeon could be notified. For example, the Triton system uses AI and infrared camera technology to analyze photos of sponges taken by an iPad in an operating or delivery room and quantify blood loss.
Improving robot function. AI techniques are being used to help improve surgical robots’ control accuracy and expand their dexterity through automation or semiautomation. “AI can detect ‘no-go’ zones and prevent instruments from entering sensitive anatomical areas,” Padoy notes, adding that AI could analyze data to improve robot design.
Sandip Panesar, MD, MSc, postdoctoral research fellow, neurosurgery, at Stanford University in Stanford, California, says that automation occurs on a scale from none (0) to full (5), where no human assistance is needed. Fully automated robots are unlikely to be an option anytime soon, although they would be a useful option in adverse conditions such as environmental disasters or war zones, where access to human expertise is limited. However, “highly automated” robots (level 4), which are capable of performing most, if not all, parts of a complex procedure, are likely on the horizon.
“AI will help ensure use of interventions that, based on data and a patient’s particular condition, will maximize recovery and reduce the probability of complications, and then get the patient home to recover sooner,” Dr Beard says. Interoperable infusion pumps in continuous communication with the electronic health record (EHR) are already in use. Infusion pump-EHR interoperability ensures that a clinician’s order is transmitted directly to the pump, reducing possible errors. In addition, an alarm notification can be forwarded to a handheld device used by a nurse. The nurse can either act on the alarm or pause it if immediate action is not needed.
“In the future, by having preset algorithms responding to different types of alarms, it’s possible that pump technology will drive additional gains in caregiver efficiency and responsiveness to changing conditions at the bedside,” Dr Beard says.
Another advantage of infusion pump interoperability is that the hospital can analyze data collected from all the pumps to optimize infusion therapy management.
“[Use of AI is] understanding patients in each stage of care and then trying to maximize efficiency and reduce costs and complications,” Dr Beard says. Other surgical applications of AI also exist (sidebar below).
Training. Identifying the optimal steps of a surgical procedure will help in training surgeons. And knowing the current step of the surgery will allow a computer to automatically display instructions to new personnel, such as what instruments are needed next, says Nicolas Padoy, PhD, associate professor on a chair of excellence research program in medical robotics at the University of Strasbourg in Strasbourg, France. AI could also enable automatic annotation of key parts of a video of the surgery, which could later be reviewed and used for educational purposes.
Daniel Hashimoto, MD, MS, surgical artificial intelligence and innovation fellow at Massachusetts General Hospital in Boston, says his team is looking at extrapolating performance data from videos to help create virtual education simulations that more effectively translate what is learned during training to actual practice.
Diagnosing. AI is being used to successfully diagnose certain conditions and to guide interventions. For example, in 2017 Bahl and colleagues reported they had developed a proof-of-concept model in which AI was used to determine that 30% of procedures to biopsy high-risk breast lesions could have been avoided. And a 2017 study from Orringer and colleagues found that AI could diagnose a brain tumor in a tissue sample obtained during surgery in just 3 to 4 minutes, a significant reduction in the time it traditionally takes for processing by the pathology department.
Analyzing. AI successes to date have been centered mostly in specialties that are image intensive, such as radiology and pathology. “Currently, the biggest application of AI in the clinical setting is radiology,” says Sandip Panesar, MD, MSc, postdoctoral research fellow, neurosurgery, at Stanford University in Stanford, California.
Dr Panesar says some research studies found that AI performed at the same level as or better than a trained radiologist. But widespread application is limited by the fact the computer is only trained to look for one type of pathology, for example, pneumonia in a chest x-ray. “However, if [use of AI in radiology] is allowed to develop, it’s pretty likely you’re going to get algorithms that can outperform humans,” Dr Panesar says.
Whende Carroll, MSN, RN-BC
Optimizing supply and instrument use. Dr Hashimoto notes that AI-driven video analysis is being used to identify the surgical instruments used during the procedure, which can help with inventory. “Knowing how many staples have been fired over the course of a case lets you know how it affects inventory without having to rely on manual efforts,” says Dr Hashimoto, who estimates that this could be in widespread use within 3 to 5 years. Tracking instruments would also help OR leaders know how often they are being used, which could help with purchasing decisions.
Staffing. Staffing is partially predicated on workflow, so the deeper understanding of complexity, numbers, and lengths of cases that AI can bring could help OR leaders better match staffing needs. Whende Carroll, MSN, RN-BC, founder of Nurse Evolution, a company that looks at how healthcare technology, data analytics, and innovation concepts can be used to improve health and how healthcare is delivered, worked with the chief nursing officer at a community hospital in using AI to predict numbers of patients in the emergency department and their acuity level, so that staffing levels were appropriate. “Adequate staffing also increases patient satisfaction and reduces staff burnout,” says Carroll, senior editor of the Online Journal of Nursing Informatics and author of an upcoming book on emerging technologies, including AI.
To reap the benefits of AI, OR leaders need to participate in its development and keep current on developments. “Whether that’s research groups or industry, it’s really critically important that everybody on the clinical care team is willing to work with these developers to provide the context of what actually matters when it comes to improving patient outcomes, logistics, and day-to-day workflow,” Dr Hashimoto says. He adds that much of the data generated by AI will be of critical importance to OR leaders. ✥
Cynthia Saver, MS, RN, is president of CLS Development, Inc, Columbia, Maryland, which provides editorial services to healthcare publications.
Bahl M, Barzilay R, Yedidia A B, et al. High-risk breast lesions: A machine learning model to predict pathologic upgrade and reduce unnecessary surgical excision. Radiology. 2017;286(3):810-818.
Kendale S, Kulkarni P, Rosenberg A D, et al. Supervised machine-learning predictive analytics for prediction of postinduction hypotension. Anesthesiology. 2018;129(4):675-688.
Orringer D A, Pandian B, Niknafs Y S, et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng. 2017;Epub. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5612414.
Panesar S, Cagle Y, Chander D, et al. Artificial intelligence and the future of surgical robotics. Ann Surg. 2019. https://journals.lww.com/annalsofsurgery/Citation/publishahead/Artificial_Intelligence_and_the_Future_of_Surgical.95173.aspx.
Rodas N L, et al. A global radiation awareness system using augmented reality and Monte Carlo simulations. Poster. European Society of Radiology. 2018. https://posterng.netkey.at/esr/viewing/index.php?module=viewing_poster&task=&pi=144787&searchkey=
Twinanda A P, Shehata S, Mutter D, et al. EndoNet: A deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging. 2017;36(1):86-97.
Twinanda A P, et al. RSDNet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE Trans Med Imaging. 2018. https://ieeexplore.ieee.org/document/8509608.
Volkov M, et al. Machine learning and coresets for automated real-time video segmentation of laparoscopic and robot-assisted surgery. 2017 IEEE International Conference on Robotics and Automation.
