AI-powered video models show promise for nursing skills assessment

Editor's Note

A new study shows that video-language models (VLMs) can reliably evaluate nursing procedures, detect errors, and provide feedback, paving the way for scalable AI-assisted training in nursing education, Cornell University September 20 reports. The paper, titled “Automated Procedural Analysis via Video-Language Models for AI-assisted Nursing Skills Assessment”, outlines the first framework applying VLMs to systematically assess nursing skills.

According to the paper, the system mimics human learning by progressing from basic action recognition to fine-grained segmentation and advanced reasoning. It identifies missing or misordered steps, explains why a procedure deviates from expected practice, and generates structured, interpretable feedback. This approach could ease the burden on faculty, who currently provide subjective and time-intensive evaluations, while enabling consistent, standardized assessments across institutions.

The research team curated a large dataset, NurViD, with over 1,500 nursing videos covering 51 procedures and 177 distinct actions. Using this foundation, they trained a VLM across four stages: procedure recognition, dense action segmentation with captions, missing event prediction, and sequence order correction. In tests, the model achieved notable gains over baselines, including a 51.7% improvement in fine-grained segmentation accuracy and a 55.4% improvement in detecting missing actions. Sequence error detection nearly doubled in performance compared with the base model.

The framework proved effective across diverse tasks such as venipuncture, wound dressing, and catheterization, where precision and sequence adherence are critical. While the models still face challenges with precise temporal localization, the authors noted the system already shows potential as a supplementary assessment tool that can scale training, reduce instructor workload, and support competency-based education. Beyond nursing, the paper suggests applications in fields requiring procedural rigor, such as emergency response and medical technician certification.