Turn-taking Prediction for Human Robot Collaboration
Tian Zhou and Juan Wachs Intelligent Systems and Assistive Technologies Lab, Purdue University, West Lafayette, IN Aug 2015 - present
Description: To enable a natural and fluent human robot collaboration flow, it is critical for a robot to comprehend their human peers’ on-going actions, predict their behaviors in the near future, and plan its actions correspondingly. Specifically, the capability of prediction ahead of time is important, so that the robots can foresee the precise timing of a turn-taking event, and get ready to take over in a proactive manner instead of reactive. Such proactive behavior would save human’s waiting time and increase naturalness in collaborative.
To that end, this prject proposes the design and implementation of a proactive turn-taking prediction algorithm, catered for human robot collaboration scenarios. Specifically, a Robotic Scrub Nurse (RSN) system which can understand surgeon’s multimodal communication cues and perform turn-taking predictions is developed. The developed algorithm was tested on a self-collected dataset of simulated OR communications between nurses and surgeons. The proposed turn-taking prediction algorithm is found to be significantly superior to its algorithmic counterparts, and is even better than human baseline in prediction accuracies when little partial input is given (less than 35% of full action, approximately 0.7 seconds). After the beginning stage, the algorithm can achieve comparable performances with humans.
Publications: Zhou, Tian., & Wachs, Juan 2016. Early Turn-taking Prediction in the Operating Room. In 2016 AAAI Fall Symposium Series about Artificial Intelligence for Human-Robot Interaction, p. 117-123. [pdf] [full talk video]
Zhou, Tian., & Wachs, Juan (accepted). Early Prediction for Physical Human Robot Collaboration in the Operating Room. Autonomous Robots.
Videos (system overview):
Videos (recorded talk at 2016 AAAI Fall Symposium):