Robust Sim2Real Transfer with the da Vinci Research Kit:

A Study On Camera, Lighting, and Physics Domain Randomization

Mustafa Haiderbhai, Radian Gondokaryono, Thomas Looi, James M. Drake, Lueder A. Kahrs

Medical Computer Vision and Robotics Lab, The Wilfred and Joyce Posluns CIGITI @ SickKids, University of Toronto

Abstract

Autonomous surgical robotics is a growing area of research, with advances being made in the areas of vision and control. Central to this research is the need for simulations to facilitate data collection and simulate learning environments for Reinforcement Learning (RL) agents. Recent simulators have facilitated RL policy generation, but lack a robust sim2real pipeline and a proven vision-based policy that can use any type of camera including the da Vinci Surgical System (dVSS) Endoscope.

To solve this, we build a ROS-based sim2real pipeline that incorporates a Unity3D da Vinci Research Kit (dVRK) simulation, modular kinematics, and shared interfaces. We examine the vision-based task of cube pushing, and train RL policies to execute in real life through Domain Randomization. Our experiments evaluate model success in simulation and two camera systems: OAK-1 and the dVSS Endoscope. Our results indicate that Domain Randomization is effective at bridging the sim2real gap, and even extends to the difficult endoscope scenario.

We achieve 100% transfer success rate on both OAK-1 and the dVSS Endoscope, with gains of over 60% compared to a base model with no Domain Randomization. We examine the various randomization parameters, including lighting, camera, and physics variables, and determine that all parameters play a significant role in bridging the sim2real gap. Testing across extreme lighting and camera configurations not seen in simulation, our models continue to perform well, with 85% accuracy on the OAK-1 camera. Our future work will extend to other tasks and more complex policies to take advantage of stereo-camera imaging.

The Sim2Real Gap

Models trained purely in simulation will show 100% success rate inside the simulation, but will fail when tranferred to an equivelant real life setup.

Simulation

SUCCESS

Regular (Oak-1) Camera

FAILURE

da Vinci Endoscope

FAILURE

Novel Simulation

To accomplish successful sim2real transfer, we build a novel simulator with modern and accurate dVRK physics. Our simulation is built on the Unity3D game engine, backed by nVidia PhysX physics, and modern rendering capabilities.

dVRK Modeling

Responsive image

Dynamics

Responsive image

Modular Kinematics

Responsive image

Custom Collision Meshes

Responsive image

Sim2Real Pipeline

Our Sim2Real pipeline builds off of our concurrent work in creating a modular robotics control framework. The modularity of our framework as well as a modular kinematics solver allows us to flexibly output to any modality, and accept input from any source. The entire pipeline can easily be swapped between simulation and real components at any step, providing easy and seamless sim2real testing and execution.

Domain Randomization

We employ Domain Randomization of various parameters to train robust agents including:

Light Direction/Intensity

Responsive image

Camera Pose

Responsive image

Cube Mass/Scale/Friction

Responsive image

After Domain Randomization

With sufficient domain randomization during simulated training, the sim2real gap can be overcome robustly.

Standard

Extreme Lighting Scenario

Extreme Camera Perspective

da Vinci Endoscope

Bridging the sim2real gap robustly allows us to execute with the da Vinci Endoscope, which is often ignored in autonomous surgical robotics in favour of commercial cameras not meant for surgical applications. The da Vinci endoscope has it's own light source and the narrow focused light and noisy image create a large gap for traditionally trained agents.

BibTeX Citation

@article{haiderbhai2022dvrksim2real, author = {Haiderbhai, Mustafa and Gondokaryono, Radian and Looi, Thomas and Drake, James M. and Kahrs, Lueder A.}, title = {Robust Sim2Real Transfer with the da Vinci Research Kit: A Study On Camera, Lighting, and Physics Domain Randomization}, journal = {International Conference on Intelligent Robots and Systems (IROS)}, year = {2022},}