Surgical Simulation Prototype for SkySurgery

The Challenge

New methods are being developed to train surgeons who perform robot-assisted surgery. Tony Reina of SkySurgery LLC approached us with a unique and interesting challenge: to develop a prototype for a simulation used to assist surgeons performing live robotic laparoscopic surgery.

Traditionally, a mentoring surgeon uses a telestrator to instruct the trainee. Telestration allows the mentor to draw on top of the operative video image and explain the surgical maneuvers through lines, circles, and arrows.

SkySurgery’s goal was to come up with the next generation of this technology by adding virtual surgical instruments and physics simulation that allows the mentor to demonstrate complex maneuvers of the surgical tools. Mentors would then be able to use these virtual tools to cut, retract, and even suture the operative video as if it were real human tissue.

Bernard demonstrating the use of the Surgical Skin Simulator prototype. Note that the webcam video feed would normally be replaced by the laparoscopic camera inserted into the patient's body, but we had a hard time finding volunteers.

The Algorithms & Prototype

Since this project needed a custom physics simulation to be implemented, we devised several algorithms:

  • Creating the initial skin mesh (a set of triangles)
  • Adding an additional point to the skin mesh (additional triangles) to allow grabbing and dragging an arbitrary point.
  • The actual movement behavior of the skin after being pulled – treating the triangle’s edges as springs using Hooke’s Law.
  • Cutting the skin when using the scalpel tool.
  • Tearing the skin when it’s pulled hard.
  • Removing redundant points.

We documented these to allow SkySurgery to formalize a patent application.

An image extracted from the patent application, illustrating a part of the algorithm used when making an incision.

An image extracted from the patent application, illustrating a part of the algorithm used when making an incision.

We then implemented the algorithms in the prototype, which was done in Unity3D. This work was broken up into four iterations, each time refining the algorithms and adding new features.

The skin behavior tuning options within the prototype.

The skin behavior tuning options within the prototype.

In the last iteration, we added support for the Razer Hydra – a pair of controllers from which an accurate three dimensional position and rotation can be obtained – resulting in a very intuitive way of controlling the simulator, allowing advising surgeons to literally pick it up and use without training.

Left: The Razer Hydra used in the Surgical Skin Simulator prototype. Right: The skin mesh is composed of a number of triangles.

The Razer Hydra used in the Surgical Skin Simulator prototype

The Result

SkySurgery partnered with Foresight Imaging LLC to patent this technology and incorporate it into Foresight’s TIMS Consultant, an FDA-approved telemedicine device that allows the mentor and trainee collaborate in realtime over a standard internet connection. They firmly believe that the new device greatly expands and enhances the capabilities of telepresence for both medical and non-medical applications.

In April, SkySurgery and Foresight Imaging presented the prototype at the Innovation, Design, and Emerging Alliances in Surgery conference. This poster summarizes the concept in medical terms.

7 thoughts on “Surgical Simulation Prototype for SkySurgery”

  1. along says:

    Hello, is there any source code for this project? Could you please send me the unity project project?

    1. Bernard says:

      Hi Along, the prototype and source code actually belong to our client. However, if this interests you I may be able to introduce you to them if you send me a message through the contact form with some more information. We’re always happy to help!

  2. along says:

    hello,My Email is,Thank you.

  3. zhangshuguang says:

    Hi,how do you make it ?use unity ?could you tell me what tool do you use ?

  4. Bernard says:

    Hi Zhangshuguang, we use Unity indeed.

  5. Anshul says:

    I am an university student working on surgical simulation project in unity, can I get the algorithm or source control of this project.

  6. Bernard says:

    Hi Anshul, thank you for reaching out. We received a few queries of people interested in the source code, and our client did confirm that we can release it. We will do it in the next couple of weeks. Feel free to reach out through our contact form in case you’d like an update on this.

Leave a comment

Your email address will not be published. Required fields are marked *