Advertisement

Forget Pokemons – in world first, Hongkonger applies augmented reality to surgery

Medical graduate Catherine Chan, armed with a master’s degree in medical imaging, is fine-tuning AR application that solves the biggest problem in keyhole surgery – having to look at images on two screens simultaneously

Reading Time:2 minutes
Why you can trust SCMP
0
A computer screen shows combined real-time images from diagnostic probes during minimally invasive surgery. The technology was spearheaded by Catherine Chan. Photos: K.Y. Cheng
Elaine Yauin Beijing

A Chinese University of Hong Kong graduate has pioneered the use of augmented reality for medical imaging during minimally invasive surgery, removing the need to look at multiple images which can be a distraction during surgery.

Catherine Chan Po-ling, an orthopaedic surgeon at Prince of Wales Hospital in Sha Tin, says the pioneering technology has solved one of the limitations of minimally invasive, or keyhole, surgery that has dogged surgeons.

Demonstrating with a pig liver, she explains: “To check whether the liver has cancerous cells, a doctor has to use an ultrasound probe to see beneath the surface of the liver. There are two cameras showing images. One, from an ordinary camera, shows the surface of the liver. The other is the camera from the ultrasound probe which shows the same black-and-white images expectant mothers see of their fetuses. The doctor has to look at two images on two screens a while manoeuvring two probes at the same time. This requires a lot of hand-eye coordination. This has long been the biggest problem regarding minimally invasive surgery.”

Catherine Chan Po-ling.
Catherine Chan Po-ling.
In the hit augmented reality game Pokemon Go, a digital cartoon monster image is laid over real images. With the new technology pioneered by Chan, augmented reality combines the images from the ordinary camera and the ultrasound probe using computer vision tracking technology.

“While monsters pop up on phone screens of Pokemom Go players, combined real-time images of the probed body parts pop up on the screen.”

Chan was the first graduate sent by the university to take a master’s degree in medical robotics and image-guided intervention at Imperial College London last year. It was part of a move by the university to boost collaboration between the medical engineering and medical fields after the establishment of the Chow Yuk Ho Technology Centre for Innovative Medicine last year.
Professor Joseph Sung, vice-chancellor of the Chinese University of Hong Kong, demonstrates the endoscopic surgical robotic system at the Chow Yuk Ho Technology Centre for Innovative Medicine.
Professor Joseph Sung, vice-chancellor of the Chinese University of Hong Kong, demonstrates the endoscopic surgical robotic system at the Chow Yuk Ho Technology Centre for Innovative Medicine.
The AR software was written by engineers at Imperial College, says Chan, “but they just invented the software without any clinical input. Among my 14 classmates at the college, only three [including me] have a background in medicine. Representing Chinese University and working under the guidance of my supervisors at Chinese University, we are the first in the world to put AR technology into medical applications.
Advertisement