Use of 5G technology for oncological surgery streaming

doi: 10.56294/dm2023126

ORIGINAL

Use of 5G technology for oncological surgery streaming

Uso de tecnología 5G para la transmisión de cirugías oncológicas

Nicolás Bravo¹ *, Inti Paredes¹*, Luis Loyola²*, Gonzalo Vargas²*

¹Fundación Arturo López Pérez, Unidad de Informática Médica y Data Science, Departamento de Investigación del Cáncer, Santiago, Chile.

²Allm Inc., Tokyo, Japan.

Cite as: Bravo N, Paredes I, Loyola L, Vargas G. Use of 5G technology for oncological surgery streaming. Data and Metadata 2023;2:126. https://doi.org/10.56294/dm2023126.

Submitted: 12-08-2023 Revised: 06-10-2023 Accepted: 10-12-2023 Published: 11-12-2023

Editor: Prof. Dr. Javier González Argote

ABSTRACT

This paper discusses the benefits of surgery streaming and tele-mentoring, as well as the use of 5G technology in surgical procedures. The paper describes the advantages of using wireless 5G broadband as a low-latency and large-bandwidth capacity connection, which can solve problems with cables and large equipment in the surgery room. The Chilean oncology clinic Fundación Arturo López Pérez coordinated an international project with Japanese companies NTT Data and Allm Inc. to implement a proof of concept using 5G technology for the transmission of an oncological surgery. This project consisted of the installation of a local 5G network, its configuration and testing, and the realization of the first broadcast of a robotic partial nephrectomy in Latin America using the 5G broadband. The paper provides details on the hardware infrastructure and components used in the project.

Keywords: 5G; Surgery Broadcast; Surgery Streaming; Surgery Education; Telementoring; Medical Informatics.

RESUMEN

En este artículo se analizan las ventajas de la cirugía en streaming y la tele-mentoría, así como el uso de la tecnología 5G en los procedimientos quirúrgicos. Se describen las ventajas de utilizar la banda ancha inalámbrica 5G como conexión de baja latencia y gran capacidad de ancho de banda, que puede resolver los problemas de cables y grandes equipos en el quirófano. La clínica oncológica chilena Fundación Arturo López Pérez coordinó un proyecto internacional con las empresas japonesas NTT Data y Allm Inc. para implementar una prueba de concepto utilizando tecnología 5G para la transmisión de una cirugía oncológica. Este proyecto consistió en la instalación de una red 5G local, su configuración y pruebas, y la realización de la primera transmisión de una nefrectomía parcial robótica en Latinoamérica utilizando la banda ancha 5G. El artículo proporciona detalles sobre la infraestructura de hardware y los componentes utilizados en el proyecto.

Palabras clave: 5G; Transmisión De Cirugía; Streaming De Cirugía; Educación En Cirugía; Telementoring; Informática Médica.

INTRODUCTION

The aim of this paper is to present a proof of concept for the use of 5G technology in surgical streaming. The transmission of surgical procedures is a valuable tool for training future physicians, and it has traditionally been done through face-to-face apprenticeship. However, technological advances in remote video recording and transmission have eliminated the physical barrier, allowing the techniques of an expert surgeon to be documented. The emergence of these technologies has revealed the possibility of tele-mentoring during the real-time broadcast of a surgical procedure, requiring the use of telecommunication technologies that deliver the lowest possible latency. The use of 5G technology is crucial for this purpose, as it provides a large bandwidth capacity comparable to wireless connections, while also bringing low communication latency. This paper presents a proof of concept using 5G technology for the transmission of an oncological surgery. The paper details the installation of a local 5G network in a surgery room of Fundación Arturo López Pérez (FALP), the network configuration and testing, and the realization of an official transmission of a robotic partial nephrectomy. The paper also presents related work on surgical transmission experience and highlights the minimum aspects that must be met in the broadcast of a clinical procedure.

METHODS

Related Work

Surgical transmission experience is not new, having documented experiences by academia since 1996.^(1,2) In 2019 Lacy et al. performed a transmission of 2 laparoscopic gastrointestinal surgeries through a dedicated 5G signal, connecting the image of the ward with a telementor located 4 kilometers away in a fixed point. In this experience, the 5G network managed to obtain a latency of 140 ms and a bandwidth of 100 MB/s, allowing a smooth and comfortable interaction between the surgeon and the telementor.⁽³⁾

The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) at its 2019 summit reached consensus on the minimum aspects that must be met in the broadcast of a clinical procedure, highlighting 3 key aspects:^(4,5,6)

● Transmission latency should be less than 450 microseconds.

● A resolution of 1080p at 30 frames per second is sufficient for most surgical applications.

● Telementoring technology should allow the mentor to annotate the video of the transmission (telestration) for better communication with the team in the room.

Development

Due to the benefits of surgery broadcast and 5G technology, FALP has coordinated an international project to implement a proof of concept of using 5G technology for the transmission of an oncological surgery. We worked in collaboration with the Japanese companies NTT Data and Allm Inc, in which the former was responsible for the installation and configuration of the equipment to set up a local 5G network in a FALP surgery room, and the latter was responsible for providing the devices and software necessary for the transmission of the procedure.

The project consisted of 3 stages:

● Installation of local 5G network in a surgery room within FALP.

● Network set-up and testing to obtain the best configuration for the transmission of an oncological surgery.

● Closing of the proof of concept with the realization of an official transmission of a robotic partial nephrectomy.

Installation of local 5G network

As the first stage, which took place during the second half of 2022, the hardware infrastructure was imported from Japan, hardware necessary for the installation of the local 5G network in a FALP's surgery room. This was achieved thanks to the support of NTT Data, which, in addition to procuring and sending the Fujitsu equipment, provided the Japanese engineers needed to configure it, closing this stage with the installation of a 5G antenna on the ceiling of FALP's surgery room.

The hardware infrastructure of the local 5G network was manufactured and provided by Fujitsu, consisting mainly of dedicated hardware and software for the proof of concept of this project, using a 4,7GHz broadband. The main components of the infrastructure are:

● PRIMERGY RX2530 M5. 5G core unit and base station central unit.

● Dedicated equipment as a 5G distribution unit.

● Indoor Radio Aggregation HUB for antenna connection.

● Indoor DAS Antenna pRadio.

● PRIMERGY RX1330 M4. Element management system.

Network configuration and test broadcasts

The second stage included the network survey and its correct configuration, where 10 urology surgical procedures were transmitted, measuring the throughput and latency of the transmissions, as well as testing capture devices for 3 different surgical scenarios: robotic, laparoscopic and open surgery.

Tests broadcasts performed:

● 6 robotic partial nephrectomies.

● 2 laparoscopic radical nephrectomies.

● 1 open radical prostatectomy.

● Retro-peritoneal tumor removal and vascular reconstruction of the kidney.

In figure 1 we show the general equipment configuration used during each testing and official broadcast. Depending on the procedure, the broadcast of the main video feed has a resolution of 1080p at 30 frames per second. The components of the broadcast equipment are:

a. For robotic surgeries we use the video output of DaVinci Surgery Equipment.

b. For laparoscopic surgeries we use the image of the laparoscopic turret.

c. For open surgeries we use a smart glasses Vuzix M400 with integrated camera capturing the surgeon's point of view.

d. The smart glass connects through Bluetooth to a Microsoft Display Bluetooth-HDMI capture card to send its camera feed throughout HDMI.

e. All procedures transmit the anesthesia machine's vital signs monitor at 720p resolution at 30 frames per second.

f. A HDMI-Ethernet H.264 encoder STRASSE HVS-3000 transforms the HDMI signal of the main video feed and the vital signs monitor feed to an Ethernet signal.

g. All surgeries broadcast the environment of the surgery room through 2 PTZ IP Hikvision 2MP dome cameras installed in mobile stands for better handling. Both feeds transmits using Ethernet with a resolution of 720p at 30 frames per second.

h. A POE Ethernet Switch combines the Ethernet signals of the HDMI encoder and IP cameras into a single Ethernet signal.

i. An APAL 5G MiFi router transforms the output Ethernet signal of the Switch into a 4,7GHz signal compatible with the local 5G signal installed in FALP's surgery room.

j. The dedicated 5G antenna in the ceiling receives the signal of the MiFi router to deliver the data to the stream processing server.

Figure 1. FALP’s surgery room broadcast equipment configuration

Broadcast platform

For the streaming of the video feed of the surgery, we use the communication platform Join developed by Allm Inc.⁽¹⁾Figure 2 shows the infrastructure between the surgery room and this web platform. Join provides a private and authenticated room to watch the 4 video feeds (main camera, vital signs monitor and 2 surgery room cameras) at once or one at a time fullscreen. Additionally, it allows annotations over a video with a pencil for clearer communication with every viewer, a feature that can improve the experience between the tele-mentor and the surgeon. The platform allows the broadcast of audio, but during this proof of concept we decide not to use this feature to respect the privacy of the surgery staff.

Figure 2. Network infrastructure between surgery room and JOIN platform. The feed is sent to a gateway server inside the clinic which encodes the video in HLS protocol and sends it to a cloud server, where Join distributes the feed to all viewers⁽¹⁾ https://www.allm.net/en/join/

RESULTS

Official broadcast for proof of concept

On March 13, 2023, as shown in figure 3, an official transmission was made integrating 4 video feeds of a robotic partial nephrectomy: 2 dome cameras to transmit the surgery room environment, the image from the vital signs monitor of the anesthesia machine and the camera of the DaVinci Surgical Team. Figure 1 shows the configuration of the surgery room streaming setup.

Figure 3. FALP's official surgery broadcast of a robotic partial nephrectomy using Local 5G network. Top feeds show the surgery room dome cameras. Lower left feed shows the vital sign monitor of the anesthesia machine, and the lower right feed shows the view of the Davinci Surgery Equipment

Network measurements

After selecting the final stream setup (camera resolutions and streamed frames per second) described in figure 1, we measure the Frames per Second (FPS) and the latency of the main stream feed (DaVinci surgery device) from 5 different Chile: La Florida, Maipu, Providencia, Viña del Mar and Concon. We measure the streaming of two robotic partial nephrectomies, the first one using FALP's Local 5G and the second one plugging the network switch directly to an Ethernet port inside the surgery room, therefore not using the Local 5G.

Table 1 shows the video experience in the viewer web browser depends on the user network, but overall, the frames per second of the DaVinci video feed were stable around 30 FPS except using Rural 4G, with no apparent difference between streaming by Local 5G or Ethernet wired. The aspect that suffers the most is the latency, with almost 2 seconds more using an Ethernet wired connection instead of Local 5G.

Table 1. Frames per second (FPS) and Latency (seconds) measurements from 5 different Chile locations with different network availability. Final stream setup.
Network	Local 5G		Wired
Network	FPS (average)	Latency [s] (average)	FPS (average)	Latency [s] (average)
Urban 5G	24-30 (27)	1-2 (1,7)	23-30 (26)	2-3 (2,4)
Urban 4G	25-31 (27)	1-2 (1,4)	15-30 (25)	1-3 (1,7)
Urban optical line	23-30 (26)	1-2 (1,5)	23-30 (26)	2-3 (2,3)
Rural optical line	25-31 (28)	1-5 (1,6)	25-30 (28)	1-2 (1,2)
Rural 4G	15-30 (24)	2-8 (4,2)	18-30 (26)	2-8 (3)

DISCUSSION

The project concludes with the first surgery in Latin America broadcast using 5G technology, which, from a practical standpoint, was a success for the technical and medical teams involved.

As shown in table 1, the broadcast latency using 5G did not meet the suggested 400 ms of SAGES. This difference may be due to a key factor in the video feed pipeline: video re-encoding at the surgery room level and the streaming server level. This computationally demanding task adds overhead to the video broadcast, resulting in additional time added to the feed latency. We are currently researching better re-encoding approaches for other broadcast-related projects, and testing with different protocols (RTSP, HLS, or WebRTC) and hardware.

Future roadmap

This first phase included the proof of concept of using the 5G network in a surgery room, evaluating equipment and capturing software available on the market. The next task is to use the experience gathered in this stage to implement different products for the benefit of public health in Chile, related to tele-mentoring of surgeries, transmission of surgeries for medical education, and storage for documentation.

Surgery education web platform

Taking advantage of the first benefit generated by the transmission of surgery, we plan to create a platform for remote surgery education. Allowing interaction with resident surgeons who may be training anywhere in the country, and giving them access to different oncological surgeries performed at FALP. This access and transmission must follow a model of confidentiality of information and effective authentication of users, thus allowing the generation of agreements with different medical education institutions. In addition, this platform seeks to provide an effective means for spectators to consult with the surgeon, without hindering the surgeon's work and concentration; achieving an even more effective transfer of the experience, and better documentation of the teaching sessions.

Tele-mentoring assistance

The surgical education platform will also open the possibility of tele-mentoring complex surgical procedures, allowing the participation of surgeons from outside the surgery room. This practice will be very useful when considering the territorial expansion of FALP, allowing the assistance of surgeons between different FALP centers in the Chilean territory.

AI-enhanced learning videos

The video of the surgeon's point of view is a focus zone where a variety of anatomical parts of the patient are visualized. In these recordings it is planned to apply, in real time, anatomical object detection models as proposed by Bamba et al.⁽⁵⁾, further enriching the transmission of the procedure for educational purposes, as well as its documentation.

CONCLUSIONS

In conclusion, the successful live broadcast of the surgery using FALP's infrastructure, NTT Data's technology supply, and Allm Inc's web broadcast solutions demonstrated the impressive capacity of these technologies to support high-quality live broadcasts. The feed was seamless, without freezing or stuttering, and the latency fell within the time window specified in Table 1. Although the latency was slightly greater than recommended by SAGES, the JOIN web platform's flexibility allows users to watch the feed from any device and location with stable and uniform latency, making it suitable for teaching surgery interns and tele-mentoring.

It is worth noting that, in order to prevent the leakage of radioactivity from radiology procedures, each FALP surgery room is surrounded by a lead layer that can significantly reduce the intensity of public 5G broadband from the exterior. Therefore, it is recommended that booster antennas be used inside healthcare buildings with these characteristics to ensure a reliable signal.

Finally, during the trial and official broadcasts, the 4,7GHz frequency broadband did not create any interference with wireless medical devices inside the surgery room. However, caution should be exercised with the signal frequencies of any surgery room to prevent interference with wireless medical devices. Overall, the success of this project provides a solid foundation for further research and development in the field of live surgery broadcasting, and we look forward to seeing more advancements in the near future.

REFERENCES

1. Moore RG, Adams JB, Partin AW, Docimo SG, Kavoussi LR. Telementoring of laparoscopic procedures: initial clinical experience. Surgical endoscopy. 1996 Feb;10:107-10.

2. Cubano M, Poulose BK, Talamini MA, Stewart R, Antosek LE, Lentz R, Nibe R, Kutka MF, Mendoza-Sagaon M. Long distance telementoring: a novel tool for laparoscopy aboard the USS Abraham Lincoln. Surgical endoscopy. 1999 Jul;13:673-8.

3. Lacy AM, Bravo R, Otero‐Piñeiro AM, Pena R, De Lacy FB, Menchaca R, Balibrea J7. 5G‐assisted telementored surgery. British Journal of Surgery. 2019 Nov;106(12):1576-9.

4. Bogen EM, Schlachta CM, Ponsky T. White paper: technology for surgical telementoring—SAGES Project 6 Technology Working Group. Surgical Endoscopy. 2019 Mar 15;33:684-90.

5. Horta-Martínez LE. 3D printing in the medical field. Seminars in Medical Writing and Education 2022;1:8–8. https://doi.org/10.56294/mw20228.

6. Bamba Y, Ogawa S, Itabashi M, Shindo H, Kameoka S, Okamoto T, Yamamoto M. Object and anatomical feature recognition in surgical video images based on a convolutional neural network. International Journal of Computer Assisted Radiology and Surgery. 2021 Nov;16(11):2045-54.

FINANCING

This project would not have been possible without the support and funding of the Ministry of Telecommunications of Japan, as well as the sponsorship of the Undersecretariat of Telecommunications of Chile.

CONFLICT OF INTEREST

The authors declare that there is no conflict of interest.

AUTHORSHIP CONTRIBUTION

Conceptualization: Inti Paredes, Nicolás Bravo, Marcela Aguirre.

Formal analysis: Gonzalo Vargas.

Acquisition of funds: Ministry of Telecommunications of Japan.

Research: Inti Paredes, Nicolás Bravo.

Methodology: Nicolás Bravo.

Project management: Nicolás Bravo (FALP), Gonzalo Vargas (Allm Inc), Aniluz Auikalil (NTT Data).

Resources: Ministry of Telecommunications of Japan.

Software: Allm Inc.

Supervision: Inti Paredes, Luis Loyola, Denise Garnica (NTT Data).

Validation: Allm Inc.

Display: Fundación Arturo López Pérez.

Drafting - original draft: Nicolás Bravo.

Writing - proofreading and editing: Nicolás Bravo.