Magnetically Guided Laser Surgery for the Treatment of Twin-to-Twin Transfusion Syndrome

Twin-to-twin transfusion syndrome (TTTS) is a severe disorder that often leads to the death of monochorionic twin fetuses, if left untreated. Current prenatal interventions to treat the condition involve the use of rigid fetoscopes for targeted laser coagulation of the vascular anastomoses. These tools are limited in their area of operation, making treatment challenging, especially in cases with anterior placentation. Herein, a robotic platform to perform this task using remote magnetic navigation is proposed. In contrast to rigid tools, the presented custom magnetic fetoscope is highly ﬂ exible, dexterous, and has considerable advantages, including safety and precision. A visual servoing algorithm that allows the surgeon to navigate in the uterus with submillimeter precision is introduced. The system has been validated on ex vivo human placentas in a setting that mimics the real intraoperative conditions.

in turn, can lead to the recurrence of TTTS. [5,12] Moreover, by lasering at other than an optimal 90°angle between the endoscope and the vessels, there is an increased risk of damaging healthy surrounding tissue. Therefore, procedure-related complications occur more frequently in cases with anterior placentas. [13] Surgical equipment for TTTS surgery has remained largely unchanged since its introduction 30 years ago. [10] Straight, rigid endoscopes can be used for the coagulation of posterior placentas. To reach anterior placentas, endoscopes that are curved or equipped with deflection mechanisms are preferred. As accessing anterior placentas is particularly challenging, alternative methods have been considered for these specific cases, including the use of side-firing laser fibers [5,14] and laparoscopic assistance. [5,15,16] Several research groups have attempted to facilitate the surgery by developing novel endoscopes with automated steering. None of these efforts have reached clinical readiness to date. [12,[17][18][19][20][21][22][23][24] In this work, we introduce a robotic platform to perform TTTS surgery using remote magnetic navigation (RMN). In RMN, magnetic fields are used to navigate devices containing magnetic material. [25] Due to the low magnetic susceptibility of tissue, these fields have no reported negative impacts on the human body, [26] and RMN is, thus, particularly promising for minimally invasive robotic surgeries. In 2001, Stereotaxis Inc. designed the first clinical system to manipulate catheters for heart ablation procedures via external magnetic fields that are generated with mobile permanent magnets. [27] In contrast to traditional cardiovascular catheters that use tendon wires to steer the orientation of the ablation tip, the magnetic catheter offers higher flexibility, stability, and precision. [28][29][30][31] While this technology is well established in clinical settings for cardiac ablation procedures, [32] researchers have only recently investigated other applications, such as navigating through the neurovascular system, [33,34] magnetic guidance for steering a catheter to a biopsy target within a pig brain, [35] steering a magnetic catheter for ophthalmic applications, [36][37][38][39] and manipulating magnetic tools for fetal surgery. [40] Other research has focused on the vascular system, [41,42] colonoscopies, [43] the cochlea, [44] or the nose. [45] Here, we use a flexible, magnetic fetoscope with an integrated high-resolution camera and a working channel for a laser fiber. The fetoscope can be readily navigated at the different possible locations of the placenta within the uterus during surgery, including the challenging position of the anterior placenta. To steer the fetoscope remotely, we use an electromagnetic navigation system (eMNS) in combination with a robotic advancer unit, as shown in Figure 1. While the eMNS generates the necessary magnetic fields to guide the orientation of the magnetic tip, the robotic advancer unit is used to insert or retract the fetoscope. This combination allows for a large workspace and provides control over the orientation of the fetoscope and its integrated laser fiber. RMN is particularly interesting for TTTS surgery, as it enables precise manipulation of highly flexible, small tools, which increase patient safety. In addition, our platform offers many of the advantages of conventional robotic approaches, such as tremor reduction, increased dexterity, and automated and teleoperated manipulation. To automate steering, visual servoing techniques have been extensively used for several decades in the Figure 1. Overview of magnetically guided fetal surgery. The eMNS is positioned near the patient. The electromagnetic coils embedded in the eMNS create the necessary field to deflect the magnetic fetoscope within the patient's uterus. During the procedure, the fetoscope can be guided to the desired location using visual feedback from the camera as well as from commonly used ultrasound probes. The robotic advancer unit allows the surgeons to accurately insert the fetoscope into the uterus. Combined with the magnetic field, this allows them to place it precisely at a desired distance to the placenta. The laser can be activated at any time, similar to conventional fetoscopes. industrial context using rigid robotic manipulators that operate in a well-controlled environment. [46] However, their use for medical soft continuum robots remains extremely challenging to date. This is due both to the complexity and variability of the visual and physical environment of the endoscope, and to the soft robot's dynamic behavior. [47] There is, however, a clear trend toward supervised autonomous actions of robotic endoscopes, as visual servoing has shown the potential to automate repetitive tasks such as wound closure and anastomosis. [48] Our main contributions to this article include: 1) the design of a fully functional magnetic fetoscope, 2) the derivation and implementation of a dedicated visual servoing approach for navigating magnetic devices with a distal camera, 3) a method to update the Jacobian matrix that captures the variability of the physical environment due to changes in fetoscope strain or inaccuracies in the eMNS, as well as 4) a demonstration of these contributions by performing a semi-automated magnetically guided ablation on ex vivo human placentas in a setting that mimics the environment of a conventional procedure.
Following the introduction, two modes of operation are introduced in the Results section. A teleoperated mode allows the surgeon to have direct control over the movements of the fetoscope, and a second visual servoing mode allows the surgeon to select targets on the fetoscopic image to reach the desired location. We then present the performance of our system and experimental results conducted on ex vivo human placentas. In the discussion, the results are examined and in the conclusion, we present a short summary and future prospects of this work. Finally, the methods are presented. Figure 2 shows the setup and components used in this study. They are described in detail in Section 5. The externally applied magnetic fields were generated by the MagHead, a custom eMNS, which uses three electromagnets in a parallel triangular arrangement to generate magnetic fields. By regulating the currents in each coil, it can generate magnetic fields for manipulation.

Remote Magnetic Navigation
The fetoscope contains permanent magnets at the distal tip, such that a magnetic torque can modify its orientation. The magnetic torque t i ∈ ℝ 3 in (N⋅m) applied on a single permanent magnet i is given by where m i ∈ ℝ 3 denotes the magnetization vector of the permanent magnet in (A⋅m 2 ) and bðp i Þ ∈ ℝ 3 denotes the externally applied magnetic field's flux density at the location p i of the magnet in (T). The magnetic tip tends to align in the direction of the applied magnetic field due to the magnetic torque, which is used to manipulate the orientation of the catheter. The angle between the catheter orientation and the field orientation depends on the stiffness of the catheter and the magnitude of the magnetic field.
The magnetic field was controlled by transmitting the necessary currents to the eMNS via a TCP/IP connection that was established using the open-source robot operating system (ROS). A set of currents i d ∈ ℝ 3 that produce the desired magnetic field b d at a desired position p d can be found using a calibrated magnetic model. [49] where A † m ðp d Þ ∈ ℝ 3Â3 is the pseudoinverse of the actuation matrix A m at the position p d and M b d ∈ ℝ 3 is the desired field vector in the so-called eMNS frame (subscripted "M") that is attached to the eMNS and in which the magnetic model was calibrated. For simplicity, we assume the magnetic field is homogeneous (i.e., spatially invariant) within the eMNS workspace W 3 so that For the purpose of magnetic calculations, we can, therefore, assume p d to be a fixed position at the center of the workspace. We control the magnetic field in a closed loop, so inaccuracies coming from the model are rejected by the controller. Previous work with this [40] and other eMNSs have shown that this simplification generates satisfying results. [36,38,39,50] During the experiments, the field magnitude jjb d jj was constant at b mag ¼ 18 mT, which is approximately two orders of magnitude smaller than a clinical magnetic resonance imaging (MRI) system.

Teleoperation
To allow manual teleoperation of the fetoscope, we used the Phantom Omni haptic input device ( Figure 2A). The manual operation mode would typically be used to move the fetoscope to a broader area on the placenta before engaging the automated control mode described in Section 2.3. The most intuitive steering with the Phantom Omni's stylus occurs when the motion of the stylus corresponds directly to that of the catheter in the magnetic field. However, the stiffness of the catheter, gravity, and inhomogeneities of the field can lead to a misalignment between the input field angle of the Phantom Omni and the fetoscope orientation. The Phantom Omni was initially calibrated in multiple configurations to create a reference table. The reference table maps the orientation of the stylus to the magnetic field, which leads to corresponding angles between the stylus and the catheter tip. During the experiments, this reference table was interpolated online to apply the correct input field for the desired orientation of the fetoscope, as shown in our previous work. [40] The insertion length of the catheter can be adjusted with buttons located on the Phantom Omni's stylus. Furthermore, the gravity of the stylus was haptically compensated to make it virtually weightless for the user.

Visual Servoing
To automate the positioning of the catheter tip with high precision, we introduce an automated visual servoing approach that is shown in Figure 3A. The aim is to control the vertical and horizontal translations of a feature that is located at . Therefore, the goal is to find an appropriate field b d such that the induced torque described in Equation (1) creates the desired movements in the image.
The rotation of the magnetic field b d that leads to this motion is expressed as a chain of intrinsic rotations with a coordinate system x 0 -y 0 -z 0 (subscripted "B"), which is fixed on the field vector b d . As shown in Figure 3B, we chose our coordinate frame such that B b d ¼ ½b mag 0 0 T . Each successive rotation and the associated change of the heading of b d can then be fully parameterized with two intrinsic Euler angles α 1 and α 2 , which describe rotations around the z 0 and y' axes. With u ¼ ½α 1 α 2 T , we can find a Jacobian matrix J ∈ ℝ 2Â2 that approximates the first-order differential system behavior around the current heading of b d [51] I p : f ¼ ÀJu : (4) assuming the relative motion between the placenta and the fetoscope's tip is only caused by the magnetic actuation. When J has full rank, we can directly relate the parameterized angular velocities u : of the magnetic field to the feature velocity I p : f in the image coordinates with Figure 2. Magnetic fetal surgery platform used for the ex vivo study. A) Experimental setup to validate our approach. A model of a pregnant woman and uterus (scale bar corresponds to uterus size) that contain an ex vivo placenta is placed near the eMNS. The magnetic fetoscope is inserted into the uterus through a 10 French trocar, typically used in a clinical setting. The laser fiber is integrated into the magnetic fetoscope and can be guided to the desired location, visualized with the integrated camera. B) The fetoscope is comprised of different sections of stiffness to increase stability and maximize maneuverability (see Section 5). Permanent magnets embedded at the tip are accompanied by reinforcement coils to ensure uniform bending of the fetoscope. C) The fetoscope is axially actuated with a customized robotic advancer unit. The lateral actuation is achieved by applying an adequate magnetic field at the location of the tip. To facilitate surgeries on anterior placentas, the tool was designed to achieve an upward bending angle of 90°at a realistic (15-25 cm) distance from the eMNS.
www.advancedsciencenews.com www.advintellsyst.com For fetoscopic laser coagulation, we assume that the position of the laser in the image frame is constant and centered in the middle of the image at I p ref ¼ ½0 0 T . To bring a tracked feature I p f to the center of the image, we define the control error I e ∈ ℤ 2 as Combining Equation (5) and (6), we arrive at the differential equation governing the dynamics of our control system To exponentially decrease the error in Equation (7) with minimal overshoot and static error, we introduce a proportionalintegral-derivative (PID) control law that is governed by where K p,e , K i,e , and K d,e are constant scalars that are found experimentally. Thus, to bring a tracked feature I p f to the center of the image, we can calculate the necessary control inputs u : with where I e $ ∈ ℝ 2 is the adapted control error that can be calculated with Equation (8). The control input u : is translated into a magnetic field vector b d by intrinsically rotating the previous field vector by the increment ∂u ¼ ½ ∂α 1 ∂α 2 T with ∂α 1 ¼ α : 1 ∂t and ∂α 2 ¼ α : 2 ∂t in their respective axes. Since the implementation of the controller is realized in discrete time, we denote Δu ¼ ½Δα 1 Δα 2 T as their discrete counterparts. We can A) The aim is to control the position of a feature I p f in the fetoscope image. This can be achieved by moving the fetoscope with a magnetic torque t ¼ m Â b d . The desired magnetic field that needs to be generated by the eMNS is calculated with a linearized inverse kinematics model that is valid in a moving reference frame {B}. The intrinsic rotations of the magnetic field in that frame can be parameterized with two angles α 1 and α 2 . These two control inputs are then transformed into the input M b d in the inertial eMNS frame {M} to generate the desired field. The Jacobian estimator provides the Jacobian matrix J to the controller by monitoring the feature trajectory I p f . The measured control error I e as well as the detected misalignment γ of the Jacobian estimator is subjected to a proportional-integral-derivative (PID) control law. B) The x-y-z eMNS frame is inertial, while the x 0 -y 0 -z 0 frame is a moving coordinate system attached to the field vector b d . The magnetic torque t is generated by the interaction of the embedded magnets with magnetization m and the magnetic field b d . C) The feature trajectory I p f is monitored in the fetoscope image coordinate system (I). A poorly estimated Jacobian matrix leads to an increased angle γ between the actual and the desired heading of the feature trajectory.
www.advancedsciencenews.com www.advintellsyst.com characterize this intrinsic rotation R Δ ∈ ℝ 3Â3 with two matrices R z 0 ∈ ℝ 3Â3 and R y 0 ∈ ℝ 3Â3 that describe the elemental rotations around the z' and y' axes of the moving x 0 -y 0 -z 0 frame ( Figure 3B). With our incremental control input Δu, we can write with and To calculate the rotation matrix R k ∈ ℝ 3Â3 that describes the rotation from {B} to {M} at the time step k ≥ 1 in the discrete implementation of the controller, we apply the iterative update law.
where R 0 defines the initial orientation of the magnetic field and Δu k is the angle increment to be performed at step k. Finally, we can use this transformation to express our desired field vector in the inertial eMNS frame as The visual servoing controller is typically initialized after moving to a desired area via teleoperation.

Jacobian Matrix Estimation
The inverse Jacobian matrix J À1 is initially estimated by performing two perpendicular motions with an angular velocity ω in the moving x'-y'-z' frame. [51] Given the two inputs u  (5) can be reformulated to include two motions. We can then calculate an estimateĴ À1 of the Jacobian matrix witĥ Due to the nonlinear kinematic behavior of the magnetic fetoscope, the estimate of the Jacobian matrix in Equation (5) is only valid within a limited area of operation. To address this problem, we employed a method to continuously update the Jacobian matrix during the procedure. As shown in Figure 3C, a poor Jacobian matrix leads to feature trajectories that move in a spiral toward the central target (white dashed line). Even though the PID controller can often overcome a poor Jacobian matrix, this leads to a slow control algorithm and, unnecessary, distracting movements of the fetoscope. To overcome this problem, we introduced a model-free approach where we measured the misalignment angle γ during motion ( Figure 3C). In the discrete controller, this was implemented by monitoring the feature trajectory and extracting γ k with the law of cosines.
γ ¼ arccos jj I e kÀn jj 2 þ jj I e k À I e kÀn jj 2 À jj I e k jj 2 2jj I e kÀn jj ⋅ jj I e k À I e kÀn jj (16) where γ k denotes the misalignment angle at time step k, I e k ∈ ℤ 2 denotes the error in the current time step in the controller, and I e kÀn ∈ ℤ 2 denotes the measured error n ≥ 1 time steps ago. The choice of n is subjected to a trade-off between noise minimization and the ability of the measurement to capture recent changes in γ. We control the fetoscope by rotating the magnetic field b d to suitable positions. To simplify, we assume that a misalignment due to a poor Jacobian matrix can be updated with a rotation matrix. The misalignment γ k acts as an estimate for how much the Jacobian matrix should be rotated. Similarly to Equation (8), the aim is to achieve an exponentially stable dynamics in γ k by applying a PID law.
where K p,γ , K i,γ , and K d,γ are constant scalars that are found experimentally. Depending on the orientation of the misalignment, we can then update the inverse Jacobian matrix clockwise or counterclockwise. In practice, this update does not happen instantly, but is applied to the Jacobian matrix with an exponential decay profile over three time steps to avoid interfering with the visual servoing PID controller derived in Equation (8). In each of these time steps, we, therefore, apply a portion of γ $ k that we denote Δγ $ k . We can then update our Jacobian matrix witĥ where k denotes the current time step in the controller. The computation of the exponential decay profile can be found in Section 5.

Visual Servoing Performance
To demonstrate that our proposed visual servoing algorithm works in various locations of the placenta within the uterus, we placed paper patches of photographed placentas in anterior, posterior, sideway, and frontal locations with respect to the entry point. We imprinted four small visual markers in a 10 mm square to have normalized ground-truth targets ( Figure 4A,H).
To simulate realistic conditions for the feature tracker (see Section 5), the markers are small, relative to the size of the tracking window (height ¼ 100 pixels and width ¼ 100 pixels). The fetoscope was moved via teleoperation to the respective locations in the uterus and placed in front of the first marker with the visual servoing algorithm by selecting the marker directly in the fetoscopic image (see Movie S1, Supporting Information). Subsequently, the fetoscope was moved to the other targets in a counterclockwise fashion (targets labeled from I to IV in Figure 4C,H). The controller threshold, i.e., the precision of the visual servoing algorithm, was chosen as three pixels.
The path of the tracked feature I p f is shown in Figure 4C-G. They show normalized feature coordinates for each of the four tracked markers in the anterior (C) and (D), posterior (E), sideways (F), and frontal (G) position of the fetoscope with respect to its entry point as shown in Figure 4A. The four areas represent the most common possible locations of the placenta in the conventional procedure. To capture the real geometry, the model uterus was designed realistically according to a uterus at about 24 weeks of gestation. Due to the normalization, the distance between the center and any peripheral target point (both marked with crosses þ) corresponds to the real distance (10 mm) between two markers on the image of the placenta. As shown in Figure 4D-G, the Jacobian matrix was initially calibrated at the starting location, whereas Figure 4C shows the result of the Jacobian matrix calibrating approximately 800 pixels off from the starting point at the anterior location. This graph indicates that a poor Jacobian calibration leads to spiral movements and, thus, increased time between targets ( Figure 4B), if the Jacobian matrix remains constant. In contrast, a controller that makes use of the Jacobian matrix update pipeline can correct the feature trajectory rapidly such that the final target is approached in a straight line ( Figure 4C, zoom-in). Nevertheless, in the case of a well-calibrated Jacobian matrix, the updated algorithm will not lead to improved times between tracked targets ( Figure 4B). Figure 4H shows a typical feature trajectory when navigating the magnetic fetoscope on the anterior side of the uterus.

Ex Vivo Placenta Experiments
Two successive experiments on ex vivo human placentas were conducted to evaluate the viability of magnetic fetoscope laser surgery for TTTS. The preparation of the placentas and the model www.advancedsciencenews.com www.advintellsyst.com in which they were embedded are described in the Methods section. Two fetal surgeons who are experts in the conventional manual procedure performed the procedures using our system. As shown in Figure 5, the placentas were ablated by first navigating the fetoscope to the target area in the uterus via teleoperation. We evaluated both anterior and posterior locations of the placenta and, thus, the fetoscope was either pointed down or up during the respective experiments. Figure 4A shows the anterior location with letter C and the posterior location with letter E. The entry point (trocar) for the procedure is shown in Figure 2A. Once the surgeons steered the fetoscope to the desired location in the uterus, the visual servoing control mode was activated to achieve accurate positioning of the tip (see Movie S2, Supporting Information). The target ( I p f ) was selected manually in the fetoscopic image displayed on the screen. The robotic advancer unit was used to control the insertion of the catheter, which, combined with the magnetic field, can be used to control the distance between the laser fiber and the tissue. To achieve a desired distance to the placenta at a desired location in the uterus, it may, therefore, be necessary to iterate between insertion and visual servoing (see Figure 5). This strategy allows the operator to benefit from both the accuracy of the magnetic manipulation, as well as their intuition for the required distance between the placenta and the laser fiber to achieve an optimal outcome of the ablation. Once the desired position was reached, the laser was activated via a foot pedal to ablate the vessels on the placenta. Similarly, to the conventional procedure, the magnetic fetoscope does not move during ablation. After the operation, the water was drained from the uterus model and the placentas were removed for histological analysis. Figure 6A shows the placenta that was situated in the posterior location before (left) and after (middle) the ablation. The black arrows indicate the positions where the surgeons targeted the laser using the visual servoing algorithm. The differences in vessel structure from the ablation are highlighted with a close-up. To validate that the vessels were completely closed by the procedure, a contrast agent was injected in the proximal and distal locations of the targeted artery. A radiography of the injected placenta confirmed that the vessels had been completely coagulated and were impermeable to perfusion, indicating a successful ablation ( Figure 6A, right, white arrows). Figure 6B shows the results of the magnetically guided laser surgery on the anterior placenta. In this configuration, the magnetic fetoscope was bent %90°to achieve a perpendicular incident angle of the laser beam toward the placenta. We can see that the targeted vessel was ablated successfully, while the surrounding tissue remained intact (black arrows). Similar to the aforementioned, the ablated artery was injected with a contrast agent proximal to the location of the ablations. The white arrows in the radiography image of Figure 6B indicate the ablations that inhibited further perfusion of the blood vessels in that area.
A fundamental challenge in TTTS laser surgery is to target the anastomoses without damaging surrounding tissue. To validate that the magnetic fetoscope can be positioned precisely enough to avoid such damage, we analyzed the cell structure of the ablated artery. The black arrows in Figure 6C mark the ablated artery (right) that is highlighted in Figure 6A and an adjacent untreated segment of the same vessel (left). The untreated artery, sampled directly proximal to the location of the ablation, shows a healthy structure. The same artery in the region of ablation shows a complete coagulation with disruption of the adjacent amnion membrane, which is a typical histological finding in a successful ablation. Furthermore, the surrounding tissue remains unharmed in all directions, indicating a precise positioning of the magnetic fetoscope.

Discussion
There is a growing need for steerable and flexible endoscopes to visualize and ablate anastomoses on anterior placentas. [4,52] To address this need, a robotic platform that enables precise and intuitive navigation of a magnetic fetoscope anywhere in the uterus is presented. We utilized a shared control strategy that combines teleoperation and visual servoing. The semi-automated control scheme leaves the task of adjusting the insertion of the fetoscope and the target selection to the surgeon's manual commands, whereas the fetoscope positioning is automatically controlled by a vision-based algorithm.
This control method enables the surgeon to directly target locations in the fetoscopic image; thus, the lateral precision (precision of the laser on the placenta) of this approach is www.advancedsciencenews.com www.advintellsyst.com essentially only limited by the resolution of the camera, as the spatial resolution of the generated magnetic field from our custom eMNS far exceeds the related resolution in the fetoscopic image.
For the chosen threshold of three pixels, the lateral precision of the fetoscopic laser on the placenta would amount to approximately 120 μm at a distance of 10 mm. It is possible that during operation, the channel and spatial reliability tracking (CSRT) tracker moves several pixels from its initial location. However, the surgeon can reselect the exact desired location after moving to the vicinity. If the tracker is occluded by an obstacle such as a fetus, it automatically turns off the visual servoing algorithm and keeps the magnetic field constant. As the magnetic fetoscope is highly flexible, unwanted physical contact with a fetus would inflict minimal force. In these cases, the surgeon has to move the fetus and reselect the desired target. Furthermore, the tracker is limited in patients with almost opaque amniotic fluid. However, in these cases, the amniotic fluid is routinely replaced with a temporary fluid that improves visibility. [8] During the initial experiments, the surgeons preferred a relatively high insertion speed (12 mm s À1 ) when using the robotic advancer unit in the teleoperation mode, whereas lower insertion speeds (3 mm s À1 ) were preferred for high-precision tasks. The black arrows mark the same artery after magnetically guided laser ablation. Right: The proximal and distal ends of the artery are injected with a contrast agent. A radiography reveals that the artery is no longer perfused after the ablation. B) Experiments on the anterior placenta. Left: Area of healthy vessels to be ablated is marked with black arrows. Middle: The same vessels in the area marked with black arrows are completely coagulated after the ablation. Right: The radiography image reveals that the area no longer supports blood flow. The white arrows indicate the beginning of the ablated vessels, i.e., the location where the perfusion of the area has been prevented. C) Microscopic image of sliced and colored arteries before and after magnetically guided ablation. Left: The image of the healthy artery (black arrows) was taken at the location directly proximal to the start of the ablated region. Right: The sliced lasered artery (black arrows) shows a ruptured amnion due to the coagulation of the tissue. The surrounding tissue remained intact, indicating a precise and minimally invasive ablation.
www.advancedsciencenews.com www.advintellsyst.com In challenging procedures, the control algorithm for steering the fetoscope via feature selection on the image allows the surgeon to focus all their attention on the ablation of the correct vessels. This increases the chance of success for the surgery and its accessibility: the training required to perform this operation could be significantly less than for the current procedure, which would facilitate exporting this surgery from highly specialized centers to more remote geographical areas. Furthermore, the surgeon can perform the procedure from a remote location and is not constrained to the operating theater.
We demonstrated that the magnetic fetoscope is effective for ablating placentas situated in an anterior location. A successful photocoagulation of these placentas relies on a complete visualization as well as an incident angle of 90 ∘ between the laser beam and the targeted vessels. Therefore, these cases are often inoperable because of the insufficient dexterity of rigid tools. In contrast, it is possible to move the magnetic fetoscope to any location without the typical kinematic constraints of rigid tools. Furthermore, the incident angle of the fetoscopic laser beam can be adjusted with a combination of advancing the fetoscope and modifying the field orientation. We believe that this semiautonomous approach is an efficient trade-off between intuitive control and the ability of the surgeon to quickly adapt to new circumstances. The effectiveness of this type of co-manipulation approach has been previously confirmed by the navigation of a magnetic endoscope. [43] The results of our visual servoing experiments indicate that we can precisely move the fetoscope to any location within the uterus. To ensure a consistent performance, we continuously update the Jacobian matrix that describes the system behavior. Although previous studies have been dedicated to the Jacobian matrix for steering magnetic continuum devices, [50,53] we found that these methods are difficult to implement in practice as they are highly dependent on parameter tuning. In contrast, our method only involves tuning three parameters of the PID controller.
Previous work in the field of flexible, fully actuated fetoscopes has focused on tendon-driven systems. [12,17] Tendon-driven systems are difficult to miniaturize and require multiple powered actuators to be mounted on or near the fetoscope. For fully actuated fetoscopes, it can, therefore, be challenging to accommodate these robotic devices in the highly dynamic operating environment. However, the actuation for magnetically guided devices is generated externally and transmitted wirelessly. This allows us to manufacture miniaturized, light, and highly flexible tools that can minimize the risk of injury to the fetus and the mother. In addition, they could reduce the abdominal strain and the associated risk of preterm premature rupture of membranes [54] or premature delivery. [6] A previous study introduced a method to intuitively steer a tendon-driven fetoscope in a cooperative manner [12] by repurposing a commercially available fetoscope and testing it in an in silico environment. This system reports a misalignment error of %5°. For a typical laser distance of 10 mm to the surface, this would amount to a lateral precision on the placenta of %900 μm. Therefore, magnetic actuation has considerable advantages over other flexible actuation methods, mainly for enhanced precision, the ability to use easy-to-manufacture miniaturized fetoscopes, and increased procedural safety.

Conclusion
Fetoscopic laser coagulation for TTTS is an established surgical technique for treating malformations that pose a significant threat to unborn twins. However, the associated procedurerelated complication rate is dependent on the location of the placenta and the associated shortcomings of rigid surgical tools. To approach this challenge, we demonstrated the design, manufacture, and control of a miniature (3.1 mm OD) magnetic fetoscope via an eMNS. The semi-automated control method enables the surgeon to safely, precisely, and quickly target vessels by selecting them in the fetoscopic image. A wider range of movement can be executed via teleoperation by means of a haptic input device. We demonstrated that our proposed Jacobian matrix update method ensures a robust performance of the controller in different locations of the placenta within the uterus. The system was successfully validated on two human ex vivo placentas in a medically relevant setting that included a clinical laser, medical trocars, a to-scale uterus model filled with water, and an eMNS that is compatible with the operating theater. To our knowledge, we presented the first robotic system to operate on an anterior placenta in a fully enclosed, realistic model, while providing a full histological analysis to show the effectiveness of the procedure.
Future work will be directed toward animal trials and, potentially, human patients. Specifically, with novel, high-resolution miniature image sensors infiltrating the market in the future, the design of the magnetic fetoscope can be further optimized to increase the visibility for the surgeon. In the future, novel imaging technologies could be embedded in the magnetic fetoscope to enhance its performance. For example, optical ultrasound could potentially provide the distance of the tool from the placental vessels, [55] as well as a method to track the progress of the ablation. [56]

Experimental Section
Experimental Setup: The experimental setup to simulate magnetically navigated TTTS surgery ex vivo is shown in Figure 2A. A to-scale phantom of a pregnant woman and a uterus was created (3D printed, polycarbonate-fused deposition modeling, fits in a 250 mm Â 250 mm Â 250 mm box) and placed in close proximity to the eMNS. The ex vivo human placentas were glued to anterior and posterior locations before filling the closed uterus model with water. Human placentas were obtained from patients with written consent and approval from the Ethical Committee of the District of Zürich (study Stv22/2006). The magnetic fetoscope was inserted into the uterus through a commercially available 10 French trocar (Transcot, Switzerland), typically used in this type of surgery. During the experiments, the operator guided the fetoscope to the desired location either via teleoperation or visual servoing before activating the laser using a pedal. The laser unit (Kerr lens mode-locked Nd:YAG laser MY 60) that was interfaced with our magnetic fetoscope was a clinical system, capable of delivering a 60 W laser beam at 1064 nm. For the purpose of this study, we used a maximal power of 20 W. The contrast agent (Angiofil, MediLumine Inc., Canada) was injected into the placental vessels for validation. [57] Electromagnetic Navigation System: The eMNS is capable of generating magnetic fields up to 20 mT in magnitude in any direction within a 25 cm side cube centered at 13 cm from the surface of the coils. It is capable of generating these magnetic fields at frequencies of up to 5 Hz. During the experiments, we typically operated at distances between 15 and 25 cm from the eMNS.
Navigation: The teleoperation mode was facilitated using the Phantom Omni (3D Systems, USA) haptic device, while the visual servoing control mode was accomplished using a standard computer mouse to click targets on the image acquired by the fetoscope. The teleoperation mode reduced hand tremors by using a digital low-pass filter and the mechanical friction of the Phantom Omni, as well as the dynamic response of the eMNS (it takes time for the eMNS controller to adapt the current in the electromagnetic coils). The insertion of the catheter was controlled using a custom-built robotic advancer unit that was mounted on a Fisso (Baitella, Switzerland) arm. During the operation, the fetoscopic view was displayed on a screen as shown in Figure 2A. This surgery would normally include an ultrasound probe to localize the fetoscope and the moving fetus. However, since we operated in a static environment, it was not required for this study.
Magnetic Fetoscope: The design of the magnetic fetoscope is shown in Figure 2B. The tubular shaft of the fetoscope comprises a lumen that holds camera cables and a polymide tube (ID ¼ 635 μm, OD ¼ 660 μm, Zeus Inc., USA) to guide the laser fiber (OD ¼ 500 μm, KLS Martin, Germany) to the tip. The tip consists of a camera with an integrated light-emitting diode (LED) illumination array with variable intensity (universal serial bus camera, MD-T1001SLH-120-01, 400 Â 400, 30 fps, 1.75 μm pixel size, and focal length ¼ 0.418 mm, Misumi Electronic Corp., Taiwan). To facilitate maneuverability of the tip, the distal part of the fetoscope is flexible and contains multiple neodymium-iron-boron ring magnet segments (sintered, axially magnetized, OD ¼ 3 mm, ID ¼ 1.7 mm, and segment lengths ¼ 2 mm Â 10 mm and 1 mm Â 4 mm, X-Magnets, China), which are separated by custom reinforcement coils made of stainless steel wire (200 μm, zivipf.com, Germany). They were encapsulated with a soft Pebax (Durometer ¼ 35 D, Nordson Medical, USA) layer. To provide stability when the catheter is inserted deep into the uterus, the proximal part was encapsulated with a more rigid Pebax (Durometer ¼ 72 D, Nordson Medical, USA) jacket. The components of the fetoscope were chosen to allow it to bend upward at a 90°angle (see Figure 2C) with a field strength of %18 mT. This enables the magnetic fetoscope to operate on a placenta located on the anterior side of the uterus with sufficient dexterity.
Robotic Advancer Unit: As shown in Figure 2C, the robotic advancer unit consists of a custom, 3D printed casing that enables quick insertion and removal of the endoscope during operation. A Maxon EPOS microcontroller (EPOS4 Compact 24/1.5 EtherCAT, Maxon Motors, Switzerland) allows velocity control of the robotic advancer unit's motor. The insertion speed can be set to levels between 3 and 12 mm s À1 , depending on the surgeon's requirements.
Visual Feedback: The key to the overall performance of the vision-based magnetic fetoscope control algorithm is reliable visual feedback. The intrinsic properties of the fetoscope camera are initially calibrated to correct for distortion. The distorted coordinate Ipf is, therefore, subjected to a nonlinear, rectifying transformation I p f ¼ f Ipf . [58] To initialize the inverse Jacobian matrix according to Equation (15), we applied two separate perpendicular field rotations Δu 1 and Δu 2 , each of a total of 22.5 ∘ over time t ¼ 1.5s. During the motion of the field, we used the modified image to detect translational shifts that occurred between two images with the Fourier shift theorem. [59] To detect the movement Δ I p 1 and Δ I p 2 in the image, we took the mean of the translational shifts that occurred during the movement of the field. In discrete time, Equation (15), therefore, becomeŝ After the Jacobian matrix was initialized, the features I p f were tracked using a discriminative correlation filter with CSRT. [60] Both the feature tracker and the control loop were operated at 20 Hz. The CSRT output coordinates were filtered with a moving average (window size of three samples) without introducing significant delays in the control system.
Controller Parameters: For this study, we tuned two sets of PID parameters for the visual servoing and Jacobian matrix estimation controllers. We found K p;e ¼ 0.025, K i;e ¼ 0.047, and K d;e ¼ 0.0085, as well as K p,γ ¼ 0.2, K i,γ ¼ 0.1, and K d,γ ¼ 0, respectively. Furthermore, the Jacobian matrix update was only active when the misalignment angle γ ≥ 5 ∘ and the tracked feature were more than 20 pixels from the target in the center, i.e., jjp f jj > 20. To calculate γ k in Equation (16), we looked n ¼ 5 frames in the past. The update of the Jacobian matrix is applied over three successive steps in Equation (18) with an exponential decay profile.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.