application of the differential transformation for cross-device co-registration of two preclinical imaging modalities

Effective Co-Registration of Two Preclinical Imaging Modalities Across Devices

Different preclinical imaging modalities have distinct advantages and disadvantages. For instance, X-ray computed tomography (CT) is well-suited for examining anatomical structures with different radio densities, such as bones and lungs. It is widely used due to its rapid acquisition speed, high three-dimensional resolution, relative ease of image assessment, and versatility with or without contrast agents1,2,3. Magnetic resonance imaging (MRI) provides the most versatile soft tissue contrast without ionizing radiation4. On the other hand, tracer-based modalities like positron emission tomography (PET), single-photon emission computed tomography (SPECT), fluorescence-mediated tomography (FMT), and magnetic particle imaging (MPI) are established tools for quantitatively assessing molecular processes, metabolism, and the biodistribution of radiolabeled diagnostic or therapeutic compounds with high sensitivity. However, they lack resolution and anatomical information5,6. Therefore, more anatomy-oriented modalities are typically paired with highly sensitive ones having their strength in tracer detection7. These combinations make the quantification of tracer concentrations within a specific region of interest possible8,9. For combined imaging devices, modality co-registration is usually a built-in feature. However, it is also useful to co-register scans from different devices, e.g., if the devices were purchased separately or if a hybrid device is not available.
This article focuses on cross-modality fusion in small animal imaging, which is essential for basic research and drug development. A previous study10 points out that this can be achieved with feature recognition, contour mapping, or fiducials markers (fiducials). Fiducials are reference points for accurately aligning and correlating images from different imaging modalities. In special cases, fiducials can even be dots of Chinese ink on the skin of nude mice11; however, often, an imaging cartridge with built-in fiducial markers is used. While this is a robust and well-developed method10, using it for every scan presents practical problems. MRI-detectable fiducials are often liquid-based and tend to dry out during storage. PET requires radioactive markers, the signal of which decays according to the emitter&#39;s half-life period, which is usually short for biomedical applications, necessitating preparation shortly before the scan. Other issues, such as the mismatch in the dynamic range of the signal from the fiducial marker and the examined object, strongly impact in vivo imaging. The wide range of dynamic contrast requires frequent adaptation of the marker signal strength to the object being examined. Consequently, while a weak marker signal may not be detected in the analysis, a strong marker signal may create artifacts that impair image quality. Additionally, to consistently include the markers, the field of view must be unnecessarily large for many applications, potentially leading to higher radiation exposure, larger data volumes, longer scan times, and, in some cases, lower resolution. This may affect the health of laboratory animals and the quality of the generated data.
Transformation and differential transformation 
An image dataset consists of voxel data and metadata. Each voxel is associated with an intensity value (Figure 1A). The metadata includes a transformation specifying the dataset placement in the imaging device&#39;s coordinate system (Figure 1B) and the voxel size used to scale the coordinate system. Additional information, such as device type or scan date, can be optionally stored in the metadata. The mentioned transformation is mathematically called a rigid body transformation. Rigid body transformations are used to change the orientation or position of objects in an image or geometric space while preserving the distance between each pair of points, meaning the transformed object retains its size and shape while being rotated and translated in space. Any series of such transformations can be described as a single transformation consisting of a rotation followed by a translation. The formula used by the software to move from the data coordinate to the metric target coordinate is shown in Figure 1C, where R is an orthonormal rotation matrix, d and v are voxel indices and sizes, and t is a 3 x 1 translation vector12. The rotation is detailed in Figure 1D.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/65701/65701fig01.jpg" /> 
Figure 1: 2D representation of an image dataset structure and placement in a global coordinate system. (A) An image dataset consists of voxel data and metadata. The transformation specifying the placement and voxel size are essential metadata components. (B) The image is rendered into the device&#39;s coordinate system. The necessary transformation to place the object consists of a rotation (blue) followed by a translation (green). (C) To move from the data coordinate to the target coordinate, the software uses this formula where R is an orthonormal rotation matrix, d, and v are voxel indices and sizes, and t is a 3 x 1 translation vector. (D) A rotation matrix (blue in plane A) represents the linear transformation of rotating points. Multiplying a point&#39;s coordinates by this matrix results in the new, rotated coordinates. <a href="https://www.jove.com/files/ftp_upload/65701/65701fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
A differential transformation is a rigid body transformation converting coordinates from one coordinate system to another, e.g., from PET to X-ray microtomography (&#181;CT), and it can be computed using fiducial markers. At least three common points - the fiducials - are selected in both coordinate systems. From their coordinates, a mathematical transformation can be derived that converts the coordinates. The software uses the least squares method, which provides a best-fit solution to a system of equations with errors or noise in the measured data. This is called the Procrustes Problem13 and is solved using singular value decomposition. The method is reliable and robust because it leads to a unique and well-defined solution (if at least three non-collinear markers are given). Six free parameters are calculated: three for translation and three for rotation. In the following, we will use the term transformation matrix even though it technically consists of a rotation matrix and translation vector.
Each imaging device has its own coordinate system, and the software calculates a differential transformation to align them. Figure 2A,B describe how the differential transformation is determined, while Figure 2C,D describe how it is applied. The images of both modalities can have different dimensions and keep them in the process, as is shown in the example image with the fusion of CT and PET in Figure 2E.
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/65701/65701fig02.jpg" /> 
Figure 2: Differential transformation. (A-D) Simplified to 2D. While applicable to other modalities, it is assumed that the modalities are CT and PET for this example. (A,C) A CT image with a red bounding box is positioned in the coordinate system. Applied to the same coordinate system, the PET image with a yellow bounding box is positioned deviating.(B) Using fiducial markers that can be located in both CT and PET, a differential transformation T can be determined. This is symbolized by the arrow. The differential transformation matrix is stored. (D) The previously saved differential transformation matrix T can then be applied to each PET image. This results in a new transformation that replaces the original transformation in the metadata. (E) A CT image fused with a PET image. The transformations in the metadata of both images refer to the same coordinate system. <a href="https://www.jove.com/files/ftp_upload/65701/65701fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Method and requirements 
For the presented method, a phantom containing markers visible in both modalities is scanned in both devices. It is then sufficient to mark these fiducials in the proposed software to calculate a differential transformation between the two modalities. The differential transformation must be created individually for each pair of devices. It can be saved and later applied to any new image, thereby eliminating the need for fiducial markers in subsequent scans. The final placement of the image in the coordinate system of another device can again be described as a transformation and be stored in the metadata of the image, replacing the original transformation there.
Four requirements for this method can be formulated: (1) Multimodal phantom: A phantom containing markers visible in both modalities must be available. A large selection of phantoms is commercially obtainable, and the use of 3D printing for phantom construction has been widely described14, even including the incorporation of radioisitopes15. The phantoms utilized in the following examples are listed in the Table of Materials. At least three non-collinear points are required16. The markers might be cavities that can be filled with an appropriate tracer, small objects made of a material that is easily detectable in each modality, or simply holes, cuts, or edges in the phantom itself as long as they can be identified in both modalities. (2) Multimodal carrier: A carrier, such as a mouse bed, is needed that can be fixed in a reproducible position in both devices. Ideally, it should not be possible to use it in a reversed position to avoid errors. The carrier is particularly important for in vivo imaging because it is needed to transport a sedated animal from one imaging device to the other without changing its position. Based on our experience, sedated mice are more likely to change their position in a flat mouse bed compared to a concave-shaped one. Additionally, a custom 3D-printed jig for holding the mouse&#39;s tibia to minimize motion has been previously suggested17. (3) Self-consistency: Each imaging device must provide the rotation and translation of the reconstructed volume in its reference frame in a reproducible and coherent manner. This also means that a coordinate system for the entire device is preserved when only a small region is scanned. It is part of the protocol to test an imaging device for its self-consistency. (4) Software support: The proposed software must be able to interpret the metadata (voxel size, translation, orientation) stored with the reconstructed volume provided by the device. The volume can be in DICOM, NIfTI, Analyze, or GFF file format. For an overview of various file formats, see Yamoah et al.12.
While the co-registration of two modalities is described, the procedure is also applicable to three or more modalities, for example, by co-registering two modalities to one reference modality.

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities

Multimodal Cross-Device and Marker-Free Co-Registration of Preclinical Imaging Modalities

We develop software tools for preclinical imaging, including image reconstruction, fusion, segmentation, and quantification. Our software is meant to be user-friendly, efficient, and compatible with any small animal imaging device to enable reproducible quantitative analysis of the acquired multimodal image data. Automation is an important topic in our field because manual analysis is often too time-consuming, error-prone, and hardly reproducible.Software-based automation can make the analysis efficient and reproducible, but the challenge is in making the automation robust enough. Small animal imaging devices provide volumetric image data of anesthetized laboratory mice and rats, and allow non-invasive assessment of pathological and physiological changes. Integrated multimodal devices combine the strengths of multiple modalities, such as Micro-CT and PET.It is also possible to combine two separate devices. We developed software for the automated fusion of special modalities, such as Micro-CT and FMT. We also provided efficient software for interactive image fusion, and defined a new file format to store the image data, considering aspects from data curation.Last year, we published an approach for retrospective respiratory gating under high throughput settings. Our protocol resides in a transformation matrix, which is generated using a calibration step that involves visible structures such as fiducial markers. Then, the image fusion can be automated using this transformation matrix, which eliminates the need for markers.

Application of the Differential Transformation for Cross-Device Co-Registration of Two Preclinical Imaging Modalities

To begin, launch the analysis software. Import the CT file as an underlay by clicking on file underlay and load underlay. In the ensuing dialogue, select the CT image file to open it.Load the PET file as an overlay by sequentially selecting file overlay and load overlay. In the dialogue that appears, choose the PET image file. To enable the display of both layers, go to menu view followed by layer visibility and check both underlay and overlay.To load and apply the previously saved differential transformation matrix, click on fusion overlay transformation and then load and apply transform. Then select the file containing the matrix from the calibration process to open it.

application-differential-transformation-for-cross-device-co

Research

JoVE Journal

Bioengineering

156 Views. Per iniziare, avvia il software di analisi. Importare il file CT come sottoposto facendo clic su sottoposto file e caricare sottoposto. Nella finestra di dialogo seguente, seleziona il file immagine CT per aprirlo.Caricare il file PET come overlay selezionando in sequenza l'overlay del file e l'overlay del caricamento. Nella finestra di dialogo che appare, scegli il file immagine PET. Per abilitare la visualizzazione di entrambi i livelli, vai alla visualizzazione del menu seguita dall...