Using neural network for restoring the lost surface of skull bones

Background

Cranioplasty is a fairly common neurosurgical operation, which, as a rule, does not require high-tech equipment in the operating room. However, despite their superficial simplicity, such interventions may employ advanced medical technology. Each defect of the skull bones is unique, which explicates the introduction of custom implants into the practice of reconstructive neurosurgery starting from the mid-1990s. A specialized software environment for three-dimensional (3D) modeling is required for designing such products. These activities are performed by medical engineers with competence in the fields of technology, medicine, and anatomy [1], or by physicians with skills and experience in such programs [2]. Despite the more than thirty-year history of custom modeling in neurosurgery, specialists employ utilitarian software products that are widely used in technical industries: MeshMixer, Blender, Autodesk 3ds Max, InVesalius, Geomagic Design X, Materialize Mimics and 3 Matic, etc. Existing solutions for constructing models of implants are focused primarily on the skills of the engineering technologist rather than the physician. Hence, on the one hand, medical professionals have to learn the basics of working in these programs. On the other hand, the strategy for scientific and technological development of the Russian Federation approved by the Decree No. 642 by the Russian Federation President of 1 December 2016 commands the need for transition to personalized medicine, high-tech health care system and technology, advanced digital intelligent manufacturing technologies, and the development of big data processing systems, machine learning and artificial intelligence. The first part of this strategy has already become firmly established in practical medicine, and the geography of the use of custom implants for repairing defects in the skull bones gradually expands [2-4]. At the same time, its second aspect related to the medical industry is in most cases at the stage of conceptual development. In the course of research activities carried out from 2018 at the Ya.L. Tsivyan Research Institute of Traumatology and Orthopedics of Novosibirsk, A.P. Ershov Institute of Information Systems of the Siberian Branch of the Russian Academy of Sciences (Novosibirsk), and AcademGene LLC (Novosibirsk), we have developed an approach to use neural networks, which made it possible to automatically create the lost surface of the skull bones based on the patient’s multislice computed tomography (MSCT) data. The developed algorithm can be used for designing custom implants in clinical practice, which would allow integrating the listed strategies into an ecosystem of personalized medical devices.

The goal of our study was to assess the sensitivity, specificity and accuracy of the developed digital algorithm for restoring the lost surface of the skull bones.

Materials and Methods

Designing neural network. As a basis for the algorithm aimed at constructing the lost surface of the skull, we decided to use a neural network using convolutional layers on an icosahedral spherical grid (ISG), adapting and improving it for the task at hand. The determination of the skull missing area was performed at the stage of converting the polygonal grid into a distance vector on the ISG. As a result of this procedure, the algorithm was determining a set of characteristics of the ISG vertices with which the skull defect was associated. With this goal in mind, we calculated the distances from the vertices of the ISG to the 3D model of the skull along the directrices from these vertices to the center. In this case, the distances corresponding to the directrices falling into the defect area exceeded one (since they traveled a distance greater than one from the ISG vertex to the center and intersected the surface of the skull behind it) and could be associated with the defect area. Small breaks (in terms of their area or number of points) that were present on the surface due to scanning errors or were natural breaks were not considered a defect. The size of the area threshold for cutting off such small surface breaks was identified empirically.

Training a neural network. A sample of 70 STL files (stereolithography) containing polygonal models of skulls, which were previously converted from MSCT DICOM data of patients without skull bone defects, was divided into three parts: 50 copies for training the neural network, and 10 copies each for the test sample and validation sample. The model was trained to repair surfaces on skulls with artificially created damage. Using a sphere of random radius, two spatial models of the sphere and the skull were superimposed in a random area with a random degree of intersection, thereby creating artificial damage. Such approach facilitated the training performed on a relatively small sample of data – i.e., the same STL model with different localization and diameter of the damage was used for training purposes. In total, in the course of neural network training, 6,000 epochs were completed (each with a sample size of 13 models). Consequently, 50 models used for training at the first stage yielded 78,000 variants of skulls with artificial damage for training the neural network.

The interface of the developed software is presented in the form of a web application (https://www.autobone.nprog.ru/), which allows an authorized user (surgeon and/or medical modeling engineer) using the function of constructing and analyzing a model of the reconstructed skull via the Internet. The application is based on the previously described digital algorithm.

Evaluation of lost surface repair using a reference skull. This group comprised 13 pairs of DICOM series of patient MSCT images. Each patient was represented by two series of examinations: the first one was performed prior to craniectomy, whereas the second was obtained after it. DICOM data were converted into STL models. Then the developed algorithm was used for the series with defects to model skull repair. The model was then placed into a 3D modeling program (Materialize Mimics) to reconstruct the lost surface of the skull. A 3D model of the same patient’s skull, but without a defect (i.e., a reference skull), was placed in this program as well. Then, for the purpose of control, a visual assessment of the model’s fit to the reference model was performed by highlighting the models in different colors and searching for deviations of the model from the reference model. To determine the adequacy of the resulting implant curvature, we measured the distance from the lower surface of the reconstructed skull to the outer surface of the entire skull. The actual area of the skull defect was determined in a similar way: by placing two volumetric STL models of the skull in the same block of a 3D modeling program, fitting them with each other, checking the quality of fit, and then subtracting the surface of the intact skull from the projection of the defect area on the model of the skull with the defect.

Assessing the accuracy of the digital algorithm. To assess the quality of the neural network in terms of automatic conversion of MSCT DICOM data into polygonal 3D models, automated removal of artifacts in MSCT data, automated detection of the defect area in the skull bones, measurement of defect area size, and construction of the lost surface of the skull, we processed 222 series of MSCT DICOM patients with defects of the skull bones using a specialized web service (https://www.autobone.nprog.ru/).

Statistical analysis and criteria for evaluating effectiveness. For descriptive statistics, we used the mean and standard deviation of the mean (M±σ), the median and its lower and upper quartiles (Me [Q1; Q3]), and minimum and maximum values of variables analyzed via Statistica 10 software package. Correct detection of the defect area was interpreted as sensitivity. The accurate choice of reference points for constructing an implant was categorized as specificity. The precise visual curvilinearity of the constructed surface was regarded as accuracy. The curvilinearity of the implant was assessed as good, satisfactory, or unacceptable. In the first case, there was a proper visual curvature of the implant (according to the neurosurgeon’s opinion, without making any precise measurements), and such curvature did not require modification in 3D modeling editors (Figure A). In the second case, there was a largely adequate curvilinearity (Figure B) requiring minor additional processing in some areas of the implant using a 3D modeling environment. In the third case, we observed curvilinearity (Figure C) that did not fit well to the anatomical region; hence the implant required further significant modification.

Figure. Examples of repairing the lost surface of the skull:  A – good result; B – satisfactory result; C – unacceptable result. 1 – in the lower part of the restored surface; sections of the zygomatic arch were selected as reference points; 2 – in the lower part, sections of the zygomatic arch and a section of the petrous pyramid were selected as reference points, which led to a noticeable deformation of the surface

Results

Lost surface repair using a reference skull. In one case out of thirteen, a significant deviation in the implant curvature was noted (19.77 mm) with a defect area of 43.3 cm2, which was due to incorrectly identified reference points for constructing the surface. In this case, the highest ratio of the curvature maximum deviation of the reconstructed surface to the defect area was noted at 0.5%. Due to considerable error, this case was excluded to ensure correct statistical processing, and the analysis was performed on 12 pairs of 3D models of the skull (Table 1).

Table 1. Results of measurements during the experiment with a reference skull

As can be seen from the data in Table 1, the maximum error of determining the defect area corresponded was 2.3 cm2, while the mean value of the error and its median did not exceed 1 cm2. In two cases, the algorithm determined the area to be larger (0.6 cm2 and 2.0 cm2). In other cases, the area of the skull bone defect determined by the algorithm was on average 1% less than the reference value. The maximum surface curvature deviation (3.89 mm) corresponded to a defect area of 95 cm2.

Accuracy of the digital algorithm. During the data loading (a total of 222 DICOM series of MSCT examinations), we observed a single case without conversion to an STL file (and, accordingly, without reconstruction of the lost surface). This was due to the small number of sections (less than 100) of the skull bones in a series of MSCT, which did not allow building a 3D model. In seven cases, the surface was not restored, which was caused either by the small size of the defects or their location in the area of the anterior wall of the frontal sinus with intact posterior wall of the sinus (in these cases, the basic mathematical function of searching for a defect by projecting vectors with ISG was not performed).

In 214 cases, loading DICOM data, converting it into an STL model, and constructing the lost surface of the skull were completed successfully. Methodologically correct measurement of the defect area was implemented in 206 cases; in four MSCT series, the patients had bilateral defects, hence the algorithm determined the total area of each defect rather than separately for each location.

In 204 out of 214 cases, the defect area was determined correctly (sensitivity = 95.3%). The reference points for constructing the lost surface of the skull were accurately determined by the algorithm in 183 of 214 cases (specificity = 85.5%). The results of the lost surface repair were good in 170 cases, satisfactory in 25 cases and unacceptable in 19 cases. Consequently, the overall accuracy of the algorithm was 91.1%; based solely on good results, it was 79.4%. Taking into account that in all cases of good construction of the lost surface, the algorithm correctly identified the reference points for its construction. Hence, we concluded that the accuracy of their determination was the key to adequate reconstruction of the skull surface.

In our study, 20 observations contained artifacts that affected the volumetric STL model. Consequently, these series were excluded, and 194 studies out of 214 were used to evaluate sensitivity, specificity and accuracy. In 188 of 194 cases, the determination of the defect area was performed correctly, and the sensitivity was 96.9%. The reference points for constructing the surface were properly determined by the algorithm in 170 of 194 cases, hence the specificity constituted 87.6%. The results of surface reconstruction in 162 cases were good; in 20 cases, they were satisfactory; and in 12 cases, they were unacceptable. Thus, the overall accuracy of the algorithm with the inclusion of the satisfactory grade of accuracy was 93.8%, while based on solely good results, it was 83.5%. Eliminating models with artifacts improved all indicators (Table 2). To test the hypothesis that the accurate determination of reference points allowed the algorithm modeling a surface with the required curvilinearity, we performed an analysis of the quality of modeled surfaces only in 170 cases of correct determination of reference points. In 160 cases, curvilinearity was assessed as good, and in 9 cases, as satisfactory. There was a single case of incorrect detection of the defect area (the algorithm, in addition to the artificial defect of the skull bones, identified the area between the occipital bone and C1 vertebra as a defect), which did not affect the curvilinearity of the generated surface in the area of the true defect of the skull bones. Therefore, we concluded that specificity has a greater impact on the ultimate accuracy of the algorithm.

Table 2. Summary performance indicators of the algorithm for restoring the lost surface of the skull bones

After excluding the series with artifacts, the mean improvement in all metrics of the algorithm was 2.6%. The best result was achieved when that the reference points were correctly identified and the restored surface had good curvilinearity.

When analyzing the causes of incorrect detection of a defect in the skull bones (n=10 of the total group with n=214), we revealed that every so often this occurred in cases of defects extending to the area of the facial skeleton and encompassing the frontal sinus and/or orbital area (n=5), as well as in the presence of newly formed bone on the dura mater surface (n=3).

To measure the impact of the defect area location on the correct determination of reference points and proper construction of the surface, we analyzed 3D models where the defect was determined correctly and there were no artifacts interfering with the construction of the surface (n=188). In the case of the defect located outside the temporal region (n=61), the implant was constructed well in 59 cases; it was unacceptable in one case and satisfactory in one case as well. In the case of the defect spreading to the temporal region (n=127), the lost surface had good curvilinearity in 101 cases; in 18 cases, it was satisfactory; and in 8 cases, it was unacceptable. The accuracy of the algorithm based solely on good results was 79.5%; with the inclusion of satisfactory results, it constituted 93.7%. We established that as the defect spread to the area of the skull base, the relative proportion of incorrectly identified reference points required for reconstructing the surface increased, which led to a decrease in good results of the lost surface repair.

Analysis of the obtained data helped establishing that the low location of the defect in the temporal region, rather than the presence of artifacts in 3D models of the skull alone, increased satisfactory and unacceptable modeling results, because in the former case, the algorithm could not correctly determine the reference points and, accordingly, generate a good geometry of the lost surface (Figure C).

Discussion

The use of neural networks and machine learning approaches in various fields has increased significantly over the last decade. A number of studies in various fields of medicine grows exponentially every year. Employing neural networks in reconstructive neurosurgery is of interest from the standpoint of creating an implant for repairing a defect in the skull bones. Several studies demonstrated various technical aspects of network architecture and training [5-10]. A publication by J. Li et al. [11] presented the results of how a trained neural network functioned on the skulls of healthy people with synthetically generated lesions, implying that the developed approach has prospects for use in medical practice. C.T. Wu et al. [12] demonstrated a clinical example of the repairing a simple convexity defect in the frontal region with an implant automatically modeled by a neural network. The implant was produced from polymethyl methacrylate by casting in a silicone mold. Despite the large number of published sources on this topic, we found no data in the available literature on direct assessment of the obtained outcomes by medical specialists performing such surgical interventions.

In the current study, after training the algorithm on models of skulls with synthetic lesions, a number of validation experiments were carried out. At the first stage, the performance of the algorithm was assessed with the presence of the initial model (reference model) of the skull before the patient underwent craniectomy. This group included MSCT examinations performed prior to surgical interventions in patients with traumatic intracranial hematomas and meningeal tumors growing into the skull bones. Thus, for each patient, there was a pair of skull models: without a defect before surgery and with a defect after surgery. The results obtained during this stage demonstrated defect replacement that was extremely close to the reference model, which made it possible to move on to conducting research on a larger sample.

An evaluation of 222 MSCT examinations showed that in 3.6% of cases (n=8), the algorithm failed to correctly restore the lost surface of the skull, which was due to the localization features and small size of the defects. Another 9% (n=20) of the studies contained artifacts that did not allow the edges of the bone defect to be correctly detected, which affected the curvilinearity of the restored surface. The presence of artifacts in 3D models of the skull after their transfer from DICOM format to STL was due to their density resembling the density of the bone tissue (+400 HU). The maximum accuracy of the algorithm (96.7%) was observed when defects were located outside the temporal region, while the surface area of the defect did not affect the quality of the resulting surface (Figure A). However, in actual clinical practice, the vast majority of skull bone defects are the consequences of surgical interventions for traumatic brain injury, when trephination habitually involves the temporal region. It has been proven that the key to effectiveness is maximum resection of the temporal bone towards the base of the skull. Therefore, an important aspect of further improvement of the algorithm is training the neural network to correctly generate the lost surface in cases of defects located in lower areas.

A limitation of our study was the lack of exclusion criteria for MSCT studies used to analyze the performance of the algorithm. At the same time, the lack of initial data refining allowed identifying the causes of unsatisfactory results of reconstructing the skull surface. The obtained results made it possible to reveal additional requirements for the used DICOM data: for precise construction of the surface, they must not contain artifacts (drainages, metal bodies) near the defect area. Eliminating the effect of a newly formed bone in the defect area on the performance of the algorithm was possible by connecting an operator between the stages of constructing 3D models of the skull and the restored surface.

To implement the priority areas of high-tech health care and the widespread introduction of custom implants, new approaches are required to reduce the time spent on their modeling. Our study aimed at assessing the effectiveness of the developed algorithm based on the work of convolutional neural networks demonstrated the high potential of the created approaches for their use in a clinical practice. The algorithm can be used in medical production for the initial modeling of the lost surface of the skull, as well as in medical 3D printing laboratories at universities and national medical research centers for developing skills of modeling custom implants. Our findings will also help resolving the issue of developing specialized medical software for 3D modeling of custom implants used for repairing defects in the skull bones [2].

Conclusion

1. The accuracy of identifying the reference points (specificity) on a 3D model of the skull by the algorithm had the greatest impact on the ultimate accuracy of repairing the lost surface.

2. The maximum accuracy of the algorithm allowing the use of the resulting surfaces without additional processing in a 3D modeling environment was achieved in series without the presence of artifacts in MSCT (83.5%), as well as with defects that did not extend into the area of the base of the skull (79.5%). 

3. Further development of the proposed approach, including an increase in training samples with reference models and addition of complex defects extending to the base of the skull, will improve the accuracy of the algorithm.