Application of Computer Vision for Localization of Vertebrae on Midsagittal Computed Tomography Slices

Automation of routine operations related to medical image analysis is an important task, as it reduces the workload of radiologists. The selection of computed tomography images corresponding to the levels of specific vertebrae for assessing the patient's body composition is usually done manually, which requires additional time. The purpose is to develop an approach to solving the problem of vertebrae localization on midsagittal computed tomography slices for automatic selection of axial slices used to assess body composition. We developed an approach based on the use of a multiclass segmentation model with the U-Net family architecture and computer vision methods for images preprocessing and segmentation masks postprocessing. In order to assess the impact of input data types and model architectures on segmentation accuracy, we considered 20 approach configurations. We found that the proposed method of preprocessing input data, based on the formation of three-channel images, increases the accuracy of multiclass segmentation for four architectures out of five considered (Dense U-Net demonstrates the maximum Dice similarity coefficient of 0.8858). We also found that the proposed training set augmentation method based on skipping axial slices when forming sagittal slices improves the multiclass segmentation accuracy for models with the ResU-Net and Dense U-Net architectures. Based on the proposed approach, we implemented a software module that solves the problems of automatic determination of the positions of the cervical, thoracic and lumbar vertebrae on the midsagittal computed tomography slice, their visualization and determination of the axial slice indices corresponding to the vertebral body centers. We integrated the developed module with the program for visualization and analysis of DICOM medical files. The developed module can be used as an auxiliary tool in solving diagnostic problems.

1. Van den Broeck J., Sealy M.J., Brussaard C., Kooijman J., Jager-Wittenaar H., Scafoglieri A. The correlation of muscle quantity and quality between all vertebra levels and level L3, measured with CT: An exploratory study. Frontiers in Nutrition, 2023, vol. 10, pp. 1148809. DOI: 10.3389/fnut.2023.1148809.

2. Arayne A.A., Gartrell R., Qiao J., Baird P.N., Yeung J.M. Comparison of CT derived body composition at the thoracic T4 and T12 with lumbar L3 vertebral levels and their utility in patients with rectal cancer. BMC Cancer, 2023, vol. 23, no. 1, pp. 56. DOI: 10.1186/s12885-023-10522-0.

3. Recio-Boiles A., Galeas J., Goldwasser B., Sánchez K., Man L., Gentzler R., Gildersleeve J., Hollen P., Gralla R. Enhancing evaluation of sarcopenia in patients with non-small cell lung cancer (NSCLC) by assessing skeletal muscle index (SMI) at the first lumbar (L1) level on routine chest computed tomography (CT). Supportive Care in Cancer, 2018, vol. 26, no. 7, pp. 2353-2359. DOI: 10.1007/s00520-018-4051-2.

4. Hong J.H., Hong H., Choi Y.R., Kim D.H., Kim J.Y., Yoon J.H., Yoon S.H. CT analysis of thoracolumbar body composition for estimating whole-body composition. Insights Imaging, 2023, vol. 14, no. 1, pp. 69. DOI: 10.1186/s13244-023-01402-z.

5. Теплякова А.Р., Шершнев Р.В., Старков С.О. Метод сегментации мышечной ткани на снимках компьютерной томографии на базе предобработанных трехканальных изображений. Научно-технический вестник информационных технологий, механики и оптики, том 24, вып. 4, 2024 г., стр. 661-664. DOI: 10.17586/2226-1494-2024-24-4-661-664. / Teplyakova A.R., Shershnev R.V., Starkov S.O. Method of muscle tissue segmentation in computed tomography images based on preprocessed three-channel images. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2024, vol. 24, no. 4, pp. 661-664 (In Russian). DOI: 10.17586/2226-1494-2024-24-4-661-664.

6. Qadri S.F., Lin H., Shen L., Ahmad M., Qadri S., Khan S., Khan M., Zareen S.S., Akbar M.A., Heyat M.B.B., Qamar S. CT-Based Automatic Spine Segmentation Using Patch-Based Deep Learning. International Journal of Intelligent Systems, 2023, vol. 2023, pp. 2345835. DOI: 10.1155/2023/2345835.

7. Khandelwal P., Collins L., Siddiqi K. Spine and Individual Vertebrae Segmentation in Computed Tomography Images Using Geometric Flows and Shape Priors. Frontiers in Computer Science, 2021, vol. 3. DOI: 10.3389/fcomp.2021.592296.

8. Cheng P., Yang Y., Yu H., He Y. Automatic vertebrae localization and segmentation in CT with a two-stage Dense-U-Net. Scientific Reports, 2021, vol. 11, no. 1. pp. 22156. DOI: 10.1038/s41598-021-01296- 18.

9. Saeed M.U., Dikaios N., Dastgir A., Ali G., Hamid M., Hajjej F. An Automated Deep Learning Approach for Spine Segmentation and Vertebrae Recognition Using Computed Tomography Images. Diagnostics, 2023, vol. 13, no. 1, pp. 2658. DOI: 10.3390/diagnostics13162658.

10. Vania M., Mureja D., Lee D. Automatic spine segmentation from CT images using Convolutional Neural Network via redundant generation of class labels. Journal of Computational Design and Engineering, 2019, vol. 6, no. 2, pp. 224-232. DOI: 10.1016/j.jcde.2018.05.002.

11. Qadri S.F., Ai D., Hu G., Ahmad M., Huang Y., Wang Y., Yang J. Automatic Deep Feature Learning via Patch-Based Deep Belief Network for Vertebrae Segmentation in CT Images. Applied Sciences, 2019, vol. 9, no. 1, pp. 69. DOI: 10.3390/app9010069.

12. Lu H., Li M., Yu K., Zhang Y., Yu L. Lumbar spine segmentation method based on deep learning. Journal of Applied Clinical Medical Physics, 2023, vol. 24, no. 1, pp. e13996. DOI: 10.1002/acm2.13996.

13. Li B., Liu C., Wu S., Li G. Verte-Box: A Novel Convolutional Neural Network for Fully Automatic Segmentation of Vertebrae in CT Image. Tomography, 2022, vol. 8, no. 1, pp. 45-58. DOI: 10.3390/tomography8010005.

14. Свидетельство о государственной регистрации программы для ЭВМ. Программа для визуализации и анализа медицинских файлов DICOM, Шершнев Р. В. (RU), Теплякова А. Р. (RU). № RU2024617422; заявл. 23.03.2024; опубл. 02.04.2024, Бюл. №4. / Shershnev R.V., Teplyakova A.R. Program for visualization and analysis of DICOM medical files. Certificate of state registration of a computer program RU2024617422, registered 23.03.2024, published 02.04.2024 (in Russian).

15. Kaur R., Juneja M., Mandal A. K. A comprehensive review of denoising techniques for abdominal CT images. Multimedia Tools and Applications, 2018, vol. 77, no. 17, pp. 22735-22770. DOI: 10.1007/s11042-017-5500-5.

16. Tomczak K., Czerwińska P., Wiznerowicz M. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge. Contemporary Oncology, 2015, vol. 19, no. 1A, pp. A68-A77. DOI: 10.5114/wo.2014.47136.

17. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015, vol. 9351, pp. 234-241. DOI: 10.1007/978-3-319-24574-4_28.

18. Oktay O., Schlemper J., Folgoc L.L., Lee M.J., Heinrich M.P., Misawa K., Mori K., McDonagh S.G., Hammerla N.Y., Kainz B., Glocker B., Rueckert D. Attention U-Net: Learning Where to Look for the Pancreas. arXiv preprint, 2018. DOI: 10.48550/arXiv.1804.03999.

19. Diakogiannis F.I., Waldner F., Caccetta P., Wu C. ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, vol. 162, pp. 94-114. DOI: 10.1016/j.isprsjprs.2020.01.013.

20. Punn N. S., Agarwal S. Inception U-Net Architecture for Semantic Segmentation to Identify Nuclei in Microscopy Cell Images. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2020, vol. 16, no. 1, pp. 12. DOI: 10.1145/3376922.

21. Cai S., Tian Y., Lui H., Zeng H., Wu Y., Chen G. Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network. Quantitative Imaging in Medicine and Surgery, 2020, vol. 10, no. 6, pp. 1275-1285. DOI: 10.21037/qims-19-1090.

22. Löffler M.T., Sekuboyina A., Jacob A., Grau A.-L., Scharr A., Husseini M.E., Kallweit M., Zimmer C., Baum T., Kirschke J.S. A Vertebral Segmentation Dataset with Fracture Grading. Radiology: Artificial Intelligence, 2020, vol. 2, no. 4. pp. e190138. DOI: 10.1148/ryai.2020190138.

23. Liebl H., Schinz D., Sekuboyina A., Malagutti L., Löffler M.T., Bayat A., El Husseini M., Tetteh G., Grau K., Niederreiter E., Baum T., Wiestler B., Menze B., Braren R., Zimmer C., Kirschke J.S. A computed tomography vertebral segmentation dataset with anatomical variations and multi-vendor scanner data. Scientific Data, 2021, vol. 8, no. 1, pp. 284. DOI: 10.1038/s41597-021-01060-0.

24. VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images / A. Sekuboyina, M.E. Husseini, A. Bayat et al. // Med Image Anal, 2021, vol. 73, pp. 102166. DOI: 10.1016/j.media.2021.102166.

25. Dataset with segmentations of 117 important anatomical structures in 1228 CT images (Version 2.0.1) [Data set] / J. Wasserthal // Zenodo. 2023. DOI: 10.5281/zenodo.10047292.

26. Свидетельство о государственной регистрации программы для ЭВМ. Программа для обучения моделей сегментации мышечной ткани по снимкам компьютерной томографии, Теплякова А. Р. (RU), Шершнев Р. В. (RU). № RU2024612322; заявл. 18.01.2024; опубл. 31.01.2024, Бюл. №2. / Teplyakova A.R., Shershnev R.V. Program for training muscle tissue segmentation models from computed tomography images. Certificate of state registration of a computer program RU2024612322, registered 18.01.2024, published 31.01.2024 (in Russian).