|
Guest editors Behrouz Shabestri, Mark Anastasio, Baowei Fei, and Frédéric Leblond provide an overview of the JBO Special Series on Artificial Intelligence Machine Learning in Biomedical Optics. |
|
Artificial Intelligence (AI) methods, including machine learning (ML) and deep learning (DL), are quickly evolving, and impacting a very wide range of scientific endeavors. Biomedical optics is no exception and AI methods are currently transforming our discipline on an almost daily basis. From changing data acquisition1,2 and image reconstruction methods,3 to segmentation and interpretation of optical images,4 AI methods are providing improved solutions to established problems and enabling new problems to be addressed. Structured light can be combined with AI methods to probe and interpret the interaction of light with biological tissues. For example, the coupling of AI methods with hyperspectral and multispectral systems can enable the detection of specific molecular signatures in tissue, cells, and biofluids.5,6 Supervised ML/DL methods are well-suited for this purpose, since they can implicitly learn high-dimensional image statistics and complicated mappings that describe optimal decision strategies for a variety of inferences of relevance to basic science and clinical applications. Enhancing advanced optical methods with AI will enable the clinical translation of new optical sensing and imaging technologies. Label free optical imaging, such as stimulated Raman histology, hyperspectral imaging, and convolutional neural networks (CNNs), has been successfully employed for intraoperative automated brain tumor diagnosis with near real-time detection.7,8 Integrating ML/DL methods with optical methods such as coherent anti-Stokes Raman scattering imaging, optical colonoscopy and fluorescence lifetime imaging has shown to be effective in the differential diagnosis of lung cancer,9 colorectal cancer,10 and cervical neoplasia,11 respectively. Another AI-enabled game-changer will be the use of DL methods for computational staining of label-free optical images, resulting in all-digital histopathology.12–14 In clinical decision making, where accuracy and timing can be critical, spatial frequency domain imaging coupled with ML has been employed for predicting the severity of burn injuries.15 The combination of multi-photon imaging with ML/DL has further enabled improved lymphedema diagnosis,16 skin cancer screening17 and atopic dermatitis.18 ML combined with emerging feature engineering approaches has become the mainstay in tissue, cells, and biofluids interrogation in spectroscopic methods. Examples of such applications range from neurosurgical guidance using spontaneous Raman spectroscopy for cancer detection19 to detection of aggressive variants of prostate cancer in pathology using Raman micro-spectroscopy.20 Merging optical coherence tomography (OCT) imaging with AI provides a unique opportunity to analyze this plethora of information and assist in making clinical decisions in the field of ophthalmology with applications in retinal imaging,21 glaucoma22 and age-related macular degeneration.23 Most recently, AI methods are proving to be invaluable for a variety of tasks related to the detection and management of COVID-19.24–26 Combining AI with optical breathalyzers may yield a rapid and accurate test for COVID-19, which is currently lacking and greatly needed. This JBO special series brings together late breaking research that describe the use of artificial intelligence in biophotonic applications, with an emphasis on ML and DL approaches. The series highlights the important role that ML and DL methods are playing in accelerating the development of innovative biophotonic technologies. This series is timely, for it comes as a growing number of the biomedical optics scientific community are starting to tackle the multiple challenges associated with the responsible adoption of AI methods. Issues such as robustness, reliability, and interpretability remain largely unaddressed but are critical for safe and effective deployment of AI-enabled biophotonic imaging and sensing systems. We hope you enjoy this special series, which includes the following twelve articles:
ReferencesV. Sitzmann et al.,
“End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging,”
ACM Trans. Graphics, 37
(4), 1
–13
(2018). https://doi.org/10.1145/3197517.3201333 ATGRDF 0730-0301 Google Scholar
A. Muthumbi et al.,
“Learned sensing: jointly optimized microscope hardware for accurate image classification,”
Biomed. Opt. Express, 10
(12), 6351
–6369
(2019). https://doi.org/10.1364/BOE.10.006351 BOEICL 2156-7085 Google Scholar
C. Belthangady and L. A. Royer,
“Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction,”
Nat. Methods, 16
(12), 1215
–1225
(2019). https://doi.org/10.1038/s41592-019-0458-z 1548-7091 Google Scholar
C. S. Lee et al.,
“Deep-learning based, automated segmentation of macular edema in optical coherence tomography,”
Biomed. Opt. Express, 8
(7), 3440
–3448
(2017). https://doi.org/10.1364/BOE.8.003440 BOEICL 2156-7085 Google Scholar
S. Ortega et al.,
“Hyperspectral imaging for the detection of glioblastoma tumor cells in H&E slides using convolutional neural networks,”
Sensors, 20
(7), 1911
(2020). https://doi.org/10.3390/s20071911 SNSRES 0746-9462 Google Scholar
M. Halicek et al.,
“Hyperspectral imaging of head and neck squamous cell carcinoma for cancer margin detection in surgical specimens from 102 patients using deep learning,”
Cancers, 11
(9), 1367
(2019). https://doi.org/10.3390/cancers11091367 Google Scholar
T. C. Hollon et al.,
“Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks,”
Nat. Med., 26
(1), 52
–58
(2020). https://doi.org/10.1038/s41591-019-0715-9 1078-8956 Google Scholar
H. Fabelo et al.,
“Deep learning-based framework for in vivo identification of glioblastoma tumor using hyperspectral images of human brain,”
Sensors, 19
(4), 920
(2019). https://doi.org/10.3390/s19040920 SNSRES 0746-9462 Google Scholar
W. Sheng et al.,
“Combining deep learning and coherent anti-Stokes Raman scattering imaging for automated differential diagnosis of lung cancer,”
J. Biomed. Opt., 22 106017
(2017). https://doi.org/10.1117/1.JBO.22.10.106017 JBOPFO 1083-3668 Google Scholar
D. Zhou et al.,
“Diagnostic evaluation of a deep learning model for optical diagnosis of colorectal cancer,”
Nat. Commun., 11 2961
(2020). https://doi.org/10.1038/s41467-020-16777-6 NCAOBW 2041-1723 Google Scholar
J. Gu et al.,
“Quantitative diagnosis of cervical neoplasia using fluorescence lifetime imaging on haematoxylin and eosin stained tissue sections,”
J Biophotonics, 7
(7), 483
–491
(2014). https://doi.org/10.1002/jbio.201200202 Google Scholar
M. Schnell et al.,
“All-digital histopathology by infrared-optical hybrid microscopy,”
Proc. Natl. Acad. Sci. USA, 117
(7), 3388
–3396
(2020). https://doi.org/10.1073/pnas.1912400117 Google Scholar
M. E. Kandel et al.,
“Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments,”
Nat. Commun., 11 6256
(2020). https://doi.org/10.1038/s41467-020-20062-x NCAOBW 2041-1723 Google Scholar
M. Halicek et al.,
“Head and neck cancer detection in digitized whole-slide histology using convolutional neural networks,”
Sci. Rep., 9 14043
(2019). https://doi.org/10.1038/s41598-019-50313-x SRCEC3 2045-2322 Google Scholar
R. Rowland et al.,
“Burn wound classification model using spatial frequency-domain imaging and machine learning,”
J. Biomed. Opt., 24 056007
(2019). https://doi.org/10.1117/1.JBO.24.5.056007 JBOPFO 1083-3668 Google Scholar
Y. V. Kistenev et al.,
“Application of multiphoton imaging and machine learning to lymphedema tissue analysis,”
Biomed. Opt. Express, 10
(7), 3353
–3368
(2019). https://doi.org/10.1364/BOE.10.003353 BOEICL 2156-7085 Google Scholar
M. J. Huttunen et al.,
“Multiphoton microscopy of the dermoepidermal junction and automated identification of dysplastic tissues with deep learning,”
Biomed. Opt. Express, 11
(1), 186
–199
(2020). https://doi.org/10.1364/BOE.11.000186 BOEICL 2156-7085 Google Scholar
P. Guimarães et al.,
“Artificial intelligence in multiphoton tomography: atopic dermatitis diagnosis,”
Sci. Rep., 10
(1), 7968
(2020). https://doi.org/10.1038/s41598-020-64937-x SRCEC3 2045-2322 Google Scholar
É. Lemoine et al.,
“Feature engineering applied to intraoperative in vivo Raman spectroscopy sheds light on molecular processes in brain cancer: a retrospective study of 65 patients,”
Analyst, 144
(22), 6517
–6532
(2019). https://doi.org/10.1039/C9AN01144G ANLYAG 0365-4885 Google Scholar
A.-A. Grosset et al.,
“Identification of intraductal carcinoma of the prostate on tissue specimens using Raman micro-spectroscopy: a diagnostic accuracy case–control study with multicohort validation,”
PLoS Med., 17
(8), e1003281
(2020). https://doi.org/10.1371/journal.pmed.1003281 1549-1676 Google Scholar
C. S. Lee et al.,
“Generating retinal flow maps from structural optical coherence tomography with artificial intelligence,”
Sci. Rep., 9
(1), 5694
(2019). https://doi.org/10.1038/s41598-019-42042-y SRCEC3 2045-2322 Google Scholar
H. Fu et al.,
“A deep learning system for automated angle-closure detection in anterior segment optical coherence tomography images,”
Am. J. Ophthalmol., 203 37
–45
(2019). https://doi.org/10.1016/j.ajo.2019.02.028 AJOPAA 0002-9394 Google Scholar
T. Schlegl et al.,
“Fully automated detection and quantification of macular fluid in OCT using deep learning,”
Ophthalmology, 125
(4), 549
–558
(2018). https://doi.org/10.1016/j.ophtha.2017.10.031 Google Scholar
R. Vaishya et al.,
“Artificial intelligence (AI) applications for COVID-19 pandemic,”
Diab. Metab. Syndrome Clin. Res. Rev., 14
(4), 337
–339
(2020). https://doi.org/10.1016/j.dsx.2020.04.012 Google Scholar
F. Shi et al.,
“Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19,”
IEEE Rev. Biomed. Eng., 14 4
–15
(2021). https://doi.org/10.1109/RBME.2020.2987975 Google Scholar
C. Carlomagno et al.,
“COVID-19 salivary Raman fingerprint: innovative approach for the detection of current and past SARS-CoV-2 infections,”
Sci. Rep., 11 4943
(2021). https://doi.org/10.1038/s41598-021-84565-3 SRCEC3 2045-2322 Google Scholar
BiographyBehrouz Shabestari is the acting director of the Division of Health Informatics Technologies and director of the NIBIB National Technology Centers Program. He directs the NIBIB programs in optical imaging and spectroscopy, and for x-ray, electron, ion beam, and computed tomography (CT). He joined the NIBIB in 2015, after 12 years as a scientific review officer at the NIH Center for Scientific Review’s Surgical Sciences, Biomedical Imaging, and Bioengineering Integrated Review Group. There, he was responsible for the review of applications in the development of methods for a wide variety of medical imaging modalities and bioengineering, including SPECT, PET, MRI/MRS, ultrasound, CT, photonics, image-guided surgery, CAD, image recognition algorithms, as well as hybrid approaches. He has extensive experience in the area of industrial and medical imaging. Mark A. Anastasio is the Donald Biggar Willett Professor in Engineering at University of Illinois Urbana–Champaign (UIUC), where he also heads the Department of Bioengineering. His lab within the Grainger College of Engineering at UIUC, the Computational Imaging Science Laboratory, performs research in computational and theoretical image science and pursues the advancement of emerging imaging methods. He is an elected fellow of the International Society for Optics and Photonics (SPIE) and of the American Institute for Medical and Biological Engineering (AIMBE). Baowei Fei is a professor of bioengineering, the Cecil H. and Ida Green Chair in Systems Biology Science, and Dean’s Fellow at the Erik Jonsson School of Engineering and Computer Science at the University of Texas at Dallas. He is also a professor of radiology at UT Southwest Medical Center. He is the director of the Quantitative BioImaging Laboratory ( https://fei-lab.org/). He is the director of the Center for Imaging and Surgical Innovation at UT Dallas and UT Southwestern Medical Center. He is a national leader in quantitative imaging and image-guided interventions. He served as conference chair for the International Conference of SPIE Medical Imaging—Image-Guided Procedures, Robotics Interventions, and Modeling from 2017 to 2020. He is a fellow of SPIE and the AIMBE. Frédéric Leblond is a professor in the Department of Engineering Physics at Polytechnique Montréal, where he heads the Optical Radiology Laboratory. He works mainly in biomedical optics (including diffuse optics and spectroscopy), designing new surgical and pathology methods, enhancing medical imaging, and studying light propagation in biological tissues. He is the co-founder and was—until 2020—technical director of ODS Medical Inc., which is tasked with commercialization of his Raman-spectroscopy-based cancer-cell detection device. He is currently working with a number of industrial partners on development of several medical imaging techniques, fiber optical systems, and software. He also holds several patents. As his work also involves human subjects and is greatly useful to medical personnel, he has collaborative projects with many hospitals across North America. |
