Comparing percent breast density assessments of an AI-based method with expert reader estimates: inter-observer variability

Stepan Romanov; Sacha Howell; Elaine Harkness; D. Gareth Evans; Sue Astley; Martin Fergie

doi:10.1117/12.3027003

29 May 2024 Comparing percent breast density assessments of an AI-based method with expert reader estimates: inter-observer variability

Stepan Romanov, Sacha Howell, Elaine Harkness, D. Gareth Evans, Sue Astley, Martin Fergie

Author Affiliations +

Proceedings Volume 13174, 17th International Workshop on Breast Imaging (IWBI 2024); 131740N (2024) https://doi.org/10.1117/12.3027003
Event: 17th International Workshop on Breast Imaging (IWBI 2024), 2024, Chicago, IL, United States

Abstract

Breast density assessment is an important part of breast cancer risk assessment, as it has been known to correlate with risk. Mammograms would typically be assessed for density by multiple expert readers, however, interobserver variability can be high. Meanwhile, automatic breast density assessment tools are becoming more prevalent, particularly those based on artificial intelligence. We evaluate one such method against expert readers. A cohort of 1329 women going through screening was used to compare between two expert readers selected from a pool of 19, and a single such reader versus a deep learning based model. Whilst the mean differences for the two experiments were statistically similar, the limits of agreement between the AI method and a single reader are substantially lower at +SD 21 (95% CI : 20.07, 22.13) -SD 22 (95% CI : -22.95, -20.90) against +SD 31 (95% CI : 33.09, 28.91) -SD 28 (95% CI : -30.09, -25.91) between two expert readers. Additionally, the absolute intraclass correlation coefficients (two-way random multiple measures) were 0.86 (95% CI : 0.85, 0.88) between the AI and reader and 0.77 (95% CI : 0.75, 0.80) between the two readers achieving statistical significance. Our AI-driven breast density assessment tool has better inter-observer agreement with a randomly selected expert reader than two expert readers (drawn from a pool) do with one another. Additionally, the automatic method has similar inter-view agreement to experts and maintains consistency across density quartiles. Deep learning enabled density methods can offer a solution to the reader bias issue and provide consistent density scores.

Citation Download Citation

Stepan Romanov, Sacha Howell, Elaine Harkness, D. Gareth Evans, Sue Astley, and Martin Fergie "Comparing percent breast density assessments of an AI-based method with expert reader estimates: inter-observer variability", Proc. SPIE 13174, 17th International Workshop on Breast Imaging (IWBI 2024), 131740N (29 May 2024); https://doi.org/10.1117/12.3027003

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Artificial intelligence

Breast density

Breast

Breast cancer

Mammography

Risk assessment

Deep learning

Show All Keywords

Keywords/Phrases

Search In:

Publication Years