Assured AI reference architecture

Marcus Tyler; James McCeney

doi:10.1117/12.3027227

7 June 2024 Assured AI reference architecture

Marcus Tyler, James McCeney

Proceedings Volume 13054, Assurance and Security for AI-enabled Systems; 130540Q (2024) https://doi.org/10.1117/12.3027227
Event: SPIE Defense + Commercial Sensing, 2024, National Harbor, Maryland, United States

Abstract

The Cross-Industry Standard Process for the development of Machine Learning applications with Quality assurance (CRISP-ML(Q)) framework describes the full AI model lifecycle from data sourcing to deployment, along with risk mitigation procedures for each phase of development. In this work we implement an example model pipeline that utilizes CRISP-ML(Q), along with mitigations for risks documented in MITRE ATLAS and the National Institute of Standards and Technology Artificial Intelligence Risk Management Framework (NIST AI RMF) and its associated playbook, with the goal of creating a generalizable reference framework for model developers and decisionmakers. This work builds on our previous effort in cross-referencing CRISP-ML(Q) with published ML risk frameworks by developing a working and reproducible pipeline for image classification that validates our risk mitigation approach. In this pipeline, we implement multiple risk mitigations and demonstrate their effectiveness in the areas of drift detection, spam mitigation, and defense against adversarial attacks, along with demonstrations of model explainability to improve model trustworthiness. We present results from a variety of A/B mitigation tests that show the effectiveness of our mitigations, and we demonstrate full pipeline runs for Convolutional Neural Network (CNN) and Resnet18 model architectures that utilize our risk mitigation process.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Marcus Tyler and James McCeney "Assured AI reference architecture", Proc. SPIE 13054, Assurance and Security for AI-enabled Systems, 130540Q (7 June 2024); https://doi.org/10.1117/12.3027227

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available