The agency is requesting public comment on methods for monitoring and evaluating AI models after deployment to ensure continued safety and effectiveness.
The US Food and Drug Administration (FDA) has issued a Request for Public Comment to gather feedback on best practices for measuring the real-world performance of artificial intelligence (AI)-enabled medical devices. The agency is seeking information from healthcare stakeholders, including manufacturers, providers, and technology experts, on how they monitor, evaluate, and manage these technologies after deployment in clinical environments.
The initiative addresses the challenge of “performance drift,” where an AI model’s effectiveness can degrade over time. According to the FDA document, factors such as changes in clinical practice, patient demographics, data inputs, and user behavior can impact system performance, potentially leading to bias or reduced reliability. The agency notes that current evaluation methods, often based on static or retrospective testing, are not designed to predict device behavior in dynamic clinical settings.
This effort follows a November 2024 meeting of the FDA Digital Health Advisory Committee, where participants discussed the need for robust evaluation strategies for AI-enabled devices. The FDA is particularly interested in practical approaches that are currently in use, supported by real-world evidence, and applied in patient- or provider-facing settings.
Key Areas for Comment
The request outlines several key areas for comment that are relevant to healthcare technology management professionals, including:
- Performance Metrics: What indicators are used to measure safety, effectiveness, and reliability in clinical use.
- Evaluation Methods: What tools and processes are used for proactive, post-deployment monitoring of AI device performance.
- Data Sources: Which data sources, such as electronic health records, device logs, and patient-reported outcomes, are used for ongoing evaluation.
- Response Protocols: What triggers additional assessments or interventions when performance degradation is detected.
- Infrastructure: What technical, operational, or organizational infrastructure is needed to support real-world performance evaluation.
The FDA states the feedback is for discussion purposes and does not represent draft guidance or policy changes at this time. Interested parties can submit comments until Dec 1, 2025, through the federal regulations website under Docket No FDA-2025-N-4203.
ID 132056545 © Wrightstudio | Dreamstime.com
