A public-private partnership between the Digital Medicine Society (DiMe) and the U.S. Food and Drug Administration (FDA) has produced a set of resources designed to guide the analytical validation of novel digital clinical measures. These resources aim to support digital health technology (DHT) developers and clinical trial sponsors in creating regulatory-grade devices for collecting health metrics in clinical studies.
Enhancing Analytical Validation
The initiative is part of the DiMe-hosted DATAcc (Digital Health Measurement Collaborative Community) and focuses on ensuring the appropriate level of analytical validation for digital measures. According to Benjamin Vandendriessche, chief delivery officer at DiMe, the newly launched information repository includes an interactive guide with introductory materials, a study builder, and in-depth guidance on statistical methodologies.
These resources complement the Verification, Analytical Validation, and Clinical Validation (V3) Framework created by DiMe in 2020, which was extended to V3+ in 2024. The V3+ framework includes a fourth component: usability validation for assessing the quality, performance, and clinical relevance of sensor-based DHTs to ensure they are “fit for purpose.” Analytical validation involves assessing the performance of an algorithm in transforming raw data from a sensor into actionable insights, often requiring a reliable reference measure.
Addressing Novel Measures
Vandendriessche notes that novel measures exist on a hierarchy, and their context of use determines the level of rigor needed for analytical validation. In cases where a good reference measure is lacking, DHT developers may need to create one. Alternatively, “anchor” measures, as termed by the FDA, can be used as an external criterion to determine if patients have experienced a meaningful change in their condition. These anchor measures may show statistical association, rather than correlation, which serves as a steppingstone for validation.
Reaching the necessary rigor in data for a primary digital endpoint in a targeted patient population with an intended clinical outcome can be a lengthy process. The foundational V3+ Framework provides a “flow of work” for evaluating sensor-based digital health technologies, such as cardiac patches that measure the heart’s electric activity, ensuring they meet or exceed established measurement tools like electrocardiograms.
Financial Implications and Digital Endpoints
DiMe’s mission includes accelerating the adoption of digital endpoints in clinical trials. An analysis with the Tufts Center for the Study of Drug Development (CSDD) revealed the potential financial benefits to sponsors in terms of reduced study size and duration. The study found that shortening trial timelines and time to market could result in a return on investment of four to six times the initial investment. Incorporating digital endpoints into phase 3 trials could lead to estimated financial gains between $24 and $40 million.
The range of sensor-based devices producing digital clinical measures is vast, spanning from well-established technologies like ECG cardiac patches to fitness trackers for predicting opioid use disorder relapse. The concept of “novel” is on a spectrum, with different approaches and statistical methodologies needed for analytical validation when the units of measurement and references are not directly comparable or when no good reference is available.
Statistical Methodologies and Scenario Planning
The new set of resources aims to guide decision-making during the design of analytical validation studies. These resources include an interactive flipbook covering the basics of analytical validation of novel digital clinical measures and a detailed document guiding developers through building a statistical plan for their analytical validation study.
A second set of resources offers in-depth considerations and recommendations for statistical methodologies applicable when traditional approaches are unsuitable. This is particularly useful for biostatisticians using simulation tools to explore scenarios in silico before initiating a study.
Engagement with the FDA and Future Roadmap
Engagement with the FDA is highly encouraged as DHT developers and clinical trial sponsors navigate the decision-making process regarding their planned analytical validation strategy. Many novel digital measures have the potential to impact drug development and address national-level clinical needs.
DiMe’s DATAcc recently released a core set of digital measures and resources for Alzheimer's disease and related dementias, focused on establishing a baseline for measuring aspects in clinical trials that are truly meaningful to patients. This reflects a concern that developers and sponsors may overlook health aspects, such as the ability to perform everyday activities and manage disease symptoms, if they are overly focused on regulatory compliance and financial outcomes.
In a follow-up project with the FDA, DiMe is assembling an end-to-end pipeline of resources into a comprehensive process map for implementing digital endpoints in clinical trials. This master roadmap, expected by the end of 2025, will benefit both newcomers and experienced professionals in the field. It aims to streamline the process and provide answers to common questions the FDA currently addresses on an ad hoc basis.
The guidance on assessing the accuracy and reliability of algorithms powering digital health technologies represents a significant advancement by providing all clinical trial stakeholders with a shared, open-access resource to streamline their conversations with regulators.
