The Blinded Central Reading of Study Endpoints by an Adjudication Committee (CEC) could improve the quality of the Trial Outcomes by a reduction of the judgments variability.
Clinicians Investigators can be biased in their Endpoints evaluation due to individual conditions (e.g. local medical practices) or by their complete knowledge of the trial goals. Endpoint adjudication can overcome this bias and increase the data quality by applying a strict Quality Control on the processes. The most effective way of detecting signals of issues or bias in real time is by using a set of well-designed Quality Control Metrics. Inter-variability and Intra-variability among Reviewers, Queries to Sites concerning the Adjudication Dossier, Time lapses between milestones, re-submissions of events for source data changes and disagreement rate between the Investigator and the Committee are among the most frequently used indicators.
THE REVIEWERS INTER-VARIABILITY QUALITY CONTROL METRICS
The Reviewers Inter-Variability is a key Quality indicator of the Endpoint Adjudication process. Inter-Variability can be defined as the Discordance Rate in the Assessments of different Committee Members (EAC Reviewers). Ideally, this rate should be as low as possible but due to the heterogeneous fields of application of the Adjudication (differences in TAs / Endpoints definitions or Medical Records provided) it has not been possible so far to establish a common benchmarking value.
It is therefore recommended to monitor the Inter-Variability rate throughout the Study and verify that it remains coherent with an expected rate without obvious outliers. A particularly high level or an unexpected variation in the Reviewers Discordance Rate could signal an issue in the Charter design or in the process implementation (e.g. an unclear endpoint definition, a problem in the tools used for the measurements, an issue with the Committee composition or with the Medical Records used for the Adjudication Dossier).
THE REVIEWERS INTRA-VARIABILITY QUALITY CONTROL METRICS
The Reviewers Intra-Variability is a fundamental Quality indicator of the Endpoint Adjudication process. Intra-Variability can be defined as the Discordance Rate in the Assessments of the same Events delivered at different time, by the same committee or a different one. Ideally, this rate should be as low as possible and in case a disagreement between Original and QC Event the eAdjudication platform detects it allowing to carry out in-depth analyzes.
It is highly recommended to include this process in any endpoint adjudication study, as it ensures that assessments are as reliable as possible and detects problems already during the early study periods.
THE QUALITY CONTROL DASHBOARD in the eADJUDICATION SOLUTION
The eAdjudication Solution supports the Endpoint Adjudication Operations by collecting the Assessment made by the Reviewers. These records are then processed allowing to present the adjudicated results as a combination of the Reviewers judgments. It also allows resolving any Disagreements by guiding the Adjudication Workflow until a final Consensus is achieved.
One important feature of the eAdjudication Solution is that it can aggregate Reviewers judgments to produce useful real-time Inter-Variability Metrics. These Quality Control Metrics, rendered as Widgets, can be found in a dedicated section of the Quality Control Dashboard.
THE ADJUDICATION COORDINATOR AND THE QUALITY CONTROL DASHBOARD
Depending on the trial size and complexity, adjudication cases may be more or less easy to process entirely automatically. Completeness of information, blinding of identifiers, language translation and reviewers’ availability are some of the common challenges for an adjudication coordinator (AC) who is the “pilot” of the eAdjudication solution and the enabler of all cases resolution.
Dashboards and graphical representation of data may look nice and colorful but are not of much help unless they are correctly set and understood, unless risks are proactively identified and thresholds are set for action. The Quality Control Dashboard provides all the necessary tools for early detection of issues and allows quick verification that any correction applied has had the expected results. The Quality Control Dashboard is a valuable tool for the AC helping to keep the study moving smoothly and allowing to correct any flaws in the process, update the committee charter or improve the internal workflows.
EXAMPLES OF INTRA-VARIABILITY AND INTER-VARIABILITY METRICS WIDGETS
Study Disagreement by Reviewers & Time
- Study Closed Events by Disagreement
Pie Graph - Splitting the Adjudicated Events depending on their status after the first round of review.
In Agreement / With Major Disagreement / With Minor Disagreement - Study Closed Events by Event Type
Histogram - Splitting the Disagreement among the different Event Type / Adjudication Forms.
Total Events per Event type / Total Events with Disagreement per Event Type - Disagreement by Time
Histogram - Follow Disagreement over Time (Monthly).
Total Events per Month - Total Events with Disagreement per Month - Disagreement Rate of Closed Events
Histogram - Follow Disagreement over Level (Major and Minor).
Total Disagreement Events / Minor Disagreement Events / Major Disagreement Events - Major and Minor Disagreement per Event Type
Histogram - Splitting the Minor and Major Disagreement among the different Event type.
Minor Disagreement / Major Disagreement - Study Disagreement by Single Reviewers
Histogram - Splitting the Events closed with Disagreement among Reviewers.
Total Events per Reviewer - Total Events with Disagreement per Reviewer - Study Disagreement per Committee
Histogram - Splitting the Events closed with Disagreement among the Couples of Reviewers.
Total Events per Couple of Reviewers - Total Events with Disagreement per Couples Reviewer. - Total QC Events
Pie Graph - Splitting the Total Events in the system among Study Events and QC Events.
Events related to the study - QC Events related to Intra-Variability Analysis - Total Discrepancies
Histogram - Splitting the Total QC Events depending on their outcome, with or without Discrepancies.
Total QC Events / Events without Discrepancies / Events with Discrepancies - Discrepancies by Event Type
Horizontal Histogram - Splitting the QC Events among the different Event Type / Adjudication Forms.
Total Closed QC Events - Total Closed Events with Discrepancies with the Original Event
DOWNLOAD NOW THE ENDPOINT ADJUDICATION HANDBOOK
The Complete Manual / Reference Book with all the topics related to the Independent Endpoint Adjudication Committees Management
