Data Appraisal and Analysis

Within the Clinical Evaluation Plan (CEP), the “Appraisal and Analysis of Pertinent Data” contents are core methodological information. It aims to clearly and systematically plan how all pertinent clinical data related to the device under evaluation will be identified during the clinical evaluation process, how the suitability, quality, and scientific validity of this data will be rigorously assessed, and finally, how this appraised data will be collectively analyzed to draw conclusions regarding the device’s safety, performance, and benefit-risk profile. The plans and methods described in this chapter are key to ensuring that the entire clinical evaluation process is scientific, objective, repeatable, and compliant with regulatory requirements.

When drafting this chapter, the emphasis should be on the systematic nature, objectivity, and pre-defined criteria of the appraisal and analysis process.

Defining the Scope of “Pertinent Data” and Identification Strategy

Firstly, the CEP needs to clearly define what constitutes “pertinent data” for this specific clinical evaluation, closely aligning with the previously established clinical evaluation scope, the characteristics of the device under evaluation, its intended use, and the specific claims that need to be supported by clinical evidence.

Building on this, a brief reiteration of the main pathways planned for acquiring this pertinent data should be provided. As mentioned in other parts of the CEP (such as “Data Generated and Held by the Manufacturer,” “Literature Search Strategy,” etc.), these data sources typically include (but are not limited to):

• Preclinical data generated and held by the manufacturer (e.g., bench testing, animal studies, biocompatibility data).
• Published scientific literature pertaining to the device under evaluation or equivalent/similar devices.
• Data from manufacturer-sponsored clinical investigations (if any).
• Post-Market Surveillance (PMS) and Post-Market Clinical Follow-up (PMCF) data.
• Clinical data from equivalent devices (if applicable).
• Outputs from the risk management process.

This part should outline the specific strategy and methods for identifying all such pertinent data. Particularly for literature data, it should mention that systematic literature search and screening methodologies will be employed. Clear inclusion and exclusion criteria are crucial for ensuring the objectivity and consistency of data screening and should be outlined or referenced here.

Planning the Data Appraisal Process

Data appraisal is one of the core steps of clinical evaluation, aimed at judging the quality of the identified data and its contribution to the evaluation conclusions. The CEP must detail this appraisal process.

Pre-defined Appraisal Criteria: Emphasize that a pre-defined, objective set of appraisal criteria will be used to systematically evaluate all collected data. These criteria should ensure the transparency and consistency of the appraisal process and minimize bias as much as possible. Appraisal criteria typically cover the following dimensions:

• Relevance: Assessing whether the data directly pertains to the device under evaluation (or its equivalent/similar devices), its intended use, target population, and the specific safety and performance endpoints under evaluation.
• Quality and Methodological Soundness: Thoroughly examining the methods used to generate the data, including study design (e.g., randomized controlled trials, observational studies, case series), conduct, completeness of data collection, appropriateness of statistical analysis, and potential sources of bias (e.g., selection bias, information bias, confounding factors), the impact of random error, adequacy of information disclosure, etc.
• Contribution to Demonstration of Conformity: Based on the relevance and quality of the data, judging its weight and value in demonstrating the safety and performance of the device.

Uniform Application of Appraisal Criteria: Stress that these pre-defined criteria will be uniformly applied to all data sources (including manufacturer-held data, literature data, clinical investigation data, etc.), and that both favorable (supportive) and unfavorable (non-supportive or indicative of risk) data will be objectively appraised with equal weight.

Differential Use of Data: Recognize that not all data may be equally suitable for all evaluation purposes. For instance, data from studies with design flaws that are insufficient to demonstrate device performance might still contain information valuable for safety analysis, and vice versa. The CEP should state that this differential applicability will be considered during appraisal.

Planning the Data Analysis Strategy

Once data has been rigorously appraised and deemed suitable, the CEP must plan how this data will be collectively analyzed to form overall conclusions about the device’s safety, performance, and benefit-risk profile.

• Evidence Synthesis and Integration: Explain how appraised evidence from various sources (e.g., preclinical, clinical investigations, literature, PMS) will be systematically integrated and synthesized to build a comprehensive and coherent body of evidence.
• Identification and Handling of Clinical Evidence Gaps: Describe how, during data analysis, any potential gaps in clinical evidence (i.e., certain aspects of safety or performance not adequately supported by data) or inconsistencies between different data sources will be identified. Also, outline the planned approach to address these situations, for example, by adjusting PMCF plans to gather further data, reassessing relevant risks in risk management, or clearly acknowledging limitations in the conclusions.
• Benefit-Risk Analysis: This is one of the ultimate goals of data analysis. The CEP should clearly state that the data analysis will lead to a comprehensive assessment of the benefit-risk profile of the device under evaluation, and whether this profile is acceptable when considered against the current state-of-the-art (SOTA) and available alternative treatments.
• Demonstration of Conformity with General Safety and Performance Requirements (GSPRs): The data analysis plan should be directly aimed at demonstrating the device’s conformity with the relevant GSPRs of the MDR (particularly GSPR 1 and GSPR 8).

Justification for the Level of Clinical Evidence

As per MDR Article 61, the manufacturer must specify and justify the level of clinical evidence necessary to demonstrate conformity with the relevant GSPRs for their device. This part of the CEP should prospectively address this.

Declaration of Evidence Level: Clearly state what level and type of clinical evidence is considered sufficient and appropriate by the manufacturer, based on the characteristics of the device under evaluation (e.g., novelty, risk class, technological maturity) and its intended use.

Rationale for Justification: Provide a detailed rationale supporting this judgment of evidence level. For example, for mature technology, low-risk devices, greater reliance might be placed on literature data and preclinical data; whereas for high-risk or innovative devices, dedicated pre-market clinical investigation data might be necessary.

Overview of Evaluation Objectives: An outline can be provided of what specific aspects this clinical evaluation aims to confirm through the planned level of evidence, for example:

• Whether the device performs as intended and specified by the manufacturer.
• Whether the risk profile of the device is comparable to that of similar devices or the current SOTA.
• Whether the device is safe for both patients and users.
• Whether the overall benefit-risk profile of the device is acceptable.

Iterative Nature of the Process and Final Reporting

It should be clarify that the data appraisal and analysis process can be iterative. New information discovered or new questions raised during the appraisal may necessitate adjustments to the clinical evaluation scope, refinements to the plan, or even supplementary data collection and analysis.

Finally, the CEP should briefly state that the complete process of identifying, screening, appraising, and analyzing all pertinent data, along with the final conclusions, will be fully and transparently documented in the Clinical Evaluation Report (CER). The CER will, based on these analyses, comprehensively describe the device’s safety, clinical performance, clinical benefits, and make a final judgment on whether sufficient clinical evidence has been obtained to support the device’s intended use and conformity.