Literature Appraisal Criteria

Literature appraisal can be assisted by AI tools, significantly reducing the time required for literature analysis and improving data extraction accuracy. The tool configuration follows the grading methodology described below. Other grading methods (e.g., GRADE, SORT, OCEBM Levels of Evidence) are not supported by the AI tools.

General Method

When reading and analysing full-text literature, evidence level appraisal criteria are established per Annex 5 "Considerations for Establishing Data Appraisal Criteria" of the Technical Guidance on Clinical Evaluation of Medical Devices (NMPA). Evidence levels are ranked from highest to lowest as follows. This method is equivalent to that described in IMDRF MDCE WG/N56FINAL:2019.

1. Randomised controlled clinical trials / meta-analyses / systematic reviews
2. Cohort studies
3. Case-control studies
4. Case series studies

To determine the relevance and contribution of clinical data to demonstrating the safety, clinical performance, and/or efficacy of the subject device, suitability and contribution assessment criteria are established per Annex 6 "Reference Assessment Methods." Data included in the analysis must be from the "subject device" and/or "same-kind medical device (equivalent or comparable device)." Literature from "other products" is excluded at the screening stage. To ensure scientific rigour, each criterion is scored quantitatively; literature with a total suitability score ≥ 8 is included. If any single criterion scores 1, the literature is excluded.

Data Suitability Assessment Criteria

Relevance Criterion	Description	Grading (Score)
Appropriate product	Is the data from the subject device?	D1 Subject device: 3; D2 Same-kind device (equivalent/comparable): 2; D3 Other product: 1
Appropriate scope	Is the scope of application the same?	A1 Same: 3; A2 Minor deviation: 2; A3 Major deviation: 1
Appropriate patient population	Does the patient population represent the intended use population (age, sex, etc.) and clinical condition (disease state and severity)?	P1 Applicable: 3; P2 Limited: 2; P3 Different population: 1
Acceptable reporting/data summary	Does the data contain sufficient information for a reasonable, objective assessment?	R1 High quality: 3; R2 Minor deficiencies: 2; R3 Insufficient information: 1

Contribution assessment uses a weighted approach. The more first-tier ratings (answered "Yes") a dataset has, the greater the weight of evidence it provides.

Data Contribution Assessment Criteria

Data Type	Is the study design appropriate?	Grading
Source data type (SD)	Is the trial design appropriate?	T1 Yes; T2 No
Outcome measures (OM)	Do the reported outcome measures reflect the intended performance of the device?	O1 Yes; O2 No
Follow-up (FU)	Is the follow-up period sufficient to evaluate treatment effects and identify complications?	F1 Yes; F2 No
Statistical significance (SS)	Is statistical analysis of the data provided and appropriate?	S1 Yes; S2 No
Clinical significance (CS)	Are the observed effects clinically meaningful?	C1 Yes; C2 No

EU MDR Special Requirements

State-of-the-Art Literature Search and Analysis Requirements

State-of-the-Art Research Questions

SOTA assessment begins in the CEP and is elaborated in the CER. Research questions should enable retrieval of:

• Background on the specific diagnostic or therapeutic approach, and safety and performance assessment
• Studies on alternative treatments for the indication

Device-Specific Research Questions

Research questions are based on the device's claimed performance, primarily including:

• Assessment of the safety and performance of the target device compared to similar products or alternatives on the market
• Determination of whether new clinical trials are needed to support clinical safety and efficacy claims

SOTA Literature Scoring Criteria

SOTA-relevant literature is scored as follows:

1. Base score (1 point): General discussion of disease management and relevant patient population
2. Additional scoring items:
   • Benefit-risk comparison of at least two treatment approaches (+2 points)
   • Study relates to use in the EU or other Western countries (+3 points)
   • Study discusses risks/complications of the evaluated product or similar products (+4 points)
3. Literature scoring ≥ 6 points will be included in the SOTA discussion. Justification for inclusion or exclusion must be provided in the CER.

Evidence Quality Assessment

MDCG 2020-6 Ranking (strongest to weakest):

1. High-quality clinical study results covering all device variants, indications, patient populations, treatment duration, etc.
2. High-quality clinical study results with some gaps
3. Results from high-quality clinical data collection systems (e.g., clinical trial registries)
4. Results where studies may have methodological limitations but data are quantifiable and acceptability can be demonstrated
5. Equivalence data (reliable/quantifiable)
6. SOTA assessment, including clinical data from similar devices
7. Complaint and vigilance data; compiled data
8. Active post-market surveillance (PMS) data, e.g., from surveys
9. Case reports on the target device
10. Non-clinical data demonstrating conformity with common specifications related to device safety and performance
11. Simulated use/animal/cadaver testing involving healthcare professionals or other end users
12. Preclinical and bench testing / standards conformity testing

Article Quality Assessment — Scientific Contribution

• Mandatory high-quality criteria: Study design (SD): Yes; Statistical significance (SS): Yes
• Additional quality indicators: Follow-up (FU), Outcome measures (OM), Clinical significance (CS)
• Refined grading: Grade A: all additional indicators Yes (FU+OM+CS = 3); Grade B: at least two Yes; Grade C: one Yes

High-quality classification:

• MDCG 2020-6 ranking 1–5
• Meets high-quality study design criteria (RCT, systematic review, meta-analysis, high-quality cohort study)
• SD = Yes AND SS = Yes

Literature meeting high-quality criteria with at least two of FU, OM, CS = Yes is considered the highest level of evidence and is prioritised for establishing quantifiable SOTA benchmarks.