Performance Evaluation Plan (PEP) under EU IVDR 2017/746
Performance Evaluation Plan (PEP) under IVDR 2017/746
1. Introduction to Performance Evaluation under IVDR
The performance evaluation process is one of the requirements that ensure an in vitro diagnostic (IVD) device meets the safety and performance requirements of the regulation. According to IVDR, the manufacturer shall demonstrate that the device performs according to the purpose declared and resolves all the relevant safety and performance questions. Three major pillars are involved in this process: scientific validity, analytical performance, and clinical performance. (Ref: IVDR Article 56, Annex XIII Part A)
Legal Basis: Articles and Annex References
Some of the most important regulations include:
IVDR Art. 56: Requires that performance evaluation shall be planned, performed and documented in such a way that clinical evidence proves the intended purpose. Also requires continuous updating during the lifetime of the device.
Annex I, particularly Sect. 9: Outlines general safety and performance requirements (GSPR) regarding the performance characteristics (analytical and clinical) that need to be met.
Annex XIII, Part A: Sets out the information that needs to be included in the PEP (Performance Evaluation Plan), including criteria, procedures, characteristics, analytes, etc.
MDCG-2022-2: Sets forth the general principles for clinical evidence scientific validity, planning of the performance evaluation, performance follow up after marketing.
Importance in the IVD Lifecycle and in CE Marking
The PEP is crucial prior to launching the product to market since it contributes to the technical documentation, which will be reviewed by the Notified Body. If the PEP is deficient, this can delay the process of CE marking.
Post market, the document facilitates continuous data collection and assessment (PMPF/PMS) to confirm continuing compliance and safety. As per the IVDR, in particular high risk devices (Class C&D), the information needs to be updated regularly.
Risk Management tool: through identification of possible failure (performance analytical or clinical) and preparedness for monitoring/mitigation/enhancements.
2. Definition of Performance Evaluation Plan (PEP)
What is a Performance Evaluation Plan?
Performance Evaluation Plan refers to a document based plan that:
Describes the purpose of the device (what it is claimed to detect/measure, who and where, etc.)
Lists which performance characteristics (in accordance with Annex I Part 9) should be demonstrated.
Includes information about the methodologies, statistics, acceptance criteria, reference materials, etc., used for the generation of the evidence.
Outlines time frames, tasks, etc.
In simple words, the plan of how evidence will be collected under IVDR.
When and Why It Is Needed
When: Prior to placing the device on the market, as part of the technical documentation before marketing. Regularly updated throughout the device’s lifecycle, especially class C&D devices.
Why: In order to conform with IVDR requirements, specifically GSPR. In order to prove scientific validity, reliability, safety, usefulness of the performance claims. Otherwise, there is risk of failure to meet the regulatory expectations or the request of additional data from the Notified Body.
Relation to the Performance Evaluation Report (PER)
PEP is where you lay the groundwork: what will you be doing, how, when, using which criteria.
PER is the output: having done all the work, collected the data, now report on what you have found out, whether the device complies with these criteria and what the limitations are. Documentation of the literature, data, study design, statistical methods, results, etc
Regulatory requirement: according to IVDR, PER is part of the technical documentation. The PER should be kept up to date according to the PEP.
3. Regulatory Requirements for PEP: Article by Article Breakdown
Here are the main Articles / Annexes relevant, and what they require of the PEP:
Legal Provision | Key Requirements Related to PEP |
|---|---|
Article 56 | Must plan, conduct, document performance evaluation; clinical evidence must support intended purpose; evidence must be continuous; must justify level of clinical evidence appropriate to device characteristics and intended purpose; must update throughout lifecycle. |
Article 57 | Governs clinical performance studies: when required, design, safety, conduct, ethics. PEP may include whether such studies are needed or justified. |
Annex I Section 9 | Identifies which performance characteristics are relevant: analytic sensitivity/specificity, limit of detection, measurement range, etc. PEP must map which of these characteristics apply, and how to test them. |
Annex XIII Part A | Contains explicit details about what a PEP must include: device intended use, analytes, reference materials, patient group definitions, methods & statistical tools, state of the art, benefit risk acceptability, milestones, acceptance criteria, etc. |
4. Scientific Validity: Planning and Documentation
Definition of Scientific Validity under IVDR: Proving an association of the analyte or biomarker with a specific clinical or physiological condition. That's a fundamental one.
When planning validity assessment:
Literature review/systematic review. Find existing peer reviewed publications, expert opinion, meta analyses. Describe methodology: what databases were used (Pubmed, EmBase), how you searched for literature, selection criteria, critical evaluation.
Gap identification. What is still unknown: specificity, groups not covered, contradictory data. Plan new studies if needed.
Proof of concept/pilot data. In case of a new analyte or ambitious claims, small studies could prove association.
In terms of documentation, include the scientific validity section in PER, which includes all the above. It should be documented according to IVDR Annex XIII.
5. Performance of Analysis: Parameters and Methods
Performance of analysis is the ability of the instrument to deliver the claimed measurement under normal (and even stressed) operating conditions. The parameters of the performance include:
Precision: repeatability (same operator, same instrument), reproducibility (different conditions).
Accuracy or trueness: against reference methods.
Limit of detection (LOD) and limit of quantitation (LOQ).
Linearity/measurement range.
Specificity/analytical specificity: avoiding the cross reactivity of similar substances.
Interferences.
Stability: of reagents, sample preparation, and more.
The methods include:
Using certified reference materials or validated measurement procedure.
Determining appropriate sample size and statistical analysis (for instance, confidence intervals and margin of error).
Pre defining acceptance criteria: which tolerable errors should be allowed and performance threshold should be achieved.
Clinical Performance: Study Design and Rationale
Rationale: To demonstrate that the outcome of the diagnostic test performed by the device is consistent with the clinical state in the targeted population.
Study design considerations include the following:
Comparators: Gold standard tests or other diagnostic tests.
Population and setting: Demographic, disease incidence rates, types of specimens
Endpoints: Sensitivity, specificity, predictive value when applicable.
Prospective or retrospective study design: Depending on feasibility and appropriateness.
Incorporation of real world evidence or literature: When appropriate (for example in low risk devices or when clinical testing is difficult). Needs to be well justified.
Ethical and Safety Considerations: Specimen collection and handling, patient consent, and minimal risk
7. Performance Evaluation Strategy: Combined Approach
Scientific validity and analytical as well as clinical performance should not be evaluated separately; rather, they should be considered together. For instance, during a clinical performance evaluation, analytical performance could also be determined under practical conditions.
There must be a process of versioning and trigger to update. If there is new literature on the subject, or users’ complaints suggest problems with performance, or at some point (particularly for Class C and D devices, where IVDR requires yearly updating), the plan can be reviewed.
A risk based approach to performance evaluation should be taken, which means that the higher the risk, the stronger evidence should be provided.
8. Relationship Between PEP, PER and PMS / PMPF
Post Market Surveillance (PMS) refers to all actions taken after market introduction for collecting data on device performance, adverse events, complaints, etc.
Post Market Performance Follow up (PMPF) involves planned studies and data collection for evaluating performance claims, monitoring device performance, detecting performance drift.
The data collected through PMS/PMPF are then used in updating PEP (in case of identification of new risks or evidence) and revision of PER.
IVDR mandates for Class C & D devices that PER should be updated at least once every year.
9. Common Mistakes and Good Practice in PEP Development
Exclusion of contrary or negative data: Neglecting literature or research that contains any negative findings is biased and undermines the scientific nature demanded by Annex XIII, Part A.
Ambiguous acceptance criteria: The inclusion of phrases such as ‘good sensitivity’ with no quantifiable limits does not comply with the scientific analysis criteria of Annex I, Section 9.1(a).
Insufficient description of the intended purpose or population: Lack of a clear definition might cause a mismatch between the data obtained in the research and the actual use of the device. This might lead to a non compliance issue (see Article 56(1) and Annex I, Section 20).
No post market planning: Failure to formulate any Post Market Performance Follow up (PMPF) and Surveillance Plan contradicts Annex XIII, Part B and Articles 78 to 81.
Failure to revise the documents: Failure to revise the PEP/PER in case of additional data/device changes/updated standards conflicts with Article 56(6).
Good Practices
Document all pertinent information, regardless of whether it is positive or negative, and provide justification thereof.
Establish risk based acceptance criteria for all analyses and clinical parameters.
Ensure that the PEP harmonizes with the risk management documentation and that responsibilities for post market surveillance/PM decision making procedures are clearly defined.
Keep up to date versions of the PEP and PER by updating them appropriately.
10. Templates and Checklists for PEP Preparation
Device Description and Intended Purpose
Describe in detail the in vitro diagnostic device (its trade name, model, catalogue number and the precise statement of the intended purpose). This paragraph should also describe the medical condition(s), the indication(s) and the user profile (professional, point of care or self testing) according to Annex I, Section 20.
Target Population and Use Environment
Describe the target population (age range, disease status, other demographic factors), the use environment (laboratory, clinical, home). Identify specimen types, and any particular specimen collection, transport and storage requirements.
Definition of Analytes or Markers
Describe in a scientifically sound way all analytes or markers to be detected/measured by the device.
Show clinical relevance of the analytes/markers according to Article 2(38) and Annex XIII Part A.
Reference Materials, Reference Measurement Procedures and Standards
List reference materials, reference measurement procedures, standards which will be used to achieve metrological traceability.
Define traceability to the appropriate reference standards, according to Annex I, Section 9.1.a.
State of the Art and Relevant Standards
Provide a review of state of the art for the intended diagnostic purpose including scientifically and clinically recognized guidelines, harmonized standards and common specifications.
Benchmark device or technology to be used for comparison in the performance evaluation process.
Analytical Performance Characteristics
Describe all performance characteristics of the analysis that are relevant to the test device, and outline procedures for testing them which include:
Accuracy and trueness
Precision (repeatability and reproducibility)
Sensitivity and specificity
Limit of detection (LOD) and limit of quantification (LOQ)
Linearity and measuring range
Interferences and cross reactivity
The above should be described in terms of study design, statistical analysis and acceptance criteria. The present section should be consistent with Annex I, Section 9 and Annex XIII, Part A.
Clinical Performance Plan
Specify the approach to proving clinical performance of the medical device through clinical performance studies, literature review, or real life evidence.
Specify all details of the design of the study, including the expected clinical end points, comparators, sample size, selection and exclusion criteria, in compliance with Article 57 and Annex XIII, part A.
Benefit Risk Assessment and Integration of Risk Management
Provide a summary of the benefit risk assessment of the medical device by integrating the risk management process according to ISO 14971 and Article 10 of Annex I, Section 3.
Show how the risk management file contributes to the PEP.
Timeframe, Milestones and Responsibilities
Provide the timeline of planned activities for each phase of the performance evaluation with the corresponding start and finish dates.
Determine the responsible persons or departments for the fulfillment of each task.
Post Market Surveillance (PMS) and Post Market Performance Follow Up (PMPF)
Describe the strategy of gathering and evaluation of post market data to prove the continued safety and performance of the medical device.
Describe how the data from the PMS and PMPF will be incorporated in subsequent updates of the PEP and PER in compliance with Article 56, Annex XIII, part B, and articles 78-81.
Version Control
Provide the version control strategy for the PEP.
Include a record of changes to the PEP indicating the reason for the change, its date and responsible person/department.
11. Performance Evaluation Plan under IVDR FAQ's
Is each IVD device expected to have a PEP?
Yes. As per the IVDR regulation, it is necessary for all IVD devices, with any exemption, very specifically, to undergo a performance evaluation. It is mandatory for the manufacturer to establish a PEP in technical documentation.
What is the frequency of PEP/PER update?
Class C and D IVD devices: PER needs to be updated annually or if there are any updates regarding the post market, PMPF, which has any impact on the performance or safety.
For low risk category: triggered updates (based on new data, changes in the state of the art, intended use/performance concerns).
Is PER always mandatory?
Yes. The PER is mandatory and a part of technical documentation. Even if based on literature and indirect data, the PER will include justification, methods, results, limitations.
12. Conclusion
The PEP is an essential element for compliance with the IVDR because this document serves as the proof of understanding the necessary performance requirements and its evaluation, as well as the performance sustainability.
An effective PEP, followed by good performing PER and excellent PMS/PMPF, will make the process of regulatory assessment easier and minimize the possibility of non conformities.
Future trends: use of real world data, guidance of notified bodies/common specifications, updated standards (analytical and clinical), increased focus on traceability/reproducibility of results.
Key References
IVDR (EU) 2017/746, especially: Article 56: Performance evaluation and clinical evidence
Article 57: General requirements regarding performance studies
Annex I: General Safety and Performance Requirements (GSPR), e.g. Section 9 on performance characteristics
Annex XIII, Part A: Performance Evaluation and Performance Studies; Part B: Post Market Performance Follow Up (PMPF)
MDCG‑2022‑2: Guidance document published 27 January 2022, setting out general principles of clinical evidence, the performance evaluation process, the role of risk management, PEP, PER, continuous updating, etc. (Public Health)
Other posts
Don't let European red tape stall your vision. We simplify complex EU building regulations so you can focus on creation. Explore our blog for the clarity you need during your project and the insights required for post completion compliance. Read on for smoother approvals and smarter builds across Europe.