Moving through the benchmarks

Although NICE never stipulated that an implant should move from a 3 to a 10 or a 13 year rating, ODEP decided the system would fail patients if manufacturers did not regularly submit data to support the continued use of their product at each of the time points.


ODEP have always realised that early data is less reliable in terms of predicting long term success of product than more mature data from a longer study. ODEP also profess  that by having benchmarks at time points between 3 and 13 years means that manufacturers will keep a careful watch on their product’s performance.


Thus, the system has evolved, with manufacturers notifying ODEP at the time of the implant being introduced into the market for a pre-entry rating and then with time and with success of the implant achieving the 13A* (Hips and Knees) or 10A (Shoulders) rating. (Pre-entry A* is awarded to a product which is going through the Beyond Compliance process).


It is not acceptable for an implant to “rest” and not continue to move through the benchmarks at the allocated time periods. If the implant does not continue to move up through the ratings it could be withdrawn. ODEP realise that data collection is time consuming and ODEP does understand that companies can face difficulties in obtaining data to deadline. We know that there is often a delay between the request and the delivery of data from some registries. Thus, a year of grace is available for manufacturers to obtain the necessary data for their next submission. ODEP will inform manufacturers when they are expected to submit and if after the appropriate reminders, the company does not submit their product will be de-registered.


With the maturity of registries ODEP will make benchmarks, past the 10 or the 13 year timepoint, available. Basically, these will keep pace with most of the national joint registries. 


After an implant has achieved the highest benchmark, the manufacturer is expected to refresh their data every three years thereafter, to ensure good vigilance and surveillance of performance through data analysis.  Regular reminders are issued to companies when their implants are approaching these gateways.



Conclusion of Appeals

It is ODEP's normal policy to consider Appeals very carefully at the notified Appeals meeting. And the decision at these Appeals meetings are almost always final, and further contact about an implant should be made at the next meeting for that particular type of implant.


What will ODEP accept as a modification?


Or… Will ODEP allow a manufacturer to keep a product’s present benchmark when they want to modify the device?


It has always been ODEP’s view that if:-

  • the implant is made of a new material,
  • the working surface (the surface that is in contact with the patient’s skeleton) has been changed,
  • the shape has been changed,

it will be usually be regarded as a new device and will have to start the ODEP process from the beginning, the pre-entry stage.

It has often been argued that there is no such thing as a “minor modification”. There have been instances of “minor modifications leading to major changes in performance, often for the worse”.


Manufacturers are well advised to let ODEP know of any “modification” they wish to make to one of their devices and it is much better that ODEP is informed rather than ODEP finding out.


Some modifications are acceptable and acceptance is allowed on an implant by implant basis after exhaustive discussion.


Camouflage is a word that entered Registry parlance when the problems with stemmed metal on metal hip replacements were finally identified as being dangerous. At that time registries, categorized hip replacements as being either cemented, uncemented or hybrid. Hybrid is where one part of the joint was cemented and the other was not. The overall results for hybrid uncemented or cemented stems with uncemented cups was good. Most of these incorporated a metal head on a HDP bearing within a metal acetabular shell. At that time, all bearing surfaces came under the 3 types of hip replacement. The numbers were very large and when metal on metal bearings came in their poor results were camouflaged by the excellent results of the other bearing combinations.


Since then there have been other examples of camouflage or an implant being in the shadow of a large number of well performing ones.  If a very effective knee replacement, which has been used very successfully thousands of times had one of its attributes altered there would be a risk of this attribute being unnoticed in a registry that was only looking at brands. The small number of the brand with this new part to it could fail repeatedly before the numbers were sufficient to alter the revision rate or PTIR (Patient Time Incident Rate) and come to the notice of the manufacturer or registry.


Thus, it is important that manufacturers declare all the product codes and their descriptors in an ODEP submission as increasingly ODEP are looking at product codes rather than just brands alone.



Data Sources


ODEP welcomes data from any source, where the data has been validated by one way or another. When ODEP started, there were few registries and most data came from peer reviewed, published studies. Increasingly manufacturers have made use of registry data, particularly where it is available through such electronic portals such as the NJR’s supplier feedback. Particularly for the early benchmarks such the 3 and 5 benchmarks ODEP welcomes more than one type of data. 


Registry data


Manufacturers are welcome to use data from Registries where the linkage between primary and revision procedures is secure. The data needs to be “cut” to the time point for which the submission is prepared. The International Society of Arthroplasty Registries (ISAR) have introduced standards for registries and ODEP takes careful note of the standing of a particular registry. ODEP is concerned about using data from a registry that does not account for patients moving from a hospital in one area to another hospital outside the catchment area of the registry involved.


Clinical Trials, RCTs, etc.


ODEP welcomes data from clinical trials. It is thought to be reliable and it is often good supporting evidence, particularly for early benchmarks, to be added to early registry data. When it all comes from the originator’s centre it needs added data from “other centres” to qualify for an A or A* rating.


Publications, Podium presentations and posters.


ODEP expects any publications, podium presentations and posters to only be submitted when they have been previously peer reviewed.


“In house data analysis”


Some manufacturers have first class in house data collecting systems. Unfortunately they are not subjected to external validation and it is thought possible for patients to escape from their follow up process.


ODEP realise that no system is full proof and that allowances have to sometime be made. It is important for manufacturers to make sure they don’t use data more than once. For example, data that is submitted from a clinical  trial cannot be  extracted  again  from a poster or podium presentation or even a registry.


BC Section Menu Left