Bundling and Camouflage
In this section we have tried to define these two terms, Camouflage and Bundling and describe how they effect our management of the data that is supplied by manufacturers for ODEP benchmarks. ODEP uses these terms in the context of all the data which they receive but particularly in the context of registry data.
Over the past few years we have seen the massive advantages in collecting “Big Data” in registries. We now have some brands of joint replacement have been used in tens of thousands and overall, the results are usually excellent with a revision rate well inside the benchmarks set by ODEP.
What has also become apparent is that a large cohort of can potentially contain a subset of patients who have had some sort of “slight” modification of the original brand or a line extension which are "Bundled" into the results of the main brand. Sometimes there may be differences in the performance of a brand when it is used in a particular way. We know that sometimes patients in these subsets can do less well than the standard implant and occasionally much worse to the point of gaining outlier status.
Thus we seek to try and prevent poor results being camouflaged by perfectly acceptable results, whilst acknowledging the advantages of bundling when it is done in a safe and regulated manner.
How do Camouflage and Bundling affect Benchmarking?
Accurate and worthwhile benchmarking relies on
a) An adequate size of the cohort
b) Purity of the cohort, ie the implants in the cohort are all the same
The practical issues for a manufacturer submitting data for a benchmark for their implant include:
The guidelines set out by the benchmarking organisation
Availability and granularity of available data
The issues that this paper wishes to address
The Balance between the necessity of bundling and camouflaging poor performance
One example of what ODEP seeks to achieve is how to deal with a proportionately small number of implants in a cohort that have significant differences from the standard implant but whose numbers are inadequate to form a separate submission for a benchmark in their own right. It is acknowledged that for the most part these implants fall within the design designated by the brand. An example would be a stem brand that had 180 and 240 length options but these were not utilized very often but were clearly important implants in terms of the portfolio. ODEP would not want to make it difficult for a surgeon to have a long stem prosthesis, (or similar example of need) available when its use was indicated.
Another example is where ODEP considers it perfectly reasonable for a manufacturer to bundle in another set of very similar data to achieve the required cohort size. An example would be a knee replacement which can be used with or without resurfacing the patella. ODEP would need to know whether the implant was equally safe / successful for patients, whether the patella was resurfaced at the index operation or not. However it might be that one or both cohorts were not large enough to support separate submissions to ODEP and the manufacturer needed to use both sets to make up the number.
It must be stressed that bundling is only allowed if the Panel consider that it is almost certainly not a significant patient safety issue and that the rules ODEP set out are observed. It has to be accepted that decisions of this type are are based on educated opinions and cannot be guaranteed. They can always be kept under review.
The Questions that need to be asked to ensure bundling is as safe as possible
What is the number of implants required in each of the smaller (bundled in) and larger (main) cohorts for ODEP feel able to benchmark?
What is an acceptable revision rate in the smaller group?
A small number of implants in a cohort which have one major difference from the rest of the cohort. Examples would include:
Sizes outside the normal integral range
A cup with multiple holes instead of “no holes”
A oval patella rather than a round one
If there were no revisions in the small group bundling would not be a problem
If the revision rate for the whole (bundled) cohort there would not be a problem
If the revision rate for the small group was the same as the larger group and both were inside the benchmark there would not be a problem
If the revision rate in the small group was sufficiently high to push the revision rate above the benchmark revision rate it would not be a problem as the submission would be rejected.
Essentially, the issues surround how many patients there should be in a subgroup to be confident that camouflage is not possible.
For ODEP to reduce the risks of Camouflage the panel will always take careful note of the product code list (with all the descriptors added), to make sure there are no product codes for implants that would fall outside the range that ODEP would expect to accept for that brand.
It will be easier for manufacturers to obtain the correct data about a product from a registry that has the necessary granularity to achieve this.
NB The guidelines for bundling for individual TJRs will be found under each TJR.