Editor’s Spotlight/Take 5: How Large a Study Is Needed to Detect TKA Revision Rate Reductions Attributable to Robotic or Navigated Technologies? A Simulation-based Power Analysis

Here’s a personality test. What is your reaction to the following claim: “Navigated, robot-assisted [your favorite procedure here] results in more-precise implant alignment than does manual surgery.”

It seems to me that a prudent reaction would be to ask more questions: How much more precise? What advantages might a patient notice, if any, from this increase in precision? Are there potential harms or added costs associated with the new approach?

Our specialty’s reaction has been otherwise. Despite the fact that no well-designed study about a new implant-alignment tool, navigation system, or surgical robot has been shown to improve patient-reported outcomes or implant durability, and despite the fact that the best studies on those innovations—including systematic reviews, network meta-analyses, registry reports, and long-term follow-up studies of randomized trials—have found no differences that a patient might perceive [^{1, 2, 8, 9, 13}], the use of expensive, unproven technology in our specialty is growing fast. The global use of robotic-assisted knee arthroplasty, for example, will reach approximately 700,000 procedures per year by 2030, sustaining a compound annual growth rate of 8% from 2021 to 2030 [⁵].

I can only speculate as to why this might be. Certainly, as a tech-driven specialty, I think many of us enjoy thinking about ways that new tools may make us more effective; why not try them out? I can attest that robotic and navigated systems are enjoyable to operate, and they feel—indeed are—very precise when used correctly. These sensations certainly provide positive feedback to the surgeon, and may contribute to a surgeon’s decision to continue to use them. It also seems likely that gut-level intuition plays a role here. Accelerometers make everything from our cellphones to the airplanes we fly in more capable; why wouldn’t they be useful to guide our hands—or a robot’s hands—through a complicated operation in three dimensions and six degrees of freedom? Finally, it’s possible that some of the buzz is driven by intellectual property interests or industry consulting relationships, though I’d hate to believe this is a large part of it.

Regardless, there are important reasons not to use new approaches until they’re proven superior in ways patients might perceive: costs, surgical time, and perhaps most importantly, the risks inherent to uncertainty and novelty. Unintended consequences can be severe, surprising, and anything but hypothetical. Real patients—folks’ moms and dads—get hurt when we use something we think works well rather than making sure we know it works well [^{10, 12}]. I’ve come to believe that surrogate endpoints, like the percentage of implant-alignment outliers, need to be viewed with great skepticism when we’re dealing with generally well-behaved operations like primary joint arthroplasty. We need to ask for more than just improved precision before deciding to use these tools in our practices. Improved implant survivorship would be nice.

For this reason, I was so excited to read this month’s Editor’s Spotlight article, “How Large a Study Is Needed to Detect TKA Revision Rate Reductions Attributable to Robotic or Navigated Technologies? A Simulation-based Power Analysis” [⁶], which was performed by a high-powered clinical-methodological combo team from the University of British Columbia. Using robust mathematical simulation approaches and source data from a systematic search process, they projected that technology-assisted TKA (robotic and/or navigated) may yield slight advantages in terms of implant survivorship over the very long run.

How slight and how long? The authors’ simulations suggested a survivorship edge associated with new tech over conventional TKA of 1.4% to 2% at 15 years. At 10 years, they calculated a number needed to treat of 100 patients. This means one would have to perform—and pay for—100 robotic TKAs in place of that many conventional TKAs in order to avert one revision procedure.

Back to the personality screen: Does this mean that we should perform a randomized trial enrolling more than 5000 patients and running it out into the second decade in order to be adequately powered to verify that such a small difference actually materializes? Or does it make more sense to say that such a small—and, at this point, hypothetical—difference is unlikely to be worth the cost and time even to do such a study? (To say nothing of the cost, time, and risk of buying, deploying, and learning how to use all those robots or navigation systems).

This study is important even if you do not replace knees, and regardless of whether you want your next operation performed by human hands or those of a robot. I say so because the methods the authors used help us to think about how, when, and with what kind of evidentiary support we might decide to try something new in the operating room. Another angle in this paper, which I think will have wide appeal, is that while we’re used to seeing studies end with the suggestion “More research is needed …”, the fact is this may not always be true. If we need 5000 patients and 15 years to find a difference in favor of an expensive robot or navigation system, that difference must be very small, indeed, and almost certainly not worth searching for with a big, resource-intensive trial. We should focus our attention elsewhere.

These are provocative topics. Join me as I discuss them with Matthew Hickey BEng and PhD candidate, and Bassam Masri MD, authors of “How Large a Study Is Needed to Detect TKA Revision Rate Reductions Attributable to Robotic or Navigated Technologies? A Simulation-based Power Analysis,” in the Take 5 interview that follows.