Canine Cruciate Registry and the Minimally Clinical Important Difference

Quality Improvement Box Set: MCID Pam Mosedale BVetMed FRCVS QI Clinical Lead
John Innes BVSc PhD CertVR DSAS(Orth) FRCVS RCVS Specialist in Small Animal Surgery (Orthopaedics)
RCVS Knowledge:
Welcome to the Quality Improvement Box Set by RCVS Knowledge, a series of webinars, podcasts, and video interviews for practices and practitioners.
Pam Mosedale:
Hi everyone. Today I’m going to talk to John Innes. John is director of movement referrals and an honorary professor at the Liverpool university and a real guru on orthopedics. And I’m very excited to talk to John about the news that our knowledge, our Canine Cruciate Registry data from it has been used in research. Hi John, how are you?
John Innes:
I’m very well, Pam. Thanks for inviting me.
Pam Mosedale:
Great. So John, do you want to tell us a little bit about why you think Canine Cruciate Registry is important first of all?
John Innes:
Vets have been arguing about how to fix cruciates since the cruciate rupture was discovered, really. And there’s been quite an evolution of different techniques I suppose, I’ve been interested in North Weeks for 30 plus years or so, and I’ve seen that evolution probably there are more techniques being used these days than perhaps ever before, but I think now we’re in the era of being able to collect large data sets. I think it would be really advantageous for our profession and our patients if we knew what were the best techniques, what were the risk factors for different types of dogs, what are the differences between within those techniques that make important differences and those sorts of granular details that can be really important to vets and to our clients and to their pets.
Pam Mosedale:
Yes, absolutely. I mean, I agree, having mean been in practice more than 40 years, the techniques seem to come and go, don’t they? So great to actually know what the better ones are. So how did collecting this data in the Cruciate Registry come about?
John Innes:
Yeah, well I guess we owe a lot to others in the profession for that. I think one of the instigators clinically was a friend and colleague of mine, Mark Morton, who’s a royal college specialist in orthopedics. And I remember Mark several years ago talking to me saying, “Do
you think there would be an appetite for a Cruciate Registry?” Because he knew that I’d been involved with a Total Hip Registry about 12 years ago or so. And I thought it was a great idea, but I thought it was a bit of a mountain to climb.
But fair play to Mark and then others that have come along to help and of course our RCVS knowledge for having the foresight and the energy to back this idea, because it is a big project and we’re looking to engage with any vet in the UK that’s performing cruciate surgery. So that’s a lot of people. And so the systems have to be very thorough and I think the team that have been put together by our RCVS knowledge, the expertise that’s come in from different disciplines has been great to see. I’ve enjoyed watching from the sidelines to some extent, but all credits to those involved, it’s great to see it up and running.
Pam Mosedale:
Because they have joint registries and these kind of things quite a lot in human medicine, don’t they?
John Innes:
Yeah, absolutely. I think I was looking into this a few weeks ago, and I think it was this, the Swedes actually who first came up with this concept in human medicine, and they actually made it compulsory for hospitals in Sweden to submit data, otherwise they wouldn’t get the compensation for the surgeries they performed. So it was linked to their income, so therefore very good engagement. And then it spread to the UK and the US. I think some insurers in the US were requiring registry data in the UK. I think now hospitals have to submit data for total knee and hip replacements and in Australasia and it’s spreading gradually around the world. And putting the patient at the centre of the clinical outcome is really important.
Pam Mosedale:
I think that’s really important and longer term clinical outcomes as well when it comes to our patients, not just what happens straight after the surgery.
John Innes:
And of course, I think if you look back in the relevant literature, most surgeons tend to publish their results some six to 12 months after the surgery, which is when actually for many conditions that’s you have the best outcome with. And I know from my own PhD that actually if you follow dogs long-term with cruciate rupture, you start to see the impact of the arthritis coming in several years down the line. So that’s another advantage of this registry. We’ll be able to look at the long-term natural history of what happens to these dogs after surgery.
Pam Mosedale:
That’ll be really useful when discussing these things with clients, won’t it?
John Innes:
Yeah. Absolutely. And of course it’s about expectations. It’s also about recognizing that a dog has a cruciate surgery and that’s not the end of the story. That dog very often has got many years of its life to live and that care has to pass from perhaps the surgeon to GP vets. And it’s understanding that whole journey I think will be really useful.
Pam Mosedale:
So how does it actually work then, the Cruciate Registry, and how do they actually assess the outcomes of the surgery?
John Innes:
The structure is any vet registered in the UK can become part of the registry. If you’re doing cruciate surgery, go onto the website and register. You can find that on our RCVS Knowledge website. And then when you are presented with a clinical case, you can discuss this briefly with the client to get their consent to participate. And then once you’ve done the surgery, you can register that particular case on the registry. It’s all done online. It’s very straightforward, very quick to do. And if you are a busy surgeon, you could get a delegate to do that, one of your admin team or a nurse to help you. And then the clients will automatically get validated questionnaires emailed to them at set intervals and then set intervals after the surgery. And these questionnaires are validated questionnaires that have been tested and validated previously.
Pam Mosedale:
So how are these outcomes validated that they use?
John Innes:
Well, briefly, Pam, I mean I could go on for hours about that, but I won’t. The validation of these questionnaires borrows from the science of psychology essentially, which is well-founded ways of doing this, but there are different types of validity that are tested. And then you have to test the reliability and the responsiveness and refine the questionnaire to the smallest number of questions, which gives you the best information. The two questionnaires that are used in the Cruciate Registry are the Liverpool Osteoarthritis in Dogs questionnaire, which I know very well, because I was the developer, but also the Canine Orthopedic Index, which is from the American College of Veterinary Surgeons, three or four papers on that have been published showing its validation. So putting those two questionnaires together gives us a bit of cross reference and cross validation.
Pam Mosedale:
So the owners fill in these questionnaires, but how is that data interpreted from there?
John Innes:
The great thing about the registry is that it’s all done automatically by the software platform that RCVS knowledge have invested in. So that the questionnaires will go automatically to the client. So the vets and nurses don’t have to worry about that once they’ve registered the case. The data then comes back into the data set into a dashboard, and I believe surgeons are able to see their own data and then they can also then benchmark their data against the anonymized data from elsewhere in the registry. So that’s a real advantage. It helps with clinical audit and the data from previous work on the different outcomes measures, what the different scores mean in terms of disease severity. The research you alluded to at the beginning of this conversation is about understanding the change in score a little bit more.
Pam Mosedale:
So that’s an interesting point you just made about clinical audit, because of course it’s now in RCVS practice standards at GP level. So anybody doing these surgeries in general practice, not just in a referral practice, can join in this, can’t they? And this data can be really useful for them, not only for their own benefit, but when it comes to things like their practice standards assessment too.
John Innes:
Yeah. Absolutely. It’s a great tool to make it easier to collect and analyse those data, because a lot of the work is done for you and it’s free of charge. I mean, what a great service from us.
Pam Mosedale:
Exactly.
John Innes:
Our RCVS knowledge. So why not get involved? Absolutely.
Pam Mosedale:
Exactly. We all love something free. And the paper that you’ve written, minimal clinically important differences for the load and the Canine Orthopedic Index outcome measures somebody who’s not an orthopedic, never been an orthopedic surgeon. And what does that mean?
John Innes:
Yeah, well, I don’t think it’s particularly an orthopedic thing and I have to learn about this myself a couple of years ago. So when you have these validated questionnaires, it’s important to understand what is the smallest change that a patient, or in our case, the proxy for the patient is the client considers to be meaningful? Because in these questionnaires, you inevitably will get some variance, some noise, little bit of error if you like, because there are human beings involved. And that’s how it always is.
So we have to understand where’s the threshold between that inherent error or noise and what’s actually the minimal clinically important difference? And so that’s what we’ve done in this study. And we’ve used data from the Cruciate Registry, which is a really valuable data set for us. And we’ve been able to show what is the change in score that clients consider to be clinically meaningful. Now, of course, in many cases, the dogs has had changed much more than the minimal difference, but we’re trying to understand where that threshold is and those estimates are useful to vets in terms of interpreting the change in the score.
Pam Mosedale:
So these are CROMS and are they client reported outcome measures? Is that right?
John Innes:
Yeah, I mean that’s an acronym that I use. And it’s really, because in human medicine where this is a whole industry now, they have the acronym of PROMS, which is Patient Reported Outcomes Measures. We can’t get our patients to respond in a meaningful way, unfortunately, but we use the client as a proxy. And interestingly, there are examples in human medicine where they have to do that too. And there are studies to show that’s a valid thing to do. So in areas such as pediatrics or in neuro disability where carer might have to respond on behalf of a patient, there are multiple studies showing that that’s a valid thing to do. And we’ve demonstrated that with clients with dogs before. And there’s some studies in cats too.
Pam Mosedale:
And it’s interesting, isn’t it? Because owner’s perception of what they think is a good result can vary massively according to whether it’s the couch potato Labrador or a working Collie or something like that, can’t it? So this is giving a way to actually measure that.
John Innes:
Yes, you’re absolutely right. You touched then on a point that’s important in that these questionnaires have to be answered by the same person each time. You can’t pass them around the family, because obviously different human beings have a different perspective on things. So it’s important to have the same person scoring each time. But yeah, you will get glass half empty and half full people. That’s part and parcel of measuring outcomes in this way. But that’s why it’s important to have thousands and thousands of subjects in such registry. So the great thing about this RCVS Knowledge Registry is that we’ve got the whole of the UK veterinary profession involved if they want to. And that should give us a huge data set in the long run.
Pam Mosedale:
And how are these MCIDs? Are they called minimally, clinically important differences? Actually estimated?
John Innes:
Well, I don’t want to bore-
Pam Mosedale:
Without being too technical.
John Innes:
Yeah. I don’t want to bore your listeners, Pam, here. I mean, because essentially it’s statistics and I know of when you mention that to veterinary students, for example, they’ll start to fall asleep. But there are different ways, and there’s no consensus on the perfect way to do this. And in the paper that we’ve published, which is open access, so people can go and read it if they want to. It’s in Plus One. Then we’ve used six different methods. They’re split into two categories, distribution based and anchor based, distribution based are purely looking at the way that the data are distributed and using techniques like receiver operated characteristic curve techniques like that.
The anchor based methods, which some argue are more clinically relevant, are based on analysing the client’s response to the anchor question, which essentially is a sort of fundamental question, how is your dog now compared to before surgery? Is it much better, slightly better, the same, slightly worse, and much worse? And then you use those categories to inform the MCID estimates. So it’s a little bit technical and it took a while for me to get my head around it, but I’m an orthopedic surgeon, so it would take a while to get my head around it.
Pam Mosedale:
But you said you touched on there on it being the second method being more clinically relevant. So what do you think how this work will be relevant to clinical practice?
John Innes:
Well, what we have done in the summary of the paper to try and give it some clinical context, although there are some caveats here, and I’ll come onto those, but we’ve tried to give some working numbers. So if you’re using load for example, we’re saying that a change in score of four points or more should be clinically meaningful. If you’re using the Canine Orthopedic Index, we’re saying change in 14 points should be clinically meaningful. There are different scales for both of those questionnaires, hence the different numbers. But further work is needed in different contexts. So this is in the context of cruciate surgery. It would be nice to
do these studies in other contexts as well, such as, let’s say arthritis or hip disease or elbow disease. But it’s a start and it gives people a little bit of confidence. If they’re seeing an individual clinical case and they’re using these CROMS in their clinic, then they can say, well, it’s moved by five points on load. I think that’s probably, it’s clinically meaningful. So if you’re using some sort of intervention, it’s giving you some confidence around the change in score.
Pam Mosedale:
Sounds very exciting for the future of these things can be developed to use in other areas too.
John Innes:
And they already are. So load is used in the Canine Hip Registry, which is a BVOA initiative. So us Brits should pat ourselves on the back a little bit, because we are sort of leading the way a little bit in veterinary medicine here. And what’s also coming down the track and our RCVS knowledge have led the way here as well, is partnering with companies that automate the collection of these data. So I mentioned that the questionnaires are going out automatically to the client, but then the data coming back is then automatically collated into a dashboard so that you can see it in summary, but you can also drill down into individual patients. And that automation will probably get better as these things start to integrate with practice management systems, et cetera. So I think things should get easier and easier for us as vets to look at our patients and how they’re doing.
Pam Mosedale:
I think that’s a huge advantage that it calculates all that automatically rather than somebody having to sit there, because they just wouldn’t do it, would they? That’s the trouble.
John Innes:
Well, we’re all short on time, aren’t we?
Pam Mosedale:
Yeah. Especially at the moment. Yeah, exactly.
John Innes:
Absolutely. Yeah.
Pam Mosedale:
So as the owner of a Labrador that did have a ruptured cruciate, which did have had a TPLO surgery and did fine, why is it important for owners then?
John Innes:
Well, I think owners, they’re not expecting vets to be miracle workers, but they are expecting vets to be professionals, and they’re expecting the profession as a whole to be doing its best. And I think with the availability of information these days and some of that information can be disinformation on the internet, I think it’s really important that the profession provides high quality evidence for what we’re doing. And also we can have realistic benchmarks. No surgeon gets a 100% success, no surgeon has no complications, but there are acceptable levels of success and acceptable levels of complication, and it’s good to know where they are. And initiatives, the registry will help us understand those issues, give good data to clients, set expectations. And I think it’s all to do with partnering with clients, isn’t it? And I think this is a step in that direction.
Pam Mosedale:
Yeah, I agree. I think the general public are more used to it now I’ve got to that age where some of might, not me thank goodness, but some of my friends have been having various joints replaced. And you can look up your surgeon, can’t you, and look at their on my NHS website and things. So I think the public, the clients are starting to expect it a little bit. And it must be so much, it’ll be so good from the prognosis point of view, won’t it?
John Innes:
Yeah, absolutely. You can look up your surgeon, but it is complicated in that obviously some surgeons are recognized within their field as being very good. And so people send them the really difficult cases. So you have to factor in lots of different variables there. But I think certainly sometimes people might look at their own data and say, oh, well maybe I need a little bit of training on something here, because my results don’t look as good as the average or something like that. So I think if you can learn from what others are doing, great for everyone, because surgery can be stressful and having good training is… I’ve certainly learned from colleagues younger than me that are doing things. I think, oh, that’s a great idea. And I’ve gone and learned from them. And I think it, we’ve all got to be open-minded about that.
Pam Mosedale:
And quality improvement, which is what the Cruciate Registry is part of, is all about that. It’s all about, no, just about improving outcomes and care, but about learning from each other and learning together as a team.
John Innes:
Absolutely.
Pam Mosedale:
So would you encourage vets to sign up for the Canine Cruciate Registry then if they’re doing cruciate surgery?
John Innes:
Yeah, I really would. I mean, I think projects like this, they’re not exposing you to any sort of risk. You’re participating, adding to the data set, it’s helping you audit your cases, keep track of them, helping you learn. And being part of a big data project like this can have a real sense of satisfaction. And you could say to your clients, I’m part of the Cruciate Registry. We’re monitoring things here, we’re professional about what we’re doing, and we want to have good standards. And I think that’s a good position to be in.
Pam Mosedale:
Absolutely. And you don’t have to be a referral orthopedic surgeon, do you, to submit your data?
John Innes:
No, absolutely not. We know that the majority of cruciates performed are not done by specialists. They’re done by perhaps people with an interest in orthopedics. And there’s absolutely nothing wrong with that. And people can be doing a great job doing that. So yeah, everyone that’s doing cruciate surgery can get involved. Whether it’s your first cruciate or your 10000th, doesn’t really matter.
Pam Mosedale:
Brilliant, thank you. Well, I think that’s our message, isn’t it? Just get involved and submit your data.
John Innes:
Absolutely.
Pam Mosedale:
That’s been so interesting. Thank you very much. That’s brilliant. Thanks for your time, John.
John Innes:
Thank you, Pam. Nice to speak to you as always.
RCVS Knowledge:
For further courses, examples, and templates for quality improvement, please visit our quality improvement pages on our website @ourcvsknowledge.org.