Latest / Deep Dive Ortho — Orthopaedic Surgery, AI & Digital Health Podcast. / Guide to Clinical Research Methods

Transcript

Guide to Clinical Research Methods · Deep Dive Ortho — Orthopaedic Surgery, AI & Digital Health Podcast.

• 0:00
  how can we really know if one treatment is actually
• 0:02
  better than another? Especially in orthopedics,
• 0:05
  it's not just guesswork, is it? No, absolutely
• 0:07
  not. It relies on, well, a really meticulous
• 0:11
  process of gathering solid evidence. And that's
• 0:13
  what we're diving into today, isn't it? The foundations,
• 0:16
  the methods behind clinical orthopedic research.
• 0:19
  Precisely. How we actually build that evidence
• 0:21
  base. We've got a stack of source material here,
• 0:24
  research papers, notes, and we're going to pull
• 0:26
  out the really crucial bits about how studies
• 0:28
  are designed, run, and importantly, how you can
• 0:32
  understand them. Which is key for anyone in the
• 0:35
  field, really, whether you're doing the research
• 0:37
  or using it to make clinical decisions. And I'm
• 0:39
  thrilled we have Professor Mo Imam with us to
• 0:41
  guide us through all this complexity. Welcome.
• 0:45
  Thank you. It's a pleasure. Thinking about how
• 0:47
  we know what we know, it's fundamental, isn't
• 0:50
  it? It gives you real insight into the evidence
• 0:52
  you're using every day. It really does. It's
• 0:54
  like knowing how the engine works. So over this
• 0:57
  deep dive, we'll cover the ethics, the absolute
• 1:00
  must do's. Non -negotiable. And then the practicalities
• 1:04
  of study design. Things like sample size, blinding,
• 1:08
  how do we measure. what actually matters to patients.
• 1:11
  Yes. Outcome measures are critical. And then
• 1:13
  understanding the data, the statistics, the different
• 1:16
  study types. Right. And we'll touch on the bigger
• 1:18
  picture, too. Things like multi -center trials,
• 1:20
  registries. Indeed. And even the economics of
• 1:23
  health care, which is becoming increasingly important.
• 1:25
  The whole ecosystem, really. Fantastic. A proc
• 1:28
  and deep dive. So let's start with a concept
• 1:31
  that's everywhere now. Evidence -based medicine.
• 1:35
  EBM. What does it actually mean? Where did it
• 1:37
  come from? OK. EBM. Well, at its heart, it's
• 1:41
  about using the best current evidence conscientiously,
• 1:45
  explicitly, you know, really thoughtfully when
• 1:47
  making decisions about individual patients. That
• 1:50
  definition from Gordon Geyette and the McMaster
• 1:52
  Group back in 91 really crystallized it. But
• 1:56
  it's not just about the research papers, is it?
• 1:58
  No, that's a common misconception. It's a three
• 2:00
  -legged stool, really. It's integrating that
• 2:02
  best external evidence with your own clinical
• 2:04
  expertise and crucially with the patient's values
• 2:07
  and preferences. All three are vital. That makes
• 2:09
  sense. A balance. Exactly. And while that definition
• 2:13
  is relatively recent, maybe late 20th century,
• 2:16
  the idea of looking systematically at outcomes.
• 2:19
  Well, that goes back much further. The source
• 2:21
  material mentions Dr. Ernest Codman. Ah yes,
• 2:24
  Codman. His story is quite something, isn't it?
• 2:27
  His end results idea. It really is. He was proposing,
• 2:31
  back in the early 1900s, that doctors should
• 2:33
  follow every patient to see the final outcome.
• 2:36
  Every single one. Every one. And if the result
• 2:39
  wasn't good, they needed to investigate why.
• 2:43
  Systematically tracking failures to learn from
• 2:45
  them. Sounds completely logical to us now. Why
• 2:48
  was that so radical then? Well, It flew in the
• 2:50
  face of the established practice, which was much
• 2:52
  more based on authority and tradition, perhaps
• 2:55
  anecdote, scrutinizing every failure publicly
• 2:58
  almost. It wasn't the done thing. So it caused
• 3:01
  friction. Significant friction. The source has
• 3:04
  mentioned it cost him his job at Massachusetts
• 3:05
  General Hospital. Wow. But his insistence on
• 3:08
  patient outcomes as the ultimate measure, well,
• 3:11
  that's now seen as a cornerstone, a really foundational
• 3:13
  step towards the empirical medicine we practice
• 3:15
  today. A difficult path for him, but crucial
• 3:18
  for progress. So we have Codman's early ideas,
• 3:21
  then the formal EBM definition. How did it become
• 3:25
  so widespread? It was a confluence of factors,
• 3:28
  really. You had the intellectual framework from
• 3:30
  Guyatt and others, but what really powered its...
• 3:33
  almost explosive integration was the simultaneous
• 3:37
  revolution in information technology. Ah, the
• 3:40
  rise of computers and databases. Exactly. Suddenly,
• 3:43
  you could search huge amounts of literature,
• 3:45
  manage large data sets, synthesize evidence in
• 3:48
  ways that were just impossible before. The technology
• 3:50
  made the EBM concept practically achievable on
• 3:53
  a grand scale. So the tech unlocked the potential
• 3:56
  of the idea. Precisely. It meant a scientific,
• 3:59
  data -driven approach could genuinely complement
• 4:02
  and sometimes challenge traditional ways of thinking
• 4:04
  in clinical practice. But before we even think
• 4:06
  about generating that data, there's the ethical
• 4:08
  foundation. It's mentioned as absolutely non
• 4:10
  -negotiable. Why is protecting research participants
• 4:13
  so critical, especially in orthopedics? It's
• 4:16
  paramount. Fundamentally, clinical research involves
• 4:19
  asking individuals to potentially accept some
• 4:21
  risk, whether physical, psychological, or to
• 4:24
  their privacy, for the benefit of others, for
• 4:27
  future patients. The ethical framework is there
• 4:29
  to ensure the rights, dignity and well -being
• 4:32
  of those participants. always come first. Oh.
• 4:35
  Always. We're guests of our patients, as you
• 4:38
  said. Absolutely. And overseeing this are institutional
• 4:40
  review boards, IRBs, or ethics committees. Yes,
• 4:44
  IRBs or IECs, depending on where you are. Their
• 4:46
  job is essentially to be the advocates for the
• 4:49
  research participants. They scrutinize the research
• 4:51
  plan, the protocol. To ensure. To ensure risks
• 4:54
  are minimized and justified by potential benefits,
• 4:58
  that patient selection is fair and, truthfully,
• 5:00
  that the informed consent process is robust.
• 5:03
  and truly allows for an informed choice. Their
• 5:06
  mandate, particularly in the U .S., comes from
• 5:08
  principles like those in the Belmont report,
• 5:10
  formalized in regulations like the Common Rule.
• 5:12
  They're gatekeepers, ensuring research is conducted
• 5:15
  ethically. Does every single study need IRB approval
• 5:17
  then, or are there exceptions? Not absolutely
• 5:20
  everything. The sources point out that things
• 5:23
  like internal quality improvement projects, perhaps
• 5:26
  some types of case reports focusing on just one
• 5:28
  or two patients. These might not always meet
• 5:31
  the formal definition of research designed to
• 5:34
  create generalizable knowledge. So they might
• 5:36
  not need formal review. They might not. But,
• 5:39
  and this is a critical piece of advice, if there
• 5:41
  is any doubt at all, you absolutely must consult
• 5:43
  your IRB or ethics committee before you start
• 5:46
  anything. It protects everyone involved. Better
• 5:48
  to ask than to make assumptions. Sound advice.
• 5:52
  And a cornerstone of that protection is informed
• 5:55
  consent. It's called a fundamental pillar. What
• 5:58
  does informed really mean here? It's about genuine
• 6:01
  understanding and voluntary choice. It's not
• 6:03
  just giving someone a leaflet. You have to ensure
• 6:06
  the potential participant truly grasps the study's
• 6:09
  purpose, what procedures are involved, the potential
• 6:12
  risks and benefits, what alternatives exist,
• 6:16
  and critically, that taking part is entirely
• 6:19
  voluntary and they can withdraw at any point
• 6:21
  without any penalty, without affecting their
• 6:24
  normal care. They need to be able to explain
• 6:26
  it back to you in a way. It's an ongoing conversation,
• 6:29
  not just a signature on a form. That sounds like
• 6:31
  it could be tricky, especially with certain patient
• 6:34
  groups. It absolutely can be. The sources highlight
• 6:37
  the need for extra care with vulnerable populations
• 6:40
  of children, people with cognitive impairments,
• 6:43
  perhaps even situations where there's a power
• 6:45
  imbalance, like if the researcher is also the
• 6:47
  treating clinician. So how do you manage that?
• 6:49
  Well, for those lacking capacity, you need proxy
• 6:53
  consent from a legal representative. But even
• 6:55
  then, the ethical obligation is to respect the
• 6:58
  individual's potential views and best interests
• 7:01
  as much as possible. It requires real sensitivity
• 7:04
  and ensuring the process isn't coercive in any
• 7:06
  way. Respecting autonomy seems key. Now, once
• 7:09
  someone's in a study, what about their data?
• 7:12
  Protecting confidentiality? Hugely important.
• 7:15
  Data protection regulations like EPA in the U
• 7:17
  .S. or GDPR in Europe are very strict. In practice,
• 7:21
  it means using study codes instead of names on
• 7:23
  data forms. keeping records secure, ensuring
• 7:26
  data is accurate and legible. And the consequences
• 7:29
  if things go wrong? They can be severe, as the
• 7:31
  source material makes clear. Data breaches can
• 7:34
  lead to reputational damage, research being halted,
• 7:37
  significant fines, and potentially even legal
• 7:39
  action against researchers. Trust is incredibly
• 7:42
  hard to regain once lost. A heavy responsibility.
• 7:46
  Beyond patient data, there's the issue of conflicts
• 7:48
  of interest, COI. How is that defined in a research
• 7:51
  context? The Institute of Medicine definition
• 7:53
  is widely used. It's basically when circumstances
• 7:57
  create a risk that a researcher's professional
• 7:59
  judgment about a primary interest, like patient
• 8:01
  welfare or research integrity, might be unduly
• 8:04
  influenced by a secondary interest. Like financial
• 8:06
  gain. That's the most common one, yes. especially
• 8:09
  with industry funding. You often have the researcher,
• 8:12
  their institution, and the company providing
• 8:13
  funding. Their interests might not always align
• 8:16
  perfectly. An ethical issue arises if financial
• 8:19
  ties could potentially bias how the research
• 8:22
  is conducted, analyzed, or reported. And industry
• 8:25
  funding is pretty significant, isn't it? It is.
• 8:27
  The source notes figures like 71 % of R &D funding
• 8:31
  in the U .S. coming from industry. And there's
• 8:33
  documented evidence, quite a lot of it, actually
• 8:35
  showing that industry -sponsored trials are statistically
• 8:39
  more likely to report results favorable to the
• 8:42
  sponsor's product. That doesn't necessarily mean
• 8:43
  the research is flawed though. Not necessarily,
• 8:46
  no, but it highlights a potential for bias that
• 8:49
  absolutely must be managed transparently. How
• 8:51
  do researchers navigate that? Are there guidelines?
• 8:54
  Yes. Professional bodies like the AAOS provide
• 8:57
  guidance. For example, suggesting researchers
• 8:59
  shouldn't trade stock in the funding company
• 9:01
  during the project. Or, if a surgeon has designed
• 9:04
  an implant and receives royalties, research on
• 9:06
  that specific implant should ideally be done
• 9:08
  by someone completely independent. What about
• 9:11
  consulting fees? Legitimate consulting is fine,
• 9:14
  provided the compensation is fair market value
• 9:16
  for the work done and, crucially, It's fully
• 9:19
  disclosed. Transparency is key. Disclosure and
• 9:22
  management, not necessarily prohibition. Are
• 9:26
  there system -level responses to COI concerns?
• 9:28
  Yes. A major one is mandatory clinical trial
• 9:31
  registration. Public databases like clinicaltrials
• 9:34
  .gov, ISRCTN, or the WHO registries, they require
• 9:39
  researchers to outline their study design, their
• 9:42
  methods, their primary outcome measures before
• 9:45
  they start recruiting anyone. How does registering
• 9:47
  beforehand help? It locks the plan in publicly.
• 9:50
  It makes it much harder to engage in things like
• 9:52
  publication bias, where only positive results
• 9:54
  get published, or selective outcome reporting,
• 9:56
  where maybe you change your main outcome measure
• 9:58
  after you've seen the data because another one
• 10:00
  looks better. So it increases transparency. Hugely.
• 10:03
  It holds researchers accountable to their original
• 10:05
  plan. Many major journals following the ICMJE
• 10:09
  guidelines now won't even consider publishing
• 10:11
  a trial unless it was registered beforehand.
• 10:13
  A powerful incentive for good practice. Right,
• 10:18
  so ethical foundations are set. Let's move to
• 10:20
  designing the study itself. It all starts with
• 10:23
  a research question, doesn't it? Absolutely.
• 10:25
  A clear, focused question is the starting point.
• 10:27
  It might come from clinical observation, spotting
• 10:29
  a gap in the literature, wanting to follow up
• 10:32
  on earlier work. But how do you know if it's
• 10:34
  a good question? One worth pursuing. There's
• 10:37
  a useful framework called the finer criteria.
• 10:40
  Is the question feasible? Can you actually do
• 10:41
  it? Interesting? Will people care? Novel? Does
• 10:44
  it add something new? Ethical? Can it be done
• 10:47
  ethically? And relevant? Does it matter to practice
• 10:50
  or policy? Finer. That's neat. Feasible, interesting,
• 10:55
  novel, ethical, relevant. Exactly. It helps you
• 10:57
  refine your initial idea into something concrete
• 10:59
  and justifiable. Once you have that finer question,
• 11:02
  how do you structure it for a study plan? The
• 11:04
  PICOT format is incredibly useful here, especially
• 11:07
  for clinical questions. Population, who are the
• 11:09
  patients? Intervention, what's being tested?
• 11:12
  Comparator, what's it being compared against?
• 11:14
  Outcome, what are you measuring? And timeframe,
• 11:17
  over what period? PICOT, population, intervention,
• 11:20
  comparator, outcome, timeframe. Using PICOT clarifies
• 11:24
  exactly what the study is about. It makes communication
• 11:26
  much clearer. So PICOT defines the question.
• 11:29
  Then comes the study protocol. The master plan.
• 11:32
  Precisely. The protocol lays out everything in
• 11:35
  detail. The background, the objectives, the exact
• 11:38
  methods for recruitment, intervention, data collection,
• 11:41
  the statistical analysis plan, ethical considerations,
• 11:44
  timelines, everything. And the key point the
• 11:47
  source makes is this must be finalized before
• 11:50
  you recruit the first participant. Why is that
• 11:53
  timing so critical? It prevents people from changing
• 11:55
  the rules halfway through the game. essentially.
• 11:58
  If the methods or analysis plan aren't fixed
• 12:00
  beforehand, there's a risk, conscious or unconscious,
• 12:04
  of making changes based on early results, which
• 12:06
  can introduce bias. It ensures the study is conducted
• 12:09
  systematically as planned. And a crucial part
• 12:11
  of that plan is sample size calculation, getting
• 12:13
  the numbers right. Yes, vital for statistical
• 12:15
  power. Power is the study's ability to detect
• 12:18
  a true difference or effect if one actually exists.
• 12:21
  And if the sample size is too small, then the
• 12:23
  study is underpowered. You risk a type 2 error,
• 12:26
  a false negative. You might conclude there's
• 12:28
  no difference between treatments when, in reality,
• 12:31
  there is one. But your study just wasn't big
• 12:34
  enough to find it. Is that a common problem in
• 12:36
  orthopedics? Being underpowered? The source suggests
• 12:39
  it might be, yeah. Estimates suggest maybe up
• 12:42
  to half of orthopedic RCTs could be underpowered.
• 12:45
  Which makes interpreting those negative trials,
• 12:48
  the ones finding no difference, really tricky.
• 12:51
  Because you don't know if it's truly no difference
• 12:54
  or just missed. Exactly. And remember, the power
• 12:57
  calculation itself is based on estimate assumptions
• 13:00
  about how big the effect will be and how much
• 13:02
  variability there is. If those assumptions are
• 13:05
  off, your actual power might be lower than planned.
• 13:07
  So there's always some uncertainty. Do researchers
• 13:10
  build in a buffer? Yes, it's pretty standard
• 13:13
  practice to inflate the calculated sample size
• 13:16
  by, say, 10 or 20 percent just to account for
• 13:19
  people dropping out or maybe not recruiting quite
• 13:22
  as many as hoped. It's a pragmatic way to safeguard
• 13:24
  the power. Makes sense. Alongside sample size,
• 13:28
  selecting the right patients is obviously key.
• 13:30
  And then in trials, randomization comes in. Right.
• 13:33
  Appropriate patient selection defines who the
• 13:35
  results apply to. Randomization in RCTs is the
• 13:39
  best tool we have to minimize selection bias
• 13:41
  and confounding by assigning patients to groups
• 13:44
  purely by chance. You hope to. You hope to make
• 13:47
  the groups as similar as possible at the start
• 13:49
  in terms of both things you know about, like
• 13:51
  age, severity, and things you don't. So any difference
• 13:55
  you see at the end is more likely due to the
• 13:57
  treatment itself. not baseline differences between
• 14:00
  the groups. So randomization tackles bias head
• 14:02
  -on. It's crucial for tackling selection bias
• 14:05
  and confounding in RCTs, yes. Bias can creep
• 14:08
  in anywhere, though design, data collection,
• 14:10
  analysis. Selection bias is a big one, where
• 14:13
  the groups start off different in a systematic
• 14:14
  way. That's why historical controls are generally
• 14:17
  a bad idea comparing today's patients to yesterday's.
• 14:20
  Too many other things have changed. Okay. Now,
• 14:22
  the sources make a point about allocation concealment
• 14:24
  being distinct from randomization. What's that
• 14:27
  about? Ah, yes, a really important distinction.
• 14:31
  Randomization generates the random sequence who
• 14:34
  should go into which group. Allocation concealment
• 14:37
  is the practical step of shielding that sequence
• 14:39
  from the people enrolling participants. It ensures
• 14:43
  they don't know which group the next eligible
• 14:45
  patient will be assigned to before they decide
• 14:47
  to enroll them. Why does that matter if the sequence
• 14:50
  is already random? Because if the recruiter knows
• 14:52
  the next assignment, say the next one gets the
• 14:55
  new drug, they might consciously or subconsciously
• 14:58
  steer certain types of patients towards or away
• 15:00
  from that next slot. Maybe hold back a sicker
• 15:03
  patient if the good treatment is next, or rush
• 15:05
  to enroll someone if the placebo is next. It
• 15:08
  undermines the randomization. So knowing the
• 15:10
  next step can reintroduce bias. Exactly. And
• 15:14
  the source notes that older methods, like sealed
• 15:16
  envelopes, weren't foolproof. They could sometimes
• 15:19
  be predicted or tampered with, and studies using
• 15:21
  them poorly tend to overestimate treatment effects.
• 15:24
  So what's the gold standard for allocation concealment
• 15:26
  now? Centralized remote methods. Using a secure
• 15:30
  telephone service or an online system, managed
• 15:33
  independently from the trial site. The person
• 15:36
  enrolling the patient contacts the central service
• 15:38
  after confirming eligibility and only then are
• 15:41
  they told the group assignment. It completely
• 15:43
  removes the possibility of influencing the allocation.
• 15:46
  That makes total sense. Keeping it separate keeps
• 15:48
  it clean. Another bias -reducing technique is
• 15:51
  blinding or masking. Who gets blinded? Ideally,
• 15:55
  as many people involved as possible. Blinding
• 15:57
  prevents knowledge of the treatment allocation
• 15:59
  from influencing behavior or assessments. So
• 16:02
  you might blind the participants, subjects, the
• 16:04
  clinicians delivering the care, the people assessing
• 16:06
  outcomes, and the data analysts. But in orthopedics,
• 16:10
  blinding the surgeon seems difficult. Often impossible,
• 16:14
  yes. The surgeon usually knows exactly what procedure
• 16:16
  they performed. That's a major challenge in surgical
• 16:18
  trials. So what can you do? You focus on blinding
• 16:21
  everyone else. Patients can sometimes be blinded,
• 16:24
  perhaps with a sham procedure, though that raises
• 16:26
  its own ethical points. Outcome assessors can
• 16:29
  definitely be blinded. Have someone not involved
• 16:31
  in the surgery or care collect the follow -up
• 16:33
  data, especially PROMs. You can mask scars on
• 16:37
  x -rays, for example. And blinding the data analysts.
• 16:40
  You mentioned that earlier. Yes, that's usually
• 16:41
  the easiest part. Just give them the data labeled
• 16:43
  group A and group B. Yet the source... points
• 16:47
  out, it hasn't always been well reported in the
• 16:49
  past. It's a simple step to prevent bias in how
• 16:52
  the data is handled. All these techniques, randomization,
• 16:55
  blinding, sham procedures, link back to an ethical
• 16:57
  principle. Clinical equipoise. What is that exactly?
• 17:01
  Equipoise is the ethical justification for doing
• 17:04
  an RCT in the first place. It means there must
• 17:06
  be genuine uncertainty within the expert medical
• 17:09
  community about which treatment arm in the trial
• 17:11
  is actually better. It's not about the individual
• 17:15
  researcher's belief, but a state of collective
• 17:17
  uncertainty. So if everyone already knows one
• 17:20
  treatment is superior, then it's unethical to
• 17:22
  randomize. You shouldn't knowingly assign patients
• 17:25
  to what is believed to be an inferior treatment.
• 17:28
  Equipoise insurer's randomization only happens
• 17:31
  when there's a real question about the best approach,
• 17:33
  making both options ethically acceptable within
• 17:36
  the trial context. It's particularly relevant
• 17:39
  when considering sham surgery. You need genuine
• 17:41
  uncertainty about the active procedures benefit
• 17:44
  over the sham. Randomize only when you truly
• 17:46
  don't know. Okay, so we've designed the study
• 17:49
  carefully. Now we need to measure the results.
• 17:51
  The science of outcome measurement. Absolutely
• 17:53
  critical. Outcome measurement is how we quantify
• 17:56
  the effect of the treatment. Is the patient better?
• 17:59
  How much better? It's essential for showing value,
• 18:01
  comparing care, seeing if evidence -based practice
• 18:04
  actually works. The WHO defines it as the change
• 18:07
  in health due to an intervention. And how do
• 18:09
  we measure these outcomes? What are the main
• 18:11
  types? Broadly, two categories. Objective or
• 18:14
  observer -reported outcome measures OROMs. These
• 18:17
  are things measured by a clinician or a device.
• 18:19
  Range of motion, strength, maybe using an instrument
• 18:21
  like the KT1000 for the knee laxity, functional
• 18:24
  tests like hop tests. And the second type? Subjective.
• 18:27
  or patient reported outcome measures, PROMs.
• 18:30
  These come directly from the patient, usually
• 18:32
  via questionnaires, about their pain, function,
• 18:34
  quality of life, symptoms, their perspective.
• 18:37
  What are the limitations of the objective OROMs?
• 18:40
  Well, instrumented measures can be precise, but
• 18:42
  also expensive or complex. Functional tests try
• 18:45
  to mimic real life, but can be affected by patient
• 18:48
  effort or technique, sometimes making them less
• 18:51
  reliable. And crucially, an objective measure
• 18:53
  might not capture what the patient actually cares
• 18:55
  about. The KT -1000 example is good. It measures
• 18:59
  forward, backward, and knee laxity well after
• 19:01
  ACL surgery, but not rotational stability, which
• 19:04
  might be the patient's main problem. Which leads
• 19:06
  us to PROMs, patient -reported outcomes. They
• 19:10
  seem increasingly important. Hugely important
• 19:12
  because they capture the patient's experience.
• 19:14
  How do they feel? How well can they function
• 19:17
  in daily life? It's about patient -centered care.
• 19:19
  What does treatment success look like from their
• 19:21
  point of view? Are all PROMs the same? No, there
• 19:23
  are different types. Generic ones like the SF
• 19:25
  -36 or SF -12 measure overall health. They're
• 19:29
  good for comparing across different diseases
• 19:31
  or picking up broad effects, but they might not
• 19:33
  be very sensitive to changes in a specific joint
• 19:36
  problem. So you also have specific ones? Yes,
• 19:39
  condition -specific or region -specific PROMs.
• 19:42
  Things like the COS for knees, the WOOS for shoulders,
• 19:45
  the Harris hip score. These are designed to be
• 19:47
  really sensitive to changes related to that specific
• 19:50
  condition. They're generally more responsive.
• 19:53
  The choice depends on what you're studying. Choosing
• 19:55
  the right measure is fruitful then. What makes
• 19:57
  a PROM or any outcome measure a good measure?
• 19:59
  What properties does it need? Three key psychometric
• 20:03
  properties. First is reliability. Is the measure
• 20:06
  consistent? If you measure the same thing twice,
• 20:08
  do you get the same result? Assuming nothing
• 20:09
  has changed. You need reliability to trust that
• 20:13
  any change you see is real, not just measurement
• 20:15
  noise. Different kinds of reliability. Yes, test
• 20:18
  or retest reliability. Interrater between different
• 20:21
  assessors. Interrater, same assessor. Internal
• 20:23
  consistency. Do items on a scale hang together?
• 20:26
  We quantify it with stats like ICC or Cronbach's
• 20:29
  Alpha. For trials, you often want reliability
• 20:32
  above .9. Reliability is consistency. What's
• 20:35
  next? Validity. Does the measure actually measure
• 20:39
  what it claims to measure? Does your knee score
• 20:41
  truly reflect knee function? There's criterion
• 20:44
  validity. compared to a gold standard, construct
• 20:47
  validity does it behave as theory predicts, discriminant
• 20:50
  validity does it not correlate with unrelated
• 20:52
  things. Reliability, validity, and the third.
• 20:56
  Responsiveness. Can the measure detect change
• 20:58
  when it actually occurs, especially clinically
• 21:01
  meaningful change? A measure could be reliable
• 21:03
  and valid, but if it's too blunt to pick up improvements,
• 21:07
  it's not much use in a trial. Right. Reliability,
• 21:09
  ability, responsiveness. So you've got your scores
• 21:11
  changing. How do you know if the change matters?
• 21:14
  Statistical significance isn't everything, you
• 21:16
  said. Exactly. It's where MCID and MDC come in.
• 21:18
  Hugely important concepts. MCID. MCID is the
• 21:22
  minimally clinically important difference. It's
• 21:24
  the smallest change in score that patients actually
• 21:27
  perceive as beneficial or important. The smallest
• 21:30
  change that makes a real difference to them.
• 21:33
  MDC is the minimum detectable change, the smallest
• 21:35
  change that's statistically likely to be real,
• 21:38
  not just measurement error. So MCID is about
• 21:40
  patient perception, MDC is about measurement
• 21:43
  precision. You've got it. And ideally, for a
• 21:46
  change to be truly meaningful, it needs to be
• 21:48
  statistically significant and exceed the MCID.
• 21:51
  The source gives a great example. A trial finds
• 21:53
  a four -point difference on the IKDC score, P
• 21:56
  .05, statistically significant. But if the MCID
• 21:59
  for the IKDC is known to be, say, 11 .5 points,
• 22:03
  that four -point difference is statistically
• 22:04
  real but probably doesn't feel important to the
• 22:06
  patient. That really clarifies the difference
• 22:08
  between statistical noise and meaningful improvement.
• 22:11
  Is there another patient -focused concept emerging?
• 22:13
  Yes. The patient -acceptable symptom state, or
• 22:16
  PASS. It's usually a single question. Is your
• 22:19
  current state acceptable to you? It's a threshold.
• 22:21
  Are you feeling well or satisfactory, regardless
• 22:24
  of how much you've changed? It complements change
• 22:27
  scores by asking about the absolute state achieved.
• 22:30
  Interesting. A different way of looking at success.
• 22:32
  Okay, so we've collected the outcome data. Now
• 22:34
  we need to analyze it. Statistics. What are the
• 22:38
  basics we need to grasp? First, know your data
• 22:41
  type. Is it continuous, like age or height? Or
• 22:45
  categorical, like yes, no, or mild, moderate,
• 22:47
  severe? That dictates the stats you can use.
• 22:49
  Then you use descriptive statistics to summarize
• 22:52
  your sample frequencies, mean, median, mode for
• 22:55
  the central point, standard deviation, or range
• 22:57
  for the spread. It gives you a picture of who
• 22:59
  was in the study. And then for comparing groups
• 23:00
  or looking for relationships. You choose inferential
• 23:03
  statistical tests based on your question and
• 23:05
  data type, comparing two group means, maybe a
• 23:07
  t -test, three or more groups, ANOVA, looking
• 23:11
  for an association, correlation, predicting an
• 23:13
  outcome, regression. The test must match the
• 23:16
  question. When we get the results, the p -values,
• 23:18
  the confidence intervals, how do we interpret
• 23:21
  them beyond just p .05? Right, the p -value just
• 23:24
  tells you the probability of seeing your result
• 23:26
  if there was truly no effect. Less than .05 usually
• 23:30
  means statistically significant, unlikely due
• 23:33
  to chance alone. But the confidence interval,
• 23:36
  the CI, is often more informative. Why the CI?
• 23:39
  It gives you a range of plausible values for
• 23:41
  the true effect in the population. A 95 % CI
• 23:44
  means we're 95 % confident the true value lies
• 23:47
  within that range. It tells you about the precision.
• 23:50
  Narrow CI means more precise estimate. Wide CI
• 23:53
  means less precise. And crucially, you can compare
• 23:55
  the CI to the MCID. Does the entire range of
• 23:58
  the CI fall below the MCID? Or does it comfortably
• 24:01
  cross it? That gives you a much better sense
• 24:04
  of clinical significance than the p -value alone.
• 24:06
  The CI gives magnitude and precision. Beyond
• 24:09
  the numbers, presenting data visually. Tuft's
• 24:12
  principles. Yes, Edward Tuft's ideas about graphical
• 24:14
  excellence and visual integrity are key. Show
• 24:16
  the data clearly. Avoid clutter, chart junk.
• 24:19
  Make sure the graphics accurately represent the
• 24:21
  numbers without distortion. Maximize the data
• 24:23
  ink ratio, essentially. And the Cleveland and
• 24:26
  McGill hierarchy. What's that telling us? It's
• 24:29
  about how accurately we perceive different visual
• 24:32
  elements. We're best at judging position along
• 24:34
  a common scale, like bar charts. Less good with
• 24:38
  angles or areas, like pie charts. So it guides
• 24:41
  choices. Bar charts are often better than pie
• 24:43
  charts for comparing proportions because we judge
• 24:45
  length more accurately than angle or area. Practical
• 24:48
  tips for figures and papers, then? Make them
• 24:51
  stand alone. The legend should explain everything
• 24:54
  needed to understand the figure without going
• 24:56
  back to the text. Define abbreviations. Keep
• 24:59
  it clean. Focus on the data. Let the substance
• 25:01
  shine through, not fancy graphics. Brilliant.
• 25:04
  Okay, we've looked deep into individual studies.
• 25:06
  But knowledge builds up over time. How do we
• 25:08
  handle multiple studies? Different designs? The
• 25:11
  hierarchy of evidence. Right. The hierarchy concept
• 25:14
  ranks study designs based on how well they protect
• 25:16
  against bias, especially when looking at cause
• 25:19
  and effect. RCTs are generally at the top for
• 25:22
  intervention questions because randomization
• 25:24
  minimizes bias. Below that, you have observational
• 25:28
  studies. Expert consensus often shapes these
• 25:31
  hierarchies and reporting guidelines like consort
• 25:33
  for RCTs. RCTs are the gold standard for interventions
• 25:36
  then? Generally, yes. for asking, does treatment
• 25:40
  X work better than treatment Y? But they have
• 25:43
  drawbacks, as we discussed. They do. Not always
• 25:45
  possible or ethical. Strict criteria can limit
• 25:48
  generalizability. How will the results apply
• 25:50
  to typical patients? And the underpowering issue
• 25:53
  means we have to be careful interpreting no -difference
• 25:55
  findings. So observational studies have a key
• 25:58
  role, too. Cohorts, case controls. Absolutely
• 26:01
  essential. When RCTs aren't feasible, they provide
• 26:04
  vital information, especially on prognosis, risk
• 26:06
  factors, real -world effectiveness. They often
• 26:09
  generate the hypotheses that RCTs then go on
• 26:12
  to test. They complement each other. You mentioned
• 26:14
  case control studies are research in reverse.
• 26:16
  Yes. That's a good way to think about it. You
• 26:19
  start with people who have the outcome, cases,
• 26:21
  and people who don't, controls. And then you
• 26:23
  look backwards in time to see if exposure to
• 26:25
  potential risk factors differed between the groups.
• 26:27
  and case reports or series. Lowest level of evidence,
• 26:31
  but still useful. Level V, yes. Not great for
• 26:35
  proving cause and effect, but invaluable for
• 26:37
  reporting something new. A rare side effect,
• 26:40
  a novel surgical technique, an unusual presentation.
• 26:44
  They alert the community, spark ideas, generate
• 26:47
  hypotheses for better studies later on. Often
• 26:49
  the very first signal of something important.
• 26:52
  So a whole ecosystem of study designs, how do
• 26:54
  we bring all this disparate evidence together?
• 26:56
  Systematic reviews. Exactly. That's the next
• 26:59
  level up, synthesizing primary studies. What's
• 27:01
  the difference between a quick narrative review
• 27:03
  and a proper systematic review? A narrative review
• 27:06
  is more like an expert summary. They pick key
• 27:09
  papers, give their perspective. It's useful,
• 27:12
  but can be subjective. A systematic review is
• 27:15
  a research project in itself. It uses explicit
• 27:18
  predefined methods to find all relevant studies
• 27:20
  on a specific question, assess their quality
• 27:23
  using tools like Cochrane's risk of bias or minors,
• 27:26
  and then synthesize the findings transparently.
• 27:30
  It aims to be objective and reproducible. And
• 27:32
  a meta -analysis fits in where? A meta -analysis
• 27:35
  is the statistical part you can sometimes do
• 27:37
  within a systematic review. If the included studies
• 27:40
  are similar enough in patients, interventions,
• 27:43
  outcomes, you can pool their numerical results
• 27:45
  together to get a single overall estimate of
• 27:48
  the effect. This pooled result is usually more
• 27:50
  precise than any individual study. Often shown
• 27:53
  in those forest plots. Yes, the forest plot is
• 27:55
  the classic way to visualize a meta -analysis.
• 27:57
  Each study gets a line showing its result and
• 28:00
  confidence interval, and a diamond at the bottom
• 28:02
  shows the pooled result. It lets you see the
• 28:04
  individual studies and the overall picture at
• 28:06
  a glance. But what if the studies being pooled
• 28:08
  are really different? Heterogeneity. Yes, that's
• 28:11
  a major issue. Heterogeneity means the studies
• 28:14
  aren't all measuring the same underlying effect.
• 28:16
  It could be due to differences in patients, clinical
• 28:19
  study methods, methodological, or just more variation
• 28:23
  in results than you'd expect by chance. Statistical.
• 28:26
  We measure statistical heterogeneity with things
• 28:28
  like the I squared statistic. And if I squared
• 28:30
  is high? High heterogeneity means you have to
• 28:33
  be very cautious about pooling the results. The
• 28:36
  average might not mean much if the studies are
• 28:38
  too dissimilar. You might explore why they differ
• 28:40
  using subgroup analyses, but the source rightly
• 28:43
  warns against just chucking out studies to make
• 28:45
  heterogeneity go down that can introduce its
• 28:48
  own bias. It requires careful judgment. Synthesis
• 28:51
  is complex, too. Okay, shifting gears slightly
• 28:54
  to practicalities. Multi -center trials needed
• 28:56
  for big numbers, but sound like a nightmare to
• 28:59
  run. They are incredibly challenging, yes. The
• 29:02
  benefits are clear bigger samples, more generalizable
• 29:04
  results, but the logistics. getting ethics approval
• 29:08
  from every single site. The source mentions the
• 29:10
  Jupiter trial experience, how delays can mount
• 29:13
  up. Central IRBs might help streamline that.
• 29:15
  What else makes them tough? Standardizing everything.
• 29:18
  making sure every site does the procedure the
• 29:20
  same way, measures outcomes identically, fills
• 29:23
  in the forms correctly. Communication is absolutely
• 29:26
  key, keeping everyone connected and motivated
• 29:28
  across different hospitals or even countries.
• 29:31
  Lessons from trials like PIVL highlight the need
• 29:34
  for constant contact, maybe face -to -face meetings.
• 29:37
  And agreeing on authorship early on is crucial
• 29:40
  when you have so many contributors. A massive
• 29:42
  coordination effort. What about registries? How
• 29:44
  do they fit in? Registries collect data on routine
• 29:47
  practice. They track outcomes for large numbers
• 29:50
  of patients undergoing certain procedures like
• 29:52
  joint replacements or ACL reconstructions in
• 29:54
  the real world. They're brilliant for long -term
• 29:57
  follow -up, spotting rare problems, and seeing
• 29:59
  how things work outside the controlled setting
• 30:01
  of an RCT. The Norwegian knee ligament registry
• 30:04
  is a great example. Do registries have weaknesses?
• 30:07
  Data quality can vary. Completeness might depend
• 30:10
  on whether reporting is mandatory. Local resources.
• 30:12
  And comparing outcomes between centers using
• 30:15
  registry data needs careful statistical adjustment
• 30:18
  because the patients might be very different.
• 30:20
  Case mix. Finally, economic studies. Why the
• 30:23
  focus on cost effectiveness now? Because healthcare
• 30:25
  resources aren't infinite. Costs are rising.
• 30:29
  And there's a push toward value -based care,
• 30:31
  getting the best outcomes for the money spent.
• 30:33
  So we need to evaluate not just if a treatment
• 30:36
  works, but if it's worth the cost compared to
• 30:39
  alternatives. The value equation. outcome divided
• 30:42
  by cost. How do you approach that? You need to
• 30:44
  measure both the costs, surgery, hospital stay,
• 30:47
  rehab, complications, and the outcomes. Clinical
• 30:50
  scores, quality of life. But the crucial thing
• 30:52
  the source highlights is defining the perspective.
• 30:55
  Are you looking at costs to the patient, the
• 30:57
  hospital, the NHS or insurer? Society as a whole.
• 31:01
  Why does the perspective matter so much? Because
• 31:04
  it changes what costs and benefits you include.
• 31:06
  A societal perspective includes lost productivity,
• 31:09
  which a hospital perspective wouldn't, the best
• 31:11
  or most cost -effective option can look very
• 31:13
  different depending on whose costs and benefits
• 31:15
  you count. The example in the source about different
• 31:18
  bone -cutting tools makes this really clear.
• 31:19
  The cheapest tool isn't necessarily the fastest
• 31:22
  or most accurate. You need clear criteria. A
• 31:25
  really important nuance we've covered so much.
• 31:28
  From EBM origins, ethics, study design, outcomes,
• 31:32
  stats, synthesis, practicalities, how do researchers
• 31:35
  share all this work? Dissemination is key. usually
• 31:38
  through peer -reviewed publications, scientific
• 31:41
  papers, and presentations at conferences like
• 31:43
  posters or talks. Tips for writing a good paper.
• 31:46
  Follow the standard IMRAD structure. Intro, methods,
• 31:49
  results, discussion. Avoid common pitfalls like
• 31:52
  trying to include absolutely everything, repeating
• 31:55
  yourself, or overstating your conclusions. Don't
• 31:58
  claim causation if your design doesn't support
• 32:00
  it. Outline first, work with co -authors, manage
• 32:03
  references properly, create clear figures and
• 32:06
  legends, and discuss authorship early. Abstracts
• 32:09
  and posters. Abstracts are the concise, pitch
• 32:11
  -grab attention, convey the key message. Posters
• 32:14
  need to be visually clear, tell the story graphically,
• 32:17
  but have enough detail for someone to understand
• 32:18
  the basics, even if you're not standing right
• 32:20
  there. Good formatting, clear fonts, and always
• 32:23
  disclose conflicts of interest. And presentations.
• 32:25
  Tell a story. The discussion is vital. Interpret
• 32:28
  your findings, place them in context, acknowledge
• 32:30
  limitations, be ready for the Q &A. Lastly, scientific
• 32:34
  meetings. More than just listening to talks.
• 32:36
  Oh, much more. They're vital for learning, seeing
• 32:39
  unpublished work, refining your thinking, prepare
• 32:41
  beforehand, check the program, abstracts. But
• 32:45
  the networking, that's equally crucial. Networking.
• 32:49
  Yes. connecting with peers, senior figures, potential
• 32:51
  mentors. Especially for junior people, the source
• 32:54
  emphasizes this. You build collaborations, get
• 32:57
  advice, find opportunities. That face -to -face
• 33:00
  interaction is still incredibly valuable, perhaps
• 33:02
  even more so in our digital age. Watching senior
• 33:05
  colleagues interact is also a great way to learn
• 33:07
  professional norms. This has been incredibly
• 33:09
  comprehensive, a real journey. It has, from how
• 33:12
  do we know, through ethics. design like randomization
• 33:15
  and blinding, measuring patient outcomes with
• 33:17
  PROMs and MCID, interpreting data, understanding
• 33:20
  different study types and synthesis, ready for
• 33:22
  the practicalities of multi -center trials, registries,
• 33:25
  and sharing the work. And for you listening as
• 33:27
  professionals in this area, grasping these principles
• 33:30
  isn't just academic. It's absolutely vital for
• 33:33
  critically evaluating the research that informs
• 33:35
  your practice every single day. It helps you
• 33:38
  make better, more informed decisions. Knowing
• 33:40
  how the evidence is generated allows you to be
• 33:43
  a much more critical and insightful consumer.
• 33:46
  You can ask the right questions about a study's
• 33:48
  methods and understand the true clinical weight
• 33:50
  of its findings. If this deep dive has been helpful,
• 33:53
  we'd really appreciate it if you could rate the
• 33:55
  show and perhaps share it with colleagues, maybe
• 33:57
  on LinkedIn or X, and to leave you with something
• 33:59
  to think about. We've seen the power of RCTs
• 34:02
  for controlling bias and the richness of registry
• 34:05
  data for real -world insights. How might linking
• 34:08
  these two worlds, combining large -scale registry
• 34:11
  data with embedded randomized comparisons, perhaps
• 34:14
  change how we understand treatment effectiveness
• 34:16
  across the diverse reality of orthopedic practice
• 34:19
  in the future?