Latest / Deep Dive Ortho — Orthopaedic Surgery, AI & Digital Health Podcast. / A Pen and Paper Introduction to Statistics

Transcript

A Pen and Paper Introduction to Statistics · Deep Dive Ortho — Orthopaedic Surgery, AI & Digital Health Podcast.

• 0:00
  Imagine this. You're an experienced surgeon.
• 0:03
  You're looking at an open tibial fracture, one
• 0:05
  of the most common severe injuries really. You're
• 0:09
  asked to classify it using a standard system,
• 0:11
  something you've done, well, countless times.
• 0:14
  Yes, standard practice. Now imagine several of
• 0:17
  your equally experienced colleagues are doing
• 0:18
  the exact same thing for the very same fracture.
• 0:22
  How often do you think you'd all agree? on the
• 0:24
  precise classification. You'd hope pretty often,
• 0:27
  wouldn't you? You would, wouldn't you? Maybe
• 0:28
  90%, 100%. Well, astonishingly, research has
• 0:31
  shown that even among highly experienced orthopedic
• 0:34
  trauma surgeons, using a widely accepted standard
• 0:37
  system, agreement can be as low as 60%. 60%.
• 0:41
  That's quite low. It really is. Think about just
• 0:44
  classifying what the injury is. That presents
• 0:46
  a significant challenge. And that's just the
• 0:48
  starting point, you know? before you even begin
• 0:50
  to figure out the absolute best way to treat
• 0:53
  it and definitively know if that treatment actually
• 0:55
  worked. Welcome to the deep dive. This is where
• 0:58
  we take a stack of source material, the articles,
• 1:01
  the research papers, the notes that you've shared
• 1:02
  with us, and we, well, we unpack the vital insights,
• 1:06
  the crucial nuggets of knowledge, really. We
• 1:09
  want to help you become truly well -informed
• 1:11
  on complex topics without, you know, drowning
• 1:14
  in information overload. A very common problem
• 1:17
  these days. It is. Our mission today is to navigate
• 1:19
  the sometimes murky, always intricate waters
• 1:22
  of clinical research. We're going to embark on
• 1:25
  a deep dive focusing on how studies are designed.
• 1:27
  the fascinating, and yes, sometimes frustrating,
• 1:30
  world of data analysis, and perhaps most crucially,
• 1:33
  how we actually measure whether a treatment has
• 1:35
  achieved a meaningful outcome, all with a particular
• 1:38
  eye on the challenges often seen within orthopedics.
• 1:40
  A field with its own unique set of hurdles, certainly.
• 1:44
  Exactly. It's about pulling out those essential
• 1:46
  pieces that make evaluating medical evidence
• 1:48
  less daunting and hopefully more insightful for
• 1:51
  you. And joining me today to guide us through
• 1:53
  this material is an expert with a remarkable
• 1:56
  ability to synthesize diverse information, to
• 1:59
  spot those subtle but critical patterns and,
• 2:02
  well, to connect the dots between the seemingly
• 2:04
  academic world of research methodology and its
• 2:07
  real world implications for clinical practice,
• 2:10
  for informed decision making. Happy to be here
• 2:12
  and try to unravel some of it. Great. He's here
• 2:14
  to help us understand not just what the studies
• 2:16
  say, but why it truly matters for the decisions
• 2:19
  you make. Right, let's jump straight in with
• 2:22
  a rapid -fire setup, just to preview some key
• 2:24
  angles of our deep dive. Thinking about professionals
• 2:27
  interpreting research, what in your view is the
• 2:30
  single most important concept they need to grasp
• 2:32
  right away? Ah, the single most important. I'd
• 2:35
  have to say it's understanding the inherent limitations.
• 2:39
  Limitations imposed by the study design and the
• 2:41
  potential for bias. You see, every study has
• 2:44
  weaknesses. Recognizing where a particular study
• 2:47
  sits within the hierarchy of evidence, along
• 2:50
  with its specific risks of bias, that's the absolute
• 2:53
  cornerstone of responsible interpretation. So
• 2:56
  you can't take it at face value? Never. If you
• 2:59
  don't see the flaws, you can't truly trust the
• 3:01
  findings. It's fundamental. All right. Fundamental
• 3:03
  indeed. Next question. Why is the choice of how
• 3:07
  you measure results sometimes just as or even
• 3:11
  more critical than the treatment itself? Oh,
• 3:13
  absolutely critical. Because if your outcome
• 3:15
  measure isn't truly meaningful, reliable, and
• 3:17
  valid, you know, within the context of the study
• 3:20
  and for the patients involved, then whatever
• 3:22
  findings you report about the treatment's effect
• 3:24
  become highly questionable. So you could get
• 3:25
  a significant result, but... Exactly. You might
• 3:28
  see a statistically significant change in, say,
• 3:31
  a lab value. But if that doesn't translate into
• 3:34
  a tangible improvement in a patient's function
• 3:36
  or quality of life, what they actually care about,
• 3:39
  then the measure, and by extension, the result,
• 3:42
  might lack clinical relevance. The measurements
• 3:45
  simply have to align with what matters. What
• 3:47
  matters to the patient, ultimately? Necessarily.
• 3:49
  Okay. And finally, stepping back from the statistics
• 3:52
  and the measurements for a moment, what's an
• 3:54
  ethical cornerstone of human research that simply
• 3:56
  cannot, under any circumstances, be overlooked?
• 3:59
  Informed consent. It's the absolute bedrock principle.
• 4:03
  It demonstrates fundamental respect for participant
• 4:05
  autonomy, ensuring individuals fully comprehend
• 4:09
  the study's purpose, the methods, the potential
• 4:11
  risks and benefits, all the available alternatives.
• 4:15
  And confirming their decision to participate
• 4:17
  is truly voluntary, free from coercion that is
• 4:20
  completely non -negotiable. So even a perfectly
• 4:22
  designed study. Without robust, genuine, informed
• 4:25
  consent, even the most methodologically brilliant
• 4:28
  study is built on ethically shaky ground. It's
• 4:30
  just unacceptable. Powerful start, thank you.
• 4:33
  Okay, let's untack this in more detail, then.
• 4:35
  We begin our deep dive into designing and analyzing
• 4:38
  studies with a concept that's often the first
• 4:41
  framework you encounter when trying to make sense
• 4:43
  of medical evidence, the hierarchy of evidence.
• 4:47
  What exactly is this and why is it such a foundational
• 4:49
  tool? Right, the hierarchy of evidence. It's
• 4:52
  essentially a structured way, a ranking system
• 4:54
  really, for different types of study designs.
• 4:58
  It ranks them based on their inherent methodological
• 5:01
  quality, particularly in terms of their ability
• 5:05
  to minimize bias and, crucially, establish causality,
• 5:09
  especially when you're evaluating the effectiveness
• 5:11
  of interventions. So it helps you judge reliability.
• 5:14
  Exactly. It helps us as readers of research quickly
• 5:17
  gauge the relative reliability and trustworthiness
• 5:19
  of a study's findings compared to other types
• 5:22
  of studies. The version from the Oxford Center
• 5:24
  for Evidence -Based Medicine is one of the most
• 5:26
  widely used and recognized frameworks for this.
• 5:28
  So it's like a pyramid or maybe a ladder with
• 5:31
  different study types at different levels. What
• 5:33
  sits at the very top when we're talking about
• 5:35
  individual studies looking at treatments? Precisely.
• 5:38
  At the pinnacle, representing the highest level
• 5:41
  of primary evidence for therapeutic interventions,
• 5:43
  you typically find randomized controlled trials,
• 5:46
  RCTs. They're generally classified as level one
• 5:48
  evidence. And why level one? What makes them
• 5:51
  the top? The reason they're placed there is fundamentally
• 5:54
  about their design's ability to reduce selection
• 5:56
  bias, which, as we said, is a major threat to
• 6:00
  validity. As you descend the hierarchy, you encounter
• 6:03
  other steady designs. Controlled observational
• 6:06
  studies sit below RCTs, and their exact level
• 6:08
  can depend on factors like whether they collected
• 6:11
  data prospectively looking forward in time, or
• 6:13
  retrospectively looking back at existing data.
• 6:16
  And lower down. Further down still, you find
• 6:18
  study types with higher inherent risks of bias.
• 6:20
  things like case series, uncontrolled studies,
• 6:23
  and finally, at the lowest level, typically level
• 6:25
  five, you have things like expert opinion, anecdotal
• 6:28
  evidence, or studies based purely on mechanisms
• 6:30
  of action rather than actual clinical outcomes
• 6:32
  in patients. Okay. You mentioned RCTs are the
• 6:35
  gold standard for primary evidence. What is it
• 6:37
  about that randomization process that makes them
• 6:39
  so powerful, especially against bias? Well, the
• 6:43
  fundamental strength of an RCT lies in the random
• 6:45
  allocation of participants to different groups.
• 6:48
  Typically the group receiving the new treatment
• 6:50
  and a comparison or control group. The idea is
• 6:53
  that by randomizing you ensure that on average
• 6:56
  the groups are comparable right at the start
• 6:58
  of the study. Comparably what way? In terms of
• 7:01
  both known factors that might influence the outcome
• 7:03
  like age, severity of disease, other health conditions,
• 7:07
  and crucially unknown factors that you might
• 7:10
  not even... be aware of or be able to measure.
• 7:13
  This comparability means that if you observe
• 7:15
  a significant difference in outcomes between
• 7:17
  the groups at the end, you could be much more
• 7:19
  confident that the difference is due to the intervention
• 7:21
  being studied rather than some pre -existing
• 7:23
  difference between the groups. So it isolates
• 7:26
  the effect of the treatment itself. It's the
• 7:28
  most robust way we have to try and establish
• 7:30
  that cause and effect relationship between a
• 7:32
  treatment and an outcome. That logic holds for
• 7:35
  many medical fields, but the source material
• 7:37
  correctly points out that even this gold standard
• 7:39
  faces significant limitations, particularly in
• 7:41
  surgical disciplines like orthopedics. What are
• 7:44
  some of the practical logistical hurdles there?
• 7:47
  Ah, yes. The reality of applying RCTs to surgery
• 7:50
  is challenging, to say the least. They are notoriously
• 7:54
  difficult and often prohibitively expensive to
• 7:57
  organize and run. They involve a substantial
• 8:00
  workload for the research team, often over a
• 8:02
  very long duration, which can lead to logistical
• 8:05
  nightmares and real challenges in just completing
• 8:07
  the study as planned. And beyond just the cost
• 8:10
  and time. Beyond the practicalities, there are
• 8:13
  profound difficulties with recruitment. Both
• 8:15
  surgeons and patients can struggle with the concept
• 8:18
  of randomization. A surgeon might have a strong
• 8:21
  clinical preference for one technique over another
• 8:23
  based on their experience, right? And randomization
• 8:26
  removes their autonomy to choose what they feel
• 8:29
  is best for that patient. Similarly, a patient
• 8:32
  might have heard about a new technique and specifically
• 8:34
  want that one. or they might simply be unwilling
• 8:37
  to accept the possibility of being randomized
• 8:40
  to an older or perhaps less preferred method,
• 8:43
  or even a non -surgical option, if that's part
• 8:45
  of the comparison. This resistance from both
• 8:48
  sides can severely impede recruitment and make
• 8:51
  surgical RCTs incredibly hard to complete successfully.
• 8:55
  And this brings us directly to a really critical
• 8:57
  ethical principle, clinical equipoise. This isn't
• 9:01
  just academic jargon. It's absolutely fundamental
• 9:03
  to the ethics of randomizing patients in the
• 9:05
  first place. Okay, clinical equipoise. How does
• 9:07
  that tie into justifying randomization ethically?
• 9:10
  What does it mean? Clinical equipoise means there
• 9:12
  must be genuine uncertainty within the expert
• 9:15
  medical community, including the investigators
• 9:17
  conducting the study, about whether one arm of
• 9:20
  the experiment is definitively superior to another.
• 9:22
  So if doctors already think one treatment is
• 9:24
  better. Exactly. If there is already a clear
• 9:27
  consensus, Based on existing evidence that one
• 9:30
  treatment is better or safer than another, it
• 9:33
  would generally be considered unethical to randomize
• 9:35
  patients to the treatment believed to be inferior.
• 9:38
  Equipoise is the necessary ethical precondition.
• 9:41
  It allows researchers to justify subjecting patients
• 9:43
  to the potential risks of an experimental treatment
• 9:46
  or randomizing them away from potentially beneficial
• 9:49
  treatment precisely because the best course of
• 9:52
  action is genuinely unknown. So if the investigator
• 9:55
  thinks they know the answer. If you, as the investigator,
• 9:59
  truly believe one treatment is superior, you
• 10:01
  have an ethical obligation to provide that treatment,
• 10:04
  not randomize patients away from it. Establishing
• 10:06
  and maintaining that state of genuine uncertainty
• 10:09
  equipoise throughout the trial is absolutely
• 10:11
  vital. Right, and the source material mentions
• 10:14
  a particularly challenging application of this
• 10:16
  principle in orthopedics, sham surgeries. That
• 10:19
  sounds ethically complex just from the name.
• 10:21
  It is indeed, and it highlights that ethical
• 10:24
  tension beautifully. SHAM surgeries are essentially
• 10:27
  the surgical equivalent of a placebo and a drug
• 10:29
  trial. They involve performing part of a surgical
• 10:32
  procedure, or perhaps a procedure that mimics
• 10:34
  the experience, but crucially omitting the actual
• 10:38
  therapeutic step. Like making the cut but not
• 10:40
  doing the repair. Exactly. Making incisions perhaps,
• 10:44
  but not performing the repair or the decompression
• 10:46
  you're studying. The scientific rationale is
• 10:49
  powerful. They allow for a truly blinded comparison
• 10:52
  because neither the patient nor the assessing
• 10:54
  clinician knows whether the full procedure was
• 10:56
  actually performed. This helps to isolate the
• 10:59
  true effect of the surgical intervention itself
• 11:01
  from the placebo effect, you know, the psychological
• 11:04
  impact of simply having undergone any procedure.
• 11:07
  But the ethical tension is undeniable. You are
• 11:10
  performing an invasive non -therapeutic procedure
• 11:13
  that carries risks anesthesia risks, infection,
• 11:17
  nerve damage potentially on a patient who may
• 11:19
  as a result be denied an effective treatment
• 11:22
  by being in that sham group. Now current literature
• 11:25
  and ethical guidelines do suggest that sham surgeries
• 11:28
  can have a place in research, but only under
• 11:31
  very stringent conditions. What sort of conditions?
• 11:33
  These include strict adherence to that principle
• 11:36
  of clinical equipoise. Genuine uncertainty about
• 11:39
  efficacy is paramount. rigorous informed consent,
• 11:43
  where the patient fully understands they might
• 11:44
  receive a sham procedure, is absolutely essential.
• 11:48
  And there needs to be very careful consideration
• 11:49
  of the risks versus the potential benefits of
• 11:52
  gaining definitive knowledge from the trial.
• 11:54
  It's a very sensitive area, but sometimes it's
• 11:56
  considered necessary to truly determine if a
• 11:58
  surgical intervention works beyond just the placebo
• 12:00
  effect and the natural history of the condition.
• 12:03
  Fascinating, and it definitely highlights the
• 12:05
  complexities researchers face. So, beyond these
• 12:08
  primary studies like RCTs, we also rely heavily
• 12:11
  on secondary and filtered research things like
• 12:13
  systematic reviews and meta -analyses. The hierarchy
• 12:17
  of evidence traditionally places these above
• 12:19
  individual studies, even RCTs. Why is that the
• 12:22
  case? Yes, they are often positioned at the top
• 12:24
  because they represent a synthesis or a summary
• 12:26
  of the best available primary evidence on a specific
• 12:29
  clinical question. A systematic review is a rigorous
• 12:32
  process. It involves a comprehensive and ideally
• 12:35
  unbiased search for all relevant primary studies
• 12:38
  addressing a particular question, followed by
• 12:41
  a critical appraisal and synthesis of their findings.
• 12:43
  So it aims to be exhaustive and transparent.
• 12:45
  Exactly. The aim is to reduce bias in the review
• 12:49
  process itself by being transparent about the
• 12:52
  search strategy, the inclusion criteria, the
• 12:55
  quality assessment, and how the results were
• 12:57
  combined. A meta -analysis takes this a step
• 13:00
  further. If the studies included in a systematic
• 13:03
  review are sufficiently similar in terms of their
• 13:05
  populations, interventions, outcome measures,
• 13:08
  that sort of thing, requiring a degree of homogeneity,
• 13:11
  then a meta -analysis can statistically pool
• 13:14
  the quantitative data from these studies. And
• 13:16
  the benefit of pooling the data? It increases
• 13:19
  the sample size and the statistical power. This
• 13:22
  potentially allows researchers to detect smaller
• 13:24
  effects or provide a much more precise estimate
• 13:27
  of the treatment effect than any single study
• 13:29
  could on its own. And when pooling those statistics
• 13:31
  in a meta -analysis, the source mentions fixed
• 13:34
  effects versus random effects models. What's
• 13:36
  the key difference in what those models assume
• 13:38
  about the data? Ah yes, that comes down to what
• 13:41
  the model assumes about the underlying effect
• 13:44
  size across the included studies. A fixed effects
• 13:48
  model Something like the Mantle -Hensel method
• 13:50
  assumes there is one true single effect size
• 13:53
  for the intervention across all the populations
• 13:55
  represented in the included studies. It assumes
• 13:59
  any variation in results seen between the studies
• 14:01
  is purely due to random chance sampling error.
• 14:05
  It essentially calculates a weighted average
• 14:06
  giving more weight to larger studies. Okay, and
• 14:09
  random effects. In contrast, a random effects
• 14:12
  model, like the Dersimonian and Laird method,
• 14:15
  is generally considered more conservative and,
• 14:17
  frankly, often more realistic in clinical research.
• 14:20
  It assumes that the true effect size might actually
• 14:22
  vary to some degree from study to study, not
• 14:25
  just because of random chance, but also because
• 14:27
  of actual differences in the population studied,
• 14:29
  perhaps the way the intervention was applied,
• 14:31
  the settings, and so on. It accounts for this
• 14:33
  between -study heterogeneity in addition to the
• 14:36
  within -study random variation. So if the studies
• 14:39
  are quite different, If there's significant heterogeneity
• 14:42
  in the results of the included studies, a random
• 14:45
  effects model is usually more appropriate, and
• 14:47
  it will typically produce wider confidence intervals,
• 14:51
  reflecting the greater uncertainty due to this
• 14:53
  underlying variation between studies. So systematic
• 14:56
  reviews and meta -analyses sound like powerful
• 14:58
  tools for summarizing evidence. But the source
• 15:01
  material also presents important criticisms,
• 15:03
  suggesting they are, well, only as good as the
• 15:05
  studies they include. What are the main caveats
• 15:08
  we should be aware of when reading them? This
• 15:10
  is an absolutely crucial point, and it really
• 15:13
  underpins that adage you mentioned. Secondary
• 15:16
  evidence is only as strong as the studies that
• 15:18
  comprise it. A beautifully conducted systematic
• 15:21
  review, meticulously done, can still yield misleading
• 15:24
  conclusions if the primary studies it includes
• 15:27
  are methodologically flawed, have high risks
• 15:29
  of bias, or are simply not relevant to current
• 15:32
  practice. So pooling bad studies doesn't make
• 15:35
  the result good. Precisely. including non -randomized
• 15:38
  studies when good RCTs exist, pooling data from
• 15:42
  studies with vastly different patient populations
• 15:44
  or interventions, or including studies that are
• 15:47
  technically sound but clinically irrelevant to
• 15:49
  the question at hand. All these things can compromise
• 15:53
  the validity and the practical impact of the
• 15:55
  review or meta -analysis. Just because you've
• 15:58
  pooled a lot of data doesn't automatically mean
• 16:00
  the answer is right or useful. If you're pooling
• 16:03
  weak evidence, you might just get a more precise
• 16:05
  estimate of the wrong answer. Readers really
• 16:07
  need to look critically at the methods section
• 16:09
  of any systematic review, understand how the
• 16:11
  studies were identified, how their quality was
• 16:13
  assessed, and whether it was genuinely appropriate
• 16:16
  to pool their results statistically in a meta
• 16:18
  -analysis, given the potential for heterogeneity.
• 16:21
  That brings us right back to the concept of bias,
• 16:24
  which you highlighted at the start as being so
• 16:26
  fundamental. You mentioned selection bias as
• 16:28
  particularly important in study design. Could
• 16:31
  you elaborate on that and maybe give a few more
• 16:33
  examples of where it can creep in, potentially
• 16:35
  distorting findings even early on? Certainly.
• 16:38
  Bias, remember, is a systematic error. It's not
• 16:41
  random noise. It's a consistent slant in the
• 16:45
  study design or conduct that leads to results
• 16:47
  that are consistently different from the true
• 16:49
  effect. Selection bias, as we discussed, arises
• 16:53
  when there are systematic differences in the
• 16:54
  characteristics of the groups being compared
• 16:56
  before the intervention even happens. This can
• 16:59
  happen if participants aren't recruited or allocated
• 17:01
  to groups in a truly random or comparable way.
• 17:04
  So beyond just using historical controls, which
• 17:06
  you said is always bad. Yes. Using historical
• 17:08
  controls is almost always problematic due to
• 17:11
  changes over time in care, diagnosis, everything.
• 17:15
  But selection bias can occur much more subtly.
• 17:17
  For example, if a study is conducted only at
• 17:20
  a highly specialized tertiary referral center,
• 17:23
  the patients recruited there might have more
• 17:25
  complex or severe forms of the condition than
• 17:28
  the average patient you'd see in a general hospital
• 17:30
  with that same condition. Comparing outcomes
• 17:32
  in this highly selected group to outcomes reported
• 17:35
  elsewhere could introduce selection bias. Similarly,
• 17:39
  if the method of recruiting patients differs
• 17:40
  between groups, Perhaps one group is recruited
• 17:43
  from an outpatient clinic and another from an
• 17:45
  inpatient ward, you might inadvertently select
• 17:47
  patients with different levels of mobility or
• 17:49
  overall health status. Even in an RCT, if the
• 17:53
  process of randomization isn't properly concealed,
• 17:56
  meaning the researchers enrolling patients know
• 17:58
  or can guess which group the next patient will
• 18:00
  be assigned to, they might consciously or unconsciously
• 18:03
  steer certain types of patients into one group
• 18:05
  over the other based on prognostic factors. That
• 18:08
  undermines a whole point of randomization and
• 18:10
  reintroduces selection bias. And the sources
• 18:12
  mention other types of bias too. Yes, absolutely.
• 18:16
  Detection bias, for instance. This occurs when
• 18:18
  the outcome is assessed differently between the
• 18:20
  groups. If the assessors, the people measuring
• 18:23
  the outcome, are aware of which treatment group
• 18:25
  a patient received, their expectations might
• 18:28
  influence their assessment. Blinding the assessors
• 18:31
  to the treatment group is the key way to mitigate
• 18:33
  this. Then there's reporting bias, which refers
• 18:36
  to the selective reporting of findings. For example,
• 18:39
  only publishing results that are statistically
• 18:41
  significant or favorable to a particular treatment
• 18:44
  while suppressing negative or null findings.
• 18:47
  This seriously distorts the overall evidence
• 18:50
  landscape, making treatments look better than
• 18:52
  they perhaps are. That reporting bias sounds
• 18:54
  particularly concerning, especially when we think
• 18:56
  about the pressure to publish. The source material
• 18:59
  calls out p -hacking as an unethical practice
• 19:02
  linked to this. What exactly is p -hacking and
• 19:05
  why is it considered so damaging to the integrity
• 19:07
  of research? P -hacking is a really problematic
• 19:10
  form of data manipulation. It's often driven
• 19:13
  by that intense pressure researchers feel to
• 19:16
  find statistically significant results, typically
• 19:19
  that p -value below .05, because significant
• 19:23
  results are generally easier to publish. It's
• 19:25
  not usually outright fabrication of data, but
• 19:28
  it involves playing with the data or the analysis
• 19:31
  methods after the data have been collected, specifically
• 19:33
  until a desired p -value is achieved. How might
• 19:36
  someone do that? Examples include running multiple
• 19:39
  different statistical tests on the same dataset,
• 19:41
  and only reporting the one that happens to yield
• 19:43
  a significant result. Or, perhaps collecting
• 19:46
  more data points if the initial analysis isn't
• 19:48
  significant, and then rerunning the analysis
• 19:50
  until it crosses the threshold. Selectively excluding
• 19:53
  outlier data points that might dilute the effect
• 19:56
  is another way. or measuring many, many different
• 19:58
  outcomes, but only reporting the few that happen
• 20:01
  to show a significant difference by chance. And
• 20:03
  why is that so bad? It sounds like just exploring
• 20:05
  the data. The reason p -hacking is so damaging
• 20:08
  and considered unethical is that it fundamentally
• 20:11
  violates the logic of hypothesis testing and
• 20:14
  the p -value itself. The p -value is the probability
• 20:18
  of observing your data. or more extreme data,
• 20:21
  assuming the null hypothesis is true, and, crucially,
• 20:24
  assuming the test and analysis plan were decided
• 20:26
  upon before you saw the data. When you p -hack,
• 20:30
  you're essentially searching through the data
• 20:31
  until you find any pattern that looks significant
• 20:33
  just by chance, and then you report it as if
• 20:36
  it were the primary finding you set out to discover
• 20:38
  all along. So it inflates the false positive
• 20:40
  rate. Massively. It hugely inflates the chance
• 20:43
  of reporting false positive results, finding
• 20:45
  an effect that isn't really there. It misleads
• 20:47
  other researchers, clinicians, and ultimately
• 20:49
  patients about the true state of evidence. It's
• 20:52
  a distortion of reality driven by chasing a specific
• 20:55
  statistical outcome, and it severely undermines
• 20:58
  the trustworthiness of the research process.
• 21:00
  Absolutely. It's a stark reminder that transparency
• 21:03
  and ethical conduct are just as important as
• 21:06
  getting the statistics right. Speaking of statistics,
• 21:10
  let's delve into some foundational concepts that
• 21:12
  are essential for interpreting the number studies
• 21:14
  report, starting with the basic distinction between
• 21:17
  samples and populations. Right. In research,
• 21:20
  we're almost always interested in understanding
• 21:22
  something about a large group, the population.
• 21:26
  They could be all patients with a certain type
• 21:28
  of fracture, all people undergoing a specific
• 21:30
  type of surgery, or maybe all adults in a country.
• 21:33
  However, it's rarely feasible or even possible,
• 21:36
  to study every single member of that population.
• 21:39
  So instead we study a smaller, hopefully representative
• 21:41
  subset, and that's our sample. And the goal is
• 21:44
  to generalize from the sample. Exactly. The goal
• 21:47
  of research is to use the data we collect from
• 21:49
  our sample to make inferences or generalize conclusions
• 21:52
  back to the larger population from which it was
• 21:54
  drawn. This is why how you select your sample
• 21:57
  is so critically important, going right back
• 21:59
  to that issue of selection bias. Random sampling.
• 22:02
  where every member of the population theoretically
• 22:04
  has an equal chance of being included, is the
• 22:07
  ideal because it helps ensure the sample is representative
• 22:10
  of the population, making your inferences more
• 22:12
  reliable. When we describe our sample data, we
• 22:16
  often use descriptive statistics. Data can be
• 22:19
  continuous, meaning it can take any value within
• 22:21
  a range, like height or blood pressure, or it
• 22:24
  can be categorical, meaning it falls into distinct
• 22:26
  categories, like sex, blood type, or fracture
• 22:29
  grade. For continuous data, the mean of the average
• 22:32
  is a common way to describe the center of the
• 22:34
  data. The source has also mentioned variance,
• 22:36
  which is a measure of dispersion. It tells you
• 22:38
  how spread out the individual data points are
• 22:41
  around that mean. How is variance calculated?
• 22:43
  It's calculated as the average of the squared
• 22:45
  differences of each data point from the mean.
• 22:48
  Measures of dispersion like variance and its
• 22:51
  square root, the standard deviation, are crucial
• 22:54
  because just Knowing the average isn't enough,
• 22:56
  you really need to know how much the data varies
• 22:58
  around that average. And how do these concepts
• 23:01
  then lead us into hypothesis testing? Hypothesis
• 23:05
  testing is the formal process researchers use
• 23:07
  to determine if the results observed in their
• 23:09
  sample provide enough evidence to support a conclusion
• 23:11
  about the wider population. It typically involves
• 23:14
  setting up two competing statements. The null
• 23:17
  hypothesis, often written as H, and the alternative
• 23:20
  hypothesis, H or sometimes H. What do those usually
• 23:22
  state? The null hypothesis usually states there
• 23:25
  is no effect or no difference between the groups
• 23:27
  being compared. For example, the new treatment
• 23:30
  has no effect on healing time compared to the
• 23:32
  old one. The alternative hypothesis states that
• 23:35
  there is an effect or a difference. For example,
• 23:37
  the new treatment does improve healing time.
• 23:40
  We then perform a statistical test on our sample
• 23:42
  data to see how likely it is we'd observe results
• 23:44
  like ours if the null hypothesis were actually
• 23:47
  true in the population. Okay, and this is where
• 23:49
  the famous p -value comes in, the number that
• 23:51
  often gets so much attention. We often hear researchers
• 23:54
  chasing that magic p .05, but what does the p
• 23:59
  -value actually represent? Ah, yes. This is probably
• 24:02
  one of the most commonly misunderstood concepts
• 24:04
  in statistics, and it's absolutely vital to get
• 24:07
  it right. The p -value is the probability of
• 24:10
  observing data from your sample that is as extreme
• 24:13
  as or more extreme than what you actually measured,
• 24:17
  assuming that the null hypothesis is true. Can
• 24:19
  you give an example? Okay, so if you conduct
• 24:21
  a study comparing two treatments and you get
• 24:22
  a p -value of, say, 0 .03, it means that if there
• 24:26
  were truly no difference between those treatments
• 24:28
  in the entire population, if the null hypothesis
• 24:30
  were correct, you would still expect to see a
• 24:33
  difference as large as or larger than the one
• 24:35
  you observed in your sample only 3 % of the time,
• 24:37
  just due to random chance or sampling variability.
• 24:41
  That traditional threshold of 0 .05 or 5 % for
• 24:45
  statistical significance is largely arbitrary.
• 24:48
  It's a convention that arose historically. If
• 24:50
  your p -value is less than this threshold, you
• 24:52
  typically reject the null hypothesis and conclude
• 24:55
  that your results provide statistically significant
• 24:57
  evidence against the null hypothesis and therefore
• 25:00
  in favor of your alternative hypothesis. But
• 25:02
  what is the p -value not? Crucially, it's absolutely
• 25:05
  critical to understand what the p -value is not.
• 25:08
  It is not the probability that the null hypothesis
• 25:10
  is true. It is not the probability that you are
• 25:13
  wrong if you reject the null hypothesis, although
• 25:16
  it's related to that concept. And, very importantly,
• 25:20
  it tells you absolutely nothing directly about
• 25:21
  the size or the clinical importance of the observed
• 25:24
  effect. As the source notes, a small p -value
• 25:27
  indicates that your observed data are unlikely
• 25:30
  if the null hypothesis is true, so it provides
• 25:32
  evidence against the null. You can think of it
• 25:34
  as a sort of currency for statistical robustness.
• 25:38
  Smaller p -values mean stronger evidence from
• 25:40
  the specific sample against the null. But even
• 25:43
  a very low p -value, say 0 .001, means there's
• 25:47
  still a 0 .1 % chance of seeing results that
• 25:49
  extreme if the null were true. In some contexts,
• 25:52
  like drug safety, even a 1 % chance, p0 .01,
• 25:56
  of being wrong about a harmful effect could be
• 25:58
  catastrophic if applied to a large population.
• 26:00
  So necessary but not sufficient. Exactly. Statistical
• 26:03
  significance is often a necessary condition for
• 26:05
  claiming an effect, but it's rarely sufficient
• 26:07
  on its own to make clinical decisions. You mentioned
• 26:10
  that many statistical tests compare groups by
• 26:12
  looking at their variances. Could you give us
• 26:14
  a simplified view of how that works? Yes, many
• 26:17
  common tests, particularly those comparing the
• 26:20
  means of different groups, are fundamentally
• 26:22
  based on analyzing variance. A key concept here
• 26:26
  is partitioning variance. Take ANOVO, for instance,
• 26:30
  analysis of variance, which is used when you're
• 26:31
  comparing the means of three or more groups.
• 26:34
  ANOVO essentially breaks down the total variation
• 26:36
  you see in the data into different sources. What
• 26:39
  sources? Primarily, the variation between the
• 26:42
  groups, which might be due to the treatment effect
• 26:44
  you're interested in, and the variation within
• 26:46
  each group, which is generally considered due
• 26:48
  to random error and individual differences among
• 26:51
  participants in the same group. It then compares
• 26:54
  the magnitude of the variance between the groups
• 26:56
  relative to the variance within the groups. This
• 27:00
  comparison is often done using something called
• 27:02
  the F -statistic, which follows a specific statistical
• 27:06
  distribution called the F -distribution. Fundamentally,
• 27:08
  it's a ratio of two variances. And if the variance
• 27:11
  between groups is large, If the variance between
• 27:14
  groups is significantly larger than the variance
• 27:16
  within groups, the test concludes there's a statistically
• 27:19
  significant difference somewhere among the group
• 27:21
  means. It's about seeing if the groups are more
• 27:24
  different from each other than individuals tend
• 27:27
  to be within their own group. Okay, so p -values
• 27:30
  give us a sense of statistical significance,
• 27:32
  of the likelihood of results being due to chance
• 27:34
  if the null hypothesis is true. But the source
• 27:36
  material strongly advocates looking beyond the
• 27:39
  p -value, especially at confidence intervals,
• 27:41
  or CIs. Why are CIs so vital, perhaps even more
• 27:45
  so than the p -value, for interpreting clinical
• 27:48
  relevance? Confidence intervals are absolutely
• 27:50
  critical for moving beyond just that binary,
• 27:53
  significant or not significant decision to understand
• 27:56
  the potential magnitude and the precision of
• 27:58
  the treatment effect, and therefore its clinical
• 28:01
  meaningfulness. A confidence interval, most commonly
• 28:03
  a 95 % CI, provides a range of values within
• 28:07
  which the true effect size in the larger population
• 28:09
  is likely to lie, based on the results observed
• 28:12
  in your sample. It gives you a sense of the uncertainty
• 28:14
  around your estimate of the effect. So it gives
• 28:17
  a range, not just a single number or a yes -no.
• 28:20
  How do we interpret that range? The key to interpreting
• 28:23
  a CI lies in where that range falls, relative
• 28:26
  to two important points. Zero. If you're looking
• 28:29
  at a difference between groups, zero means no
• 28:32
  difference. And, critically, relative to what
• 28:34
  is considered a clinically important difference.
• 28:37
  That's the minimum change that patients or clinicians
• 28:40
  would consider meaningful and worthwhile in practice.
• 28:43
  Let's maybe walk through a few scenarios to make
• 28:45
  it clear. If your 95 % CI for the difference
• 28:48
  between two treatments does not include zero,
• 28:50
  that aligns with a statistically significant
• 28:52
  result, typically P0 .05. The entire range suggests
• 28:57
  a difference likely exists. But then you must
• 28:59
  look at where that range lies. If the entire
• 29:02
  CI, both the lower and upper bounds, is above
• 29:04
  what you've defined as a clinically important
• 29:06
  difference, then you can be more confident that
• 29:08
  the treatment not only has a statistically significant
• 29:10
  effect, but also one that is likely to be clinically
• 29:12
  worthwhile. Okay, what if it does include zero?
• 29:15
  If the CI includes zero, your result is not statistically
• 29:18
  significant. typically PP .05. This means your
• 29:22
  data don't provide enough evidence to reject
• 29:24
  the null hypothesis of no difference. The true
• 29:27
  effect could plausibly be zero, or it could even
• 29:30
  point in the opposite direction if the CI spans
• 29:32
  both negative and positive values. This outcome
• 29:35
  is essentially uncertain based on your data.
• 29:37
  Right. And what about situations where it's significant,
• 29:39
  but maybe not clinically meaningful? or vice
• 29:42
  versa. Exactly. Consider a scenario where the
• 29:45
  p value is, say, 0 .06, just missing that conventional
• 29:49
  0 .05 cutoff, so not statistically significant.
• 29:52
  The 95 per cci for the difference between groups
• 29:55
  might be, for example, negative 1 .5 plus 8 .5
• 29:58
  points on a pain scale, where you've decided
• 30:00
  a clinically important difference is five points.
• 30:02
  Now, the c i includes zero, consistent with p
• 30:04
  .05, but look at the upper end. It includes values
• 30:07
  like 8 .5, which are well above your clinically
• 30:09
  important threshold of five. important. It suggests
• 30:12
  that while your study didn't find statistically
• 30:14
  significant evidence against the null, the results
• 30:17
  are still compatible with a clinically important
• 30:19
  benefit. You might conclude your study was perhaps
• 30:22
  underpowered to detect this difference definitively,
• 30:25
  or the result remains uncertain, but you certainly
• 30:28
  can't confidently say the treatment is ineffective
• 30:30
  based on that CI. Now flip that. Consider a study
• 30:34
  with a p -value of 0 .04 statistically significant,
• 30:37
  but the 95 % CI for the effect size is, say,
• 30:41
  plus 0 .5, plus 2 .5 points on that same pain
• 30:45
  scale, where five points is the clinically important
• 30:47
  difference. Here, the CI does not include zero.
• 30:50
  It's statistically significant, but the entire
• 30:52
  range, even the upper limit of 2 .5, is well
• 30:55
  below the five -point threshold you decided was
• 30:57
  clinically important. So significant, but maybe
• 31:00
  not worth it. Precisely. You have a statistically
• 31:02
  significant result, but the CI strongly suggests
• 31:05
  that the true effect, while likely existing,
• 31:08
  is probably too small to be clinically meaningful
• 31:10
  for patients. Or, one last scenario. Imagine
• 31:14
  a study where the p -value is high, say, a .20.
• 31:17
  clearly not significant, and the CI for the effect
• 31:20
  is, say, negative 3 plus 1 .5. This CI includes
• 31:24
  zero, consistent with the non -significant p
• 31:26
  -value. And importantly, the upper limit of the
• 31:29
  plausible effect, 1 .5, is well below your clinically
• 31:32
  important difference. In this case, both the
• 31:35
  p -value and the CI point towards the treatment
• 31:37
  likely being ineffective in any clinically meaningful
• 31:39
  way. So the CI gives much more context. Absolutely.
• 31:42
  The CI gives you not just a yes -no about statistical
• 31:44
  significance, but a range of plausible values
• 31:46
  for the actual effect size. This allows you to
• 31:48
  judge the precision of the estimate. A wide CI
• 31:50
  means less precision, more uncertainty. A narrow
• 31:53
  CI means more precision and, crucially, to directly
• 31:56
  assess its clinical relevance in a way the p
• 31:58
  -value alone simply cannot. Always, always look
• 32:00
  at the CI. That was a really clear explanation,
• 32:03
  thank you. a deep dive into how studies are designed
• 32:06
  and the statistical language we really need to
• 32:09
  understand to interpret them properly. That transitions
• 32:12
  us perfectly into our next segment. The critical
• 32:15
  challenges of measuring what truly matters in
• 32:17
  clinical outcomes and the fundamental ethical
• 32:19
  standards that have to underpin all human research.
• 32:22
  You highlighted earlier that choosing the right
• 32:24
  outcome measure is paramount. Let's expand a
• 32:26
  bit on the different domains of outcomes and
• 32:28
  why that choice is so vital. Right. Outcomes
• 32:31
  in healthcare are multifaceted, aren't they?
• 32:33
  We can think about them in several broad domains.
• 32:35
  There are clinical outcomes which relate directly
• 32:37
  to the patient's health status and the biological
• 32:40
  or functional effects of the intervention. Things
• 32:42
  like pain levels, range of motion, maybe fracture
• 32:45
  healing time on an x -ray, presence of infection,
• 32:48
  or even mortality. So the direct health impact.
• 32:51
  Yes. Then there are process of care outcomes.
• 32:54
  which look more at how care is delivered, things
• 32:57
  like length of hospital stay, readmission rates,
• 33:00
  resource utilization. Patient satisfaction is
• 33:03
  another crucial domain, though it's important
• 33:04
  to note it's also distinct from clinical outcomes.
• 33:07
  And finally, there are economic outcomes, focusing
• 33:11
  on costs, both the direct costs of the treatment
• 33:13
  itself, and perhaps indirect costs, like lost
• 33:16
  productivity for the patient. Selecting the correct
• 33:18
  outcome measure or perhaps measures for a particular
• 33:21
  study is absolutely critical because it must
• 33:23
  align with the research question being asked
• 33:25
  and be meaningful within the context of the intervention
• 33:27
  being studied and the patient population involved.
• 33:31
  If you're setting a surgery designed to reduce
• 33:33
  pain and improve function, measuring only, say,
• 33:36
  an inflammatory marker in the blood might be
• 33:38
  scientifically interesting, but it's unlikely
• 33:40
  to be clinically relevant on its own. And this
• 33:42
  links to the idea of value in healthcare. Very
• 33:45
  much so. This is becoming even more prominent
• 33:47
  with the shift we're seeing toward value -based
• 33:50
  healthcare models, moving away from simply paying
• 33:53
  for the volume of services provided to paying
• 33:55
  for the value delivered. And value is often defined,
• 33:58
  quite simply, as the outcomes achieved relative
• 34:01
  to the costs incurred to achieve them. If you
• 34:04
  can't accurately and meaningfully measure the
• 34:07
  outcomes that genuinely matter to patients and
• 34:09
  to the healthcare system, you simply can't demonstrate
• 34:12
  value. So, the measure must capture what the
• 34:15
  intervention is intended to change, and that
• 34:18
  change must be relevant either to the patient's
• 34:20
  life or to the efficiency of the healthcare system.
• 34:23
  Okay, so assuming we know what we want to measure,
• 34:25
  what properties define a good outcome measure?
• 34:28
  What should we be looking for when a study reports
• 34:30
  using a particular scale or test? The two most
• 34:33
  fundamental properties you need to look for are
• 34:35
  reliability and validity. Think of reliability
• 34:38
  as consistency or repeatability. A reliable instrument
• 34:42
  produces the same result, or very similar results,
• 34:45
  when measuring the same stable phenomenon under
• 34:47
  the same conditions. But measurements aren't
• 34:49
  always perfect? Never. Measurement always includes
• 34:52
  some degree of error. Whether that's error introduced
• 34:55
  by the person taking the measurement we call
• 34:57
  that rater error or error from the instrument
• 34:59
  itself, perhaps it's not calibrated properly.
• 35:02
  There's also random fluctuation or variability
• 35:05
  in the patient's state, even over short periods.
• 35:08
  Reliability is concerned with how much of the
• 35:10
  observed variation in measurements is due to
• 35:13
  these errors versus how much reflects the true
• 35:15
  underlying change in the thing you're actually
• 35:18
  trying to measure. Are there different types
• 35:20
  of reliability? Yes, several. Interrater reliability
• 35:23
  refers to the degree of agreement between two
• 35:26
  or more different observers or clinicians measuring
• 35:28
  the same thing. Interrater reliability is the
• 35:31
  consistency of measurements taken by the same
• 35:33
  observer on different occasions. When assessing
• 35:36
  reliability, we talk about concepts like agreement,
• 35:38
  concordance, repeatability, essentially, how
• 35:40
  consistent is the measurement. Okay, so that's
• 35:42
  reliability consistency. What about validity?
• 35:45
  Validity, on the other hand, is about accuracy.
• 35:48
  It's the degree to which an instrument truly
• 35:50
  measures what it is intended to measure. It answers
• 35:53
  the fundamental question, are we actually measuring
• 35:55
  the concept we think we are measuring? The source
• 35:59
  makes a very important point here. Validity dictates
• 36:02
  what you are able to do with the test results.
• 36:05
  If a measure is valid, you can confidently use
• 36:07
  its results to make decisions or draw conclusions
• 36:10
  about the specific construct it measures, like
• 36:12
  knee function or pain level. And reliability
• 36:15
  is needed for validity. Critically, yes. A measure
• 36:18
  that is inaccurate or unreliable cannot be valid.
• 36:23
  If you're not getting consistent results when
• 36:24
  you repeat the measurement, low reliability,
• 36:27
  you certainly can't be accurately measuring the
• 36:30
  true underlying state, validity. Think about
• 36:32
  it. If your bathroom scale gives you wildly different
• 36:35
  readings every time you step on it, it's unreliable,
• 36:38
  and therefore it's not a valid measure of your
• 36:40
  true weight. Establishing validity is often more
• 36:43
  challenging than reliability and involves accumulating
• 36:46
  evidence from various sources. Things like content
• 36:48
  validity, does it cover all aspects of the concept?
• 36:51
  Construct validity, does it relate to other measures
• 36:54
  as expected? Criterion validity, does it correlate
• 36:56
  with a gold standard? To show the measure behaves
• 36:59
  in ways consistent with the theoretical concept
• 37:01
  it's supposed to capture. Right. Let's explore
• 37:04
  some of the different measurement types then,
• 37:06
  starting with patient -reported outcome measures,
• 37:08
  or PROMs, which seem increasingly prominent in
• 37:10
  clinical research, particularly orthopedics.
• 37:13
  Indeed they are. PROMs are standardized, validated
• 37:16
  questionnaires that are completed directly by
• 37:18
  patients themselves. They are specifically designed
• 37:21
  to capture the patient's own perspective on their
• 37:23
  health status, their functional limitations,
• 37:26
  their symptoms like pain or stiffness, and their
• 37:28
  overall well -being or quality of life. And why
• 37:31
  the shift towards using these more? Well, the
• 37:34
  rise of PROMs really reflects a growing recognition
• 37:36
  that the patient's experience and their functional
• 37:38
  status are often the most important outcomes,
• 37:41
  particularly in areas like orthopedics, where
• 37:43
  improving quality of life, reducing pain, and
• 37:46
  restoring function are frequently the primary
• 37:48
  goals of treatment. They provide a more holistic
• 37:51
  picture that clinician -based measures or imaging
• 37:53
  alone might completely miss. The sources list
• 37:56
  several common examples, many specific to musculoskeletal
• 37:59
  health. Things like simple visual analog scales,
• 38:02
  VAS, or numeric rating scales for pain are very
• 38:06
  common. Then you have condition -specific scores
• 38:08
  like the Oxford hip score or Oxford knee score,
• 38:12
  the International Knee Documentation Committee,
• 38:14
  IKDC, subjective form, which is widely used for
• 38:17
  knee function, the hip disability and osteoarthritis
• 38:20
  outcome score, HOS, The Western Ontario and McMaster
• 38:23
  University's Osteoarthritis Index, the Lomax,
• 38:25
  there for hip and knee osteoarthritis symptoms
• 38:27
  and function. For upper limb problems, there's
• 38:30
  the disabilities of the arm, shoulder, and hand,
• 38:33
  DAS, a questionnaire. The Marks Activity Level
• 38:35
  Score is another example, specifically assessing
• 38:37
  how physically active a patient is. And what
• 38:39
  makes a good prom beyond reliability and validity?
• 38:43
  Desirable properties also include acceptability
• 38:45
  to patients. Are the questions understandable?
• 38:47
  Is it too burdensome or long to complete? Feasibility
• 38:51
  and practice is also key. How much time does
• 38:53
  it take to administer and score? The health assessment
• 38:57
  questionnaire, HAQ for instance, is noted as
• 39:00
  being relatively fast while something like the
• 39:02
  SF -36, a widely used general health status measure,
• 39:06
  can vary quite a bit in completion time depending
• 39:07
  on how it's administered on paper, online, or
• 39:10
  via an interview. It's also really important
• 39:12
  to distinguish PROMs, which measure health outcomes,
• 39:16
  from patient -reported experience measures, or
• 39:17
  PRMs. PRMs measure aspects of the care process
• 39:20
  itself, things like communication with staff,
• 39:23
  cleanliness of the facilities, waiting times,
• 39:25
  not the health outcome. And as the source points
• 39:27
  out, studies have generally shown quite a weak
• 39:29
  association between PRMs and PROMs, meaning a
• 39:32
  patient can report a poor experience of care,
• 39:34
  but still have a good clinical outcome, or vice
• 39:36
  versa. they capture fundamentally different things.
• 39:39
  Okay, that clarifies PROMs. And then we have
• 39:42
  the measures taken by clinicians or observers.
• 39:45
  What are some examples there and what are the
• 39:47
  particular challenges associated with those?
• 39:49
  Right, these are the more traditional objective
• 39:51
  or perhaps semi -objective measures performed
• 39:53
  by healthcare professionals. Common examples
• 39:56
  in orthopedics include goniometry to measure
• 39:59
  joint range of motion. Now while using a goniometer
• 40:01
  is generally considered more reliable than just
• 40:03
  visual estimation, it still has limitations,
• 40:06
  especially for small joints, and the measurement
• 40:09
  technique used is absolutely crucial for consistency.
• 40:12
  Radiographic measurement of angles and alignments
• 40:14
  is often the most accurate, but of course is
• 40:16
  limited by radiation exposure and isn't practical
• 40:19
  for frequent follow -up. Muscle power is very
• 40:22
  commonly assessed using the Medical Research
• 40:23
  Council MRC grading system from M0 no contraction,
• 40:28
  up to M5, normal power against full resistance.
• 40:32
  This is widely used but can have subjective elements,
• 40:35
  particularly when distinguishing between grades
• 40:37
  M4, movement against gravity with some resistance,
• 40:40
  and M5, normal. Sensory tests, like two -point
• 40:43
  discrimination to assess nerve function, require
• 40:45
  really careful technique. Applying too much pressure,
• 40:47
  for instance, can give inaccurate results. Grip
• 40:50
  strength measurement needs standardization. Usually
• 40:52
  multiple temps are taken, and whether you report
• 40:55
  the mean or the best result depends on the protocol.
• 40:58
  And crucially, the dynamometer, the instrument
• 41:00
  itself, needs regular calibration to ensure its
• 41:03
  accuracy. And what about classification systems?
• 41:05
  You mentioned the issue with tibial fracture
• 41:07
  classification right at the start. Yes, that's
• 41:10
  a significant challenge, particularly relevant
• 41:12
  to the example you used. Classification systems
• 41:15
  are used everywhere in orthopedics to categorize
• 41:18
  the severity or type of injuries or conditions.
• 41:21
  We have the AO classification for fractures,
• 41:23
  Schatzker for tibial plateau fractures, Gustio
• 41:26
  and Anderson for open fractures, Cheren for assessing
• 41:29
  soft tissue injury around closed fractures. While
• 41:32
  these systems provide a necessary framework for
• 41:34
  communication and research, their reliability,
• 41:37
  particularly that inter -observer reliability,
• 41:40
  the agreement between different clinicians looking
• 41:42
  at the same case can be surprisingly low. Like
• 41:44
  that 60 % figure. Exactly. That 60 % agreement
• 41:48
  rate among experienced orthopedic trauma surgeons
• 41:50
  classifying open tibial fractures using the standard
• 41:53
  Castillo and Anderson system is a stark reminder.
• 41:57
  Even with defined criteria, subjective interpretation
• 42:00
  can lead to considerable disagreement. And if
• 42:02
  different surgeons classify the same injury differently,
• 42:05
  how can you consistently use that classification
• 42:07
  either as a baseline characteristic to ensure
• 42:09
  groups are comparable, or even as an outcome
• 42:12
  measure in research? It introduces significant
• 42:14
  measurement error and reduces the study's power
• 42:16
  and credibility. Other observer -based measures
• 42:19
  mentioned include trauma severity scores like
• 42:21
  TRS, which combine physiological data, like blood
• 42:24
  pressure, and anatomical injury data to predict
• 42:26
  survival after major trauma. Cosmesis, or the
• 42:30
  aesthetic appearance, is another important outcome,
• 42:32
  particularly in specialties like plastic surgery
• 42:34
  or pediatric orthopedics, for instance after
• 42:36
  limb -lengthening procedures. Measuring cosmesis
• 42:39
  is often subjective and can be controversial,
• 42:41
  despite its undeniable psychosocial implications
• 42:44
  for patients. There's a recognized need for validated
• 42:46
  cosmetic outcome measures, which are feasible
• 42:49
  to develop using things like standardized photography
• 42:51
  and scoring systems. For infections like osteomyelitis,
• 42:55
  classification systems exist. The source mentions
• 42:58
  Waldvogel's system, which is more descriptive
• 43:00
  compared to May's system, which is designed more
• 43:03
  to guide treatment and predict prognosis. And
• 43:05
  finally, cost -utility analysis using outcomes
• 43:08
  like Quality Adjusted Life Years, or QLYs, is
• 43:11
  a critical economic outcome measure. A QALY combines
• 43:14
  the length of life gained with a measure of its
• 43:16
  quality, often derived from patient utility scores
• 43:19
  based on questionnaires like the EQ5D. It's highly
• 43:22
  relevant in orthopedics because improving quality
• 43:24
  of life is often the primary benefit of many
• 43:26
  interventions, and 2ALYs allow comparison of
• 43:29
  the cost -effectiveness of interventions across
• 43:31
  different areas of healthcare. That provides
• 43:33
  a really comprehensive overview of the measurement
• 43:35
  challenges. Let's transition now to the equally
• 43:38
  critical foundation of ethical practice in human
• 43:41
  research. The source material outlines several
• 43:43
  key pillars, starting with informed consent.
• 43:46
  We touched on it in the rapid fire, but let's
• 43:48
  delve deeper into what constitutes truly informed
• 43:51
  consent. Informed consent is, without any exaggeration,
• 43:56
  non -negotiable. It's the primary ethical principle
• 43:58
  that upholds the autonomy, the self -determination
• 44:01
  of potential research participants. For consent
• 44:04
  to be truly informed, the participant must be
• 44:07
  provided with all relevant information about
• 44:09
  the study. This includes its purpose, the study
• 44:11
  procedures, what exactly will happen to them,
• 44:14
  how often for how long, any potential risks,
• 44:16
  both common and rare, serious and non -serious.
• 44:20
  Potential benefits, both perhaps directly to
• 44:22
  them, but also importantly to future patients
• 44:24
  or scientific knowledge. What are the alternative
• 44:26
  treatments or procedures available outside the
• 44:28
  study? What are their rights as a participant,
• 44:30
  including the absolute right to withdraw at any
• 44:33
  time without penalty? and information about confidentiality
• 44:36
  and how their data will be handled and protected.
• 44:39
  So it's not just giving them a leaflet? Absolutely
• 44:41
  not. It's not enough just to provide the information,
• 44:44
  perhaps in a lengthy document. The participant
• 44:47
  must also understand it. This often involves
• 44:49
  the researcher checking their comprehension,
• 44:52
  maybe asking them to explain key aspects of the
• 44:54
  study back in their own words. And crucially,
• 44:57
  their decision to participate must be entirely
• 45:00
  voluntary, free from any coercion or undue influence.
• 45:04
  This voluntary aspect can be particularly challenging
• 45:07
  in situations where there is a significant power
• 45:09
  imbalance perhaps between a physician and their
• 45:12
  patient, a professor and their student, or an
• 45:15
  employer and employee. The consent process should
• 45:17
  really be an ongoing conversation, not just a
• 45:20
  one -off signature on a form. And the source
• 45:22
  mentions there are specific populations considered
• 45:25
  vulnerable who require additional protections
• 45:27
  in research. Yes, absolutely. Certain groups
• 45:30
  are deemed vulnerable because various factors
• 45:33
  such as compromised autonomy, cognitive limitations
• 45:36
  being institutionalized, or facing unequal power
• 45:39
  dynamics may affect their ability to give truly
• 45:41
  free and informed consent, or might make them
• 45:44
  more susceptible to coercion or exploitation.
• 45:47
  These groups are afforded extra safeguards in
• 45:50
  research regulations. Who falls into these categories?
• 45:53
  Examples typically include minors, children under
• 45:55
  the legal age of consent, adults with impaired
• 45:58
  mental capacity who would require proxy consent
• 46:01
  from a legally authorized representative, pregnant
• 46:04
  women, fetuses, and neonates due to potential
• 46:07
  risks to the developing child, and prisoners
• 46:09
  due to the constraints on their liberty and potential
• 46:12
  for coercion within the institutional setting.
• 46:14
  For children, specific rules apply, often including
• 46:17
  the need for ongoing permission or assent from
• 46:19
  the child themselves, depending on their age
• 46:21
  and maturity level, in addition to permission
• 46:23
  from their parents or legal guardians. There
• 46:26
  are also very strict conditions under which parental
• 46:28
  permission might be waived, such as in certain
• 46:30
  emergency research settings, but these are tightly
• 46:32
  regulated and reviewed. Another critical ethical
• 46:36
  consideration highlighted is conflicts of interest.
• 46:39
  These seem particularly prevalent or at least
• 46:41
  discussed more often in research with industry
• 46:44
  ties. What constitutes a conflict and why are
• 46:46
  they problematic for research integrity? A conflict
• 46:49
  of interest arises when a person's primary professional
• 46:52
  duty and in research that's primarily upholding
• 46:55
  the integrity of the study and protecting the
• 46:57
  welfare of the participants could be improperly
• 47:00
  influenced by a secondary interest. This secondary
• 47:03
  interest is very often financial, but it could
• 47:06
  also be non -financial things, like seeking academic
• 47:08
  promotion, gaining professional recognition,
• 47:11
  or even just a strong personal belief in the
• 47:13
  value of one's own work or invention. In industry
• 47:16
  -funded research, you often have at least three
• 47:18
  parties involved. The researcher, their institution,
• 47:22
  like a university or hospital, and the funding
• 47:24
  corporation. Each party has its own interests,
• 47:27
  which may not always perfectly align. The corporation
• 47:29
  naturally wants the research to show their product
• 47:31
  is effective and safe. The institution might
• 47:34
  benefit financially from research grants or intellectual
• 47:37
  property rights. The researcher might be funding
• 47:39
  for their lab, publications which help their
• 47:41
  career, and perhaps personal financial benefits
• 47:43
  like consultancy fees or royalties. And the problem
• 47:46
  is the potential for bias. Exactly. Financial
• 47:49
  conflicts of interest are a major concern because
• 47:52
  there's a considerable body of evidence suggesting
• 47:54
  they can subtly, or sometimes even overtly, bias
• 47:58
  the research process and its outcomes. For example,
• 48:01
  a researcher receiving significant royalties
• 48:03
  from an implant they helped develop might, consciously
• 48:06
  or unconsciously, be biased towards designing
• 48:09
  studies or interpreting results in a way that
• 48:11
  favors that implant. Professional bodies, like
• 48:14
  the AAOS, American Academy of Orthopedic Surgeons,
• 48:18
  mentioned at the source, often have principals
• 48:20
  recommending stringent measures to manage these
• 48:22
  conflicts. These might include things like avoiding
• 48:25
  trading stocks in the funding company during
• 48:26
  the trial, and having research on one's own royalty
• 48:29
  -generating products conducted by an independent,
• 48:32
  disinterested third party if possible. The evidence
• 48:35
  base as noted in the source does suggest a correlation.
• 48:37
  Clinical trials with significant industry funding
• 48:40
  or author ties are statistically more likely
• 48:43
  to report results favorable to the industry sponsor
• 48:45
  compared to independently funded trials. Now
• 48:48
  this doesn't automatically mean all industry
• 48:50
  funded research is biased, but it certainly highlights
• 48:53
  the inherent risk and the absolute need for transparency
• 48:56
  and robust management strategies for these conflicts.
• 48:59
  Full disclosure is the minimum standard. It certainly
• 49:02
  seems many of these ethical principles we take
• 49:04
  for granted now were solidified in response to
• 49:07
  some quite shocking past abuses in research.
• 49:11
  The source material provides some sobering historical
• 49:13
  examples. Yes. Tragically, many of the current
• 49:17
  ethical regulations and oversight mechanisms
• 49:19
  we have are direct responses to egregious historical
• 49:23
  violations of ethical principles. The Tuskegee
• 49:25
  syphilis experiment, which ran for decades in
• 49:28
  the US, is perhaps the most infamous example.
• 49:30
  African -American men with syphilis were deliberately
• 49:33
  denied effective treatment penicillin long after
• 49:36
  it became widely available, purely so researchers
• 49:38
  could study the natural progression of the untreated
• 49:40
  disease. Participants were actively deceived
• 49:43
  and profoundly harmed. The Stanford Prison Experiment,
• 49:47
  though different in nature and smaller scale,
• 49:49
  highlighted the ethical failure of researchers
• 49:51
  to adequately protect participants and to stop
• 49:53
  the study when individuals were clearly experiencing
• 49:55
  severe psychological distress and felt unable
• 49:57
  to withdraw. And then there was Project DemCultra,
• 50:01
  involving CIA -sponsored experiments often conducted
• 50:03
  on unwitting human subjects, using drugs and
• 50:05
  psychological manipulation techniques in the
• 50:07
  mid -20th century. And these led to real changes
• 50:10
  in oversight. Absolutely. Public outcry and subsequent
• 50:13
  investigations into these and other abuses led
• 50:15
  to significant reforms. In the U .S., the National
• 50:19
  Research Act of 1974 was a landmark piece of
• 50:21
  legislation. It established the National Commission
• 50:24
  for the Protection of Human Subjects of Biomedical
• 50:26
  and Behavioral Research. This commission produced
• 50:29
  the Pivotal Belmont Report, which outlined the
• 50:31
  three core ethical principles that now guide
• 50:34
  human research. Respect for persons. autonomy
• 50:37
  and protection of the vulnerable, beneficence,
• 50:40
  do no harm, maximize benefits, and justice, fair
• 50:43
  distribution of burdens and benefits. This eventually
• 50:45
  led to the development of the Common Rule, which
• 50:47
  is the set of federal regulations governing most
• 50:49
  research involving human subjects in the U .S.
• 50:52
  It mandates things like institutional review
• 50:54
  boards, IRBs, or ethics committees, IECs, to
• 50:58
  review and approve research protocols before
• 51:00
  they begin, requires detailed and formed written
• 51:02
  consent, and sets out specific requirements for
• 51:05
  risk benefit assessments and ensuring subject
• 51:07
  rights and welfare are protected. Internationally,
• 51:10
  guidelines like the International Council for
• 51:11
  Harmonization, ICH, Good Clinical Practice, GCP,
• 51:16
  provide a unified standard for ensuring the quality,
• 51:18
  integrity, and ethical conduct of clinical trials
• 51:21
  involving human participants globally. These
• 51:24
  historical lessons are absolutely vital reminders
• 51:26
  of why these protections are so necessary and
• 51:28
  must be rigorously upheld. So with all these
• 51:31
  layers of design, analysis, measurement, and
• 51:33
  ethics, who ultimately bears the responsibility
• 51:36
  for ensuring a study is conducted properly and
• 51:39
  ethically on the ground? The primary responsibility
• 51:42
  falls squarely on the shoulders of the principal
• 51:44
  investigator, or PI. This is the individual,
• 51:48
  usually a clinician or senior researcher, who
• 51:50
  leads the research team at a particular study
• 51:52
  site. The PI is accountable for the overall conduct
• 51:55
  of the study at their site and, critically, for
• 51:58
  protecting the rights, safety, and welfare of
• 52:00
  the research subjects under their care. They
• 52:03
  are the one ultimately answerable to the sponsor,
• 52:05
  the ethics committee, and regulatory authorities
• 52:07
  for what happens at their site. What are some
• 52:09
  of the key responsibilities of a PI, particularly
• 52:12
  under guidelines like GCP? Under GCP, the PI
• 52:15
  has a wide range of specific responsibilities.
• 52:17
  They must ensure that the study is conducted
• 52:20
  strictly in compliance with the ethically approved
• 52:22
  protocol, those core ethical principles we discussed,
• 52:25
  like informed consent, and all relevant local
• 52:28
  and international regulatory requirements. If
• 52:31
  any deviation from the protocol is necessary,
• 52:33
  for example, if a patient's immediate safety
• 52:36
  requires a different clinical action, the PI
• 52:38
  must document it thoroughly and report it appropriately.
• 52:42
  Importantly, any significant changes or amendments
• 52:44
  to the protocol generally require prior review
• 52:47
  and approval from the Ethics Committee, IRBIEC,
• 52:50
  before they are implemented. The exception is,
• 52:53
  if a deviation is immediately necessary to eliminate
• 52:55
  an unforeseen hazard to a participant, in that
• 52:58
  case, it should be done, but then documented
• 53:01
  and reported to the Ethics Committee and sponsored
• 53:03
  immediately afterwards. The PI is also responsible
• 53:06
  for ensuring subjects are kept adequately informed
• 53:08
  throughout the study and obtaining re -consent
• 53:11
  if significant new information about risks or
• 53:13
  potential benefits emerges, or if there are major
• 53:15
  changes to the study protocol that might affect
• 53:17
  their willingness to continue. They must also
• 53:20
  ensure the accuracy, completeness, and integrity
• 53:22
  of all the study data collected at their site.
• 53:25
  You mentioned recruitment earlier as being a
• 53:28
  major challenge, particularly for RCTs. How does
• 53:31
  that impact the PI specifically, and what are
• 53:33
  some common issues they might face in just managing
• 53:35
  the day -to -day trial operations? Yes. Poor
• 53:39
  recruitment and participant retention is, unfortunately,
• 53:42
  one of the most common reasons why clinical trials
• 53:44
  fail to meet their objectives or sometimes even
• 53:47
  fail to complete at all. The PI needs to be realistic
• 53:50
  when predicting recruitment rates for their site
• 53:53
  during the planning phase. They need to identify
• 53:56
  potential factors that might hinder enrollment.
• 53:59
  Maybe the protocol is extremely complex with
• 54:00
  lots of visits, perhaps the eligibility criteria
• 54:03
  are very strict, or maybe patients in their locality
• 54:06
  just aren't interested in this specific comparison
• 54:08
  being made. They need to regularly review recruitment
• 54:11
  progress against the targets. If it's lagging
• 54:14
  behind, the PI needs to proactively discuss the
• 54:17
  difficulties with their research staff and the
• 54:19
  trial sponsor. Solutions might involve providing
• 54:21
  additional training or resources to staff, looking
• 54:24
  at ways to simplify aspects of the protocol if
• 54:26
  feasible, which would need ethics approval, collaborating
• 54:29
  with other sites, or perhaps requesting an extension
• 54:31
  to the recruitment period if resources allow.
• 54:34
  It requires active management. Managing the overall
• 54:37
  trial operations involves significant risk management
• 54:39
  as well. Essential documents like the detailed
• 54:41
  protocol itself, standard operating procedures,
• 54:44
  SOPs, for various tasks like data entry or sample
• 54:47
  handling, the monitoring plan from the sponsor,
• 54:50
  and all the ethics committee approvals must be
• 54:52
  meticulously organized and in place before the
• 54:55
  very first subject is enrolled. Regulatory audits
• 54:58
  or inspections often uncover common findings.
• 55:00
  Things like inadequate monitoring by the sponsor,
• 55:04
  instances of PI noncompliance with the protocol
• 55:06
  or regulations, deficiencies in the informed
• 55:08
  consent process or documentation, and issues
• 55:11
  with accountability for the investigational product,
• 55:14
  like ensuring the correct drug or device is stored,
• 55:16
  dispensed, used, and tracked properly. A diligent
• 55:20
  PI and their research team need to be proactive
• 55:22
  in identifying and mitigating these potential
• 55:23
  risks and ready to implement swift, corrective,
• 55:26
  and preventive actions if any issues are identified
• 55:28
  during monitoring or audits. Right, so assuming
• 55:31
  the trial runs successfully, all the data are
• 55:33
  collected and analyzed. The findings then need
• 55:36
  to be communicated, typically through writing
• 55:38
  a research manuscript for publication. The source
• 55:41
  discusses the standard MRAD structure. Can you
• 55:45
  quickly break down what should ideally go into
• 55:47
  each of those sections for clarity? Yes, MRAD.
• 55:51
  It stands for Introduction, Methods, Results,
• 55:54
  and Discussion. It's the universally accepted
• 55:57
  structure for most original scientific research
• 55:59
  manuscripts because it provides a logical and
• 56:01
  predictable flow. It allows readers to quickly
• 56:04
  understand why the study was done. Introduction.
• 56:07
  How it was done. Methods. What was found. Results.
• 56:10
  And what those findings mean. Discussion. So
• 56:13
  starting with the introduction. The introduction
• 56:15
  sets the stage. It should provide the necessary
• 56:17
  background context for the research, explain
• 56:19
  the existing knowledge gap or the clinical problem
• 56:21
  the study addresses, clearly state the specific
• 56:24
  research question or hypothesis being tested,
• 56:26
  and finish by presenting the study's primary
• 56:28
  aim or objective. Then the method sounds crucial
• 56:31
  for judging quality. The method section is absolutely
• 56:33
  critical for transparency and reproducibility.
• 56:36
  It needs to describe exactly how the study was
• 56:38
  conducted in sufficient detail that another researcher
• 56:41
  could, at least in theory, replicate it. This
• 56:44
  includes detailing the study design, for example,
• 56:47
  RCT, cohort study, the setting where it took
• 56:50
  place, the participant selection criteria, inclusion
• 56:53
  and exclusion, and how participants were recruited.
• 56:57
  It needs to describe the interventions or exposures
• 56:59
  being studied, define the primary and secondary
• 57:02
  outcome measures used, and exactly how they were
• 57:04
  assessed. Information about the sample size calculation
• 57:07
  and the statistical methods used for the analysis
• 57:10
  must also be included here. Crucially, the description
• 57:13
  of the study sample itself, demographics, baseline
• 57:15
  characteristics of the participants, often presented
• 57:18
  in Table 1 belongs in the methods section, not
• 57:20
  the results. Okay, results, just the facts. The
• 57:23
  results section should present the findings objectively,
• 57:26
  without interpretation or speculation at this
• 57:28
  stage. You report the data, often using tables
• 57:30
  and figures for clarity. You should include key
• 57:33
  descriptive statistics like means, standard deviations,
• 57:37
  medians, ranges, and the results of your inferential
• 57:40
  statistical tests, making sure to include the
• 57:42
  numerical values for effect sizes, like mean
• 57:44
  differences or odds ratios, their confidence
• 57:47
  intervals, and the exact p -values. If you use
• 57:50
  the word significant in a statistical context,
• 57:53
  it absolutely must be followed by the corresponding
• 57:55
  p -value or CI to justify that claim. You should
• 57:58
  focus on reporting the main findings that directly
• 58:00
  relate to your primary study objectives, especially
• 58:03
  if space is limited. And finally, the discussion
• 58:06
  is where you interpret it all. Exactly. The discussion
• 58:09
  section is where you interpret your results and
• 58:11
  place them in the broader context. You typically
• 58:14
  start by briefly summarizing your study's most
• 58:16
  important findings, answering the research question
• 58:18
  you post in the introduction. Then you compare
• 58:21
  and contrast your results with previous research
• 58:23
  in the field. Do your findings support contradict,
• 58:26
  or perhaps add nuance to what was already known.
• 58:30
  You should discuss the clinical relevance or
• 58:32
  implications of your findings. What do they mean
• 58:34
  for patient care, clinical practice, or perhaps
• 58:37
  health policy? And this is absolutely vital for
• 58:40
  trustworthiness and scientific integrity. You
• 58:42
  must explicitly discuss your study's limitations.
• 58:45
  No study is perfect. Honestly, acknowledging
• 58:48
  the limitations, perhaps potential biases that
• 58:50
  couldn't be fully controlled, issues with measurement
• 58:53
  tools, constraints imposed by the sample size
• 58:55
  or duration, or factors affecting the generalizability
• 58:58
  of your findings significantly increases the
• 58:59
  credibility of your work. Discussing limitations
• 59:02
  also often helps to identify remaining uncertainties
• 59:05
  and generate important ideas for future research.
• 59:08
  And using reporting guidelines helps with this.
• 59:10
  Yes, adhering to specific reporting guidelines,
• 59:13
  like the Prisma Statement for Systematic Reviews
• 59:15
  and Meta -Analyses, or CONSORT for RCTs, helps
• 59:19
  ensure that all the essential information is
• 59:21
  included in a standardized way, which improves
• 59:23
  the quality and transparency of reporting. The
• 59:26
  whole submission process to a journal involves
• 59:29
  gathering all these materials, including increasingly
• 59:32
  detailed disclosures of any potential financial
• 59:34
  conflicts of interest from all authors, again
• 59:37
  to promote transparency. That was a really thorough
• 59:40
  breakdown of the entire research journey from
• 59:42
  initial design right through to publication.
• 59:45
  Let's maybe transition into a quick lightning
• 59:46
  round just to reinforce a few of the key concepts
• 59:48
  we've discussed. All right, lightning round time.
• 59:52
  Beyond the p -value, what is one specific statistical
• 59:55
  concept that professionals should always pay
• 59:57
  close attention to when they're reading research
• 59:59
  papers? Confidence intervals. Always look at
• 1:00:02
  the range of the plausible effect and crucially
• 1:00:04
  how that range relates to what you'd consider
• 1:00:06
  clinically important, not just whether it happens
• 1:00:09
  across zero or not. Good one. Okay, if you're
• 1:00:11
  reading a paper and something feels a bit off
• 1:00:14
  or maybe too good to be true, what's a potential
• 1:00:17
  red flag that might suggest reporting bias or
• 1:00:20
  maybe even p -hacking could be involved? Hmm.
• 1:00:23
  Watch out for selective reporting. For instance,
• 1:00:26
  if lots of different outcomes were measured according
• 1:00:28
  to the methods section, but only one or two are
• 1:00:31
  highlighted as significant in the results section
• 1:00:34
  without mentioning the others. or perhaps a lack
• 1:00:37
  of transparency about exactly how the statistical
• 1:00:39
  analyses were conducted, especially if the results
• 1:00:42
  seem unexpectedly strong or clean. Make sense.
• 1:00:45
  And for someone perhaps new to research ethics
• 1:00:48
  or wanting a refresher, where is a key place
• 1:00:50
  they could start looking for foundational information
• 1:00:52
  on the ethical standards for human studies? Well,
• 1:00:55
  searching for resources on the U .S. Common Rule,
• 1:00:58
  especially if working with U .S. federally funded
• 1:00:59
  research, is a good starting point there. or
• 1:01:02
  for broader international guidance that's widely
• 1:01:04
  adopted, looking up the ICH Good Clinical Practice
• 1:01:07
  GCP guidelines would be an excellent place to
• 1:01:09
  begin. They cover ethical conduct and data quality.
• 1:01:13
  Excellent. Great. Quick recap. So pulling all
• 1:01:16
  of this together now, thinking about our listeners
• 1:01:18
  trying to navigate this complex world, what are
• 1:01:21
  the key actionable takeaways they should really
• 1:01:23
  keep in mind when they're trying to evaluate
• 1:01:25
  clinical research findings for themselves? Okay.
• 1:01:28
  Key takeaways. First, I'd say use the hierarchy
• 1:01:30
  of evidence as a useful starting guide, but always
• 1:01:33
  remember its limitations. Really try to understand
• 1:01:36
  the inherent strengths and weaknesses of the
• 1:01:38
  specific study design you're looking at. RCTs
• 1:01:40
  are theoretically strong against bias, but hard
• 1:01:43
  to do well, especially in surgery. Systematic
• 1:01:46
  reviews pool evidence, but remember they're only
• 1:01:49
  ever as good as the primary studies they include.
• 1:01:52
  Be critical of the design itself. Second, scrutinize
• 1:01:55
  how the outcomes were measured. Was a patient
• 1:01:57
  reported using a validated PROM? Was a clinician
• 1:02:00
  observed using a potentially less reliable classification
• 1:02:03
  system? Understand the importance of demonstrated
• 1:02:06
  reliability and validity for those specific measures
• 1:02:09
  used in that study. Critically ask, does the
• 1:02:11
  measure actually capture what truly matters in
• 1:02:13
  clinical practice or, more importantly, to the
• 1:02:15
  patient? Third, be constantly vigilant for potential
• 1:02:18
  biases and underlying ethical considerations.
• 1:02:20
  Look for subtle signs of selection bias in how
• 1:02:23
  participants were recruited or allocated to groups.
• 1:02:25
  Think about potential reporting bias and how
• 1:02:27
  the results are presented. Are they telling the
• 1:02:29
  whole story? And always consider whether potential
• 1:02:31
  conflicts of interest, especially financial ones,
• 1:02:33
  might plausibly influence the interpretation
• 1:02:35
  or conclusions drawn by the authors. Fourth,
• 1:02:39
  make sure you go beyond the p -value. While statistical
• 1:02:43
  significance tells you something about the likelihood
• 1:02:44
  of the results being due purely to chance under
• 1:02:47
  the null hypothesis, always use the confidence
• 1:02:50
  interval. Use it to understand the potential
• 1:02:52
  magnitude and the precision of the treatment
• 1:02:54
  effect, and then use that information to gauge
• 1:02:56
  its actual clinical relevance or importance.
• 1:03:00
  And finally, I'd say look for transparency and
• 1:03:02
  honesty in the reporting itself. Does the paper
• 1:03:05
  follow a standard logical structure like MRAD?
• 1:03:08
  Do the authors openly and frankly discuss their
• 1:03:10
  studies' limitations? Acknowledging limitations
• 1:03:13
  isn't a sign of weakness, it's actually a sign
• 1:03:15
  of rigorous, trustworthy research, and it helps
• 1:03:18
  you understand where the evidence is still uncertain,
• 1:03:20
  or where more research is needed. Those are incredibly
• 1:03:23
  valuable filters, I think, for anyone to apply
• 1:03:25
  when they're trying to sift through the, frankly,
• 1:03:28
  overwhelming amount of medical literature out
• 1:03:30
  there. Thank you so much for guiding us through
• 1:03:32
  this really detailed deep dive into clinical
• 1:03:36
  research design, analysis, outcome measurement,
• 1:03:38
  and all those crucial ethical considerations.
• 1:03:41
  It's been my pleasure. It is a complex area,
• 1:03:43
  but as you say, it's absolutely vital for anyone
• 1:03:45
  involved in health care, whether research or
• 1:03:47
  clinician or even patient, to have a basic grasp
• 1:03:50
  of these principles. Absolutely. And if you found
• 1:03:53
  this deep dive into evaluating clinical research
• 1:03:56
  valuable, perhaps consider sharing it with a
• 1:03:58
  colleague who might also benefit or leaving us
• 1:04:01
  a rating wherever you happen to get your podcasts.
• 1:04:03
  It genuinely helps other professionals find the
• 1:04:06
  show. So the next time you pick up a research
• 1:04:08
  paper, maybe don't just look for that headline
• 1:04:10
  p -value or the simple conclusion in the abstract.
• 1:04:13
  Ask yourself, what did they really measure here?
• 1:04:16
  And can I actually trust that measurement? And
• 1:04:18
  perhaps more importantly, what does that confidence
• 1:04:21
  interval will really tell me about whether this
• 1:04:22
  finding actually matters for my patient or for
• 1:04:25
  my practice. That question I think is often much
• 1:04:28
  harder to answer than just looking at the statistics
• 1:04:29
  alone, but it's infinitely more important in
• 1:04:32
  the long run. That's all for this deep dive.
• 1:04:35
  Join us next time as we unpack another complex
• 1:04:37
  topic.