Latest / Pearls and Prep / T-Tests and Tummy Aches: How to Read a Research Study Like a Pro

Transcript

T-Tests and Tummy Aches: How to Read a Research Study Like a Pro · Pearls and Prep

• 0:00
  Foreign.
• 0:04
  Welcome
• 0:05
  back,
• 0:05
  my
• 0:05
  friends,
• 0:05
  to
• 0:06
  another
• 0:06
  episode
• 0:06
  of
• 0:07
  Pearls
• 0:07
  and
• 0:07
  Prep.
• 0:07
  This
• 0:08
  one's
• 0:08
  a
• 0:08
  fun
• 0:08
  one.
• 0:09
  We're
• 0:09
  going
• 0:09
  to
• 0:10
  talk
• 0:10
  about
• 0:10
  research.
• 0:11
  We're
• 0:11
  going
• 0:11
  to
• 0:11
  make
• 0:11
  research
• 0:12
  more
• 0:12
  fun.
• 0:13
  Research
• 0:13
  articles,
• 0:14
  research
• 0:14
  terms.
• 0:15
  Terms
• 0:15
  that
• 0:16
  maybe
• 0:16
  make
• 0:16
  you
• 0:16
  super
• 0:17
  confused,
• 0:17
  like
• 0:18
  P
• 0:18
  value
• 0:19
  and
• 0:19
  effect
• 0:20
  size
• 0:21
  and
• 0:21
  odds
• 0:22
  ratios.
• 0:23
  How
• 0:23
  frightening.
• 0:24
  No,
• 0:24
  we're
• 0:24
  going
• 0:25
  to
• 0:25
  demystify
• 0:25
  that.
• 0:26
  And
• 0:26
  this
• 0:27
  is
• 0:27
  not
• 0:27
  boring,
• 0:28
  so
• 0:28
  there's
• 0:28
  no
• 0:29
  need
• 0:29
  for
• 0:29
  you
• 0:29
  to
• 0:29
  fall
• 0:29
  asleep.
• 0:30
  This
• 0:30
  stuff
• 0:30
  can
• 0:31
  actually
• 0:31
  be
• 0:31
  really
• 0:31
  fun.
• 0:32
  And
• 0:32
  honestly,
• 0:32
  it
• 0:33
  makes
• 0:33
  you
• 0:33
  a
• 0:34
  little
• 0:34
  more
• 0:34
  literate.
• 0:35
  It
• 0:35
  allows
• 0:35
  you
• 0:35
  to
• 0:36
  understand
• 0:36
  what's
• 0:36
  happening
• 0:37
  and
• 0:38
  understand
• 0:39
  your
• 0:39
  research
• 0:39
  articles
• 0:40
  more
• 0:40
  richly.
• 0:41
  And
• 0:41
  it
• 0:42
  allows
• 0:42
  you
• 0:42
  to
• 0:42
  ask
• 0:42
  better
• 0:43
  questions
• 0:43
  and
• 0:43
  really
• 0:44
  understand
• 0:44
  the
• 0:44
  nuances
• 0:45
  of
• 0:45
  things
• 0:46
  that
• 0:46
  are
• 0:46
  being
• 0:46
  presented
• 0:47
  to
• 0:47
  you.
• 0:47
  It's
• 0:48
  sort
• 0:48
  of
• 0:48
  like
• 0:48
  learning
• 0:49
  a
• 0:49
  new
• 0:49
  language.
• 0:49
  And
• 0:49
  it's
• 0:50
  super
• 0:50
  fun.
• 0:50
  Now,
• 0:51
  my
• 0:51
  friends,
• 0:52
  don't
• 0:52
  get
• 0:52
  bogged
• 0:53
  down
• 0:53
  in
• 0:53
  my
• 0:53
  calculations
• 0:54
  or
• 0:54
  my
• 0:55
  math.
• 0:55
  That's
• 0:55
  not
• 0:55
  the
• 0:56
  point.
• 0:56
  Right.
• 0:56
  The
• 0:57
  point
• 0:57
  of
• 0:57
  this
• 0:57
  is
• 0:57
  not
• 0:58
  a
• 0:58
  math
• 0:59
  exercise.
• 1:00
  The
• 1:00
  goal
• 1:00
  here
• 1:00
  is
• 1:00
  for
• 1:01
  you
• 1:01
  to
• 1:01
  gain
• 1:02
  a
• 1:02
  finer
• 1:03
  conceptual
• 1:04
  appreciation
• 1:05
  of
• 1:05
  research
• 1:06
  terms
• 1:06
  and
• 1:07
  familiarity
• 1:08
  with
• 1:08
  kinds
• 1:08
  of
• 1:09
  studies
• 1:09
  and
• 1:10
  jargon
• 1:10
  and
• 1:11
  things
• 1:11
  like
• 1:11
  that.
• 1:11
  It's
• 1:11
  not
• 1:12
  for
• 1:12
  you
• 1:12
  to
• 1:13
  calculate
• 1:13
  P
• 1:14
  values
• 1:14
  or
• 1:15
  standard
• 1:15
  deviations.
• 1:16
  Leave
• 1:16
  that
• 1:16
  up
• 1:16
  to
• 1:16
  the
• 1:16
  pros,
• 1:17
  the
• 1:17
  statisticians.
• 1:18
  You
• 1:18
  are
• 1:19
  going
• 1:19
  to
• 1:19
  just
• 1:19
  have
• 1:19
  to
• 1:19
  have
• 1:20
  an
• 1:20
  understanding,
• 1:20
  though,
• 1:21
  of
• 1:21
  why
• 1:21
  we're
• 1:21
  even
• 1:22
  calculating
• 1:22
  standard
• 1:23
  deviation.
• 1:24
  What
• 1:24
  is
• 1:24
  a
• 1:24
  T
• 1:24
  test?
• 1:25
  When
• 1:25
  do
• 1:25
  we
• 1:25
  use
• 1:26
  a
• 1:26
  T
• 1:26
  test?
• 1:26
  Those
• 1:27
  things
• 1:27
  will
• 1:27
  make
• 1:27
  you
• 1:27
  literate
• 1:28
  and
• 1:28
  more
• 1:29
  excited
• 1:30
  to
• 1:30
  approach
• 1:30
  research
• 1:31
  and
• 1:31
  less
• 1:31
  intimidating.
• 1:32
  So,
• 1:33
  my
• 1:33
  friends,
• 1:33
  we
• 1:34
  are
• 1:34
  going
• 1:34
  to
• 1:34
  talk
• 1:34
  about
• 1:35
  jelly
• 1:35
  beans
• 1:35
  and
• 1:36
  tummy
• 1:36
  aches.
• 1:36
  Okay,
• 1:37
  it's
• 1:37
  more
• 1:37
  of
• 1:37
  a
• 1:37
  conversation.
• 1:38
  You
• 1:38
  know
• 1:38
  me.
• 1:39
  This
• 1:39
  is
• 1:39
  not
• 1:40
  a
• 1:40
  lecture
• 1:40
  hall.
• 1:41
  This
• 1:41
  is
• 1:41
  a
• 1:41
  conversation.
• 1:43
  And
• 1:43
  if
• 1:43
  I
• 1:44
  jump
• 1:44
  a
• 1:44
  little
• 1:44
  bit
• 1:45
  to
• 1:45
  different
• 1:46
  topics,
• 1:46
  again,
• 1:46
  we're
• 1:47
  having
• 1:47
  a
• 1:47
  conversation.
• 1:48
  I
• 1:48
  don't
• 1:48
  want
• 1:48
  you
• 1:48
  to
• 1:48
  think
• 1:48
  there's
• 1:49
  some
• 1:49
  kind
• 1:49
  of
• 1:49
  midterm.
• 1:50
  So
• 1:50
  just
• 1:50
  relax
• 1:51
  and
• 1:51
  kick
• 1:51
  back
• 1:51
  and
• 1:52
  hear
• 1:52
  about
• 1:52
  this
• 1:53
  study.
• 1:53
  Let's
• 1:53
  enjoy
• 1:54
  this
• 1:54
  together,
• 1:54
  my
• 1:55
  friends.
• 1:55
  So
• 1:56
  at
• 1:56
  Candyville
• 1:57
  Elementary
• 1:57
  School,
• 1:57
  we
• 1:57
  want
• 1:58
  to
• 1:58
  figure
• 1:58
  out
• 1:58
  if
• 1:59
  jelly
• 1:59
  beans
• 2:00
  cause
• 2:00
  tummy
• 2:01
  aches.
• 2:01
  And
• 2:02
  that's
• 2:02
  our
• 2:02
  research
• 2:03
  question.
• 2:03
  That's
• 2:03
  what
• 2:03
  we're
• 2:04
  setting
• 2:04
  out
• 2:04
  to
• 2:04
  do.
• 2:05
  Every
• 2:05
  good
• 2:05
  study
• 2:05
  should
• 2:06
  start
• 2:06
  with
• 2:06
  a
• 2:06
  research
• 2:07
  question.
• 2:07
  It
• 2:08
  should
• 2:08
  have
• 2:08
  a
• 2:08
  clearly
• 2:09
  definable
• 2:09
  goal
• 2:10
  or
• 2:10
  objective.
• 2:11
  We
• 2:11
  want
• 2:11
  to
• 2:11
  see
• 2:11
  if
• 2:12
  jelly
• 2:12
  beans
• 2:13
  cause
• 2:13
  tummy
• 2:14
  aches.
• 2:14
  That's
• 2:15
  the
• 2:15
  problem
• 2:15
  we're
• 2:15
  trying
• 2:15
  to
• 2:16
  solve.
• 2:16
  Because
• 2:16
  if
• 2:16
  jelly
• 2:17
  beans
• 2:17
  cause
• 2:17
  tummy
• 2:17
  aches,
• 2:18
  we
• 2:18
  probably
• 2:18
  don't
• 2:19
  want
• 2:19
  to
• 2:19
  serve
• 2:19
  jelly
• 2:19
  beans
• 2:20
  anymore
• 2:20
  in
• 2:20
  the
• 2:20
  cafeteria.
• 2:21
  Right
• 2:21
  now
• 2:21
  in
• 2:21
  Candyville,
• 2:22
  there's
• 2:23
  this
• 2:23
  huge
• 2:24
  free
• 2:24
  for
• 2:25
  all
• 2:25
  jelly
• 2:25
  bean
• 2:26
  machine
• 2:26
  that
• 2:26
  just
• 2:26
  spits
• 2:27
  out
• 2:27
  jelly
• 2:27
  beans
• 2:27
  anytime
• 2:28
  you
• 2:28
  want
• 2:28
  them.
• 2:28
  We're
• 2:29
  going
• 2:29
  to
• 2:29
  try
• 2:29
  to
• 2:29
  figure
• 2:29
  out
• 2:30
  is
• 2:30
  this
• 2:30
  a
• 2:30
  good
• 2:30
  idea
• 2:31
  for
• 2:31
  school
• 2:31
  policy
• 2:31
  or
• 2:32
  not.
• 2:32
  You
• 2:32
  could
• 2:33
  probably
• 2:33
  already
• 2:33
  see
• 2:34
  the
• 2:34
  parallels
• 2:34
  between
• 2:34
  a
• 2:35
  study
• 2:35
  like
• 2:35
  this.
• 2:35
  Does
• 2:36
  this
• 2:36
  antidepressant
• 2:37
  work?
• 2:37
  You
• 2:37
  start
• 2:37
  out
• 2:37
  with
• 2:38
  a
• 2:38
  problem
• 2:38
  with
• 2:38
  an
• 2:39
  area
• 2:39
  of
• 2:39
  focus.
• 2:39
  It's
• 2:40
  very
• 2:40
  specific.
• 2:41
  And
• 2:41
  we
• 2:41
  have
• 2:41
  a
• 2:41
  primary
• 2:42
  outcome
• 2:42
  measure
• 2:43
  which
• 2:43
  is
• 2:43
  tummy
• 2:43
  aches.
• 2:44
  Tummy
• 2:45
  aches
• 2:45
  are
• 2:45
  a
• 2:45
  primary
• 2:46
  outcome
• 2:46
  measure.
• 2:46
  What
• 2:47
  does
• 2:47
  that
• 2:47
  mean?
• 2:47
  It
• 2:47
  means
• 2:48
  that's
• 2:48
  the
• 2:48
  thing
• 2:48
  we're
• 2:49
  measuring
• 2:49
  this
• 2:49
  by.
• 2:50
  How
• 2:50
  do
• 2:50
  we
• 2:50
  know
• 2:50
  if
• 2:51
  jelly
• 2:51
  beans
• 2:51
  cause
• 2:52
  tummy
• 2:52
  aches?
• 2:53
  The
• 2:53
  tummy
• 2:53
  ache
• 2:54
  is
• 2:54
  our
• 2:54
  primary
• 2:55
  outcome
• 2:55
  measure.
• 2:56
  Our
• 2:56
  secondary
• 2:57
  outcome
• 2:57
  measure
• 2:58
  could
• 2:58
  be
• 2:58
  your
• 2:58
  pain.
• 2:59
  How
• 2:59
  much
• 3:00
  pain
• 3:00
  these
• 3:00
  children
• 3:01
  get
• 3:01
  when
• 3:01
  they
• 3:01
  have
• 3:02
  tummy
• 3:02
  aches.
• 3:02
  So
• 3:03
  you
• 3:03
  have
• 3:03
  a
• 3:03
  primary
• 3:03
  outcome
• 3:04
  measure.
• 3:04
  Tummy
• 3:04
  aches
• 3:05
  secondary
• 3:06
  outcome
• 3:06
  measure
• 3:06
  would
• 3:07
  be
• 3:07
  your
• 3:07
  pain
• 3:08
  on
• 3:08
  a
• 3:08
  0
• 3:08
  to
• 3:09
  10
• 3:09
  scale
• 3:09
  in
• 3:09
  your
• 3:10
  tummy
• 3:10
  from
• 3:10
  your
• 3:11
  tummy
• 3:11
  ache.
• 3:12
  Now
• 3:13
  always
• 3:13
  be
• 3:13
  mindful
• 3:14
  when
• 3:14
  you're
• 3:14
  looking
• 3:15
  at
• 3:15
  primary
• 3:15
  and
• 3:15
  secondary
• 3:16
  outcome
• 3:16
  measures.
• 3:17
  How's
• 3:17
  your
• 3:18
  study
• 3:18
  really
• 3:18
  quantifying
• 3:19
  that?
• 3:19
  Is
• 3:19
  the
• 3:19
  kid
• 3:20
  just
• 3:20
  saying
• 3:20
  I
• 3:20
  have
• 3:20
  a
• 3:20
  tummy
• 3:21
  ache?
• 3:21
  Well,
• 3:21
  okay,
• 3:22
  that's
• 3:23
  not
• 3:23
  nothing,
• 3:24
  but
• 3:25
  how
• 3:25
  do
• 3:25
  we
• 3:25
  define
• 3:26
  tummy
• 3:26
  aches?
• 3:27
  Just
• 3:27
  like
• 3:27
  we
• 3:27
  say
• 3:27
  how
• 3:28
  do
• 3:28
  we
• 3:28
  define
• 3:28
  depression?
• 3:29
  What
• 3:30
  scale
• 3:30
  are
• 3:30
  we
• 3:30
  using?
• 3:31
  What
• 3:31
  number
• 3:32
  on
• 3:32
  the
• 3:32
  scale
• 3:32
  defines
• 3:33
  depression?
• 3:34
  Are
• 3:34
  we
• 3:34
  defining
• 3:35
  depression
• 3:35
  through
• 3:36
  some
• 3:36
  kind
• 3:36
  of
• 3:36
  interview
• 3:38
  or
• 3:38
  are
• 3:38
  we
• 3:38
  doing
• 3:38
  it
• 3:38
  through
• 3:38
  an
• 3:39
  official
• 3:39
  metric
• 3:39
  like
• 3:39
  the
• 3:40
  PHQ9?
• 3:41
  In
• 3:41
  this
• 3:42
  case,
• 3:42
  how
• 3:43
  are
• 3:43
  we
• 3:43
  defining
• 3:43
  tummy
• 3:44
  aches?
• 3:44
  Just
• 3:44
  self
• 3:44
  report
• 3:45
  the
• 3:45
  child
• 3:45
  saying
• 3:46
  it
• 3:46
  or
• 3:46
  is
• 3:46
  it
• 3:47
  a
• 3:47
  visit
• 3:47
  to
• 3:47
  the
• 3:47
  nurse's
• 3:48
  office?
• 3:48
  In
• 3:48
  this
• 3:48
  case,
• 3:48
  our
• 3:49
  primary
• 3:49
  outcome
• 3:49
  measure
• 3:50
  will
• 3:50
  be
• 3:50
  a
• 3:50
  tummy
• 3:51
  ache
• 3:51
  that's
• 3:51
  actually
• 3:52
  reported
• 3:52
  to
• 3:52
  the
• 3:53
  nurse's
• 3:53
  office.
• 3:54
  Now
• 3:54
  you
• 3:54
  can
• 3:54
  already
• 3:54
  see
• 3:55
  as
• 3:55
  a
• 3:56
  researcher
• 3:56
  in
• 3:57
  training
• 3:57
  that
• 3:58
  might
• 3:58
  not
• 3:58
  be
• 3:58
  the
• 3:58
  best
• 3:59
  primary
• 3:59
  outcome
• 4:00
  measure
• 4:00
  because
• 4:00
  what
• 4:01
  if
• 4:01
  a
• 4:01
  kid
• 4:01
  has
• 4:01
  a
• 4:01
  tummy
• 4:02
  ache
• 4:02
  and
• 4:02
  he's
• 4:02
  really
• 4:02
  shy
• 4:02
  and
• 4:03
  he
• 4:03
  doesn't
• 4:03
  tell
• 4:03
  anybody?
• 4:04
  So
• 4:04
  now
• 4:04
  we're
• 4:05
  going
• 4:05
  to
• 4:05
  under
• 4:05
  report
• 4:05
  tummy
• 4:06
  aches
• 4:06
  because
• 4:07
  this
• 4:07
  kid's
• 4:07
  not
• 4:07
  going
• 4:08
  to
• 4:08
  the
• 4:08
  nurse's
• 4:09
  office.
• 4:09
  He
• 4:09
  might
• 4:10
  not
• 4:10
  even
• 4:10
  tell
• 4:10
  anyone
• 4:11
  he
• 4:11
  has
• 4:11
  a
• 4:11
  tummy
• 4:11
  ache.
• 4:11
  He
• 4:12
  might
• 4:12
  lie
• 4:13
  when
• 4:13
  someone
• 4:13
  asks
• 4:14
  him
• 4:14
  later
• 4:14
  on
• 4:14
  in
• 4:14
  the
• 4:15
  study
• 4:15
  scores.
• 4:16
  You
• 4:16
  can
• 4:16
  already
• 4:16
  appreciate
• 4:17
  some
• 4:17
  of
• 4:17
  the
• 4:17
  challenges
• 4:18
  in
• 4:18
  research
• 4:19
  and
• 4:19
  research
• 4:20
  questions.
• 4:20
  Right?
• 4:20
  Just
• 4:21
  with
• 4:21
  this,
• 4:21
  just
• 4:22
  with
• 4:22
  this
• 4:22
  quick
• 4:23
  back
• 4:23
  and
• 4:23
  forth.
• 4:24
  Now
• 4:24
  when
• 4:24
  we
• 4:24
  have
• 4:24
  a
• 4:25
  hypothesis,
• 4:25
  right
• 4:26
  hypothesis
• 4:27
  can
• 4:27
  either
• 4:27
  be
• 4:27
  directional
• 4:28
  or
• 4:28
  non.
• 4:28
  Directional.
• 4:29
  In
• 4:29
  this
• 4:29
  case
• 4:30
  we
• 4:30
  have
• 4:30
  a
• 4:30
  directional
• 4:30
  hypothesis.
• 4:31
  We
• 4:31
  believe
• 4:32
  kids
• 4:32
  who
• 4:32
  eat
• 4:32
  jelly
• 4:32
  beans
• 4:33
  will
• 4:33
  have
• 4:33
  more
• 4:34
  tummy
• 4:34
  aches.
• 4:35
  A
• 4:35
  non
• 4:35
  directional
• 4:35
  hypothesis
• 4:36
  would
• 4:36
  be
• 4:37
  there
• 4:37
  will
• 4:37
  be
• 4:37
  a
• 4:37
  difference
• 4:38
  in
• 4:38
  tummy
• 4:38
  aches
• 4:38
  between
• 4:39
  jelly
• 4:39
  bean
• 4:39
  eaters
• 4:39
  and
• 4:40
  non
• 4:40
  eaters.
• 4:40
  But
• 4:41
  it
• 4:41
  doesn't
• 4:41
  give
• 4:41
  us
• 4:41
  a
• 4:41
  direction.
• 4:42
  We
• 4:42
  didn't
• 4:42
  commit
• 4:43
  to
• 4:43
  Whether
• 4:43
  it's
• 4:43
  more
• 4:44
  or
• 4:44
  less,
• 4:45
  hence
• 4:45
  non
• 4:46
  directional.
• 4:47
  Another
• 4:47
  thing
• 4:48
  you
• 4:48
  can
• 4:48
  keep
• 4:48
  in
• 4:48
  mind
• 4:49
  is
• 4:49
  type
• 4:49
  1
• 4:50
  and
• 4:50
  type
• 4:50
  2
• 4:50
  errors.
• 4:51
  These
• 4:51
  are
• 4:51
  the
• 4:51
  types
• 4:51
  of
• 4:52
  errors
• 4:52
  that
• 4:52
  can
• 4:52
  be
• 4:52
  committed
• 4:53
  in
• 4:53
  research.
• 4:53
  A
• 4:54
  type
• 4:54
  1
• 4:54
  error
• 4:55
  would
• 4:55
  be
• 4:55
  saying
• 4:56
  that
• 4:56
  there
• 4:56
  is
• 4:56
  an
• 4:57
  effect
• 4:57
  of
• 4:57
  jelly
• 4:58
  beans
• 4:58
  on.
• 4:59
  Yes,
• 4:59
  there
• 4:59
  is.
• 5:00
  More
• 5:01
  jelly
• 5:01
  beans
• 5:01
  mean
• 5:01
  more
• 5:02
  tummy
• 5:02
  aches,
• 5:03
  and
• 5:03
  there
• 5:03
  actually
• 5:03
  is
• 5:04
  not
• 5:04
  an
• 5:04
  effect.
• 5:05
  So
• 5:05
  that
• 5:05
  would
• 5:05
  be
• 5:05
  a
• 5:05
  type
• 5:06
  1
• 5:06
  error.
• 5:06
  You
• 5:06
  thought
• 5:07
  that
• 5:07
  jelly
• 5:07
  beans
• 5:08
  increased
• 5:09
  tummy
• 5:09
  aches
• 5:09
  and
• 5:09
  had
• 5:10
  an
• 5:10
  effect
• 5:10
  on
• 5:10
  tummy
• 5:11
  aches,
• 5:11
  but
• 5:11
  it.
• 5:12
  But
• 5:12
  they
• 5:12
  really
• 5:13
  don't.
• 5:13
  Okay,
• 5:14
  And
• 5:14
  a
• 5:14
  type
• 5:15
  2
• 5:15
  error
• 5:15
  is
• 5:15
  the
• 5:15
  opposite
• 5:16
  when
• 5:16
  you
• 5:16
  said
• 5:16
  there
• 5:16
  is
• 5:17
  no
• 5:17
  effect
• 5:17
  between
• 5:17
  jelly
• 5:18
  beans
• 5:18
  and
• 5:18
  tummy
• 5:19
  aches.
• 5:19
  But
• 5:19
  guess
• 5:19
  what?
• 5:19
  There
• 5:20
  actually
• 5:20
  was
• 5:21
  an
• 5:21
  effect.
• 5:22
  And
• 5:22
  that's
• 5:23
  very
• 5:23
  difficult
• 5:24
  because
• 5:25
  a
• 5:25
  promising
• 5:26
  treatment
• 5:26
  in
• 5:27
  the
• 5:27
  real
• 5:27
  world
• 5:27
  could
• 5:27
  get
• 5:27
  abandoned
• 5:28
  too
• 5:28
  early
• 5:28
  and
• 5:29
  patients
• 5:29
  lose
• 5:29
  out
• 5:29
  on
• 5:30
  something
• 5:30
  that
• 5:30
  could
• 5:31
  have
• 5:31
  changed
• 5:31
  some
• 5:31
  lives.
• 5:31
  Because
• 5:32
  there
• 5:32
  was
• 5:32
  a
• 5:32
  type
• 5:33
  2
• 5:33
  error
• 5:34
  and
• 5:34
  that
• 5:35
  type
• 5:35
  1
• 5:35
  error,
• 5:36
  patients
• 5:36
  might
• 5:36
  be
• 5:36
  put
• 5:37
  on
• 5:37
  a
• 5:37
  new
• 5:37
  med
• 5:37
  that
• 5:37
  doesn't
• 5:38
  really
• 5:38
  help
• 5:38
  that
• 5:38
  much,
• 5:39
  wasting
• 5:39
  time
• 5:39
  and
• 5:39
  exposing
• 5:40
  them
• 5:40
  to
• 5:40
  side
• 5:40
  effects
• 5:41
  because
• 5:41
  there
• 5:41
  was
• 5:42
  a
• 5:42
  type
• 5:42
  1
• 5:42
  error
• 5:42
  or
• 5:43
  a
• 5:43
  F
• 5:43
  false
• 5:43
  positive.
• 5:44
  Okay,
• 5:45
  so
• 5:45
  just
• 5:45
  get
• 5:46
  familiar
• 5:46
  there
• 5:47
  between
• 5:47
  the
• 5:48
  difference
• 5:48
  between
• 5:48
  a
• 5:48
  type
• 5:49
  1
• 5:49
  and
• 5:49
  a
• 5:49
  type
• 5:50
  2
• 5:50
  error.
• 5:50
  A
• 5:51
  type
• 5:51
  1
• 5:51
  is
• 5:51
  a
• 5:51
  false
• 5:51
  positive
• 5:52
  and
• 5:53
  a
• 5:53
  type
• 5:53
  2
• 5:53
  is
• 5:53
  a
• 5:54
  false
• 5:54
  negative.
• 5:55
  To
• 5:55
  me,
• 5:55
  it's
• 5:55
  kind
• 5:56
  of
• 5:56
  linear,
• 5:56
  it's
• 5:56
  kind
• 5:56
  of
• 5:56
  clear,
• 5:57
  right?
• 5:57
  A
• 5:57
  false
• 5:57
  positive.
• 5:58
  We
• 5:58
  want
• 5:58
  to
• 5:58
  see
• 5:59
  that
• 5:59
  this
• 5:59
  depression
• 6:00
  med
• 6:00
  works,
• 6:00
  right?
• 6:01
  But
• 6:01
  the
• 6:01
  biggest
• 6:02
  error
• 6:03
  you
• 6:03
  can
• 6:03
  commit
• 6:04
  is
• 6:04
  saying,
• 6:04
  hey,
• 6:04
  everybody,
• 6:05
  this
• 6:05
  works
• 6:05
  and
• 6:05
  it
• 6:05
  really
• 6:06
  doesn't
• 6:06
  work.
• 6:06
  That's
• 6:06
  a
• 6:07
  classic
• 6:07
  type
• 6:07
  one
• 6:08
  error.
• 6:08
  The
• 6:08
  second
• 6:08
  kind
• 6:09
  of
• 6:09
  error
• 6:09
  is
• 6:09
  when
• 6:10
  it
• 6:10
  actually
• 6:10
  does
• 6:10
  work
• 6:11
  and
• 6:11
  you
• 6:11
  think
• 6:11
  it
• 6:11
  doesn't
• 6:12
  work.
• 6:12
  Type
• 6:12
  one,
• 6:13
  type
• 6:13
  two.
• 6:13
  Pretty
• 6:14
  interesting
• 6:14
  there.
• 6:15
  So,
• 6:15
  my
• 6:16
  friends,
• 6:16
  when
• 6:16
  we
• 6:17
  want
• 6:17
  to
• 6:17
  test
• 6:18
  this
• 6:18
  jelly
• 6:18
  bean
• 6:20
  theory
• 6:20
  and
• 6:21
  whether
• 6:21
  or
• 6:21
  not
• 6:21
  it
• 6:22
  causes
• 6:22
  stomach
• 6:22
  aches,
• 6:23
  we
• 6:23
  have
• 6:24
  a
• 6:24
  lot
• 6:24
  of
• 6:24
  different
• 6:25
  study
• 6:25
  designs
• 6:26
  we
• 6:26
  can
• 6:26
  think
• 6:26
  about.
• 6:27
  And
• 6:28
  it's
• 6:28
  kind
• 6:29
  of
• 6:29
  exciting,
• 6:29
  right?
• 6:30
  And
• 6:30
  there's
• 6:30
  many
• 6:31
  different
• 6:31
  kinds
• 6:31
  of
• 6:31
  studies.
• 6:32
  But
• 6:32
  for
• 6:32
  this
• 6:32
  study,
• 6:33
  we're
• 6:33
  going
• 6:33
  to
• 6:33
  do
• 6:33
  what's
• 6:34
  called
• 6:34
  a
• 6:34
  prospective
• 6:35
  cohort
• 6:36
  study.
• 6:36
  The
• 6:37
  prospective
• 6:37
  cohort
• 6:38
  study
• 6:38
  is
• 6:39
  when
• 6:39
  we're
• 6:39
  going
• 6:39
  to
• 6:39
  observe
• 6:40
  kids
• 6:40
  over
• 6:40
  a
• 6:40
  set
• 6:41
  period
• 6:41
  of
• 6:41
  time
• 6:42
  instead
• 6:42
  of
• 6:42
  randomizing
• 6:43
  them,
• 6:44
  right?
• 6:44
  So
• 6:44
  we're
• 6:45
  going
• 6:45
  to
• 6:45
  just
• 6:45
  observe
• 6:45
  these
• 6:46
  kids.
• 6:46
  These
• 6:46
  kids
• 6:47
  are
• 6:47
  going
• 6:47
  to
• 6:47
  go
• 6:47
  to
• 6:47
  the
• 6:47
  cafeteria
• 6:48
  and
• 6:48
  they're
• 6:48
  either
• 6:49
  going
• 6:49
  to
• 6:49
  take
• 6:49
  jelly
• 6:49
  beans
• 6:50
  or
• 6:50
  they're
• 6:50
  not.
• 6:50
  And
• 6:50
  we're
• 6:50
  going
• 6:51
  to
• 6:51
  track
• 6:51
  which
• 6:51
  kids
• 6:52
  are
• 6:52
  taking
• 6:52
  jelly
• 6:52
  beans
• 6:53
  and
• 6:53
  how
• 6:53
  much
• 6:54
  versus
• 6:54
  the
• 6:54
  ones
• 6:55
  who
• 6:55
  never
• 6:55
  take
• 6:55
  jelly
• 6:55
  beans
• 6:56
  at
• 6:56
  all.
• 6:56
  And
• 6:56
  then
• 6:56
  we're
• 6:56
  going
• 6:57
  to
• 6:57
  march
• 6:57
  over
• 6:57
  to
• 6:57
  the
• 6:57
  nurse's
• 6:58
  office
• 6:58
  at
• 6:58
  the
• 6:58
  end
• 6:58
  of
• 6:59
  the
• 6:59
  week
• 6:59
  and
• 6:59
  understand
• 7:00
  how
• 7:00
  many
• 7:00
  kids
• 7:01
  came
• 7:02
  here
• 7:02
  for
• 7:02
  tummy
• 7:02
  aches?
• 7:03
  And
• 7:03
  we're
• 7:04
  going
• 7:04
  to
• 7:04
  just
• 7:04
  correlate
• 7:04
  the
• 7:05
  data.
• 7:05
  Right?
• 7:05
  No
• 7:05
  need
• 7:05
  to
• 7:06
  get
• 7:06
  in
• 7:06
  the
• 7:06
  weeds
• 7:06
  too
• 7:06
  much.
• 7:07
  I
• 7:07
  just
• 7:07
  want
• 7:07
  you
• 7:07
  to
• 7:07
  have
• 7:07
  an
• 7:08
  appreciation
• 7:09
  for
• 7:09
  what
• 7:09
  a
• 7:10
  prospective
• 7:10
  cohort
• 7:11
  study
• 7:11
  is.
• 7:11
  You're
• 7:11
  not
• 7:12
  randomizing
• 7:12
  kids.
• 7:13
  You're
• 7:14
  observing
• 7:14
  them
• 7:15
  in
• 7:15
  their
• 7:15
  natural
• 7:15
  environment.
• 7:16
  These
• 7:16
  kids
• 7:16
  already
• 7:17
  have
• 7:17
  jelly
• 7:17
  beans
• 7:18
  they
• 7:18
  can
• 7:18
  eat
• 7:18
  every
• 7:18
  day,
• 7:19
  and
• 7:19
  you're
• 7:19
  just
• 7:19
  tracking
• 7:20
  who
• 7:20
  had
• 7:20
  them,
• 7:20
  who
• 7:21
  didn't
• 7:21
  have
• 7:21
  them,
• 7:21
  and
• 7:21
  what
• 7:22
  the
• 7:22
  tummy
• 7:22
  aches
• 7:22
  look
• 7:23
  like.
• 7:23
  And
• 7:23
  then
• 7:23
  you're
• 7:23
  going
• 7:24
  to
• 7:24
  put
• 7:24
  these
• 7:25
  statistics
• 7:25
  together.
• 7:26
  That's
• 7:26
  pretty
• 7:26
  cool.
• 7:27
  And
• 7:27
  it's
• 7:27
  also
• 7:27
  important
• 7:29
  before
• 7:29
  you
• 7:29
  do
• 7:29
  a
• 7:29
  study
• 7:30
  to
• 7:30
  have
• 7:31
  inclusion
• 7:31
  and
• 7:32
  exclusion
• 7:32
  criteria.
• 7:33
  Right.
• 7:34
  You
• 7:34
  want
• 7:34
  certain
• 7:35
  guardrails
• 7:35
  in
• 7:35
  your
• 7:35
  study.
• 7:36
  You
• 7:36
  don't
• 7:36
  want
• 7:36
  to
• 7:36
  mess
• 7:37
  up
• 7:37
  your
• 7:37
  data,
• 7:38
  and
• 7:38
  you
• 7:38
  have
• 7:38
  to
• 7:38
  design
• 7:39
  things
• 7:39
  before
• 7:39
  you
• 7:39
  even
• 7:40
  start.
• 7:40
  So
• 7:41
  we
• 7:41
  already
• 7:41
  talked
• 7:41
  about
• 7:42
  nurse
• 7:42
  visits
• 7:43
  are
• 7:43
  our
• 7:43
  objective
• 7:44
  marker
• 7:44
  whether
• 7:44
  a
• 7:44
  kid
• 7:45
  has
• 7:45
  a
• 7:45
  tummy
• 7:45
  ache
• 7:45
  or
• 7:45
  not.
• 7:46
  It's
• 7:46
  imperfect.
• 7:46
  It's
• 7:47
  not
• 7:47
  great.
• 7:47
  We're
• 7:47
  also
• 7:47
  going
• 7:48
  to
• 7:48
  do
• 7:48
  a
• 7:48
  pain
• 7:48
  rating
• 7:48
  as
• 7:49
  a
• 7:49
  subjective
• 7:49
  marker
• 7:50
  for
• 7:50
  whether
• 7:51
  these
• 7:51
  kids
• 7:52
  have
• 7:52
  pain,
• 7:52
  and
• 7:53
  that's
• 7:53
  important.
• 7:54
  We
• 7:54
  may
• 7:54
  even
• 7:55
  track
• 7:56
  absences
• 7:56
  from
• 7:57
  school.
• 7:57
  We
• 7:57
  could
• 7:57
  track
• 7:58
  how
• 7:58
  many
• 7:58
  kids
• 7:58
  were
• 7:58
  absent
• 7:59
  from
• 7:59
  school
• 7:59
  because
• 7:59
  they
• 8:00
  had
• 8:00
  stomachaches.
• 8:01
  Now,
• 8:02
  when
• 8:02
  we
• 8:02
  talk
• 8:02
  about
• 8:02
  inclusion
• 8:03
  and
• 8:03
  exclusion
• 8:04
  criteria,
• 8:05
  this
• 8:05
  is
• 8:05
  important
• 8:06
  because
• 8:06
  you
• 8:06
  don't
• 8:06
  want
• 8:06
  to,
• 8:06
  you
• 8:07
  know,
• 8:07
  in
• 8:07
  A
• 8:07
  K
• 8:07
  through
• 8:07
  12
• 8:08
  school,
• 8:08
  let's
• 8:08
  say
• 8:08
  you
• 8:08
  don't
• 8:09
  want
• 8:09
  to
• 8:09
  count
• 8:09
  toddlers
• 8:10
  or
• 8:10
  you
• 8:10
  don't
• 8:11
  want
• 8:11
  to
• 8:11
  count
• 8:11
  the
• 8:11
  daycare
• 8:12
  or
• 8:12
  the
• 8:12
  three
• 8:12
  year
• 8:12
  olds
• 8:12
  or
• 8:13
  four
• 8:13
  year
• 8:13
  olds.
• 8:13
  Maybe
• 8:14
  you
• 8:14
  just
• 8:14
  want
• 8:14
  to
• 8:14
  focus
• 8:15
  on
• 8:16
  third
• 8:16
  and
• 8:16
  fourth
• 8:16
  graders.
• 8:17
  Right?
• 8:18
  That
• 8:18
  would
• 8:18
  be
• 8:19
  very
• 8:19
  simple.
• 8:19
  Third
• 8:19
  and
• 8:19
  fourth
• 8:20
  graders.
• 8:20
  So
• 8:21
  eight
• 8:21
  years
• 8:21
  and
• 8:21
  nine
• 8:22
  years
• 8:22
  old.
• 8:22
  Maybe
• 8:22
  they
• 8:23
  drink
• 8:23
  a
• 8:23
  lot
• 8:23
  of
• 8:23
  Red
• 8:23
  Bull
• 8:24
  before
• 8:24
  school.
• 8:24
  Maybe
• 8:25
  there's
• 8:25
  just
• 8:25
  too
• 8:25
  many
• 8:26
  confounders.
• 8:27
  Confounders
• 8:28
  are
• 8:28
  things
• 8:28
  that
• 8:28
  can
• 8:28
  muddy
• 8:29
  our
• 8:29
  data
• 8:30
  and
• 8:30
  explain
• 8:31
  differences
• 8:31
  for
• 8:31
  reasons
• 8:32
  we
• 8:32
  didn't
• 8:32
  intend.
• 8:33
  Right.
• 8:33
  So
• 8:33
  that's
• 8:33
  why
• 8:33
  you
• 8:34
  might
• 8:34
  want
• 8:34
  to
• 8:34
  restrict
• 8:35
  the
• 8:35
  amount
• 8:36
  of.
• 8:37
  You
• 8:37
  want
• 8:37
  to
• 8:37
  restrict
• 8:38
  who's
• 8:38
  included
• 8:39
  in
• 8:39
  your
• 8:39
  criteria.
• 8:40
  Because
• 8:40
  too
• 8:40
  many
• 8:41
  different
• 8:41
  kinds
• 8:42
  of
• 8:42
  people
• 8:42
  introduce
• 8:43
  way
• 8:43
  too
• 8:43
  many
• 8:43
  variables.
• 8:44
  It
• 8:45
  invites
• 8:45
  too
• 8:46
  many
• 8:46
  questions
• 8:46
  about
• 8:46
  your
• 8:47
  research
• 8:47
  study
• 8:47
  that
• 8:47
  you
• 8:48
  don't
• 8:48
  need
• 8:48
  to
• 8:49
  deal
• 8:49
  with.
• 8:49
  For
• 8:49
  simplicity's
• 8:50
  sake,
• 8:50
  we're
• 8:50
  going
• 8:51
  to
• 8:51
  do
• 8:51
  third
• 8:51
  and
• 8:51
  fourth
• 8:51
  graders,
• 8:52
  ages
• 8:52
  8
• 8:53
  to
• 8:53
  9.
• 8:54
  It's
• 8:54
  more
• 8:54
  of
• 8:54
  a
• 8:54
  narrow
• 8:55
  developmental
• 8:55
  window.
• 8:56
  They
• 8:56
  have
• 8:56
  similar
• 8:56
  GI
• 8:57
  physiology.
• 8:58
  Right.
• 8:58
  You
• 8:58
  avoid
• 8:59
  toddlers
• 8:59
  or
• 8:59
  kids
• 9:00
  with
• 9:00
  different
• 9:00
  diets.
• 9:01
  You
• 9:01
  can
• 9:01
  also
• 9:02
  exclude
• 9:02
  people
• 9:02
  with
• 9:03
  any
• 9:03
  kind
• 9:03
  of
• 9:04
  food
• 9:04
  allergies
• 9:05
  or
• 9:05
  lactose
• 9:06
  intolerance
• 9:06
  or
• 9:07
  just
• 9:07
  GI
• 9:07
  sensitivities.
• 9:08
  Right.
• 9:09
  You
• 9:09
  can
• 9:09
  definitely
• 9:10
  exclude
• 9:10
  them.
• 9:10
  It's
• 9:11
  very
• 9:11
  important.
• 9:11
  You're
• 9:12
  Tracking
• 9:12
  jelly
• 9:13
  bean
• 9:13
  consumption
• 9:13
  in
• 9:13
  the
• 9:14
  cafeteria.
• 9:15
  And
• 9:15
  also
• 9:15
  any
• 9:16
  kids
• 9:16
  who
• 9:16
  have
• 9:16
  tummy
• 9:17
  aches
• 9:17
  at
• 9:17
  baseline.
• 9:17
  We're
• 9:18
  not
• 9:18
  going
• 9:18
  to
• 9:18
  include
• 9:18
  them
• 9:19
  because
• 9:19
  you
• 9:19
  already
• 9:20
  have
• 9:20
  a
• 9:20
  stomachache
• 9:20
  for
• 9:21
  whatever
• 9:21
  reason.
• 9:21
  If
• 9:21
  you
• 9:21
  have
• 9:22
  known
• 9:22
  GI
• 9:22
  issues,
• 9:23
  ibs,
• 9:23
  celiac
• 9:24
  reflux,
• 9:25
  a
• 9:25
  recent
• 9:25
  viral
• 9:26
  gastroenteritis,
• 9:27
  we
• 9:27
  don't
• 9:27
  want
• 9:27
  you
• 9:27
  in
• 9:27
  the
• 9:28
  study.
• 9:28
  You
• 9:28
  might
• 9:28
  have
• 9:29
  a
• 9:29
  stomachache
• 9:29
  for
• 9:29
  different
• 9:29
  reasons
• 9:30
  and
• 9:30
  it'll
• 9:31
  muddy
• 9:31
  up
• 9:31
  the
• 9:31
  effect
• 9:32
  of
• 9:32
  whether
• 9:32
  or
• 9:33
  not
• 9:33
  it's
• 9:33
  our
• 9:33
  jelly
• 9:33
  beans
• 9:34
  causing
• 9:34
  the
• 9:34
  stomachaches
• 9:35
  or
• 9:35
  your
• 9:35
  existing
• 9:36
  GI
• 9:36
  issues.
• 9:37
  Non
• 9:37
  compliance
• 9:38
  kids
• 9:38
  who
• 9:38
  are
• 9:38
  just
• 9:39
  non
• 9:39
  compliant,
• 9:39
  behaviorally
• 9:40
  problematic.
• 9:41
  Maybe
• 9:41
  we
• 9:41
  don't
• 9:41
  really
• 9:41
  want
• 9:42
  them
• 9:42
  in
• 9:42
  here
• 9:42
  because
• 9:43
  they
• 9:43
  might
• 9:43
  lie,
• 9:44
  take
• 9:44
  too
• 9:44
  many
• 9:44
  jelly
• 9:45
  beans
• 9:45
  and
• 9:45
  give
• 9:45
  them
• 9:46
  to
• 9:46
  other
• 9:46
  people
• 9:46
  and
• 9:46
  just
• 9:47
  cause
• 9:47
  a
• 9:47
  ruckus.
• 9:47
  We
• 9:47
  don't
• 9:48
  know,
• 9:48
  right?
• 9:48
  So
• 9:48
  non
• 9:49
  compliance
• 9:49
  behavioral
• 9:50
  problems.
• 9:50
  We
• 9:51
  might
• 9:51
  exclude
• 9:51
  certain
• 9:51
  kids
• 9:52
  at
• 9:52
  the
• 9:52
  teacher,
• 9:52
  say,
• 9:53
  hey,
• 9:53
  I
• 9:53
  don't
• 9:53
  think
• 9:53
  this
• 9:54
  person's
• 9:54
  going
• 9:54
  to
• 9:54
  really
• 9:55
  be
• 9:55
  able
• 9:55
  to
• 9:55
  comply
• 9:56
  with
• 9:56
  your
• 9:56
  study
• 9:56
  in
• 9:57
  a
• 9:57
  good
• 9:57
  way.
• 9:58
  So
• 9:58
  inclusion
• 9:58
  makes
• 9:58
  your
• 9:59
  sample
• 9:59
  size
• 10:00
  homogenous
• 10:01
  enough
• 10:01
  that
• 10:01
  the
• 10:01
  noise
• 10:02
  is
• 10:02
  minimized.
• 10:02
  That's
• 10:03
  the
• 10:03
  big
• 10:03
  takeaway.
• 10:04
  Kids
• 10:04
  are
• 10:04
  similar
• 10:04
  in
• 10:05
  age
• 10:05
  setting,
• 10:05
  pedigree
• 10:06
  physiology
• 10:07
  exclusion.
• 10:08
  You
• 10:08
  strip
• 10:09
  away
• 10:09
  those
• 10:09
  alternate
• 10:10
  explanations.
• 10:11
  So
• 10:11
  if
• 10:11
  you
• 10:11
  see
• 10:11
  tummy
• 10:12
  aches,
• 10:12
  you
• 10:12
  can
• 10:12
  more
• 10:12
  confidently
• 10:13
  point
• 10:13
  to
• 10:13
  the
• 10:13
  jelly
• 10:14
  beans.
• 10:14
  That's
• 10:14
  really
• 10:15
  the
• 10:15
  point.
• 10:16
  Let's
• 10:16
  keep
• 10:16
  moving
• 10:17
  here.
• 10:17
  We
• 10:17
  talked
• 10:17
  about
• 10:17
  different
• 10:18
  kinds
• 10:18
  of
• 10:18
  studies,
• 10:19
  right?
• 10:19
  I'm
• 10:19
  choosing
• 10:19
  a
• 10:20
  prospective
• 10:20
  cohort
• 10:21
  study.
• 10:21
  That's
• 10:21
  where
• 10:22
  we
• 10:22
  follow
• 10:22
  these
• 10:22
  kids
• 10:23
  throughout
• 10:23
  the
• 10:23
  week.
• 10:24
  We're
• 10:24
  not
• 10:24
  randomizing
• 10:25
  them,
• 10:25
  we're
• 10:26
  not
• 10:26
  designing
• 10:26
  our
• 10:26
  own
• 10:27
  experiment.
• 10:27
  We're
• 10:27
  just
• 10:28
  observing
• 10:28
  them
• 10:28
  in
• 10:28
  their
• 10:29
  environment.
• 10:29
  Prospective
• 10:30
  cohort
• 10:30
  study.
• 10:31
  Now,
• 10:31
  there
• 10:31
  are
• 10:32
  different
• 10:32
  kinds
• 10:32
  of
• 10:32
  studies,
• 10:33
  like
• 10:33
  case
• 10:33
  reports.
• 10:34
  A
• 10:34
  nurse
• 10:34
  might
• 10:35
  just
• 10:35
  write
• 10:35
  up
• 10:35
  a
• 10:35
  report
• 10:36
  about
• 10:36
  Timmy
• 10:36
  who
• 10:37
  ate
• 10:37
  40
• 10:37
  jelly
• 10:37
  beans
• 10:38
  at
• 10:38
  lunch
• 10:38
  and
• 10:39
  developed
• 10:39
  a
• 10:39
  severe
• 10:39
  tummy
• 10:40
  ache
• 10:40
  that
• 10:40
  afternoon.
• 10:41
  She
• 10:41
  writes
• 10:41
  the
• 10:42
  story
• 10:42
  up.
• 10:42
  It
• 10:42
  sparks
• 10:43
  attention.
• 10:43
  Maybe
• 10:44
  this
• 10:44
  is
• 10:44
  the
• 10:44
  first
• 10:44
  clue
• 10:45
  that
• 10:45
  jelly
• 10:45
  beans
• 10:45
  and
• 10:45
  tummy
• 10:46
  aches
• 10:46
  might
• 10:46
  be
• 10:46
  connected.
• 10:47
  But
• 10:47
  a
• 10:47
  case
• 10:48
  report
• 10:48
  is
• 10:48
  a
• 10:48
  low
• 10:49
  hierarchy
• 10:49
  of
• 10:49
  evidence
• 10:50
  because
• 10:50
  it's
• 10:50
  just
• 10:51
  a
• 10:51
  case
• 10:51
  report.
• 10:51
  It's
• 10:52
  one
• 10:52
  case.
• 10:52
  It
• 10:52
  could
• 10:53
  give
• 10:53
  us
• 10:53
  a
• 10:53
  lot
• 10:53
  of
• 10:53
  very
• 10:54
  interesting
• 10:54
  details
• 10:55
  and
• 10:55
  we
• 10:56
  can't
• 10:56
  negate
• 10:57
  it
• 10:57
  entirely,
• 10:57
  right?
• 10:58
  It
• 10:58
  could
• 10:58
  be
• 10:59
  very
• 10:59
  interesting.
• 11:00
  There's
• 11:00
  case
• 11:01
  reports
• 11:01
  out
• 11:01
  there
• 11:02
  on
• 11:02
  people
• 11:02
  using
• 11:02
  psychedelics
• 11:03
  and
• 11:03
  stuff,
• 11:04
  right?
• 11:04
  You
• 11:04
  definitely
• 11:05
  want
• 11:05
  to
• 11:05
  figure
• 11:05
  out
• 11:06
  what's
• 11:06
  more
• 11:06
  to
• 11:07
  these
• 11:07
  jelly
• 11:07
  beans.
• 11:08
  But
• 11:08
  maybe
• 11:09
  one
• 11:09
  experience
• 11:10
  as
• 11:10
  it
• 11:10
  relates
• 11:10
  to
• 11:10
  40
• 11:11
  jelly
• 11:11
  beans
• 11:11
  could
• 11:12
  be
• 11:12
  pretty
• 11:12
  instructive
• 11:13
  and
• 11:13
  it
• 11:13
  might
• 11:13
  give
• 11:13
  us
• 11:14
  really
• 11:14
  rich
• 11:14
  data.
• 11:14
  But
• 11:15
  again,
• 11:15
  it's
• 11:15
  so
• 11:15
  limited
• 11:16
  because
• 11:16
  what
• 11:16
  if
• 11:17
  that
• 11:17
  one
• 11:17
  kid
• 11:17
  already
• 11:18
  has
• 11:18
  GI
• 11:18
  issues?
• 11:19
  We
• 11:19
  don't
• 11:19
  know
• 11:19
  that.
• 11:20
  What
• 11:20
  if
• 11:20
  another
• 11:21
  kid,
• 11:21
  his
• 11:22
  best
• 11:22
  friend
• 11:22
  also
• 11:22
  had
• 11:23
  40
• 11:23
  jelly
• 11:23
  beans
• 11:23
  and
• 11:24
  they
• 11:24
  didn't
• 11:24
  have
• 11:24
  a
• 11:24
  tummy
• 11:24
  ache
• 11:25
  at
• 11:25
  all
• 11:25
  and
• 11:25
  they
• 11:25
  felt
• 11:25
  great.
• 11:26
  So
• 11:26
  we're
• 11:26
  missing
• 11:26
  out
• 11:27
  on
• 11:27
  really
• 11:27
  good
• 11:27
  data
• 11:28
  with
• 11:28
  the
• 11:28
  case
• 11:28
  reports.
• 11:29
  A
• 11:29
  case
• 11:29
  series
• 11:30
  notices
• 11:30
  12
• 11:30
  kids
• 11:31
  that
• 11:31
  come
• 11:31
  in
• 11:31
  with
• 11:31
  tummy
• 11:31
  aches
• 11:32
  after
• 11:32
  eating
• 11:33
  jelly
• 11:33
  beans
• 11:33
  at
• 11:33
  a
• 11:33
  school
• 11:34
  carnival.
• 11:34
  She
• 11:35
  summarizes
• 11:35
  all
• 11:35
  12
• 11:36
  cases.
• 11:37
  Patterns
• 11:37
  begin
• 11:38
  to
• 11:38
  emerge.
• 11:38
  All
• 11:38
  of
• 11:38
  the
• 11:39
  12
• 11:39
  kids
• 11:39
  ate
• 11:39
  jelly
• 11:40
  beans.
• 11:40
  Nine
• 11:40
  reported
• 11:41
  pain
• 11:41
  within
• 11:41
  two
• 11:42
  hours.
• 11:42
  Seven
• 11:42
  had
• 11:43
  vomiting.
• 11:44
  This
• 11:44
  is
• 11:44
  definitely
• 11:44
  stronger
• 11:45
  than
• 11:45
  one
• 11:45
  case.
• 11:45
  It's
• 11:46
  a
• 11:46
  series
• 11:46
  of
• 11:46
  kids,
• 11:47
  but
• 11:47
  there's
• 11:47
  still
• 11:47
  no
• 11:48
  control
• 11:48
  Group.
• 11:49
  Maybe
• 11:49
  12
• 11:49
  of
• 11:50
  those
• 11:50
  kids
• 11:50
  also
• 11:50
  ate
• 11:51
  hot
• 11:51
  dogs.
• 11:51
  There's
• 11:51
  no
• 11:52
  denominator.
• 11:52
  We
• 11:52
  don't
• 11:53
  really
• 11:53
  know
• 11:53
  how
• 11:53
  many
• 11:53
  kids
• 11:53
  ate
• 11:54
  beans
• 11:54
  and
• 11:54
  didn't
• 11:55
  get
• 11:55
  sick.
• 11:56
  Case
• 11:56
  series
• 11:56
  is
• 11:56
  also
• 11:57
  limited,
• 11:57
  but
• 11:57
  it's
• 11:57
  a
• 11:58
  little
• 11:58
  better
• 11:58
  because
• 11:58
  it
• 11:58
  has
• 11:59
  a
• 11:59
  little
• 11:59
  more
• 11:59
  data.
• 12:00
  We're
• 12:00
  getting
• 12:00
  a
• 12:01
  little
• 12:01
  more
• 12:01
  interesting
• 12:02
  now.
• 12:02
  A
• 12:03
  cross
• 12:03
  sectional
• 12:03
  studies.
• 12:04
  Going
• 12:04
  up
• 12:04
  the
• 12:04
  hierarchy
• 12:05
  a
• 12:05
  little
• 12:05
  bit.
• 12:06
  It's
• 12:06
  an
• 12:06
  example
• 12:07
  would
• 12:07
  be
• 12:07
  at
• 12:07
  Tuesday
• 12:07
  at
• 12:08
  lunch,
• 12:08
  every
• 12:08
  student
• 12:09
  fills
• 12:09
  out
• 12:09
  a
• 12:09
  survey
• 12:09
  and
• 12:10
  says,
• 12:10
  how
• 12:10
  many
• 12:10
  jelly
• 12:10
  beans
• 12:11
  did
• 12:11
  you
• 12:11
  eat?
• 12:12
  And
• 12:12
  do
• 12:12
  you
• 12:12
  have
• 12:12
  a
• 12:12
  tummy
• 12:12
  ache?
• 12:13
  It's
• 12:13
  a
• 12:13
  cross
• 12:13
  sectional
• 12:14
  study.
• 12:15
  I
• 12:15
  like
• 12:15
  to
• 12:15
  consider
• 12:15
  this
• 12:16
  like
• 12:16
  a
• 12:16
  snapshot
• 12:17
  study.
• 12:17
  It
• 12:17
  gives
• 12:17
  you
• 12:18
  a
• 12:18
  snapshot
• 12:18
  of
• 12:18
  what's
• 12:19
  happening
• 12:19
  right
• 12:20
  now.
• 12:20
  Notice
• 12:21
  how
• 12:21
  that's
• 12:21
  kind
• 12:22
  of
• 12:22
  neat
• 12:22
  because
• 12:22
  you
• 12:22
  catch
• 12:22
  people
• 12:23
  where
• 12:23
  they
• 12:23
  are
• 12:23
  right
• 12:24
  now.
• 12:24
  It
• 12:24
  gives
• 12:24
  you
• 12:24
  some
• 12:25
  idea
• 12:25
  of
• 12:25
  their
• 12:25
  prevalence
• 12:26
  of
• 12:26
  tummy
• 12:26
  aches.
• 12:27
  And
• 12:27
  you
• 12:27
  could
• 12:27
  say
• 12:28
  30%
• 12:28
  of
• 12:28
  all
• 12:28
  jelly
• 12:29
  bean
• 12:29
  eaters
• 12:29
  report
• 12:30
  tummy
• 12:30
  aches
• 12:30
  compared
• 12:31
  to
• 12:31
  10%
• 12:31
  of
• 12:31
  non
• 12:32
  eaters.
• 12:32
  So
• 12:32
  now
• 12:33
  we're
• 12:33
  getting
• 12:33
  a
• 12:33
  little
• 12:33
  bit
• 12:34
  warmer.
• 12:34
  We're
• 12:34
  getting
• 12:34
  a
• 12:35
  little
• 12:35
  bit
• 12:35
  more
• 12:35
  interested.
• 12:36
  But
• 12:37
  again,
• 12:37
  it's
• 12:37
  still
• 12:37
  just
• 12:38
  a
• 12:38
  snapshot.
• 12:39
  It
• 12:39
  doesn't
• 12:39
  necessarily
• 12:40
  prove
• 12:40
  cause.
• 12:41
  Did
• 12:41
  the
• 12:41
  jelly
• 12:41
  beans
• 12:42
  lead
• 12:42
  to
• 12:42
  tummy
• 12:43
  aches
• 12:43
  or
• 12:43
  did
• 12:43
  tummy
• 12:43
  aches
• 12:44
  make
• 12:44
  kids
• 12:45
  more
• 12:45
  likely
• 12:45
  to
• 12:45
  self
• 12:46
  report
• 12:46
  jelly
• 12:46
  bean
• 12:47
  eating?
• 12:47
  We
• 12:47
  don't
• 12:48
  really
• 12:48
  know.
• 12:48
  We
• 12:49
  kind
• 12:49
  of
• 12:49
  miss
• 12:49
  the
• 12:50
  time
• 12:50
  order.
• 12:50
  Did
• 12:50
  they
• 12:50
  eat
• 12:51
  them
• 12:51
  yesterday
• 12:51
  or
• 12:52
  four
• 12:52
  days
• 12:52
  ago?
• 12:52
  You're
• 12:53
  just
• 12:53
  asking
• 12:53
  right
• 12:54
  now.
• 12:54
  There
• 12:54
  are
• 12:55
  limitations
• 12:55
  with
• 12:55
  the
• 12:55
  cross
• 12:56
  sectional
• 12:56
  study,
• 12:56
  but
• 12:57
  when
• 12:57
  you
• 12:57
  think
• 12:57
  cross
• 12:57
  sectional,
• 12:58
  think
• 12:58
  snapshots.
• 12:59
  You're
• 12:59
  kind
• 12:59
  of
• 12:59
  getting
• 13:00
  an
• 13:00
  idea
• 13:00
  of
• 13:00
  what's
• 13:00
  happening
• 13:01
  right
• 13:01
  now.
• 13:02
  Case
• 13:02
  controls,
• 13:03
  Right?
• 13:03
  So
• 13:04
  a
• 13:04
  case
• 13:04
  control
• 13:04
  study,
• 13:04
  you
• 13:05
  start
• 13:05
  with
• 13:05
  100
• 13:05
  kids
• 13:06
  who
• 13:06
  currently
• 13:06
  have
• 13:07
  tummy
• 13:07
  aches.
• 13:07
  Those
• 13:07
  are
• 13:08
  cases.
• 13:08
  You
• 13:08
  match
• 13:08
  them
• 13:08
  with
• 13:09
  100
• 13:09
  who
• 13:09
  don't
• 13:09
  and
• 13:10
  you
• 13:10
  look
• 13:10
  back
• 13:10
  and
• 13:10
  say,
• 13:11
  did
• 13:11
  you
• 13:11
  eat
• 13:11
  jelly
• 13:12
  beans
• 13:12
  this
• 13:12
  week?
• 13:13
  So
• 13:13
  the
• 13:13
  control
• 13:14
  piece
• 13:14
  is
• 13:14
  there,
• 13:14
  which
• 13:15
  is
• 13:15
  neat
• 13:15
  because
• 13:15
  you
• 13:15
  have
• 13:15
  a
• 13:16
  hundred
• 13:16
  kids
• 13:16
  who
• 13:16
  you
• 13:16
  know,
• 13:17
  didn't
• 13:17
  have
• 13:17
  jelly
• 13:18
  beans
• 13:18
  and
• 13:18
  you
• 13:18
  have
• 13:18
  a
• 13:18
  hundred
• 13:19
  who
• 13:19
  do.
• 13:19
  It's
• 13:20
  efficient
• 13:20
  for
• 13:20
  rare
• 13:20
  outcomes
• 13:21
  and
• 13:22
  it's
• 13:22
  great
• 13:22
  when
• 13:22
  you
• 13:23
  can't
• 13:23
  follow
• 13:23
  a
• 13:23
  whole
• 13:24
  school
• 13:24
  for
• 13:24
  years
• 13:24
  because
• 13:25
  following
• 13:25
  a
• 13:25
  school,
• 13:26
  even
• 13:26
  in
• 13:26
  my
• 13:26
  prospective
• 13:27
  study,
• 13:27
  is
• 13:28
  a
• 13:28
  lot
• 13:28
  of
• 13:28
  legwork.
• 13:29
  You
• 13:29
  have
• 13:29
  to
• 13:30
  train
• 13:31
  the
• 13:31
  people
• 13:31
  who
• 13:31
  are
• 13:32
  recording
• 13:32
  the
• 13:32
  jelly
• 13:33
  bean
• 13:33
  consumption
• 13:34
  and
• 13:35
  really
• 13:36
  meticulously
• 13:36
  track
• 13:36
  down
• 13:37
  how
• 13:37
  many
• 13:37
  jelly
• 13:37
  beans
• 13:38
  these
• 13:38
  kids
• 13:38
  are
• 13:38
  even
• 13:39
  eating.
• 13:39
  You
• 13:39
  have
• 13:39
  to
• 13:40
  go
• 13:40
  to
• 13:40
  the
• 13:40
  nurse's
• 13:40
  office
• 13:41
  and
• 13:41
  get
• 13:41
  reports
• 13:41
  and
• 13:42
  probably
• 13:42
  do
• 13:42
  a
• 13:43
  lot
• 13:43
  of
• 13:43
  red
• 13:43
  tape
• 13:44
  privacy
• 13:44
  stuff,
• 13:45
  which
• 13:45
  is
• 13:45
  so
• 13:45
  important.
• 13:46
  All
• 13:46
  of
• 13:46
  these
• 13:46
  studies,
• 13:46
  you
• 13:47
  have
• 13:47
  to
• 13:47
  get
• 13:47
  consent.
• 13:48
  Consult
• 13:49
  your
• 13:49
  local
• 13:49
  resources
• 13:50
  for
• 13:50
  that,
• 13:50
  my
• 13:50
  friends.
• 13:50
  That's
• 13:51
  a
• 13:51
  different
• 13:51
  podcast
• 13:52
  episode,
• 13:52
  but
• 13:52
  you
• 13:53
  get
• 13:53
  my
• 13:53
  point,
• 13:53
  right?
• 13:54
  Some
• 13:54
  of
• 13:54
  these
• 13:54
  studies
• 13:55
  are
• 13:55
  much
• 13:56
  more
• 13:56
  intensive
• 13:56
  than
• 13:56
  other
• 13:57
  studies.
• 13:58
  Retrospective
• 13:58
  cohort
• 13:59
  studies
• 13:59
  are
• 13:59
  pretty
• 13:59
  cool
• 13:59
  because
• 14:00
  you
• 14:00
  can
• 14:00
  look
• 14:00
  at
• 14:00
  cafeteria
• 14:01
  records
• 14:01
  from
• 14:01
  last
• 14:02
  year
• 14:02
  and
• 14:02
  you
• 14:03
  could
• 14:03
  go
• 14:03
  and
• 14:03
  say,
• 14:03
  wow,
• 14:04
  1,000
• 14:05
  kids
• 14:05
  had
• 14:05
  jelly
• 14:06
  beans
• 14:06
  last
• 14:06
  year,
• 14:07
  and
• 14:07
  we
• 14:07
  had
• 14:08
  100
• 14:08
  of
• 14:08
  them
• 14:09
  go
• 14:09
  to
• 14:09
  the
• 14:09
  nurse's
• 14:09
  office
• 14:10
  for
• 14:11
  tummy
• 14:11
  aches.
• 14:11
  That
• 14:12
  year,
• 14:12
  you
• 14:12
  could
• 14:13
  say
• 14:13
  1,000
• 14:14
  kids
• 14:14
  didn't
• 14:14
  have
• 14:14
  jelly
• 14:15
  beans.
• 14:15
  And
• 14:16
  look,
• 14:16
  only
• 14:16
  four
• 14:16
  of
• 14:17
  those
• 14:17
  kids
• 14:17
  ended
• 14:17
  up
• 14:17
  at
• 14:17
  the
• 14:18
  nurse's
• 14:18
  office.
• 14:19
  So
• 14:19
  you
• 14:19
  have
• 14:19
  records.
• 14:20
  Retrospective
• 14:21
  cohort.
• 14:21
  You're
• 14:21
  going
• 14:21
  back
• 14:22
  in
• 14:22
  time.
• 14:22
  Retrospective
• 14:23
  cohort.
• 14:24
  It's
• 14:24
  a
• 14:24
  cohort
• 14:24
  or
• 14:24
  a
• 14:25
  group
• 14:25
  of
• 14:25
  people
• 14:25
  implicated
• 14:26
  in
• 14:26
  your
• 14:26
  study.
• 14:27
  And
• 14:27
  you
• 14:27
  have
• 14:27
  existing
• 14:28
  records.
• 14:28
  You're
• 14:29
  not
• 14:29
  randomizing
• 14:30
  again,
• 14:30
  you're
• 14:30
  not
• 14:31
  designing
• 14:31
  an
• 14:31
  experiment.
• 14:32
  What's
• 14:33
  happening
• 14:33
  is
• 14:33
  these
• 14:34
  guys
• 14:34
  already
• 14:35
  have
• 14:35
  done
• 14:35
  what
• 14:36
  they've
• 14:36
  done,
• 14:36
  and
• 14:36
  you're
• 14:37
  just
• 14:37
  collecting
• 14:37
  records
• 14:38
  now.
• 14:38
  Data
• 14:39
  could
• 14:39
  be
• 14:39
  missing
• 14:39
  or
• 14:39
  inaccurate.
• 14:40
  Right.
• 14:40
  Did
• 14:41
  the
• 14:41
  cafeteria
• 14:41
  actually
• 14:42
  record
• 14:42
  the
• 14:42
  jelly
• 14:43
  bean
• 14:43
  sales
• 14:43
  correctly?
• 14:44
  Can't
• 14:45
  really
• 14:45
  control
• 14:45
  it
• 14:46
  once
• 14:46
  it's
• 14:47
  measured
• 14:47
  already.
• 14:48
  So
• 14:48
  those
• 14:48
  are
• 14:48
  some
• 14:48
  limitations
• 14:49
  of
• 14:49
  a
• 14:49
  retrospective
• 14:50
  cohort
• 14:50
  study,
• 14:51
  but
• 14:51
  nonetheless,
• 14:52
  pretty
• 14:52
  darn
• 14:53
  interesting.
• 14:53
  Now,
• 14:53
  randomized
• 14:54
  controlled
• 14:54
  trial,
• 14:55
  You've
• 14:55
  probably
• 14:55
  heard
• 14:55
  about
• 14:55
  it
• 14:56
  a
• 14:56
  million
• 14:56
  times.
• 14:56
  It
• 14:57
  is
• 14:57
  a
• 14:57
  gold
• 14:57
  standard
• 14:58
  here
• 14:58
  in
• 14:59
  the
• 14:59
  hierarchy
• 15:00
  where
• 15:00
  you
• 15:00
  do
• 15:01
  get
• 15:01
  to
• 15:01
  randomly
• 15:01
  assign
• 15:02
  200
• 15:03
  kids,
• 15:03
  100,
• 15:04
  however
• 15:04
  many
• 15:05
  kids
• 15:05
  you
• 15:05
  want,
• 15:05
  randomly
• 15:06
  assign
• 15:06
  them
• 15:06
  to
• 15:06
  either
• 15:07
  jelly
• 15:07
  beans
• 15:07
  at
• 15:07
  lunch
• 15:08
  or
• 15:08
  no
• 15:08
  jelly
• 15:09
  beans
• 15:09
  at
• 15:09
  lunch.
• 15:09
  And
• 15:10
  then
• 15:10
  you
• 15:10
  follow
• 15:10
  them
• 15:10
  to
• 15:11
  decide
• 15:11
  to
• 15:11
  figure
• 15:12
  out
• 15:12
  if
• 15:12
  they've
• 15:12
  had
• 15:12
  tummy
• 15:13
  aches.
• 15:13
  And
• 15:13
  the
• 15:14
  reason
• 15:14
  randomized
• 15:15
  control
• 15:15
  trials
• 15:15
  are
• 15:16
  gold
• 15:16
  standards
• 15:16
  is
• 15:16
  that
• 15:17
  they
• 15:17
  distribute
• 15:17
  your
• 15:18
  confounders.
• 15:19
  If
• 15:19
  you
• 15:19
  have
• 15:20
  groups
• 15:21
  that.
• 15:21
  Let's
• 15:21
  say
• 15:22
  you
• 15:22
  just
• 15:22
  took
• 15:22
  a
• 15:22
  group
• 15:23
  from
• 15:23
  Mrs.
• 15:23
  Jones's
• 15:24
  math
• 15:24
  class
• 15:24
  and
• 15:25
  compared
• 15:25
  it
• 15:25
  to
• 15:26
  Mr.
• 15:26
  Jones's
• 15:27
  English
• 15:27
  class.
• 15:28
  The
• 15:28
  problem
• 15:29
  is
• 15:29
  that
• 15:29
  the
• 15:29
  math
• 15:30
  class
• 15:30
  might
• 15:30
  have
• 15:30
  a
• 15:31
  random
• 15:31
  group
• 15:31
  of
• 15:31
  people
• 15:32
  with
• 15:32
  really
• 15:32
  sensitive
• 15:33
  stomachs.
• 15:33
  Maybe
• 15:33
  the
• 15:34
  math
• 15:34
  class
• 15:34
  has
• 15:35
  a
• 15:35
  midterm
• 15:35
  coming
• 15:36
  up
• 15:36
  and
• 15:36
  they're
• 15:36
  already
• 15:36
  stressed
• 15:37
  and
• 15:37
  their
• 15:37
  stomachs
• 15:38
  hurts.
• 15:38
  You
• 15:38
  think
• 15:38
  it's
• 15:39
  jelly
• 15:39
  beans
• 15:39
  that
• 15:40
  are
• 15:40
  causing
• 15:40
  tummy
• 15:40
  aches
• 15:41
  in
• 15:41
  that
• 15:41
  math
• 15:41
  class,
• 15:41
  and
• 15:41
  it's
• 15:42
  Just
• 15:42
  a
• 15:42
  group
• 15:42
  of
• 15:42
  people
• 15:42
  who
• 15:43
  happen
• 15:43
  to
• 15:43
  be
• 15:44
  much
• 15:44
  more
• 15:44
  achy
• 15:45
  than
• 15:45
  the
• 15:45
  others.
• 15:46
  Well,
• 15:46
  in
• 15:46
  randomized
• 15:47
  control
• 15:47
  trial,
• 15:48
  you're
• 15:48
  blending
• 15:48
  all
• 15:49
  the
• 15:49
  classes
• 15:49
  together.
• 15:50
  You're
• 15:50
  blending
• 15:50
  all
• 15:51
  the
• 15:51
  kids
• 15:51
  who
• 15:51
  might
• 15:52
  have
• 15:52
  GI
• 15:52
  issues,
• 15:53
  You're
• 15:53
  blending
• 15:54
  all
• 15:54
  the
• 15:54
  kids
• 15:55
  who
• 15:55
  might
• 15:55
  be
• 15:55
  more
• 15:56
  stressed
• 15:56
  or
• 15:56
  did
• 15:57
  have
• 15:57
  breakfast
• 15:57
  or
• 15:57
  didn't
• 15:58
  have
• 15:58
  breakfast.
• 15:58
  It's
• 15:58
  exponentially
• 15:59
  less
• 15:59
  likely
• 16:00
  that
• 16:00
  you're
• 16:00
  going
• 16:00
  to
• 16:00
  have
• 16:01
  these
• 16:01
  weird
• 16:02
  groups
• 16:02
  of
• 16:02
  people
• 16:03
  that
• 16:03
  all
• 16:03
  have
• 16:03
  the
• 16:03
  same
• 16:04
  problem,
• 16:05
  and
• 16:05
  instead
• 16:05
  they're
• 16:06
  going
• 16:06
  to
• 16:06
  be
• 16:06
  randomly
• 16:06
  dispersed
• 16:07
  among
• 16:07
  a
• 16:07
  large
• 16:08
  group
• 16:08
  of
• 16:08
  people.
• 16:08
  And
• 16:09
  that's
• 16:09
  why
• 16:09
  this
• 16:10
  is
• 16:10
  very
• 16:10
  strong.
• 16:11
  It
• 16:11
  has
• 16:11
  the
• 16:11
  strongest
• 16:12
  internal
• 16:12
  validity.
• 16:13
  Those
• 16:13
  confounders
• 16:14
  like
• 16:14
  stress
• 16:15
  or
• 16:15
  not
• 16:15
  sleeping
• 16:15
  enough
• 16:16
  or
• 16:16
  GI
• 16:16
  issues,
• 16:17
  they're
• 16:17
  distributed
• 16:18
  evenly
• 16:18
  among
• 16:19
  all
• 16:19
  the
• 16:19
  groups.
• 16:20
  Because
• 16:20
  if
• 16:20
  they're
• 16:20
  not
• 16:21
  distributed
• 16:21
  evenly,
• 16:22
  then
• 16:22
  you're
• 16:22
  going
• 16:22
  to
• 16:22
  go,
• 16:23
  well,
• 16:23
  is
• 16:23
  it
• 16:23
  really
• 16:23
  the
• 16:24
  jelly
• 16:24
  beans
• 16:24
  or
• 16:24
  is
• 16:25
  that
• 16:25
  class
• 16:26
  more
• 16:26
  stressed?
• 16:27
  So
• 16:27
  that's
• 16:27
  why
• 16:28
  randomized
• 16:28
  controlled
• 16:28
  trial
• 16:29
  is
• 16:29
  the
• 16:29
  gold
• 16:29
  standard
• 16:30
  for
• 16:30
  studies.
• 16:32
  And
• 16:32
  when
• 16:33
  we
• 16:34
  talk
• 16:34
  about
• 16:34
  a
• 16:34
  meta
• 16:35
  analysis,
• 16:35
  we're
• 16:36
  pooling
• 16:36
  together
• 16:36
  data
• 16:37
  from
• 16:38
  dozens
• 16:38
  of
• 16:38
  studies
• 16:39
  and
• 16:39
  statistically
• 16:40
  combining
• 16:40
  how
• 16:41
  effective
• 16:41
  those
• 16:42
  interventions
• 16:43
  might
• 16:43
  be
• 16:43
  into
• 16:44
  one
• 16:44
  number.
• 16:45
  It
• 16:45
  maximizes
• 16:46
  your
• 16:46
  sample
• 16:46
  size.
• 16:47
  Meta
• 16:47
  analysis
• 16:48
  is
• 16:48
  like
• 16:48
  pooling
• 16:48
  all
• 16:49
  of
• 16:49
  this
• 16:49
  stuff
• 16:49
  together
• 16:50
  and
• 16:50
  making
• 16:51
  sense
• 16:51
  of
• 16:51
  it.
• 16:51
  And
• 16:52
  that's
• 16:52
  very
• 16:52
  important
• 16:53
  because
• 16:53
  now
• 16:53
  you
• 16:54
  have
• 16:54
  thousands
• 16:54
  of
• 16:54
  people
• 16:55
  of
• 16:55
  jelly
• 16:55
  bean
• 16:56
  eaters
• 16:56
  and
• 16:56
  non
• 16:56
  jelly
• 16:56
  bean
• 16:57
  eaters
• 16:57
  amassed
• 16:58
  into
• 16:58
  a
• 16:58
  major
• 16:59
  group.
• 16:59
  You
• 16:59
  couldn't
• 17:00
  do
• 17:00
  that
• 17:00
  in
• 17:00
  one
• 17:00
  study.
• 17:01
  How
• 17:01
  many
• 17:01
  jelly
• 17:01
  bean
• 17:02
  eaters
• 17:02
  could
• 17:02
  you
• 17:02
  really
• 17:03
  find
• 17:03
  at
• 17:03
  this
• 17:03
  Candyville
• 17:04
  Elementary
• 17:04
  School?
• 17:05
  Not
• 17:05
  many.
• 17:05
  So
• 17:06
  you're
• 17:06
  pulling
• 17:06
  this
• 17:07
  in
• 17:07
  tons
• 17:07
  of
• 17:07
  people.
• 17:08
  The
• 17:08
  problem,
• 17:08
  of
• 17:09
  course,
• 17:09
  with
• 17:09
  that
• 17:09
  is,
• 17:10
  even
• 17:10
  though
• 17:10
  it
• 17:10
  gives
• 17:10
  you
• 17:11
  great
• 17:11
  information,
• 17:11
  is
• 17:12
  that
• 17:12
  every
• 17:12
  study
• 17:12
  is
• 17:13
  still
• 17:13
  conducted
• 17:13
  differently.
• 17:14
  You
• 17:14
  have
• 17:15
  some
• 17:15
  studies
• 17:15
  that,
• 17:16
  you
• 17:17
  know,
• 17:17
  determine
• 17:18
  tummy
• 17:18
  aches
• 17:18
  differently
• 17:19
  than
• 17:19
  the
• 17:19
  other
• 17:19
  studies,
• 17:20
  right?
• 17:20
  Different
• 17:21
  design,
• 17:21
  experimental
• 17:22
  designs
• 17:22
  and
• 17:23
  ways
• 17:24
  that
• 17:24
  these
• 17:24
  researchers
• 17:24
  are
• 17:25
  going
• 17:25
  about
• 17:25
  doing
• 17:25
  this.
• 17:26
  If
• 17:26
  you
• 17:26
  have
• 17:26
  a
• 17:26
  lot
• 17:26
  of
• 17:26
  junk
• 17:27
  in
• 17:27
  there
• 17:27
  and
• 17:27
  a
• 17:27
  lot
• 17:27
  of
• 17:27
  noise,
• 17:28
  you're
• 17:28
  going
• 17:28
  to
• 17:28
  get
• 17:28
  a
• 17:28
  lot
• 17:28
  of
• 17:29
  junk
• 17:29
  and
• 17:29
  noise.
• 17:29
  But
• 17:30
  because
• 17:30
  you're
• 17:30
  allocating
• 17:31
  a
• 17:31
  lot
• 17:31
  of
• 17:31
  different
• 17:31
  things
• 17:32
  at
• 17:32
  once,
• 17:32
  you
• 17:33
  have
• 17:33
  a
• 17:33
  potential
• 17:33
  to
• 17:34
  get
• 17:34
  some
• 17:34
  rich
• 17:35
  data.
• 17:35
  So
• 17:36
  hopefully
• 17:36
  you
• 17:36
  have
• 17:36
  an
• 17:36
  idea
• 17:37
  now
• 17:37
  of
• 17:38
  what
• 17:38
  the
• 17:39
  difference
• 17:39
  is
• 17:39
  between
• 17:39
  a
• 17:40
  lot
• 17:40
  of
• 17:40
  these
• 17:40
  studies.
• 17:40
  Again,
• 17:40
  we're
• 17:41
  picking
• 17:41
  the
• 17:41
  perspective
• 17:42
  cohort
• 17:42
  study.
• 17:43
  And
• 17:43
  now
• 17:44
  in
• 17:44
  just
• 17:44
  a
• 17:44
  moment,
• 17:45
  we're
• 17:45
  going
• 17:45
  to
• 17:46
  go
• 17:46
  over
• 17:46
  descriptive
• 17:46
  statistics
• 17:47
  and
• 17:47
  where
• 17:47
  to
• 17:47
  go
• 17:48
  from
• 17:49
  here,
• 17:49
  my
• 17:49
  friend.
• 17:50
  All
• 17:51
  right,
• 17:51
  my
• 17:51
  friend.
• 17:51
  So
• 17:52
  now
• 17:52
  we're
• 17:52
  going
• 17:52
  to
• 17:52
  go
• 17:52
  over
• 17:53
  descriptive
• 17:54
  statistics.
• 17:54
  I
• 17:55
  don't
• 17:55
  want
• 17:55
  you
• 17:55
  to
• 17:55
  be
• 17:55
  intimidated
• 17:56
  by
• 17:56
  this
• 17:57
  because
• 17:57
  it's
• 17:57
  really
• 17:58
  pretty
• 17:58
  simple,
• 17:58
  right?
• 17:59
  Because
• 18:00
  we
• 18:00
  ran
• 18:00
  our
• 18:00
  study,
• 18:01
  let's
• 18:01
  say
• 18:01
  these
• 18:02
  kids,
• 18:02
  they
• 18:03
  had
• 18:03
  access
• 18:03
  to
• 18:04
  jelly
• 18:04
  Beans,
• 18:05
  they
• 18:05
  either
• 18:05
  went
• 18:05
  to
• 18:05
  the
• 18:06
  nurse
• 18:06
  or
• 18:06
  they
• 18:07
  didn't.
• 18:07
  And
• 18:08
  if
• 18:08
  they
• 18:08
  did
• 18:08
  go
• 18:08
  to
• 18:08
  the
• 18:09
  nurse,
• 18:09
  they
• 18:09
  recorded
• 18:10
  their
• 18:10
  pain
• 18:10
  scores.
• 18:11
  Maybe
• 18:11
  if
• 18:12
  they
• 18:12
  went
• 18:13
  to
• 18:13
  the
• 18:13
  nurse
• 18:13
  anyway
• 18:14
  and
• 18:14
  they
• 18:14
  didn't
• 18:15
  eat
• 18:15
  jelly
• 18:15
  beans,
• 18:15
  they're
• 18:16
  still
• 18:16
  recording
• 18:16
  pain
• 18:17
  scores.
• 18:18
  So
• 18:18
  either
• 18:18
  way,
• 18:19
  all
• 18:19
  of
• 18:19
  our
• 18:19
  data's
• 18:20
  in,
• 18:20
  right?
• 18:20
  Kids
• 18:20
  ate
• 18:20
  their
• 18:20
  jelly
• 18:21
  beans.
• 18:21
  Kids
• 18:21
  didn't
• 18:21
  eat
• 18:22
  their
• 18:22
  jelly
• 18:22
  beans.
• 19:17
  Of
• 19:17
  the
• 19:17
  ones
• 19:17
  who
• 19:17
  did,
• 19:18
  we
• 19:18
  have
• 18:24
  pain
• 19:18
  scores.
• 19:19
  Of
• 19:19
  the
• 19:20
  ones
• 19:20
  who
• 19:21
  didn't
• 19:21
  eat
• 19:21
  jelly
• 19:22
  beans,
• 19:22
  we
• 19:22
  also
• 19:22
  have
• 19:22
  pain
• 19:23
  scores.
• 19:23
  Okay,
• 19:24
  so
• 19:24
  don't
• 19:24
  even
• 19:24
  worry
• 19:24
  about
• 19:24
  the,
• 19:25
  the
• 19:26
  actual
• 19:27
  architecture
• 19:27
  of
• 19:28
  my
• 19:28
  fake
• 19:29
  study.
• 19:29
  It's
• 19:29
  not
• 19:30
  important.
• 19:30
  What
• 19:30
  matters
• 19:31
  is
• 19:31
  we
• 19:31
  have
• 19:32
  the
• 19:32
  data
• 19:32
  now.
• 19:33
  So
• 19:33
  let's
• 19:33
  talk
• 19:33
  about
• 19:34
  it.
• 19:34
  These
• 19:34
  are
• 19:34
  descriptive
• 19:35
  statistics.
• 19:36
  Mean,
• 19:37
  median,
• 19:37
  range,
• 19:38
  and
• 19:38
  standard
• 19:39
  deviation.
• 19:39
  Those
• 19:39
  are
• 19:39
  the
• 19:40
  things
• 19:40
  that
• 19:40
  we
• 19:40
  want
• 19:40
  to
• 19:41
  understand.
• 19:42
  All
• 19:42
  this
• 19:42
  is
• 19:43
  is
• 19:43
  just
• 19:43
  statistics
• 19:44
  that
• 19:44
  describe
• 19:45
  your
• 19:45
  study.
• 19:45
  They're
• 19:45
  plain
• 19:46
  numbers
• 19:46
  from
• 19:46
  your
• 19:46
  study.
• 19:47
  No
• 19:47
  guessing,
• 19:48
  no
• 19:48
  predictions,
• 19:48
  just
• 19:49
  raw
• 19:49
  data.
• 19:50
  What
• 19:50
  does
• 19:50
  mean?
• 19:50
  It's
• 19:51
  in
• 19:51
  this
• 19:52
  case
• 19:52
  the
• 19:52
  average
• 19:52
  pain
• 19:53
  score.
• 19:53
  Jelly
• 19:53
  bean
• 19:54
  kids
• 19:54
  had
• 19:54
  an
• 19:54
  average
• 19:55
  pain
• 19:55
  score
• 19:55
  of
• 19:55
  4.
• 19:56
  Non
• 19:56
  Jelly
• 19:57
  Bean
• 19:57
  eating
• 19:57
  kids
• 19:58
  had
• 19:58
  an
• 19:58
  average
• 19:58
  pain
• 19:58
  score
• 19:59
  of
• 19:59
  1
• 19:59
  when
• 19:59
  they
• 19:59
  went
• 19:59
  to
• 20:00
  the
• 20:00
  nurse's
• 20:00
  office.
• 20:01
  That
• 20:01
  might
• 20:01
  suggest
• 20:02
  that
• 20:03
  your
• 20:04
  jelly
• 20:04
  bean
• 20:04
  kids
• 20:05
  have
• 20:05
  higher
• 20:06
  pain
• 20:06
  scores.
• 20:06
  It's
• 20:07
  not
• 20:07
  conclusive.
• 20:08
  A
• 20:08
  hundred
• 20:08
  percent.
• 20:09
  And
• 20:09
  an
• 20:09
  average
• 20:10
  is
• 20:10
  also
• 20:10
  it.
• 20:11
  It
• 20:11
  can
• 20:11
  be
• 20:11
  a
• 20:12
  little
• 20:12
  bit
• 20:12
  misleading,
• 20:12
  right?
• 20:13
  Because
• 20:13
  if
• 20:13
  your
• 20:13
  average
• 20:14
  jelly
• 20:14
  bean
• 20:14
  score,
• 20:15
  a
• 20:15
  jelly
• 20:16
  bean
• 20:16
  pain
• 20:16
  score
• 20:17
  was
• 20:17
  a
• 20:18
  four
• 20:18
  for
• 20:18
  the
• 20:18
  jelly
• 20:18
  bean
• 20:19
  kids,
• 20:20
  we
• 20:20
  don't
• 20:20
  know.
• 20:20
  Right?
• 20:21
  Did
• 20:21
  all
• 20:21
  of
• 20:21
  the
• 20:21
  kids
• 20:22
  who
• 20:22
  had
• 20:22
  jelly
• 20:22
  beans
• 20:23
  report
• 20:24
  a
• 20:24
  4
• 20:24
  out
• 20:24
  of
• 20:24
  10
• 20:24
  pain,
• 20:25
  or
• 20:25
  did
• 20:25
  you
• 20:25
  have
• 20:26
  like
• 20:27
  some
• 20:27
  of
• 20:27
  them
• 20:27
  report
• 20:28
  a
• 20:28
  two
• 20:28
  or
• 20:28
  three
• 20:28
  and
• 20:29
  then
• 20:29
  like
• 20:29
  a
• 20:29
  few
• 20:30
  skewed
• 20:30
  the
• 20:30
  average
• 20:31
  because
• 20:31
  they
• 20:31
  reported
• 20:31
  a
• 20:32
  10?
• 20:32
  Well,
• 20:32
  if
• 20:33
  that's
• 20:33
  the
• 20:33
  case,
• 20:33
  that
• 20:33
  could
• 20:34
  really
• 20:34
  be
• 20:34
  weird
• 20:34
  data
• 20:35
  because
• 20:36
  why
• 20:36
  are
• 20:36
  some
• 20:36
  kids
• 20:37
  reporting
• 20:37
  10
• 20:37
  and
• 20:38
  other
• 20:38
  kids
• 20:38
  reporting
• 20:38
  two
• 20:39
  or
• 20:39
  three?
• 20:39
  Right?
• 20:39
  So
• 20:40
  that's
• 20:40
  unusual.
• 20:40
  It
• 20:41
  might
• 20:41
  not
• 20:41
  be
• 20:41
  your
• 20:41
  jelly
• 20:42
  beans.
• 20:42
  Maybe
• 20:43
  the
• 20:43
  lunch
• 20:43
  lady
• 20:43
  served
• 20:44
  a
• 20:44
  weird
• 20:44
  batch
• 20:45
  of
• 20:45
  chili
• 20:46
  that
• 20:46
  day.
• 20:46
  Maybe
• 20:47
  there's
• 20:47
  a
• 20:47
  virus
• 20:48
  going
• 20:48
  around.
• 20:49
  We
• 20:49
  don't
• 20:49
  know.
• 20:50
  So
• 20:51
  averages
• 20:51
  are
• 20:51
  important,
• 20:52
  especially
• 20:52
  if
• 20:53
  you
• 20:53
  can
• 20:53
  contrast
• 20:53
  this
• 20:54
  average
• 20:54
  to
• 20:54
  the
• 20:54
  non
• 20:55
  jelly
• 20:55
  bean
• 20:55
  eating
• 20:55
  kids.
• 20:56
  Clearly
• 20:56
  the
• 20:57
  jelly
• 20:57
  bean
• 20:57
  eating
• 20:57
  kids
• 20:58
  are
• 20:58
  having
• 20:58
  more
• 20:58
  pain.
• 20:59
  But
• 20:59
  the
• 20:59
  average
• 21:00
  is
• 21:00
  not
• 21:00
  everything
• 21:00
  you
• 21:01
  need
• 21:01
  to
• 21:01
  know.
• 21:01
  So
• 21:02
  that's
• 21:02
  why
• 21:02
  you
• 21:02
  want
• 21:03
  to
• 21:03
  look
• 21:03
  at
• 21:03
  all
• 21:03
  your
• 21:03
  descriptive
• 21:04
  statistics.
• 21:04
  The
• 21:05
  median
• 21:05
  is
• 21:05
  the
• 21:05
  middle
• 21:06
  score.
• 21:06
  If
• 21:06
  you
• 21:06
  line
• 21:06
  up
• 21:06
  all
• 21:07
  the
• 21:07
  pain
• 21:07
  scores,
• 21:07
  the
• 21:08
  middle
• 21:08
  kid
• 21:08
  in
• 21:08
  the
• 21:08
  jelly
• 21:08
  bean
• 21:09
  group
• 21:09
  had
• 21:09
  a
• 21:09
  three.
• 21:10
  Well,
• 21:10
  that's
• 21:10
  kind
• 21:11
  of
• 21:11
  interesting.
• 21:11
  That
• 21:11
  kind
• 21:12
  of
• 21:12
  helps
• 21:13
  you
• 21:13
  get
• 21:13
  rid
• 21:13
  of
• 21:13
  all
• 21:13
  these
• 21:14
  outliers
• 21:14
  that
• 21:14
  could
• 21:15
  otherwise
• 21:15
  be
• 21:15
  in
• 21:15
  your
• 21:16
  way
• 21:16
  because
• 21:16
  you
• 21:16
  know
• 21:17
  what's
• 21:17
  in
• 21:17
  the
• 21:17
  middle.
• 21:17
  Now
• 21:18
  you
• 21:18
  can
• 21:18
  tease
• 21:19
  out
• 21:19
  those
• 21:19
  averages
• 21:20
  that
• 21:20
  misled
• 21:20
  you
• 21:21
  before
• 21:21
  because
• 21:21
  two
• 21:22
  or
• 21:22
  three
• 21:22
  kids
• 21:22
  had
• 21:22
  a
• 21:22
  10
• 21:22
  out
• 21:23
  of
• 21:23
  10
• 21:23
  pain.
• 21:23
  The
• 21:24
  median
• 21:24
  is
• 21:24
  going
• 21:24
  to
• 21:24
  help
• 21:25
  orient
• 21:25
  you
• 21:25
  more
• 21:25
  to
• 21:26
  a
• 21:26
  standard
• 21:27
  middle
• 21:27
  of
• 21:27
  the
• 21:27
  road
• 21:28
  score
• 21:29
  that
• 21:30
  you
• 21:30
  can
• 21:30
  expect.
• 21:30
  A
• 21:31
  pain
• 21:31
  score
• 21:31
  that's
• 21:31
  a
• 21:32
  little
• 21:32
  more
• 21:32
  reliable
• 21:32
  in
• 21:33
  some
• 21:33
  ways.
• 21:33
  But
• 21:34
  again,
• 21:34
  I
• 21:34
  like
• 21:34
  looking
• 21:34
  at
• 21:35
  all
• 21:35
  these
• 21:35
  numbers
• 21:35
  in
• 21:35
  their
• 21:36
  totality
• 21:37
  because
• 21:37
  they
• 21:37
  help
• 21:37
  you
• 21:38
  understand
• 21:38
  stuff.
• 21:38
  You
• 21:39
  don't
• 21:39
  want
• 21:39
  to
• 21:39
  look
• 21:39
  at
• 21:39
  a
• 21:39
  research
• 21:40
  article
• 21:40
  on
• 21:40
  a
• 21:41
  medication
• 21:41
  for
• 21:41
  depression
• 21:42
  and
• 21:42
  you
• 21:42
  see
• 21:43
  this
• 21:43
  amazing
• 21:44
  average
• 21:44
  reduction
• 21:45
  in
• 21:45
  depressive
• 21:46
  scores
• 21:46
  and
• 21:46
  then
• 21:46
  you
• 21:46
  look
• 21:47
  more
• 21:47
  closely
• 21:47
  and
• 21:47
  you're
• 21:48
  like,
• 21:48
  wow.
• 21:48
  But
• 21:49
  look
• 21:49
  at
• 21:49
  the
• 21:49
  range
• 21:49
  on
• 21:50
  this.
• 21:50
  That's
• 21:50
  another
• 21:50
  form
• 21:51
  of
• 21:51
  a
• 21:51
  descriptive
• 21:51
  statistic.
• 21:52
  Look
• 21:52
  at
• 21:52
  the
• 21:52
  range.
• 21:53
  We
• 21:53
  had
• 21:53
  20
• 21:53
  people
• 21:54
  in
• 21:54
  this
• 21:54
  study,
• 21:55
  they
• 21:56
  reported
• 21:56
  massive
• 21:57
  reductions
• 21:57
  in
• 21:58
  depression
• 21:59
  and
• 21:59
  most
• 21:59
  of
• 21:59
  the
• 21:59
  people
• 22:00
  didn't.
• 22:00
  Most
• 22:00
  of
• 22:00
  the
• 22:00
  people
• 22:01
  was
• 22:01
  actually
• 22:01
  a
• 22:01
  little
• 22:01
  lackluster.
• 22:02
  So
• 22:03
  those
• 22:03
  few
• 22:04
  people
• 22:04
  who
• 22:04
  are
• 22:04
  reporting
• 22:05
  massive
• 22:05
  reductions
• 22:06
  in
• 22:06
  depression,
• 22:07
  what's
• 22:07
  going
• 22:07
  on
• 22:07
  there?
• 22:08
  Did
• 22:08
  something
• 22:08
  else
• 22:09
  reduce
• 22:09
  their
• 22:09
  depression?
• 22:10
  Are
• 22:10
  they
• 22:10
  taking
• 22:10
  other
• 22:10
  medications?
• 22:11
  Are
• 22:12
  they
• 22:12
  different
• 22:13
  kinds
• 22:13
  of
• 22:13
  patients
• 22:13
  that
• 22:14
  aren't
• 22:14
  really
• 22:14
  like
• 22:14
  the
• 22:15
  patients
• 22:15
  I
• 22:15
  see?
• 22:16
  So
• 22:16
  that's
• 22:16
  really
• 22:17
  important.
• 22:17
  Right
• 22:18
  you,
• 22:18
  the
• 22:18
  range
• 22:19
  is
• 22:19
  going
• 22:19
  to
• 22:19
  give
• 22:19
  you
• 22:19
  an
• 22:19
  idea
• 22:20
  of
• 22:20
  like
• 22:20
  where
• 22:20
  are
• 22:20
  the
• 22:20
  outliers
• 22:21
  here?
• 22:21
  And
• 22:22
  it
• 22:22
  might
• 22:22
  help
• 22:22
  you
• 22:22
  look
• 22:23
  at
• 22:23
  the
• 22:23
  study
• 22:23
  more
• 22:23
  closely
• 22:24
  and
• 22:24
  figure
• 22:24
  out
• 22:24
  why
• 22:25
  we
• 22:25
  have
• 22:25
  such
• 22:25
  a
• 22:25
  wide
• 22:26
  range
• 22:26
  here.
• 22:27
  If
• 22:27
  you
• 22:27
  have
• 22:27
  a
• 22:27
  wide
• 22:28
  range,
• 22:28
  it
• 22:28
  might
• 22:29
  not
• 22:29
  conclusively,
• 22:30
  but
• 22:30
  it
• 22:30
  might
• 22:30
  suggest
• 22:31
  some
• 22:31
  other
• 22:31
  variables
• 22:32
  at
• 22:32
  play
• 22:32
  other
• 22:33
  than
• 22:33
  your
• 22:33
  jelly
• 22:33
  beans
• 22:34
  that
• 22:34
  are
• 22:34
  causing
• 22:35
  extraordinary
• 22:35
  amounts
• 22:36
  of
• 22:36
  pain.
• 22:36
  It
• 22:36
  can
• 22:37
  skew
• 22:37
  or
• 22:37
  undermine
• 22:38
  your
• 22:38
  data.
• 22:38
  Now
• 22:39
  the
• 22:39
  standard
• 22:39
  deviation,
• 22:40
  I
• 22:40
  love
• 22:40
  this
• 22:40
  word
• 22:41
  because
• 22:41
  it's
• 22:41
  a
• 22:41
  word
• 22:41
  that
• 22:42
  means
• 22:42
  what
• 22:42
  it
• 22:42
  says
• 22:43
  it
• 22:43
  means
• 22:43
  and
• 22:43
  it's
• 22:44
  how
• 22:44
  spread
• 22:44
  out
• 22:44
  your
• 22:45
  scores
• 22:45
  are.
• 22:45
  If
• 22:46
  one
• 22:46
  kid
• 22:46
  said
• 22:46
  0
• 22:46
  and
• 22:47
  another
• 22:47
  kid
• 22:47
  said
• 22:47
  9,
• 22:48
  that's
• 22:48
  a
• 22:48
  big
• 22:48
  standard
• 22:49
  deviation.
• 22:49
  It
• 22:49
  means
• 22:50
  kids
• 22:50
  experiences
• 22:51
  were
• 22:51
  really
• 22:51
  different.
• 22:52
  And
• 22:52
  it
• 22:52
  might
• 22:53
  make
• 22:53
  your
• 22:53
  data
• 22:53
  less
• 22:54
  reliable
• 22:54
  because
• 22:54
  we're
• 22:55
  not
• 22:55
  sure
• 22:55
  that
• 22:56
  the
• 22:56
  jelly
• 22:56
  beans
• 22:56
  themselves
• 22:57
  are
• 22:57
  causing
• 22:58
  consistent
• 22:58
  pain
• 22:59
  levels.
• 22:59
  Some
• 22:59
  are
• 23:00
  lower,
• 23:00
  some
• 23:00
  are
• 23:00
  higher.
• 23:01
  That
• 23:01
  would
• 23:01
  mean
• 23:01
  a
• 23:01
  large
• 23:01
  standard
• 23:02
  deviation
• 23:03
  is
• 23:03
  making
• 23:04
  your
• 23:04
  data
• 23:04
  look
• 23:05
  a
• 23:05
  little
• 23:05
  unreliable
• 23:06
  and
• 23:06
  fishy
• 23:06
  and
• 23:06
  not
• 23:07
  conclusive.
• 23:08
  So
• 23:08
  keep
• 23:08
  that
• 23:09
  in
• 23:09
  mind
• 23:09
  as
• 23:09
  we
• 23:09
  talk
• 23:10
  about
• 23:10
  descriptive
• 23:11
  statistics.
• 23:13
  Once
• 23:13
  you
• 23:13
  have
• 23:14
  the
• 23:14
  snapshot,
• 23:15
  your
• 23:15
  descriptive
• 23:16
  statistics,
• 23:17
  the
• 23:17
  next,
• 23:18
  the
• 23:18
  next
• 23:19
  step
• 23:19
  in
• 23:19
  our
• 23:19
  process
• 23:20
  is
• 23:21
  does
• 23:21
  this
• 23:21
  data
• 23:22
  mean
• 23:22
  anything?
• 23:22
  And
• 23:23
  that's
• 23:23
  where
• 23:23
  inferential
• 23:24
  statistics
• 23:25
  come
• 23:25
  into
• 23:26
  play.
• 23:26
  Is
• 23:27
  this
• 23:27
  real
• 23:27
  or
• 23:27
  is
• 23:28
  this
• 23:28
  just
• 23:28
  lucky?
• 23:30
  We
• 23:30
  have
• 23:30
  to
• 23:30
  ask
• 23:31
  is
• 23:32
  this
• 23:32
  real
• 23:32
  or
• 23:32
  could
• 23:33
  it
• 23:33
  be
• 23:33
  just
• 23:33
  a
• 23:33
  random
• 23:34
  chance
• 23:34
  that
• 23:35
  these
• 23:35
  PAIN
• 23:35
  scores,
• 23:36
  these
• 23:36
  four
• 23:36
  out
• 23:37
  of
• 23:37
  10
• 23:37
  pain
• 23:37
  scores,
• 23:38
  are
• 23:38
  higher
• 23:38
  than
• 23:38
  the
• 23:39
  non
• 23:39
  jelly
• 23:39
  bean
• 23:39
  eaters,
• 23:40
  which
• 23:40
  are
• 23:40
  about
• 23:40
  a
• 23:40
  1
• 23:41
  out
• 23:41
  of
• 23:41
  10
• 23:41
  on
• 23:41
  average
• 23:42
  pain
• 23:42
  scores?
• 23:42
  That's
• 23:43
  where
• 23:43
  inferential
• 23:43
  Stats
• 23:44
  are
• 23:44
  going
• 23:45
  to
• 23:45
  crunch
• 23:45
  some
• 23:46
  numbers
• 23:46
  and
• 23:47
  get
• 23:47
  you
• 23:47
  some
• 23:47
  information.
• 23:48
  You
• 23:49
  could
• 23:49
  run
• 23:49
  a
• 23:49
  T
• 23:49
  test.
• 23:50
  A
• 23:50
  T
• 23:50
  test
• 23:51
  is
• 23:51
  going
• 23:51
  to
• 23:51
  compare
• 23:52
  the
• 23:52
  averages
• 23:53
  in
• 23:54
  your
• 23:54
  jelly
• 23:54
  bean
• 23:55
  eaters
• 23:55
  and
• 23:55
  their
• 23:55
  pain
• 23:56
  and
• 23:56
  versus
• 23:57
  the
• 23:57
  non
• 23:58
  jelly
• 23:58
  bean
• 23:58
  eaters.
• 23:59
  And
• 23:59
  they're
• 23:59
  going
• 23:59
  to
• 23:59
  crunch
• 24:00
  up
• 24:00
  those
• 24:00
  averages
• 24:01
  and
• 24:01
  then
• 24:01
  that
• 24:02
  T
• 24:02
  test
• 24:02
  is
• 24:02
  going
• 24:03
  to
• 24:03
  also
• 24:03
  look
• 24:03
  at
• 24:03
  the
• 24:03
  variance
• 24:04
  between
• 24:05
  those
• 24:05
  pain
• 24:06
  scores
• 24:06
  in
• 24:06
  each
• 24:06
  group.
• 24:07
  So
• 24:07
  like
• 24:07
  we
• 24:07
  talked
• 24:08
  about,
• 24:08
  if
• 24:08
  you
• 24:08
  have
• 24:08
  wild
• 24:09
  numbers
• 24:10
  in
• 24:10
  your
• 24:10
  pain
• 24:11
  group
• 24:11
  where
• 24:11
  it's
• 24:11
  like
• 24:12
  three
• 24:12
  out
• 24:12
  of
• 24:12
  four,
• 24:12
  a
• 24:13
  three
• 24:13
  out
• 24:13
  of
• 24:13
  ten,
• 24:13
  a
• 24:13
  four
• 24:13
  out
• 24:14
  of
• 24:14
  ten,
• 24:14
  a
• 24:14
  four
• 24:14
  out
• 24:14
  of
• 24:14
  ten,
• 24:15
  four
• 24:15
  out
• 24:15
  of
• 24:15
  ten,
• 24:15
  and
• 24:15
  then
• 24:15
  a
• 24:16
  ten
• 24:16
  out
• 24:16
  of
• 24:16
  ten,
• 24:16
  ten
• 24:17
  out
• 24:17
  of
• 24:17
  ten,
• 24:17
  ten
• 24:17
  out
• 24:17
  of
• 24:18
  ten,
• 24:18
  zero
• 24:18
  out
• 24:19
  of
• 24:19
  ten.
• 24:19
  Well,
• 24:19
  guess
• 24:20
  what?
• 24:20
  When
• 24:20
  you're
• 24:20
  running
• 24:20
  your
• 24:20
  T
• 24:21
  test
• 24:22
  and
• 24:22
  you
• 24:22
  see
• 24:22
  this
• 24:22
  wide
• 24:23
  standard
• 24:23
  deviation,
• 24:24
  your
• 24:25
  test
• 24:25
  is
• 24:25
  going
• 24:26
  to
• 24:26
  crunch
• 24:26
  out
• 24:26
  a
• 24:26
  product
• 24:27
  that's
• 24:27
  like,
• 24:27
  wait
• 24:27
  a
• 24:28
  minute,
• 24:28
  this
• 24:29
  is
• 24:29
  just.
• 24:30
  This
• 24:30
  may
• 24:30
  not
• 24:31
  be
• 24:31
  the
• 24:31
  jelly
• 24:31
  beans.
• 24:31
  We
• 24:32
  don't
• 24:32
  know
• 24:32
  because
• 24:33
  the
• 24:33
  data
• 24:33
  is
• 24:34
  really
• 24:34
  all
• 24:35
  over
• 24:35
  the
• 24:35
  place.
• 24:35
  The
• 24:36
  purpose
• 24:36
  of
• 24:36
  this
• 24:36
  T
• 24:37
  test
• 24:37
  is
• 24:37
  it's
• 24:37
  going
• 24:37
  to
• 24:38
  spit
• 24:38
  out
• 24:38
  a
• 24:38
  P
• 24:38
  value
• 24:39
  and
• 24:39
  the
• 24:39
  P
• 24:39
  value
• 24:40
  is
• 24:40
  going
• 24:40
  to
• 24:40
  tell
• 24:40
  you
• 24:40
  how
• 24:41
  likely
• 24:41
  it
• 24:41
  is
• 24:41
  that
• 24:41
  your
• 24:42
  results
• 24:42
  just
• 24:42
  happened
• 24:43
  by
• 24:43
  luck.
• 24:44
  Right?
• 24:44
  So
• 24:45
  our
• 24:45
  P
• 24:45
  value,
• 24:45
  let's
• 24:46
  just
• 24:46
  say,
• 24:46
  is
• 24:46
  0.01.
• 24:48
  So
• 24:48
  that's
• 24:48
  basically
• 24:49
  telling
• 24:49
  us
• 24:49
  there's
• 24:50
  only
• 24:50
  a
• 24:50
  1%
• 24:51
  chance
• 24:51
  this
• 24:51
  jelly
• 24:52
  bean
• 24:52
  effect
• 24:52
  is
• 24:53
  a
• 24:53
  fluke.
• 24:53
  Only
• 24:54
  1%
• 24:54
  chance.
• 24:55
  If
• 24:55
  we
• 24:55
  kept
• 24:55
  running
• 24:56
  this
• 24:56
  over
• 24:56
  and
• 24:56
  over
• 24:56
  and
• 24:57
  over
• 24:57
  again,
• 24:57
  there's
• 24:58
  a
• 24:58
  1%
• 24:58
  chance
• 24:59
  that
• 24:59
  this
• 25:00
  is
• 25:00
  just
• 25:00
  a
• 25:01
  weird
• 25:01
  day
• 25:01
  where
• 25:01
  all
• 25:02
  these
• 25:02
  kids
• 25:02
  randomly
• 25:03
  have
• 25:03
  stomachaches
• 25:04
  that
• 25:04
  don't
• 25:04
  have
• 25:04
  to
• 25:04
  do
• 25:05
  with
• 25:05
  jelly
• 25:05
  beans.
• 25:05
  In
• 25:05
  other
• 25:06
  words,
• 25:06
  it's
• 25:06
  pointing
• 25:06
  to
• 25:07
  the
• 25:07
  fact
• 25:07
  that
• 25:08
  this
• 25:08
  could
• 25:08
  very
• 25:08
  well
• 25:08
  be
• 25:09
  something
• 25:09
  different.
• 25:09
  It
• 25:10
  doesn't
• 25:10
  conclude
• 25:10
  that
• 25:11
  it's
• 25:11
  your
• 25:11
  jelly
• 25:11
  beans
• 25:12
  causing
• 25:12
  these
• 25:12
  stomach
• 25:13
  aches.
• 25:14
  Again,
• 25:14
  on
• 25:14
  a
• 25:14
  random
• 25:15
  day,
• 25:15
  we
• 25:16
  should
• 25:16
  not
• 25:16
  have
• 25:16
  this
• 25:17
  many
• 25:17
  tummy
• 25:17
  aches.
• 25:18
  That's
• 25:18
  what
• 25:18
  the
• 25:19
  P
• 25:19
  value
• 25:19
  is
• 25:19
  saying,
• 25:20
  right?
• 25:20
  There's
• 25:20
  only
• 25:21
  a
• 25:21
  1%
• 25:21
  chance
• 25:22
  there
• 25:22
  would
• 25:22
  be
• 25:22
  this
• 25:23
  many
• 25:23
  tummy
• 25:23
  aches
• 25:23
  on
• 25:24
  a
• 25:24
  random
• 25:24
  day.
• 25:24
  Most
• 25:25
  random
• 25:25
  days,
• 25:25
  we
• 25:26
  probably
• 25:26
  have
• 25:26
  10
• 25:27
  tummy
• 25:27
  aches
• 25:27
  in
• 25:27
  the
• 25:27
  school.
• 25:28
  But
• 25:28
  to
• 25:28
  have
• 25:29
  40
• 25:30
  or
• 25:30
  50,
• 25:30
  there's
• 25:31
  only
• 25:31
  a
• 25:31
  1%
• 25:32
  chance.
• 25:32
  And
• 25:32
  that's
• 25:33
  why
• 25:33
  your
• 25:33
  P
• 25:33
  value
• 25:33
  would
• 25:34
  be
• 25:34
  a
• 25:34
  0.01,
• 25:35
  that
• 25:36
  this
• 25:36
  jelly
• 25:36
  bean
• 25:36
  effect
• 25:37
  is
• 25:37
  just
• 25:37
  a
• 25:38
  fluke.
• 25:38
  That's
• 25:39
  fascinating.
• 25:40
  And
• 25:40
  then
• 25:40
  we
• 25:40
  move
• 25:40
  into
• 25:40
  effect
• 25:41
  size,
• 25:41
  as
• 25:41
  you've
• 25:42
  probably
• 25:42
  heard
• 25:42
  of
• 25:42
  before,
• 25:42
  Cohen's
• 25:43
  D
• 25:43
  or
• 25:44
  effect
• 25:44
  size.
• 25:45
  And
• 25:45
  that
• 25:45
  tells
• 25:46
  us
• 25:46
  if
• 25:46
  this
• 25:46
  difference
• 25:47
  is
• 25:47
  real.
• 25:48
  So
• 25:48
  if
• 25:48
  the
• 25:48
  P
• 25:48
  value
• 25:49
  tells
• 25:49
  us,
• 25:49
  is
• 25:49
  this
• 25:49
  real,
• 25:50
  is
• 25:50
  it
• 25:50
  big
• 25:51
  enough
• 25:51
  to
• 25:51
  care
• 25:51
  about?
• 25:52
  You
• 25:52
  could
• 25:52
  have
• 25:52
  a
• 25:52
  P
• 25:53
  value
• 25:53
  that
• 25:54
  says
• 25:55
  there's
• 25:55
  only
• 25:56
  a
• 25:56
  1%
• 25:56
  chance
• 25:57
  your
• 25:57
  jelly
• 25:57
  bean
• 25:57
  effect
• 25:57
  is
• 25:58
  a
• 25:58
  fluke.
• 25:59
  But
• 26:00
  if
• 26:00
  the
• 26:00
  effect
• 26:01
  size
• 26:01
  is
• 26:01
  small,
• 26:02
  who
• 26:02
  cares?
• 26:03
  You
• 26:03
  could
• 26:03
  say,
• 26:04
  this
• 26:04
  antidepressant.
• 26:05
  No
• 26:05
  way
• 26:05
  is
• 26:06
  this
• 26:06
  a
• 26:06
  fluke.
• 26:07
  This
• 26:07
  would
• 26:07
  never
• 26:07
  happen
• 26:08
  by
• 26:08
  random
• 26:08
  chance.
• 26:09
  There
• 26:09
  is
• 26:10
  some
• 26:10
  effect
• 26:10
  going
• 26:10
  on
• 26:11
  here.
• 26:11
  So
• 26:11
  great,
• 26:11
  you're
• 26:12
  really
• 26:12
  conclusive
• 26:13
  that
• 26:14
  this
• 26:14
  P
• 26:14
  value
• 26:14
  is
• 26:15
  significant.
• 26:15
  But
• 26:16
  if
• 26:16
  the
• 26:16
  effect
• 26:16
  size
• 26:17
  is
• 26:17
  low
• 26:17
  and
• 26:17
  your
• 26:18
  antidepressants
• 26:18
  only
• 26:19
  lowering
• 26:19
  depression
• 26:20
  like
• 26:20
  half
• 26:20
  a
• 26:21
  point,
• 26:21
  that's
• 26:22
  not
• 26:22
  clinically
• 26:22
  significant
• 26:23
  necessarily.
• 26:24
  Right.
• 26:24
  It's
• 26:24
  not
• 26:25
  something
• 26:25
  we
• 26:25
  really
• 26:25
  care
• 26:26
  about
• 26:26
  that
• 26:26
  much.
• 26:26
  Because
• 26:27
  is
• 26:27
  it
• 26:27
  really
• 26:27
  gonna
• 26:27
  materially
• 26:28
  help
• 26:28
  your
• 26:28
  patient
• 26:29
  if
• 26:29
  it
• 26:29
  just
• 26:30
  lowers
• 26:31
  their
• 26:31
  depressive
• 26:32
  score
• 26:32
  1
• 26:32
  point
• 26:33
  and
• 26:33
  their
• 26:33
  lives
• 26:34
  are
• 26:34
  still
• 26:34
  impairing
• 26:35
  and
• 26:35
  distressing
• 26:36
  and
• 26:36
  miserable?
• 26:36
  Not
• 26:37
  really,
• 26:37
  no.
• 26:37
  So
• 26:38
  an
• 26:38
  effect
• 26:38
  size
• 26:38
  is
• 26:38
  going
• 26:39
  to
• 26:39
  tell
• 26:39
  you
• 26:39
  how
• 26:39
  big
• 26:39
  is
• 26:39
  this
• 26:40
  difference?
• 26:40
  Is
• 26:40
  it
• 26:40
  big
• 26:40
  enough
• 26:41
  to
• 26:41
  care
• 26:41
  about?
• 26:42
  And
• 26:42
  a
• 26:42
  huge
• 26:43
  effect
• 26:43
  size
• 26:43
  of
• 26:43
  like
• 26:44
  2,
• 26:44
  which
• 26:44
  we
• 26:44
  had
• 26:45
  in
• 26:45
  the
• 26:45
  jelly
• 26:45
  bean
• 26:45
  case
• 26:46
  would
• 26:46
  be
• 26:46
  great.
• 26:47
  It's
• 26:47
  not
• 26:47
  subtle.
• 26:47
  It's
• 26:48
  a
• 26:48
  very
• 26:48
  big
• 26:48
  deal.
• 26:49
  And
• 26:49
  the
• 26:49
  effect
• 26:49
  size
• 26:50
  is
• 26:50
  crunched
• 26:51
  by
• 26:51
  finding
• 26:52
  the
• 26:52
  difference
• 26:53
  in
• 26:53
  pain.
• 26:54
  It's
• 26:54
  divided
• 26:54
  by
• 26:55
  the
• 26:55
  standard
• 26:55
  deviation.
• 26:56
  And
• 26:56
  it
• 26:56
  gives
• 26:57
  you
• 26:57
  an
• 26:57
  idea
• 26:58
  of
• 26:58
  how
• 26:58
  effective,
• 26:59
  how
• 26:59
  large
• 26:59
  this
• 26:59
  effect
• 27:00
  is.
• 27:00
  And
• 27:00
  again,
• 27:00
  I
• 27:00
  love
• 27:00
  the
• 27:01
  elegance
• 27:01
  of
• 27:01
  that
• 27:01
  because
• 27:02
  you're
• 27:02
  taking
• 27:03
  the
• 27:03
  average.
• 27:03
  So
• 27:03
  the
• 27:04
  average
• 27:04
  is
• 27:04
  way
• 27:04
  higher.
• 27:05
  Right.
• 27:05
  You
• 27:05
  have
• 27:05
  three
• 27:06
  higher
• 27:07
  pain
• 27:07
  points
• 27:08
  in
• 27:08
  the
• 27:08
  jelly
• 27:08
  bean
• 27:09
  group.
• 27:09
  Right?
• 27:09
  It's
• 27:10
  a
• 27:10
  4
• 27:10
  out
• 27:10
  of
• 27:10
  10
• 27:10
  versus
• 27:11
  a
• 27:11
  1
• 27:11
  out
• 27:11
  of
• 27:11
  10
• 27:11
  pain.
• 27:11
  It
• 27:12
  takes
• 27:12
  into
• 27:12
  account
• 27:12
  the
• 27:13
  variance
• 27:13
  within
• 27:14
  each
• 27:14
  group
• 27:15
  and
• 27:15
  that's
• 27:15
  how
• 27:16
  you
• 27:16
  find
• 27:17
  the
• 27:17
  actual
• 27:18
  effect
• 27:18
  size
• 27:18
  or
• 27:18
  the
• 27:18
  Cohen's
• 27:19
  D,
• 27:20
  which
• 27:20
  is
• 27:21
  very
• 27:21
  cool.
• 27:21
  I
• 27:21
  encourage
• 27:22
  you
• 27:22
  to
• 27:22
  play
• 27:22
  with
• 27:22
  these
• 27:23
  numbers
• 27:23
  a
• 27:23
  little
• 27:23
  bit
• 27:24
  later.
• 27:24
  But
• 27:24
  the
• 27:25
  effect
• 27:25
  sizes
• 27:26
  kind
• 27:26
  of
• 27:26
  vary,
• 27:27
  right?
• 27:27
  If
• 27:27
  you
• 27:28
  have
• 27:28
  a
• 27:28
  two
• 27:29
  effect
• 27:29
  size,
• 27:29
  that's
• 27:30
  massive,
• 27:30
  that's
• 27:30
  a
• 27:31
  really
• 27:31
  big
• 27:31
  deal.
• 27:32
  And
• 27:32
  in
• 27:32
  statistics
• 27:33
  or
• 27:33
  in
• 27:33
  your
• 27:33
  studies,
• 27:34
  and
• 27:34
  if
• 27:35
  your
• 27:35
  effect
• 27:35
  size
• 27:35
  is
• 27:36
  like.02,
• 27:37
  it's
• 27:37
  kind
• 27:37
  of
• 27:37
  subtle.0.5
• 27:38
  is
• 27:38
  more
• 27:39
  significant.
• 27:39
  When
• 27:39
  you
• 27:40
  approach
• 27:40
  one,
• 27:40
  it's
• 27:41
  definitely
• 27:41
  clinically
• 27:42
  meaningful.
• 27:43
  The
• 27:43
  more
• 27:43
  you
• 27:43
  move
• 27:44
  closer
• 27:44
  to
• 27:44
  one,
• 27:44
  certainly
• 27:45
  one
• 27:45
  and
• 27:45
  over,
• 27:46
  you're
• 27:46
  getting
• 27:46
  into
• 27:46
  large
• 27:46
  effect
• 27:47
  sizes.
• 27:48
  02,
• 27:48
  03.
• 27:49
  There's
• 27:49
  something
• 27:49
  there.
• 27:50
  057,
• 27:51
  0.8.
• 27:52
  We're
• 27:52
  getting
• 27:52
  a
• 27:52
  little
• 27:52
  bit
• 27:53
  more
• 27:53
  significant
• 27:54
  and
• 27:54
  the
• 27:54
  effect
• 27:54
  is
• 27:55
  a
• 27:55
  little
• 27:55
  bit
• 27:55
  more
• 27:55
  intense.
• 27:56
  So
• 27:56
  effect
• 27:57
  size
• 27:57
  is
• 27:57
  like,
• 27:57
  how
• 27:58
  meaningful
• 27:58
  is
• 27:58
  this
• 27:59
  for
• 27:59
  me
• 28:00
  as
• 28:00
  a
• 28:00
  clinician,
• 28:01
  as
• 28:01
  I
• 28:01
  talk
• 28:01
  about
• 28:02
  these
• 28:02
  medications
• 28:02
  and
• 28:03
  think
• 28:03
  about
• 28:03
  these
• 28:03
  medications
• 28:04
  now
• 28:04
  we
• 28:05
  have
• 28:05
  odds
• 28:05
  ratios.
• 28:06
  You've
• 28:06
  probably
• 28:06
  heard
• 28:06
  about
• 28:06
  those.
• 28:07
  The
• 28:07
  odd
• 28:07
  ratio
• 28:08
  compares
• 28:08
  the
• 28:08
  odds
• 28:09
  of
• 28:09
  getting
• 28:09
  a
• 28:09
  tummy
• 28:09
  ache
• 28:09
  between
• 28:10
  the
• 28:10
  two
• 28:10
  groups.
• 28:11
  In
• 28:11
  this
• 28:11
  case,
• 28:11
  we
• 28:11
  have
• 28:12
  an
• 28:12
  odds
• 28:12
  ratio
• 28:12
  of
• 28:12
  6.
• 28:13
  That
• 28:13
  means
• 28:13
  Jelly
• 28:14
  Bean
• 28:14
  Kids
• 28:14
  had
• 28:15
  six
• 28:15
  times
• 28:15
  the
• 28:15
  odds
• 28:16
  of
• 28:16
  getting
• 28:16
  a
• 28:16
  tummy
• 28:16
  ache.
• 28:17
  Right.
• 28:17
  So
• 28:17
  an
• 28:17
  OR
• 28:17
  ratio
• 28:18
  of
• 28:18
  six
• 28:18
  means
• 28:19
  you're
• 28:19
  six
• 28:20
  times
• 28:20
  more
• 28:20
  likely.
• 28:20
  If
• 28:21
  you
• 28:21
  see
• 28:21
  it
• 28:21
  in
• 28:21
  a
• 28:21
  depressive
• 28:22
  or
• 28:22
  anxiety
• 28:23
  journal
• 28:23
  or
• 28:23
  something,
• 28:24
  benzodiazepines
• 28:25
  are
• 28:25
  six
• 28:25
  times
• 28:26
  more
• 28:26
  likely
• 28:26
  to
• 28:26
  stop
• 28:27
  panic
• 28:27
  attacks
• 28:27
  than
• 28:27
  placebo.
• 28:28
  That
• 28:28
  would
• 28:28
  be
• 28:28
  an
• 28:29
  odds
• 28:29
  ratio
• 28:29
  of
• 28:29
  6.
• 28:30
  I
• 28:30
  made
• 28:30
  that
• 28:31
  statistic
• 28:31
  up,
• 28:32
  but
• 28:32
  you
• 28:32
  get
• 28:32
  what
• 28:33
  I
• 28:33
  mean.
• 28:33
  That's
• 28:33
  an
• 28:33
  odds
• 28:34
  ratio
• 28:34
  and
• 28:34
  a
• 28:34
  confidence
• 28:35
  interval.
• 28:35
  You've
• 28:35
  heard
• 28:36
  of
• 28:36
  that
• 28:36
  too.
• 28:36
  That
• 28:36
  tells
• 28:37
  us
• 28:37
  the
• 28:37
  range
• 28:37
  we're
• 28:38
  pretty
• 28:38
  sure
• 28:38
  the
• 28:38
  real
• 28:39
  effect
• 28:39
  lives
• 28:40
  inside
• 28:40
  of.
• 28:41
  Right.
• 28:41
  So
• 28:41
  if
• 28:42
  our
• 28:42
  odds
• 28:42
  ratio
• 28:43
  was
• 28:43
  6
• 28:44
  and
• 28:45
  the
• 28:45
  95%
• 28:46
  confidence
• 28:46
  interval
• 28:47
  was
• 28:47
  3
• 28:47
  to
• 28:48
  12,
• 28:48
  that
• 28:48
  means
• 28:49
  we're
• 28:49
  95%
• 28:50
  sure
• 28:50
  the
• 28:51
  real
• 28:51
  odds
• 28:51
  are
• 28:52
  between
• 28:52
  3
• 28:52
  and
• 28:53
  12.
• 28:53
  In
• 28:53
  other
• 28:53
  words,
• 28:54
  we're
• 28:54
  95%
• 28:55
  sure
• 28:55
  that.
• 28:56
  That
• 28:56
  the
• 28:56
  odds
• 28:56
  of
• 28:57
  a
• 28:57
  benzo
• 28:58
  reducing
• 28:58
  a
• 28:59
  panic
• 28:59
  attack
• 29:00
  is
• 29:00
  three
• 29:00
  to
• 29:01
  12
• 29:01
  times
• 29:01
  more
• 29:01
  likely
• 29:02
  than
• 29:02
  placebo.
• 29:02
  Not
• 29:03
  necessarily
• 29:03
  six,
• 29:03
  but
• 29:04
  three
• 29:04
  to
• 29:04
  12.
• 29:05
  Right.
• 29:06
  That's
• 29:06
  what
• 29:06
  we
• 29:06
  mean
• 29:07
  by
• 29:07
  confidence
• 29:07
  interval.
• 29:08
  95%
• 29:09
  chance.
• 29:09
  And
• 29:10
  you'll
• 29:10
  see
• 29:10
  a
• 29:10
  little
• 29:10
  parenthesis,
• 29:11
  three
• 29:11
  to
• 29:12
  12
• 29:12
  or
• 29:12
  three
• 29:12
  to
• 29:12
  five
• 29:13
  or
• 29:13
  three
• 29:13
  to
• 29:13
  six.
• 29:13
  And
• 29:14
  that's
• 29:14
  just
• 29:14
  telling
• 29:14
  you
• 29:14
  how
• 29:14
  likely
• 29:15
  it
• 29:15
  is
• 29:16
  in
• 29:16
  that
• 29:16
  range.
• 29:17
  Right,
• 29:17
  because
• 29:17
  it's
• 29:17
  never
• 29:17
  as
• 29:18
  simple
• 29:18
  as
• 29:19
  all
• 29:19
  benzos
• 29:20
  reduced
• 29:21
  panic
• 29:21
  attack
• 29:21
  six
• 29:22
  times
• 29:22
  more
• 29:22
  likely
• 29:22
  than
• 29:23
  placebo.
• 29:23
  It's
• 29:24
  not
• 29:24
  that
• 29:24
  simple.
• 29:24
  You
• 29:24
  have
• 29:25
  a
• 29:25
  confidence
• 29:25
  range
• 29:26
  or
• 29:26
  interval
• 29:27
  that's
• 29:27
  based
• 29:27
  on
• 29:28
  the
• 29:28
  data
• 29:28
  we
• 29:29
  described
• 29:29
  above
• 29:29
  in
• 29:30
  our
• 29:30
  inferential
• 29:31
  and
• 29:31
  descriptive
• 29:31
  statistics.
• 29:33
  Think
• 29:33
  of
• 29:33
  statistics
• 29:34
  a
• 29:34
  little
• 29:34
  bit
• 29:34
  like
• 29:34
  baking.
• 29:35
  The
• 29:35
  descriptive
• 29:36
  statistics
• 29:36
  tell
• 29:36
  you
• 29:36
  about
• 29:37
  the
• 29:37
  ingredients.
• 29:38
  The
• 29:38
  P
• 29:38
  value
• 29:39
  checks
• 29:39
  if
• 29:39
  the
• 29:39
  cake
• 29:40
  is
• 29:40
  rising
• 29:40
  by
• 29:40
  luck
• 29:41
  or
• 29:41
  not,
• 29:41
  or
• 29:42
  if
• 29:42
  there's
• 29:42
  something
• 29:42
  else
• 29:43
  at
• 29:43
  play
• 29:43
  that
• 29:43
  you
• 29:43
  put
• 29:43
  in
• 29:44
  the
• 29:44
  batter.
• 29:44
  The
• 29:45
  effect
• 29:45
  size
• 29:45
  is
• 29:45
  how
• 29:45
  big
• 29:46
  the
• 29:46
  cake
• 29:46
  actually
• 29:46
  is,
• 29:47
  and
• 29:47
  the
• 29:47
  odds
• 29:47
  ratio
• 29:48
  is
• 29:48
  how
• 29:48
  much
• 29:48
  more
• 29:48
  likely
• 29:49
  it
• 29:49
  is
• 29:49
  to
• 29:49
  taste
• 29:50
  good
• 29:50
  or
• 29:51
  bad.
• 29:51
  The
• 29:52
  confidence
• 29:52
  interval
• 29:53
  is
• 29:53
  how
• 29:53
  sure
• 29:53
  you
• 29:53
  are
• 29:54
  it'll
• 29:54
  turn
• 29:54
  out
• 29:54
  the
• 29:54
  same
• 29:55
  next
• 29:55
  time
• 29:55
  and
• 29:56
  within
• 29:56
  which
• 29:57
  parameters.
• 29:58
  So
• 29:58
  that
• 29:59
  hopefully
• 29:59
  demystifies
• 30:00
  is
• 30:01
  a
• 30:01
  lot.
• 30:01
  When
• 30:02
  you
• 30:02
  pick
• 30:02
  up
• 30:02
  a
• 30:02
  study
• 30:03
  just
• 30:04
  to
• 30:04
  wrap
• 30:04
  everything
• 30:05
  up,
• 30:05
  I
• 30:06
  want
• 30:06
  you
• 30:06
  to
• 30:06
  look
• 30:06
  at
• 30:06
  it
• 30:06
  and
• 30:07
  say,
• 30:08
  are
• 30:08
  the
• 30:08
  objectives
• 30:09
  clearly
• 30:09
  stated
• 30:10
  in
• 30:10
  this
• 30:11
  study?
• 30:11
  Does
• 30:12
  this
• 30:12
  make
• 30:12
  sense
• 30:13
  in
• 30:13
  our
• 30:13
  study?
• 30:14
  Did
• 30:14
  jelly
• 30:14
  beans
• 30:15
  increase
• 30:15
  tummy
• 30:15
  aches
• 30:16
  in
• 30:16
  school
• 30:16
  kids?
• 30:17
  Is
• 30:17
  that
• 30:18
  a
• 30:18
  question
• 30:18
  worth
• 30:19
  asking?
• 30:19
  You're
• 30:20
  probably
• 30:20
  gonna
• 30:20
  say,
• 30:20
  not
• 30:21
  at
• 30:21
  all.
• 30:21
  Who
• 30:21
  cares?
• 30:22
  I
• 30:22
  did
• 30:22
  this
• 30:22
  for
• 30:22
  fun
• 30:23
  today.
• 30:23
  But
• 30:24
  when
• 30:24
  you
• 30:24
  pick
• 30:24
  up
• 30:24
  a
• 30:24
  research
• 30:25
  article,
• 30:25
  really
• 30:25
  look
• 30:26
  at
• 30:26
  that.
• 30:26
  Does
• 30:27
  playing
• 30:28
  Cambodian
• 30:29
  style
• 30:29
  badminton
• 30:31
  reduce
• 30:33
  headaches
• 30:34
  in
• 30:34
  postmenopausal
• 30:35
  women?
• 30:36
  That
• 30:37
  might
• 30:37
  be
• 30:37
  relevant
• 30:37
  to
• 30:38
  some
• 30:38
  populations,
• 30:38
  but
• 30:39
  is
• 30:39
  that
• 30:39
  really
• 30:39
  relevant
• 30:40
  to
• 30:40
  me?
• 30:41
  Does
• 30:41
  this
• 30:41
  reflect
• 30:41
  the
• 30:42
  average
• 30:42
  patient
• 30:42
  that
• 30:43
  walks
• 30:43
  into
• 30:43
  my
• 30:43
  office
• 30:44
  that
• 30:44
  plays
• 30:45
  Cambodian
• 30:46
  badminton
• 30:47
  and
• 30:47
  Is
• 30:47
  postmenopausal
• 30:49
  maybe
• 30:49
  not,
• 30:49
  right?
• 30:50
  Is
• 30:50
  this
• 30:50
  worth
• 30:50
  me
• 30:51
  reading?
• 30:51
  Is
• 30:51
  this
• 30:51
  worth
• 30:52
  a
• 30:52
  study,
• 30:52
  even
• 30:52
  doing
• 30:53
  it?
• 30:53
  I
• 30:53
  don't
• 30:53
  know.
• 30:53
  Is
• 30:54
  this
• 30:54
  generalizable
• 30:55
  to
• 30:55
  me?
• 30:56
  And
• 30:56
  then
• 30:56
  when
• 30:56
  you
• 30:56
  look
• 30:56
  at
• 30:56
  the
• 30:57
  study,
• 30:57
  you
• 30:57
  want
• 30:57
  to
• 30:57
  ask,
• 30:58
  how
• 30:58
  did
• 30:58
  they
• 30:58
  recruit
• 30:59
  these
• 30:59
  subjects?
• 31:00
  In
• 31:00
  my
• 31:01
  study
• 31:01
  with
• 31:01
  the
• 31:01
  jelly
• 31:02
  beans,
• 31:02
  like,
• 31:02
  how
• 31:02
  did
• 31:02
  we
• 31:02
  do
• 31:02
  this?
• 31:03
  Did
• 31:03
  we.
• 31:03
  In
• 31:04
  our
• 31:04
  subjects
• 31:04
  are
• 31:04
  school
• 31:05
  kids
• 31:05
  in
• 31:05
  the
• 31:05
  cafeteria?
• 31:06
  Right.
• 31:07
  Did
• 31:07
  we
• 31:07
  explain
• 31:08
  how
• 31:09
  we
• 31:09
  picked
• 31:09
  them
• 31:09
  and
• 31:10
  what
• 31:10
  our
• 31:10
  inclusion
• 31:11
  and
• 31:11
  exclusion
• 31:11
  criteria
• 31:12
  was?
• 31:12
  Are
• 31:13
  these
• 31:13
  volunteers?
• 31:13
  Are
• 31:14
  these
• 31:14
  compliant
• 31:14
  kids?
• 31:15
  They
• 31:15
  have
• 31:15
  medical
• 31:15
  problems.
• 31:16
  Can
• 31:16
  I
• 31:16
  generalize
• 31:17
  this?
• 31:17
  Right?
• 31:18
  Is
• 31:18
  this
• 31:18
  a
• 31:18
  special
• 31:18
  school
• 31:19
  where
• 31:19
  everybody's
• 31:20
  lactose
• 31:20
  intolerant
• 31:20
  and
• 31:21
  has
• 31:21
  food
• 31:21
  allergies?
• 31:22
  Is
• 31:22
  this
• 31:22
  a
• 31:22
  school
• 31:22
  where
• 31:23
  their
• 31:23
  grades
• 31:24
  are
• 31:24
  really
• 31:24
  low
• 31:24
  and
• 31:24
  they're
• 31:25
  behaviorally
• 31:25
  a
• 31:26
  problem?
• 31:26
  And
• 31:27
  is
• 31:27
  it
• 31:27
  understaffed?
• 31:28
  Right.
• 31:29
  Maybe
• 31:29
  the
• 31:29
  nurses.
• 31:30
  There
• 31:30
  aren't
• 31:30
  enough
• 31:31
  nurses
• 31:31
  to
• 31:31
  collect
• 31:31
  data
• 31:32
  accurately.
• 31:33
  So
• 31:33
  you
• 31:33
  want
• 31:33
  to
• 31:33
  really
• 31:33
  look
• 31:34
  at
• 31:34
  this,
• 31:34
  too.
• 31:34
  When
• 31:34
  you're
• 31:35
  looking
• 31:35
  at
• 31:35
  how
• 31:36
  your
• 31:36
  data
• 31:36
  is
• 31:36
  collected
• 31:37
  and
• 31:37
  the
• 31:38
  designs
• 31:38
  of
• 31:38
  these
• 31:38
  studies,
• 31:39
  can
• 31:39
  you
• 31:39
  generalize
• 31:40
  it?
• 31:40
  Is
• 31:40
  this
• 31:41
  representative
• 31:41
  of
• 31:41
  the
• 31:42
  patients
• 31:42
  that
• 31:42
  I
• 31:42
  see?
• 31:42
  See?
• 31:43
  How
• 31:43
  many
• 31:43
  kids
• 31:44
  are
• 31:44
  we
• 31:44
  even
• 31:44
  talking
• 31:45
  about
• 31:45
  here
• 31:45
  with
• 31:45
  jelly
• 31:46
  beans?
• 31:46
  If
• 31:46
  you're
• 31:47
  reading
• 31:47
  a
• 31:47
  research
• 31:47
  study
• 31:48
  and
• 31:48
  there's
• 31:48
  like,
• 31:48
  10
• 31:49
  people
• 31:49
  trying
• 31:50
  to
• 31:50
  figure
• 31:50
  out
• 31:50
  the
• 31:50
  effects
• 31:51
  of
• 31:52
  buspirone
• 31:53
  on
• 31:53
  sexual
• 31:54
  side
• 31:54
  effects,
• 31:55
  okay,
• 31:55
  10
• 31:56
  people.
• 31:56
  It
• 31:56
  might
• 31:57
  not
• 31:57
  be
• 31:57
  that
• 31:58
  important
• 31:59
  to
• 31:59
  you
• 31:59
  because
• 31:59
  it
• 31:59
  wasn't
• 31:59
  powered
• 32:00
  adequately.
• 32:01
  We
• 32:01
  know
• 32:01
  about
• 32:01
  power,
• 32:02
  right?
• 32:02
  The
• 32:02
  more
• 32:02
  people
• 32:03
  in
• 32:03
  your
• 32:03
  sample,
• 32:04
  the
• 32:04
  more
• 32:04
  we
• 32:04
  can
• 32:05
  figure
• 32:05
  out
• 32:05
  the
• 32:06
  effect
• 32:06
  of
• 32:06
  this
• 32:07
  potentially,
• 32:07
  right?
• 32:07
  If
• 32:08
  I
• 32:08
  flip
• 32:08
  a
• 32:08
  coin
• 32:09
  and
• 32:09
  I
• 32:09
  only
• 32:09
  flip
• 32:09
  it
• 32:10
  five
• 32:10
  times,
• 32:10
  I'm
• 32:11
  more
• 32:11
  likely
• 32:11
  to
• 32:12
  get
• 32:12
  a
• 32:12
  fluke.
• 32:12
  I
• 32:12
  could
• 32:13
  get
• 32:13
  tails
• 32:13
  five
• 32:13
  times.
• 32:14
  Or
• 32:14
  if
• 32:14
  I
• 32:15
  flipped
• 32:15
  it
• 32:15
  twice,
• 32:16
  I
• 32:16
  could
• 32:16
  get
• 32:16
  tails
• 32:17
  two
• 32:17
  times.
• 32:17
  And
• 32:18
  so
• 32:18
  that's
• 32:18
  like,
• 32:19
  you
• 32:19
  know.
• 32:19
  But
• 32:19
  if
• 32:20
  I
• 32:20
  do
• 32:20
  that
• 32:20
  500
• 32:21
  times,
• 32:21
  it's
• 32:22
  basically
• 32:22
  impossible
• 32:23
  for
• 32:23
  me
• 32:23
  to
• 32:23
  get
• 32:23
  tails
• 32:24
  every
• 32:24
  time
• 32:24
  unless
• 32:25
  the
• 32:25
  coin
• 32:25
  is
• 32:25
  rigged.
• 32:26
  What
• 32:27
  that
• 32:27
  means
• 32:27
  is
• 32:28
  you've
• 32:28
  got
• 32:28
  to
• 32:28
  power
• 32:28
  your
• 32:29
  studies.
• 32:29
  The
• 32:29
  more
• 32:30
  people
• 32:31
  you
• 32:31
  put
• 32:31
  in
• 32:31
  these
• 32:32
  studies,
• 32:32
  the
• 32:32
  less
• 32:32
  likely
• 32:33
  that
• 32:34
  chance
• 32:34
  is
• 32:34
  going
• 32:34
  to
• 32:34
  rule
• 32:35
  your
• 32:35
  data.
• 32:35
  Right?
• 32:35
  The
• 32:36
  less
• 32:36
  likely
• 32:36
  I'm
• 32:36
  going
• 32:36
  to
• 32:37
  spin
• 32:37
  tails
• 32:38
  200
• 32:38
  times.
• 32:39
  But
• 32:39
  if
• 32:39
  I
• 32:40
  only
• 32:40
  do
• 32:40
  it
• 32:40
  two
• 32:40
  or
• 32:40
  three
• 32:41
  times,
• 32:41
  I
• 32:41
  could
• 32:41
  draw
• 32:41
  these
• 32:42
  huge
• 32:42
  conclusions.
• 32:42
  Well,
• 32:43
  tails
• 32:43
  is
• 32:43
  more
• 32:43
  likely
• 32:44
  than
• 32:44
  heads
• 32:44
  because
• 32:45
  I
• 32:45
  flipped
• 32:45
  it
• 32:45
  twice
• 32:46
  and
• 32:46
  I
• 32:46
  got
• 32:46
  two
• 32:46
  tails.
• 32:47
  My
• 32:47
  study
• 32:47
  wasn't
• 32:47
  adequately
• 32:48
  powered,
• 32:48
  and
• 32:49
  thus
• 32:49
  it's
• 32:49
  kind
• 32:49
  of
• 32:50
  misleading.
• 32:50
  So
• 32:50
  you
• 32:50
  have
• 32:51
  to
• 32:51
  realize
• 32:52
  samples,
• 32:53
  right?
• 32:53
  Think
• 32:53
  about
• 32:53
  that.
• 32:54
  Is
• 32:54
  this
• 32:54
  sample
• 32:54
  adequately
• 32:56
  powered?
• 32:57
  How
• 32:57
  many
• 32:58
  questions
• 32:58
  are
• 32:58
  we
• 32:58
  testing
• 32:59
  here?
• 33:00
  In
• 33:00
  this
• 33:00
  case,
• 33:00
  we
• 33:01
  tested
• 33:01
  how
• 33:02
  many
• 33:02
  people
• 33:02
  had
• 33:02
  tummy
• 33:03
  aches?
• 33:04
  What
• 33:04
  the
• 33:04
  score
• 33:04
  was
• 33:04
  of
• 33:05
  their
• 33:05
  tummy
• 33:05
  aches,
• 33:06
  right?
• 33:06
  Is
• 33:06
  now,
• 33:06
  are
• 33:07
  tummy
• 33:07
  aches.
• 33:07
  Well,
• 33:07
  Defined.
• 33:08
  That's
• 33:08
  a
• 33:08
  huge
• 33:09
  limitation
• 33:09
  of
• 33:10
  my
• 33:10
  study,
• 33:10
  because
• 33:11
  a
• 33:11
  tummy
• 33:12
  ache
• 33:12
  could
• 33:12
  just
• 33:12
  be
• 33:12
  what
• 33:12
  the
• 33:12
  kid
• 33:13
  says
• 33:13
  the
• 33:13
  tummy
• 33:13
  ache
• 33:14
  is.
• 33:14
  Oh,
• 33:14
  I
• 33:14
  have
• 33:14
  a
• 33:14
  tummy
• 33:15
  ache,
• 33:16
  right?
• 33:16
  Or
• 33:16
  is
• 33:16
  it
• 33:17
  actually.
• 33:18
  And
• 33:18
  where
• 33:19
  do
• 33:19
  we
• 33:19
  define
• 33:19
  tummy?
• 33:20
  Are
• 33:20
  some
• 33:20
  kids
• 33:21
  pointing
• 33:21
  to
• 33:21
  their
• 33:21
  back
• 33:21
  or
• 33:22
  their
• 33:22
  head
• 33:22
  on
• 33:22
  accident?
• 33:23
  And
• 33:23
  it's
• 33:23
  not
• 33:24
  really
• 33:24
  even
• 33:24
  a
• 33:24
  tummy
• 33:24
  ache?
• 33:25
  So
• 33:26
  we
• 33:26
  have
• 33:26
  to
• 33:26
  carefully
• 33:26
  define
• 33:27
  clinically
• 33:27
  what
• 33:28
  a
• 33:28
  tummy
• 33:28
  ache
• 33:28
  is.
• 33:29
  Just
• 33:29
  like
• 33:29
  when
• 33:29
  you're
• 33:30
  reading
• 33:30
  your
• 33:30
  articles,
• 33:31
  how
• 33:31
  are
• 33:31
  we
• 33:31
  defining
• 33:32
  depression?
• 33:33
  Is
• 33:33
  there
• 33:33
  an
• 33:33
  official
• 33:34
  metric
• 33:34
  that
• 33:34
  we're
• 33:35
  using?
• 33:35
  Are
• 33:35
  we
• 33:35
  always
• 33:35
  using
• 33:36
  the
• 33:36
  same
• 33:36
  metric?
• 33:36
  Are
• 33:37
  we
• 33:37
  changing
• 33:37
  metrics?
• 33:38
  Are
• 33:38
  we
• 33:38
  relying
• 33:39
  only
• 33:39
  on
• 33:39
  the
• 33:40
  clinical
• 33:40
  interview?
• 33:40
  And
• 33:41
  maybe
• 33:41
  the
• 33:41
  interview
• 33:41
  can
• 33:42
  vary
• 33:42
  between
• 33:43
  clinicians?
• 33:43
  How
• 33:44
  long
• 33:44
  are
• 33:44
  these
• 33:44
  interviews?
• 33:45
  Are
• 33:45
  they
• 33:45
  60
• 33:45
  minutes?
• 33:46
  Are
• 33:46
  they
• 33:46
  20
• 33:46
  minutes?
• 33:47
  Do
• 33:47
  these
• 33:47
  patients
• 33:48
  also
• 33:48
  have
• 33:48
  bipolar
• 33:49
  disorder?
• 33:49
  Do
• 33:49
  they
• 33:49
  have
• 33:50
  other
• 33:50
  comorbidities?
• 33:51
  Defining
• 33:51
  your
• 33:52
  population,
• 33:52
  defining
• 33:53
  what
• 33:53
  you're
• 33:54
  measuring,
• 33:54
  defining
• 33:55
  all
• 33:55
  of
• 33:55
  these
• 33:55
  things
• 33:56
  carefully
• 33:56
  and
• 33:56
  narrowly
• 33:57
  are
• 33:58
  very
• 33:58
  important.
• 33:58
  That's
• 33:58
  how
• 33:59
  you
• 33:59
  can
• 33:59
  look
• 33:59
  at
• 33:59
  these
• 34:00
  research
• 34:00
  articles
• 34:01
  with
• 34:01
  a
• 34:01
  lot
• 34:02
  more
• 34:02
  scrutiny.
• 34:03
  You
• 34:04
  got
• 34:04
  to
• 34:04
  look
• 34:04
  at
• 34:04
  how
• 34:04
  many
• 34:04
  kids
• 34:05
  dropped
• 34:05
  out.
• 34:06
  Maybe
• 34:06
  50
• 34:07
  kids
• 34:07
  dropped
• 34:07
  out
• 34:08
  of
• 34:08
  this
• 34:08
  after
• 34:08
  one
• 34:08
  jelly
• 34:09
  bean
• 34:09
  saying
• 34:09
  they
• 34:09
  were
• 34:09
  in
• 34:10
  too
• 34:10
  much
• 34:10
  pain.
• 34:10
  They
• 34:11
  went
• 34:11
  home
• 34:11
  for
• 34:11
  days.
• 34:11
  We
• 34:12
  never
• 34:12
  even
• 34:12
  measured
• 34:12
  them
• 34:13
  because
• 34:13
  they
• 34:13
  didn't
• 34:13
  show
• 34:13
  up
• 34:13
  at
• 34:13
  the
• 34:14
  nurse's
• 34:14
  office,
• 34:15
  right?
• 34:15
  So
• 34:15
  that's
• 34:15
  another
• 34:16
  reason
• 34:16
  you
• 34:16
  got
• 34:16
  to
• 34:16
  look
• 34:16
  at
• 34:17
  the
• 34:17
  primary
• 34:17
  outcome.
• 34:18
  Is
• 34:18
  the
• 34:18
  nursing
• 34:18
  office
• 34:19
  visit
• 34:19
  the
• 34:20
  best
• 34:20
  primary
• 34:20
  outcome
• 34:21
  measure?
• 34:21
  Could
• 34:22
  we
• 34:22
  have
• 34:22
  done
• 34:22
  this
• 34:23
  differently?
• 34:24
  Right?
• 34:24
  Could
• 34:24
  we
• 34:24
  have
• 34:24
  said
• 34:25
  school
• 34:25
  absentees
• 34:26
  or
• 34:27
  self
• 34:28
  report
• 34:28
  in
• 34:28
  the
• 34:28
  classroom
• 34:29
  outside
• 34:29
  of
• 34:29
  the
• 34:30
  nurse's
• 34:30
  office?
• 34:30
  If
• 34:31
  we're
• 34:31
  only
• 34:31
  capturing
• 34:32
  pain
• 34:32
  and
• 34:33
  tummy
• 34:33
  ache
• 34:33
  incidents
• 34:33
  in
• 34:34
  the
• 34:34
  nurse's
• 34:34
  office,
• 34:35
  that
• 34:35
  might
• 34:35
  be
• 34:36
  misleading.
• 34:36
  Maybe
• 34:37
  there
• 34:37
  was
• 34:37
  a
• 34:37
  holiday
• 34:38
  and
• 34:38
  we
• 34:38
  missed
• 34:38
  half
• 34:38
  the
• 34:39
  kids.
• 34:39
  Maybe
• 34:39
  there
• 34:40
  was
• 34:40
  a
• 34:40
  half
• 34:40
  day.
• 34:40
  Maybe
• 34:41
  recess
• 34:41
  was
• 34:41
  longer
• 34:42
  one
• 34:42
  day.
• 34:43
  Or
• 34:43
  again,
• 34:43
  think
• 34:44
  about
• 34:44
  all
• 34:44
  the
• 34:44
  variables
• 34:45
  at
• 34:45
  lunch.
• 34:46
  The
• 34:46
  bad
• 34:46
  chili,
• 34:47
  the
• 34:47
  lunch
• 34:47
  lady
• 34:48
  made
• 34:48
  a
• 34:48
  weird
• 34:49
  stack
• 34:49
  of
• 34:49
  pancakes.
• 34:50
  There's
• 34:50
  so
• 34:51
  many
• 34:51
  variables.
• 34:52
  And
• 34:52
  so
• 34:52
  really
• 34:52
  scrutinizing
• 34:53
  your
• 34:54
  study
• 34:54
  and
• 34:55
  having
• 34:55
  fun
• 34:55
  with
• 34:55
  it
• 34:55
  and
• 34:56
  thinking
• 34:56
  critically,
• 34:56
  can
• 34:57
  I
• 34:57
  trust
• 34:57
  this
• 34:57
  data?
• 34:58
  Can
• 34:59
  I
• 34:59
  trust
• 34:59
  what
• 34:59
  I'm
• 35:00
  seeing
• 35:00
  here?
• 35:00
  It's
• 35:01
  telling
• 35:01
  me
• 35:01
  what
• 35:01
  it
• 35:02
  thinks
• 35:02
  that's
• 35:02
  really
• 35:03
  important.
• 35:04
  The
• 35:04
  number
• 35:04
  of
• 35:04
  jelly
• 35:05
  beans
• 35:05
  these
• 35:05
  kids
• 35:05
  ate,
• 35:06
  was
• 35:06
  that
• 35:06
  measured?
• 35:07
  Did
• 35:07
  we
• 35:07
  quantify
• 35:08
  that?
• 35:08
  Do
• 35:08
  we
• 35:08
  care?
• 35:09
  Maybe
• 35:09
  jelly
• 35:09
  beans
• 35:10
  do
• 35:10
  cause
• 35:10
  tummy
• 35:11
  aches,
• 35:11
  but
• 35:11
  it's
• 35:11
  only
• 35:11
  if
• 35:11
  people
• 35:12
  have
• 35:12
  like
• 35:12
  50,
• 35:13
  and
• 35:13
  if
• 35:13
  they
• 35:13
  have
• 35:13
  only
• 35:14
  two,
• 35:14
  we
• 35:15
  can't
• 35:15
  really
• 35:15
  tell.
• 35:15
  We
• 35:15
  can't
• 35:16
  say
• 35:16
  declaratively
• 35:17
  that
• 35:17
  jelly
• 35:18
  beans
• 35:18
  cause
• 35:18
  tummy
• 35:19
  aches.
• 35:19
  We
• 35:19
  could
• 35:19
  say
• 35:20
  jelly
• 35:20
  beans
• 35:21
  at
• 35:21
  this
• 35:22
  particular
• 35:22
  quantity
• 35:23
  cause
• 35:23
  tummy
• 35:24
  aches,
• 35:24
  and
• 35:24
  we
• 35:25
  might
• 35:25
  not
• 35:25
  even
• 35:25
  be
• 35:25
  able
• 35:25
  to
• 35:26
  say
• 35:26
  that
• 35:26
  we
• 35:26
  might
• 35:26
  say
• 35:27
  40
• 35:28
  jelly
• 35:28
  beans
• 35:29
  in
• 35:29
  these
• 35:29
  types
• 35:30
  of
• 35:30
  kids
• 35:31
  who
• 35:31
  haven't
• 35:31
  eaten
• 35:32
  in
• 35:32
  three
• 35:32
  hours
• 35:32
  can
• 35:33
  cause
• 35:33
  tummy
• 35:33
  aches.
• 35:34
  You
• 35:34
  see
• 35:34
  how
• 35:34
  rich
• 35:34
  this
• 35:35
  is,
• 35:35
  how
• 35:35
  you
• 35:35
  can
• 35:35
  stratify
• 35:36
  it
• 35:36
  and
• 35:36
  pick
• 35:36
  it
• 35:37
  apart
• 35:37
  and
• 35:37
  think
• 35:37
  critically
• 35:38
  when
• 35:38
  you
• 35:39
  look
• 35:39
  at
• 35:40
  a
• 35:40
  research
• 35:40
  article.
• 35:40
  There's
• 35:41
  endless
• 35:41
  ways
• 35:42
  to
• 35:42
  pick
• 35:43
  apart
• 35:43
  a
• 35:43
  study
• 35:43
  and
• 35:44
  think
• 35:44
  about
• 35:44
  it
• 35:44
  critically
• 35:45
  and
• 35:46
  really
• 35:46
  feel
• 35:46
  like
• 35:47
  you're
• 35:47
  empowered.
• 35:47
  Your
• 35:48
  friend
• 35:48
  could
• 35:48
  say,
• 35:48
  look
• 35:49
  at
• 35:49
  this
• 35:49
  great
• 35:49
  study.
• 35:49
  Oh,
• 35:50
  my
• 35:50
  God.
• 35:50
  Wellbutrin
• 35:51
  is
• 35:51
  the
• 35:52
  best
• 35:52
  medication
• 35:52
  in
• 35:52
  the
• 35:53
  universe
• 35:53
  for
• 35:53
  depression.
• 35:54
  Yeah,
• 35:55
  Steve,
• 35:55
  two
• 35:55
  people
• 35:56
  were
• 35:56
  in
• 35:56
  that
• 35:56
  study.
• 35:57
  And
• 35:58
  what's
• 35:58
  the
• 35:58
  rate
• 35:59
  of
• 35:59
  spontaneous
• 36:00
  remission
• 36:00
  anyway?
• 36:01
  When
• 36:01
  did
• 36:01
  they
• 36:01
  measure
• 36:02
  the
• 36:02
  depressive
• 36:03
  scores?
• 36:04
  Was
• 36:04
  it
• 36:04
  four
• 36:04
  weeks
• 36:05
  later?
• 36:05
  10
• 36:05
  weeks
• 36:05
  later?
• 36:06
  18
• 36:06
  weeks
• 36:07
  later?
• 36:07
  What
• 36:08
  are
• 36:08
  the
• 36:08
  confounders?
• 36:09
  Did
• 36:09
  the
• 36:10
  patients
• 36:10
  know
• 36:10
  they
• 36:10
  were
• 36:10
  taking
• 36:11
  Wellbutrin?
• 36:12
  Was
• 36:12
  it
• 36:12
  blinded?
• 36:12
  In
• 36:12
  other
• 36:13
  words,
• 36:13
  they
• 36:13
  don't
• 36:13
  know
• 36:13
  what
• 36:13
  they're
• 36:14
  even
• 36:14
  taking
• 36:14
  versus
• 36:14
  they
• 36:15
  do,
• 36:15
  and
• 36:16
  it's
• 36:16
  unblinded.
• 36:17
  My
• 36:17
  friends,
• 36:17
  I
• 36:18
  really
• 36:18
  hope
• 36:18
  you
• 36:18
  had
• 36:18
  fun
• 36:19
  with
• 36:19
  my
• 36:19
  jelly
• 36:19
  bean
• 36:20
  case
• 36:20
  study.
• 36:21
  Remember,
• 36:21
  the
• 36:21
  numbers
• 36:22
  didn't
• 36:22
  matter.
• 36:23
  The
• 36:23
  point
• 36:23
  is
• 36:23
  that
• 36:23
  you
• 36:23
  have
• 36:24
  a
• 36:24
  conceptual
• 36:24
  appreciation
• 36:25
  for
• 36:25
  these
• 36:25
  research
• 36:27
  terms
• 36:28
  and
• 36:28
  you're
• 36:28
  more
• 36:28
  empowered
• 36:29
  to
• 36:29
  pick
• 36:29
  up
• 36:29
  an
• 36:29
  article
• 36:29
  and
• 36:30
  know
• 36:30
  what
• 36:30
  the
• 36:30
  hell
• 36:30
  it's
• 36:31
  talking
• 36:31
  about.
• 36:32
  You
• 36:32
  don't
• 36:32
  have
• 36:32
  to
• 36:32
  be
• 36:33
  a
• 36:33
  statistician
• 36:34
  to
• 36:34
  appreciate
• 36:34
  a
• 36:35
  good
• 36:35
  research
• 36:35
  article.
• 36:35
  There's
• 36:36
  so
• 36:36
  much
• 36:36
  number
• 36:37
  crunching
• 36:37
  in
• 36:37
  there
• 36:38
  that
• 36:38
  you
• 36:38
  don't
• 36:38
  even
• 36:38
  have
• 36:38
  the
• 36:39
  tools
• 36:39
  to
• 36:39
  ever
• 36:39
  use.
• 36:40
  It's
• 36:40
  good
• 36:41
  if
• 36:41
  you
• 36:41
  understand
• 36:41
  them.
• 36:42
  It's
• 36:42
  good
• 36:43
  if
• 36:43
  you're
• 36:43
  always
• 36:43
  curious
• 36:44
  and
• 36:44
  ask
• 36:44
  questions.
• 36:45
  But
• 36:45
  I
• 36:45
  hope
• 36:45
  I
• 36:45
  gave
• 36:46
  you
• 36:46
  a
• 36:46
  foundational
• 36:47
  understanding
• 36:48
  for
• 36:48
  how
• 36:48
  to
• 36:48
  read
• 36:48
  this
• 36:49
  stuff
• 36:49
  better
• 36:49
  and
• 36:49
  more
• 36:50
  critically.
• 36:51
  I
• 36:51
  really
• 36:51
  hope
• 36:52
  you
• 36:52
  guys
• 36:52
  feel
• 36:52
  empowered.
• 36:53
  Don't
• 36:54
  forget
• 36:54
  to
• 36:54
  become
• 36:54
  Patreon
• 36:55
  members.
• 36:55
  You
• 36:55
  can
• 36:55
  see
• 36:56
  a
• 36:56
  lot
• 36:56
  of
• 36:56
  this
• 36:56
  in
• 36:56
  slide
• 36:57
  deck
• 36:57
  form.
• 36:57
  Beautiful
• 36:58
  visuals
• 36:59
  that
• 36:59
  you
• 36:59
  have
• 36:59
  at
• 37:00
  your
• 37:00
  fingertips.
• 37:00
  It's
• 37:01
  so
• 37:01
  worth
• 37:01
  it.
• 37:01
  There's
• 37:01
  also
• 37:01
  a
• 37:02
  private
• 37:02
  podcast
• 37:03
  feed
• 37:03
  with
• 37:03
  private
• 37:04
  case
• 37:04
  studies
• 37:05
  where
• 37:05
  you
• 37:05
  can
• 37:05
  further
• 37:05
  empower
• 37:06
  your
• 37:06
  learning.
• 37:06
  And
• 37:07
  if
• 37:07
  you
• 37:07
  can't
• 37:07
  become
• 37:08
  a
• 37:08
  Patreon
• 37:08
  member,
• 37:08
  I
• 37:09
  understand.
• 37:10
  Go
• 37:10
  to
• 37:10
  Apple,
• 37:10
  please,
• 37:11
  and
• 37:11
  give
• 37:11
  us
• 37:11
  five
• 37:12
  stars.
• 37:12
  This
• 37:13
  podcast
• 37:13
  has
• 37:13
  gotten
• 37:14
  really
• 37:14
  big,
• 37:14
  really
• 37:14
  fast,
• 37:15
  especially
• 37:15
  among
• 37:16
  psych
• 37:16
  NP
• 37:16
  students.
• 37:17
  It's
• 37:17
  designed
• 37:18
  to
• 37:18
  get
• 37:18
  you
• 37:18
  all
• 37:19
  feeling
• 37:19
  comfortable
• 37:19
  and
• 37:19
  confident
• 37:20
  as
• 37:20
  you
• 37:20
  enter
• 37:20
  the
• 37:21
  field.
• 37:21
  Go
• 37:21
  to
• 37:21
  the
• 37:21
  Show
• 37:22
  Notes
• 37:22
  and
• 37:22
  email
• 37:23
  me
• 37:23
  suggestions
• 37:24
  about
• 37:24
  what
• 37:25
  future
• 37:25
  episodes
• 37:25
  you
• 37:26
  want
• 37:26
  me
• 37:26
  to
• 37:26
  cover
• 37:27
  that
• 37:27
  you're
• 37:27
  learning
• 37:27
  about.
• 37:28
  I
• 37:28
  love
• 37:28
  doing
• 37:28
  that.
• 37:29
  It
• 37:29
  brings
• 37:29
  me
• 37:29
  a
• 37:30
  lot
• 37:30
  of
• 37:30
  joy.
• 37:30
  I'm
• 37:31
  signing
• 37:31
  off
• 37:31
  now.
• 37:31
  It
• 37:32
  was
• 37:32
  a
• 37:32
  pleasure
• 37:32
  to
• 37:32
  be
• 37:33
  with
• 37:33
  all
• 37:33
  of
• 37:33
  you
• 37:33
  today.
• 37:34
  Be
• 37:34
  good,
• 37:35
  be
• 37:35
  safe.
• 37:36
  Much
• 37:36
  love.
• 37:37
  Bye.
• 37:37
  Bye.