Warm-up

To prepare for the main portion of today’s activity, I’d like to create 3 lists (informed by today’s readings):

  1. Common misinterpretations of p-values
  2. Advantages of p-values
  3. Drawbacks of p-values

\(~\)

Activity

Each group will lead a brief discussion (5-10 minutes) on one of the scenarios described in the section below. To prepare, your group should first:

  1. Thoroughly read and understand your assigned scenario.
  2. Identify a set of 2 or 3 key ideas that you think the class should extract from your scenario.
  3. Prepare several insightful questions to guide nuanced discussion of the scenario. You should avoid directly telling the class your conclusions regarding the scenario, and your key ideas that you noted in #2.

\(~\)

Scenarios

The following scenarios each describe an application of hypothesis testing and p-values as well as a short statistical conclusions. However, some of these conclusions are insufficient or even incorrect.

The goal of each discussion is to classify the statistical conclusion (bold text) as either:

  • Correct/appropriate
  • Incorrect/misleading

I encourage you to rely upon the information from Tuesday’s class activity and today’s readings when analyzing each scenario.

Scenarios:

  1. Prof. Miller and Steph Curry compete in a 3-pt shooting contest. The null hypothesis states both will make the same proportion of their shots. Prof. Miller makes 3 of 5 shots, while Steph makes 5 of 5, leading to a p-value of 0.17. Prof. Miller concludes that the contest provides statistical evidence that he and Steph Curry are equally good 3-pt shooters.
  2. A clinical trial randomly assigned 6000 participants suffering from heart burn to receive either Nexium (a new medication) or Prilosec (a successful drug developed in the 1980s). In the study, 88% of the group receiving Prilosec reported reduced symptoms, while 90% of the group receiving Nexium reported reduced symptoms. The corresponding p-value is 0.008. The drug company believes the new medication offers a tremendous improvement over the existing standard of care due to the study’s very low p-value.
  3. An observational study collected hemagglutination inhibition assays from recent recipients of the influenza vaccine to evaluate whether there was any relationship between the success of the vaccine and the number of hours the recipient had slept the night prior to inoculation. These assays showed that the vaccine was less successful for participants who slept fewer than 7-hours the night before compared to participants who slept 7 or more hours the night before. However, the corresponding p-value was 0.11. The researchers concluded there was insufficient evidence that sleep impacted the success of the vaccine, but it is possible that there is a small effect.
  4. In the 1970s, data on graduate applications to six different colleges at UC-Berkeley showed that males were 1.8 times more likely than females to be accepted into graduate school. A statistical test using the null hypothesis that males and females were equally likely to be accepted into graduate school at UC-Berkeley produced a statistically significant p-value. This provides statistical proof that women were discriminated against in the graduate school admissions process at UC-Berkeley.
  5. In a study conducted in the 1980s, participants with a high risk of heart attack were randomly assigned to receive either the drug clofibrate or a placebo pill. 708 of 1103 in the group receiving clofibrate adhered to their treatment (took the drug as prescribed), and these “adherers” had a death rate of 15% during the study period. In contrast, the 357 participants assigned to receive clofibrate who did not adhere to their treatment, or “non-adherers”, had a death rate of 25%, or almost double. Additionally, the 2789 individuals assigned to the placebo group had a death rate of 21%. The p-value comparing the 15% death rate among “adherers” with the 25% death rate among “non-adherers” was 0.0001, and the p-value comparing the 15% death rate among “adherers” with the 21% death rate in the placebo group was 0.0002. The researchers concluded that clofibrate significantly reduced the risk of death among individuals with a high risk of heart attack