Statistical Inference using Models

Robin Donatello

2023-10-04

Refresher on the Normal Distribution

Describes Variability

The normal distribution is used to describe the variability associated with sample statistics which are taken from either repeated samples or repeated experiments. The normal distribution is quite powerful in that it describes the variability of many different statistics such as the sample mean and sample proportions.

Sampling Distribution Theory

Turn to a neighbor and discuss your answers to the following questions. We will share out to the class after each one.

• 👥 What is a Sampling Distribution?
• 👥 What does the Central Limit Theorem state?
• 👥 What assumptions are needed for the normal approximation (model) to provide valid inference?

The Normal Distribution

Board work following IMS 13.2

• Distributional Notation
• Standardizing with Z-scores
• Normal Probabilies (qnorm, pnorm)

Quantifying variability of a statistic

Follows IMS 13.3

68-95-99.7 rule

Example

SAT scores closely follow \(N\sim(1500, 90,000)\).

About what percent of test takers score 900 to 2100? What percent score between 1500 and 2100?

Example

SAT scores closely follow \(N\sim(1500, 90,000)\). So \(\sigma = 300\).

About what percent of test takers score 900 to 2100? What percent score between 1500 and 2100?

• 900 has a z-score of -2, 1500 has a z-score of 0, 2100 has a z-score of 2

pnorm(2100, mean=1500, sd = 300) - pnorm(900, mean=1500, sd = 300)

[1] 0.9544997

Standard Error

Definition: Standard Deviation (SD)

Variability of the data values (\(x\))

Definition: Standard Error (SE)

Variability of the sample statistic (e.g. \(\bar{x}\) or \(\hat{p}\))

Margin of Error

Definition: Margin of Error (MOE)

The margin of error describes how far away observations are from their mean.

Often approximated as \(2 * SE\)

• 95% of the observations are within one margin of error of the mean.
• If the spread of the observations goes from some lower bound to some upper bound, a rough approximation of the \(SE\) is to divide the range by 4.
  • If you notice the sample proportions go from 0.1 to 0.4, the SE can be approximated to be 0.075.

Case study: Stents

This example and data comes from IMS 13.6

Show the code

stent30 <- openintro::stent30

Observed data

Consider an experiment that examined whether implanting a stent in the brain of a patient at risk for a stroke helps reduce the risk of a stroke. The results from the first 30 days of this study are summarized in the following table.

Show the code

table(stent30$group, stent30$outcome) |> addmargins()

           
            no event stroke Sum
  control        214     13 227
  treatment      191     33 224
  Sum            405     46 451

Show the code

table(stent30$group, stent30$outcome) |>  prop.table(margin=1) |>round(digits=2)

           
            no event stroke
  control       0.94   0.06
  treatment     0.85   0.15

These results are surprising! The point estimate suggests that patients who received stents may have a higher risk of stroke: \(p_{trmt}−p_{ctrl}=0.090\).

Point estimate vs Interval estimate

The point estimate for the difference in proportions \(p_{trmt}−p_{ctrl}=0.090\) is a single point estimate, based on this single sample.

A point estimate is our best guess for the value of the parameter, so it makes sense to build the confidence interval around that value.

Constructing a 95% confidence interval (CI)

When the sampling distribution of a point estimate can reasonably be modeled as having a normal distribution, the point estimate we observe will be within 1.96 standard errors of the true value of interest about 95% of the time. Thus, a 95% confidence interval for such a point estimate can be constructed:

\[\mbox{point estimate} \pm 1.96 × SE\]

We can be 95% confident this interval captures the true value.

Construct a 95% CI for the stent example

The conditions necessary to ensure the point estimate \(p_{trmt}−p_{ctrl}\) is nearly normal have been verified for you, and the estimate’s standard error is \(SE = 0.028\).

• Construct a 95% confidence interval for the change in 30-day stroke rates from usage of the stent.
• Interpret this interval in context of the problem.

\[0.090 \pm 1.96×0.028 = (0.035,0.145)\]

We are 95% confident that implanting a stent in a stroke patient’s brain increased the risk of stroke within 30 days by a rate of 0.035 to 0.145.

Important note

⚠️ it’s incorrect to say that we can be 95% confident that the true value is inside the mean.

Figure 13.11: Twenty-five samples of size n=300 were collected from a population with p=0.30. For each sample, a confidence interval was created to try to capture the true proportion p. However, 1 of these 25 intervals did not capture p=0.30.

• This is one of the most common errors: while it might be useful to think of it as a probability, the confidence level only quantifies how plausible it is that the parameter is in the interval.
• Our intervals say nothing about the confidence of capturing individual observations, a proportion of the observations, or about capturing point estimates.
• Confidence intervals provide an interval estimate for and attempt to capture population parameters.

Hypothesis test

First draft

Let’s setup a hypothesis to test if stents work to reduce the risk of a stroke.

\(H_{0}\): Stents don’t work

\(H_{A}\): Stents reduce the risk of a stroke

Hypothesis test

Revision 1:

Making it a statement about two groups

\(H_{0}\): Patients who have a stent have the same risk of a stroke as patients who don’t have a stent

\(H_{A}\): Patients who have a stent have lower risk of a stroke as patients who don’t have a stent

Hypothesis test

Revision 2:

Make it a statement using summary statistics and removing the directionality of the hypothesis

 

\(H_{0}\): The proportion of patients with a stent who have a stroke is the same as the proportion of patients without a stent who have a stroke.

\(H_{A}\): The proportion of patients with a stent who have a stroke is different than the proportion of patients without a stent who have a stroke.

Hypothesis test

Revision 3:

Writing it in symbols

Let \(p_{trmt}\) be the proportion of patients with a stent who have a stroke, and \(p_{ctrl}\) be the proportion of patients without a stent who have a stroke

\(H_{0}: p_{trmt} = p_{ctrl}\)

\(H_{A}: p_{trmt} \neq p_{ctrl}\)

Hypothesis test

Revision 3.5:

Rewriting as a difference in parameters

Let \(p_{1}\) be the proportion of patients with a stent who have a stroke, and \(p_{ctrl}\) be the proportion of patients without a stent who have a stroke

\(H_{0}: p_{trmt} - p_{ctrl} = 0\)

\(H_{A}: p_{trmt} - p_{ctrl} \neq 0\)

Calculating a test statistic

\[ Z = \frac{\mbox{point estimate - null value}}{SE}\]

\[ Z = \frac{(p_{trmt} - p_{ctrl}) - 0}{SE_{p_{trmt} - p_{ctrl}}} = \frac{.090}{.028} = 3.21 \]

\[ P(Z > 3.2) = .00068 \qquad \mbox{ (the p-value)}\]

If the true difference in proportions was 0, then the probability of observing a difference of 0.09 due to random chance is 0.00068.

https://xkcd.com/1478/

Using Confidence Intervals to test a hypothesis

We are 95% confident that implanting a stent in a stroke patient’s brain increased the risk of stroke within 30 days by a rate of 0.035 to 0.145.

Since the interval does not contain the null hypothesized value of 0 (is completely above 0), it means the data provide convincing evidence that the stent used in the study changed the risk of stroke within 30 days