You can access a significantly larger sample of the platform's content for free by logging in with your Gmail account. Sign in now to explore.

What is a two-sample t-test?

A two-sample t-test tests whether there is a statistically significant difference between the means of two populations. Assume two samples, \(X_t, X_c\); for simplicity, let us call them control and treatment samples. We can define the difference of their means as: \(\Delta = \bar{X}_t - \bar{X_c}\). Then, the two-sample t-test can be written (p.186, Kohavi, Tang, and Xu (2020)) as:

\[ T = \frac{\Delta}{\sqrt{Var(\Delta)}} \]

Let’s expand on the previous relationship:

\[ T = \frac{ \bar{X}_t - \bar{X_c}}{\sqrt{Var(\bar{X}_t) + Var(\bar{X}_c) + 2COV(\bar{X}_t, \bar{X}_c)}} \] Since the two samples are independent, the covariance of the means is zero. The variance of the mean is:

\[ Var(X_t) = Var(\frac{X_1+X_2+X_{n_t}}{n_t}) = \frac{1}{n_t^2} \bigg[ Var(X_1) + Var(X_2) + ... \bigg] = \frac{n_t}{n_t^2} s_t^2 = \frac{s_t^2}{n_t} \]

where we assumed that all samples are independent and come from the same distribution, with empirical variance \(s_t^2\). Similarly, we can obtain the same result for the control group. Hence, our t-test can be written as:

\[ T = \frac{ \bar{X}_t - \bar{X_c}}{\sqrt{\frac{s_t^2}{n_t}+ \frac{s_c^2}{n_c}}} \]

We used the **Welch’s t-test** for this definition, which assumes unequal variances between samples. For a thorough discussion and comparison with Student’s t-test, you can check https://en.wikipedia.org/wiki/Welch%27s_t-test

Hypothesis testing

- Tests for normality Easy (Hypothesis testing, Normality)
- Confidence intervals that overlap Medium (Confidence interval, Hypothesis testing)
- Confidence interval definition Easy (Confidence interval, Hypothesis testing)
- P-value definition Easy (P-value, Hypothesis testing)
- An intuitive way to write power Easy (Power, Hypothesis testing)