As is the nature of any observational study, correlation doesn't imply causation. In our Firecracker Step 1 Performance post, we showed that active Firecrackers significantly outperformed non-Firecrackers. You may wonder what is the root cause of why active Firecrackers had an average Step 1 score 14 points higher than non-Firecrackers.
Was it because medical students who actively used Firecracker were predisposed to studying more than non-Firecrackers? Or maybe because better performing students self select to use Firecracker?
The answer to the first question is no, Firecracker users reported spending the same amount of time studying for the Step 1 exam as non-Firecrackers.
Here we see that active Firecracker users (users who actively used Firecracker for over 3 months) reported spending about the same number of hours a week studying for the Step 1 exam during dedicated prep time as non-Firecrackers $latex (p=0.461)$. In the above graph the vertical black lines are the standard deviations; the distributions have about the same center and spread.
What about the second question, do better performing medical students self select to use Firecracker? To explore this question, we looked at how Firecrackers performed compared to non-Firecrackers, given the same MCAT score. Current research says that the MCAT is a moderate predictor for Step 1 performance. Our data was right in line with published research:
The MCAT is arguably a good proxy for study habits and ability. So how did Firecrackers compare to non-Firecrackers given the same MCAT score? Do Firecrackers score better because they were already better students to begin with?
The answer to this question is also no.
This graph shows that Firecracker's outperformed non-Firecrackers even given the same MCAT score almost across the board. Firecrackers in the above graph used Firecracker for 3 or more months.
Here's how it looks if we plot the lines of best fit without the points for each group:
The difference in average Step 1 scores between the two groups is very apparent when we plot the lines of best fit. A paired t-test shows that the average Firecracker score is significantly higher (8.88 points on average), even when you pair Firecrackers and non-Firecrackers by their MCAT score $latex (p<0.001)$.
The evidence of Firecracker's benefit is extremely strong. Firecracker helps you unleash your potential. Sign up for a free trial today!