Following last week, this is the last part of our 3-part blog post on Firecracker question creation. It details question curation through big data.
Big Data Curation - Phase III
Every month Firecracker students answer over 8M review questions and 150,000 patient cases, forming a vast data ecosystem. Numbers like these keep our questions under constant curative pressures and student scrutiny. We use data from this student-question ecosystem to recommend daily questions, flag questions for improvement, and generate analytics, such as question item response curves. Below is an example of item response curves from two of our questions. The graphs show the probability of successfully answering the question in relationship to knowledge mastery. The question on the left is relatively easy and doesn’t effectively discriminate mastery, which means we’ll pull it from our bank to fix it. The question on the right is more dependant upon knowledge mastery; it is also more difficult.
Next time you answer a Firecracker question, you can appreciate that you’re a part of building a world-class learning system where every detail counts. Whether part of the design, development, or data phase, each step of the process plays a vital role in Firecracker question creation.