The recommendation engine uses the following parameters for each question you've seen:
- A weighted average of all your previous review scores. Recent review scores receive heavier weights.
- The length of time that has elapsed since you last reviewed that question
- The number of times you've seen that question
Firecracker uses this information to calculate the risk that you will forget the concept associated with that question. This risk of forgetting increases over time and increases at a faster rate for questions that you've only seen a few times or that you've scored poorly on.
Once our algorithm calculates this risk of forgetting for all your questions - we create a stack of questions for you to review with the questions with the highest risk of forgetting at the top.
For a given user, this is what their stack might look like.
When you jump into questions on Firecracker, you start at the right end of the above distribution and "burn down the stack" working your way from right to left. When you do questions, they get pushed back to low priority (because you just saw them). As time elapses, questions increase in priority and move from low priority to high priority. The rate at which priorities increase is based on the inputs to our algorithm described earlier.
The priority of a given question decreases immediately after review. Priorities decrease more with higher scores and with more reviews.
Let's contrast this with a user who shows no improvement between reviews and scores a 1 with every review.
Here we can see that even without improvement between reviews, the priority of the given question decreases as the number of reviews increases.
Let's look at one more example.
Here we can see how the priorities change as your score fluctuates. By using a weighted average of all your previous recalls in our priority calculation we hone in on the signal (your true understanding of a given question) and reduce the noise of score changes between reviews.
The intervals above are not fixed. You have the option to filter questions at any level of our content tree and do the questions that you're at the highest risk of forgetting. The intervals above are just to give you a sense of how long it may be until you see a question again given a certain review history.
Data Scientist, Firecracker