Answer buttons and how to use SRS

Spaced repetition is very powerful compared to massed repetition, which is why software utilising the spacing effect is growing ever more popular. I sometimes feel like an SRS missionary, writing articles about why everybody should start using SRS and which program I prefer myself (if you don’t know what SRS is, please read these articles before reading this one). This is all very good, but it leaves many questions unanswered. For instance, how should spaced repetition software be used?

In this article, I will discuss how to review vocabulary using SRS, including how to use the various answer buttons and some other functions commonly available. I will use Anki for all examples, but other good programs will of course have similar or indeed identical features. The algorithms used to calculate the spacing between repetitions might not be identical for all programs, but they are similar. What I discuss in this article is useful beyond specific programs, so don’t put too much emphasis on the exact details.

This article isn’t meant to be a guide to what is correct, it’s rather meant to be a discussion with some personal examples and motivations to why I’m doing what I do. If you have other ideas or don’t agree, please leave a comment!

More about spaced repetition on Hacking Chinese

[add_posts tag=spaced-repetition-software show=100]

Answer buttons

Most programs make use of four buttons, typically labelled:

  1. “again”
  2. “hard”
  3. “easy”
  4. “very easy”

Without any kind of definition, these answers are completely arbitrary. Do you hit “again” even if you fail a minor point in an otherwise complex and completely correct answer? Do you hit “easy” or “very easy” when you encounter a word you feel that you know quite well? What’s the difference between “hard” and “easy”? Of course, there are no entirely correct answers here, but I will give you my own ideas. Before that, though, let’s discuss briefly why this matters at all.

The short answer is that the choice matters greatly in the long run. Even if the algorithms will more or less automatically adjust the difficulty and the intervals for you, make sure you’re not being too harsh. A quick calculation shows that if you could use “very easy” for 10% of the cards instead of “easy”, you will save many hours over a year of reviewing (how much you save depends on how much time you spend reviewing of course). This time could have been spent doing something more useful. However, the opposite is true as well. If you select “very easy” for cards you don’t know that well, you will end up failing them, thus wasting more time than you would have saved, so don’t be too lenient either.

What does all this mean, then? It means that you should assess your answers as accurately as you can. This is individual to a certain degree and requires a bit of practice. If you use a decent program, you should be able to get detailed graphs showing you how much you click the various alternatives. Here’s what my graph looks like:

Avoid perfectionism: 90% is good enough

Note that most algorithms are set to give a 10% failure rate, which is almost exactly what I have if we ignore new cards. This is because a higher rate would mean that you’re wasting time studying too many words you already know, so don’t make the mistake of thinking that a higher number is necessarily better. Perfectionism is a waste of time. If you fail 10% of the cards, you’re doing it right!

Now, most of this is self-adjusting. If you enter a bunch of difficult words, you will hit “hard” quite a lot (check my new cards, for instance). However, since the intervals of these cards will be shorter because of that, you will also learn them better, which means the interval will be adjusted so you think they are “easy” next time you see them. This is the whole point.

The answer buttons and how I use them

Here’s what I usually do (including approximate percentage for mature cards):

  1. “again” (~10%) – I really don’t know the word, or I have forgotten a crucial part of it (such as forgetting the pronunciation of a character or only having a vague idea of what it means). However, the nature of the mistake is important. If it’s something I think it’s likely I will remember next time, I might choose “hard” rather than “again” for relatively new cards (interval less than one month). The reasoning behind this is that it will just clog the review queue if I reset the interval by hitting “again”. If I make the same mistake next time, I will choose “again”. I’m convinced this saves quite a lot of time in the long run.
  2. “hard” (~20%)  – I select this answer if I can come up with the right answer, but only after considerable mental effort, such as retrieving old mnemonics or comparing with other words I know. Also, see the discussion above about using “hard” instead of “again” for mistakes that are unlikely to occur twice. I never use this for words that have longer intervals (months or years), because you simply won’t see them again for a long while even if you choose “hard”.
  3. “easy” (~65%) – This option is for cards that I recall almost instantly or feel very familiar with. These are words that wouldn’t stop me when I’m reading and don’t require much mental effort to recall. I estimate that I spend less than three seconds each on these cards; if more, they count as “hard” rather than “easy”.
  4. “very easy” (~5%) – I only use this answer when I’m completely sure that I won’t forget this word within the given interval. For new cards, it means that I have learnt it properly and really think that the current interval is much too short and is a bit wasteful. For old cards, I answer “very easy” for anything I’m sure I’d know years from now even without reviewing. Instant or intimate knowledge of the word is what generates “very easy” for me.

Look at the interval

I think it’s a mistake to be 100% consistent when reviewing vocabulary if you ignore the intervals and only look at the buttons themselves (“again”, “hard” and so on). There is a huge difference between “hard (4 days)” and “hard (4 years)”. The reason is that I can easily pick the first one if I make an easily correctable mistake on a new word because I know I will be checked again in a few days anyway. If I fail again because the correction wasn’t as easy as I imagined, I will know that and can hit “again” if I feel the interval has grown too large. If the interval is four years, however, I won’t see the card ever again (in practical terms) even if I choose “hard”, which means that I can’t be sure I’ve corrected the mistake. In this case, I would hit “again” and suck it up.

Other useful functions

There are many other features you should use to increase efficiency further:

  • Leeches are cards that drain time because you fail them over and over. Anki suspends these cards automatically after a given level. I suggest lowering this level (the default number is 16), because if you fail a card that many times, you really need to do something else. As I’ve argued elsewhere, spaced repetition is meant to reinforce what you already know, not force new knowledge through cramming. Please also refer to my article about killing leeches.
  • Suspending cards simply means that you manually remove them from the review queue and that they won’t appear until you activate them again. I do this for cards I suspect have serious problems or when I think that reviewing them is useless, perhaps because I’ve given too little information and might confuse the word with other words.
  • Marking is something I’ve started doing a lot recently, which basically means putting a tag on the word that you can easily find later. I separate active vocabulary learning from reviewing, both in time and space. I review using my phone, but I always use my computer to kill leeches, delve deeper into difficult words or sort out synonym issues. Therefore, while reviewing, I simply mark any card I might need to work more with and keep reviewing. Then, I go through these words later, using dictionaries, asking native speakers, correcting mistakes.

Some questions for the reader

  • Are your answering criteria different from mine?
  • What does your graphs look like for new/young/mature cards?
  • Do you use leeches, suspending and marking as I do?
  • Do you use any other cool tricks I haven’t mentioned?
  • Any other tips, tricks or ideas you’d like to share?

Dealing with tricky vocabulary: Killing leeches

A leech is something that drains time from your studying and will keep doing so until you do something about it. The most essential example would be a piece of vocabulary, let’s say a character, that just refuses to stick in your mind. Even though it has appeared numerous times in your spaced repetition program, you just can’t recall the correct answer, perhaps because it’s very difficult, perhaps for no readily apparent reason whatsoever. What makes a particular word turn into a leech is very difficult to say, simply because it seems to be wildly different between different learners. Every learner I’ve talked to knows about leeches, although they might not use that word. This article is about how to deal with leeches.

Other articles on Hacking Chinese related to spaced repetition:

More about spaced repetition on Hacking Chinese

[add_posts tag=spaced-repetition-software show=100]

Leeches are harmless on their own, but deadly in packs

Fortunately, we don’t need to know exactly why a leech appears, we simply need to know how to recognise it and how to do deal with it. The reason for wanting to kill leeches should be obvious: Not only do they drain time since they require much more reviewing than other flashcards, they also create a sense of frustration when you always fail a certain flashcard. Speaking of flashcards, I’m more or less assuming that you are using spaced repetition software (Anki, for instance), but you will be able to benefit from this article even if you’re old-school and still rely on pen and paper.

The problem with leeches is treacherous. One leech might perhaps drain a couple of minutes study time, which is insignificant. However, if you let the number grow, the time that get sucked in by these bastards will quickly accumulate and start becoming a real problem. This is completely unnecessary.

How to spot leeches

Spotting leeches is in one regard very easy, but in another quite difficult. It’s easy because you often get a feeling that you forget a word very often, which means that it’s probably a leech. If you’re using Anki, there is a specific function that will identify leeches for you, which means that you don’t really have to think too much about it yourself.

The default setting for the definition of a leech is that the card has been failed 16 times, which is ridiculously high in my opinion; I’ve lowered the number to 8, but possibly you could lower it even more, down to 5 or so. When this number is reached, the card is suspended and you will have to manually do something about it. This is precisely what we want.

However, noticing leeches might sometimes be more difficult than that. What if there are three characters you often confuse or you’re simply too lenient when you correct yourself, so you fool yourself into believing that it’s not a problem? Being aware of leeches and having an active strategy to combat them is essential and will probably solve most problems related to identification.

Kill the leeches!

So, you have identified a leech, but what do you do with it? It depends on what kind of leech it is. Characters that refuse to stick are usually the result of bad mnemonics. Go back and look at how the character is composed and see how you can create a new story or picture that will allow you to remember it. If it’s a multi-character word, the same is true: you need a new approach and a more vivid connection to allow you to remember the piece of vocabulary in question.

Confusing similar characters is a problem which might be a little bit harder to deal with and requires more effort. However, as I have argued elsewhere, knowing lots of individual characters is essential if you want to learn words really fast. If you have a series of characters you can’t distinguish, you have a flawed tool kit and your learning in general will also suffer. What I do in these situations is to compose a list of similar characters and then go through them systematically. Using a dictionary with a visual etymology tree is essential, because that way you can actually see similar characters right next to each other (see below).

Let’s take the characters 氐, 抵, 诋, 底 and 低 as examples. They are all pronounced more or less the same (only 低 differs in that it’s first tone). I found it very hard to remember which one of these was which, so I looked at this page over at Look at the right and you will see all the different characters that contains 氐. Going through all these and writing them down (preferably by hand) is a good way of killing related. I usually gather these collections on a single sheet of paper and then put that somewhere where I usually have spare time, like next to my bed, close to the table or in the bathroom.

Being lazy means more work, not less

Dealing with leeches is quite easy most of the time. Laziness is usually a bigger problem than a lack of ideas about or insight into how to solve the problem. What I do is that I mark or label potential leeches in some way and then ignore them for a while until I feel like I have the time and energy to deal with them. Then I spend an hour or so looking at them and weeding out any problems that might be lurking there. That way, I kill lots of leeches at once. Finish what you started before you go leech hunting.

The reason I say it’s stupid to be lazy is that you don’t save time by ignoring leeches, rather the opposite is true. Lowering the threshold of leeches in Anki will mean that you spend more time looking at character parts and killing leeches. However, this is not a waste of time, because it means that you actually learn something instead of just failing over and over. Not only is it good for your Chinese, it’s good for your self-confidence as well.

Giving up is (sometimes) an option

Even though it’s necessary to learn most words at some point and there probably is a reason why you have entered a specific word into you vocabulary list, it’s important to realise that you don’t always have to fight. Let’s say you have a word you’ve tried to learn a number of times but which simply just refuses to stick, regardless of mnemonics or any other strategy you try. In such a case, I ask myself: Do I really need this word? Is it commonly used? Can I do without it? If the answer is yes, just delete the card and get rid of the leech that way. Spend the time thus saved learning something else.

In any case, keep your eyes peeled for leeches! When you feel that you have a couple, take decisive action to get rid of them, once and for all. A few leeches are irritating, but a throng of them will kill you. Don’t let them! Take control and make you studying even more effective.

Questions to the reader

  • Were you aware of the concept of leeches before reading this article?
  • How do you handle the problem with leeches?
  • What’s your worst leech ever (you can check most reps in Anki, for instance)?
  • Do you have tactics for leech hunting than presented here?

(PS my worst leech ever is 昆, which has 62 reps, proving that I don’t always follow my own advice)

Spaced repetition isn’t rote learning

Anyone who has spent some time thinking about and experimenting with how to best learn new words will realise that rote learning isn’t a great idea. I remember teachers in elementary school recommending me to use the word list in the book, cover one column with a piece of paper, look at the other and see if I remembered the words thus hidden by the paper. If not, peek and try again. This is horribly ineffective and any other way is likely to be better.

But isn’t that what spaced repetition is all about?

I have said that spaced repetition software is the best thing since sliced bread, but isn’t flashcards and spaced repetition just another form of the rote learning from elementary school described above? Sure, we use a fancy computer program now, but what makes this the best thing ever (at least since sliced bread was patented), while the scenario in the previous paragraph should be avoided at all costs? I think there are two major differences. Before that, though, I’d like to link to some more basic articles about spaced repetition here on Hacking Chinese:

More about spaced repetition on Hacking Chinese

[add_posts tag=spaced-repetition-software show=100]

 Spaced repetition is about repetition, not learning

The main function of Anki or other similar programs is repetition. They allow you to maintain words that you have previously learnt and they allow you to do so without wasting too much time. I advice against using SRS to learn the words in the first place, so don’t just import a hundred new words and go through them until you know them, because that would indeed be repeating the same mistake as we’ve looked at above. Instead you should spend some extra time and use smart and carefully thought-out methods to learn the words as efficiently as you can. Looking at character components and individual characters to learn words very fast is a good way to do it. I have also written an article about how to learn Chinese characters as a beginner.

The point is that you use one method to memorise the words when you first encounter them, and then you use spaced repetition software to help you remember them afterwards. The program will remind you roughly when you are about to forget a word or a character; ideally you review the flashcard the second you would have forgotten it if it hadn’t popped up.

Don’t go on tilt when reviewing flashcards

If you’re serious about language learning, you will inadvertently end up with quite a number of flashcards, and sometimes you just don’t have time to keep the queue down. Cards pile up. When you finally sit down to tackle the problem, it’s easy to go on tilt and simply try to hack one’s way through the heap of cards, and when you realise that you’ve forgotten a certain card, you simply hit “again” and wait for it to turn up again. You’re on tilt. The likelihood is that you still won’t know the word, or that you will have forgotten it again a few minutes later. Don’t do this, this is falling into the same pit as we’ve talked about before.

When you answer a card incorrectly, try to be honest. Sometimes, you might miss a word because you simply made a mental blunder, misread the card or something. Do you honestly believe that you actually know the card in question? If yes, then by all means, hit “again” (or even “hard” if you really know the word and just misread something) and keep going, you should be fine. But if your answer is wavering or a bit uncertain, you should stop and examine the flashcard more carefully. If you don’t want to interrupt the flow of reviewing cards, simply suspend or mark the card and look at it after you’re done with the reviewing. Anki has a very useful system to detect leeches, use it! In fact, this kind of feature is a warning system. If the alarm goes off, it means that you’re doing something horribly wrong and you need to change strategy. You can read more about leeches and how to kill them in an upcoming article..


My conclusion is that spaced repetition, if done correctly, isn’t rote learning. Rote learning means that you associate A with B simply by repeatedly seeing or hearing the connection until you can remember it. Spaced repetition is about first learning something and then using software to review and enhance the connections you’ve already made. It’s simply a way to refresh what you already know, to enhance pathways and links between different parts of the web of Chinese in your brain. Spaced repetition software can be used to maintain and strengthen these links, but it shouldn’t be used either for building them in the first place or repairing them if they fall apart. Use your toolkit and mnemonics for those situations instead!