Effective progress or experience points? The double-edged sword of measuring your Chinese learning

Learning Chinese takes a long time, and it’s hard to stay motivated if you can’t tell whether you’re improving.

In the beginning, progress is easy to see. Every new word, character and phrase feels useful. Later, you might still be learning a great deal, but the difference is harder to notice.

Adding twenty words to a vocabulary of two hundred feels huge; adding twenty words to a vocabulary of five thousand barely registers.

Tune in to the Hacking Chinese Podcast to listen to the related episode (#301).
Available on Apple Podcasts, Spotify, YouTube and many other platforms!

Measuring progress is a great way to notice what you can’t feel

This is where measuring progress can help. A good benchmark can show you that you are still moving forward, even when it doesn’t feel that way. It can help you keep moving across the intermediate plateau, where you feel that no matter how much you’re studying, you just don’t seem to improve.

How to get past the intermediate Chinese learning plateau

But measurable progress is a double-edged sword.

What you measure shapes what you do.

• If you count flashcard reviews, you’ll be tempted to do more flashcard reviews.
• If you track streaks, you’ll protect the streak.
• If you focus on test scores, you’ll study for the test.

These things aren’t necessarily bad, but they are not the same as becoming better at using Chinese.

In this article, I’ll argue that measurable progress is both powerful and dangerous. Used wisely, it can motivate you and make your learning more effective. Used carelessly, it can trap you in activities that look productive while pulling you away from real Chinese.

Measurable progress is awesome…

Before we turn to Chinese in particular, I’d like to share with you my personal theory of measurable progress.

It started with this question: Why is it that so many people like going to the gym these days?

I like body-weight exercises myself (I practise gymnastics), and these can be done for free at home, so why pay money and spend extra time going to the gym?

I realise that the answer to this question is complex and involves many factors, but I think that measurable progress is a key component.

In a gym, each movement can be measured very precisely. We repeat more or less the same routines every time, and therefore we can see that, yes, we have added so and so many reps or so and so many kilograms since last month.

This motivates us to keep going, even though we might not feel or see the difference in our everyday lives.

The fact that the progress is measurable makes us move forward.

You can measure progress with body-weight exercises, too, of course, but it’s a lot harder and less precise.

…but it has some serious drawbacks

But let’s return to language learning. The gym of language learning is spaced repetition software (SRS for short) and other fairly mechanical ways of practising that give us detailed feedback on what we do.

Why is it that some people use SRS more than they actually read or listen to Chinese directly? Why is it that some regard SRS as a comprehensive language learning strategy, when in fact it’s just a tool among many?

Well, because it’s very measurable.

Spaced repetition software: What it is and how to use it to learn Chinese

Measurable progress supercharged by gamification

In recent years, heavily gamified apps such as Duolingo have taken the idea of measurable progress even further. Streaks, XP, levels and shiny badges create a constant stream of small rewards that feel like proof that you are moving forward.

The problem is that most of these numbers measure how much you interact with the app, not how much Chinese you engage with. In an ideal world, staying engaged in the app would mean staying engaged with the language, but if you look at how many exercises are designed, it should be clear that the main goal is to keep you tapping, not to prepare you for real communication.

In an interview with The Verge, the CEO of Duolingo even said that he prioritises engagement over language learning. The reason he gave is that you can’t teach someone who isn’t there.

This sounds reasonable, but it’s wrong. It assumes that the alternative is nothing and that Duolingo is the only way to learn Mandarin. What if the student puts on a podcast or reads a book instead?

The goal is not to use Duolingo; it’s to learn Mandarin. For you, I mean, for the CEO of Duolingo, the goal is that you keep using the app.

Used wisely, there is nothing wrong with gamification. It can help you build habits and make it easier to show up every day.  The danger appears when streaks and XP become your main way of judging progress.

You can have a perfect streak and impressive league rankings while still struggling with basic conversations in Chinese.

Are you measuring what counts?

If you care about long-term progress, you need to make sure that what you’re measuring is time spent meaningfully engaged with Chinese, not just time spent keeping a cartoon owl happy.

Measurable progress is alluring because it offers us proof of progress. We can show that we are learning, and we can share with others what we have accomplished. I have at least learnt 20 new words today. I know 100 more characters than last month.

The problem is that these programs and apps can’t supplant reading and listening. They won’t teach you how to communicate in Chinese.

They are useful tools that can help us boost vocabulary and reinforce certain other areas, but they are not substitutes for actually using the language, either in written or spoken form.

I wrote a bit more about counting what counts in this article: The importance of counting what counts when learning Chinese

The importance of counting what counts when learning Chinese

Don’t expect to do well in a multi-port event only by going to the gym

If you want to win a multi-sport event, you need to do more than just go to the gym. Still, most professional athletes use a gym, and I think that language learning is similar in many ways.

For example, spaced repetition software has a lot to offer language learners, but if SRS is your main (or perhaps only) window to the language, you’re doing something seriously wrong.

When a measure becomes a target, it stops being a good measure

All this points to a more general problem with measuring things. As soon as you start tracking a number, you also start changing how you behave.

If you count flashcard reviews, you’ll naturally try to increase the number of flashcard reviews. If you track hours spent in an app, you’ll look for ways to keep the app open.

This can be helpful when the thing you measure is very close to what you actually want, for example, pages of Chinese you’ve read. It’s much less helpful when the number you’re chasing is only loosely related to your real goal.

Goodhart’ law for Chinese learners

Economists sometimes call this Goodhart’s law: when a measure becomes a target, it stops being a good measure.

A test score can be a rough indication of your Chinese ability, but once passing the test becomes the main goal, you start learning to pass the test rather than to use the language.

The same thing happens with SRS statistics, streaks in an app or any other number you put at the centre of your learning. The more you focus on pushing that number up, the less useful it becomes as a measurement.

Using the HSK as a roadmap to learning Chinese

Not everything that can be counted counts, and not everything that counts can be counted

Another issue is that you can never measure everything that matters. Numbers are very good at capturing simple, countable things, such as how many cards you reviewed or how many days in a row you’ve opened an app.

They’re very bad at capturing things like how relaxed you feel when you listen to a podcast, how willing you are to start a conversation, or how likely you are to keep studying for many years.

These unmeasured things are a big part of what really matters. If you rely too much on what you can count, your learning can slowly drift away from your real goals simply because the important but invisible things aren’t measured.

How to measure progress without being trapped in the gym

As I have written in another article, I think that benchmarking is the way out of these dire straits. Benchmarking offers you a way to measure progress while exposing yourself to the language in a healthy way.

In case you’re not familiar with benchmarking in the sense that I use the word here, it simply means using various methods to record progress and compare with similar measurements in the future to highlight the fact that you are actually learning something, even if it doesn’t feel like that (or, in case you actually don’t improve, it’s a good way of telling you that you need to change your method).

A good place to look for inspiration is the Central European Framework of Reference for Languages (CEFR), which uses can-do statements about real-world language ability across six levels (A1, A2, B1, B2, C1, C2). For example (A1):

• Listening: I can recognise familiar words and very basic phrases concerning myself, my family and immediate concrete surroundings when people speak slowly and clearly.
• Reading: I can understand familiar names, words and very simple sentences, for example, on notices and posters or in catalogues.
• Spoken interaction: I can interact in a simple way, provided the other person is prepared to repeat or rephrase things at a slower rate of speech and help me formulate what I’m trying to say. I can ask and answer simple questions in areas of immediate need or on very familiar topics.
• Spoken production: I can use simple phrases and sentences to describe where I live and people I know.
• Writing: I can write a short, simple postcard, for example, sending holiday greetings. I can fill in forms with personal details, for example entering my name, nationality and address on a hotel registration form.

You can use this self-assessment grid (which is where the above text is quoted from), but there’s plenty of more material available about CEFR if you’re interested, especially in the Companion Volume.

Common examples of how measurable progress leads you astray

If you do use numbers, it is worth thinking about what you choose to measure. Here are some common examples of measurable progress that can be misleading if you treat them as your main targets:

• Number of characters “known”. Many learners like to say that they know a certain number of characters, often based on a list or an app. This sounds very concrete, but “know” can mean many different things: recognise in isolation, write from memory, understand in common words, or use in context. The number also says nothing about how useful those characters are, or how easily you can read real texts. If you treat this number as a target, it’s tempting to chase rare characters just to make the total bigger instead of consolidating the common characters that appear all the time in real Chinese. In short, knowing characters is a very small part of what you need to be able to read Chinese.
• Proficiency tests such as HSK. A test score or level can be a useful benchmark and sometimes a practical requirement, but it only measures how well you perform on that particular test, on that day, with that format and content. It doesn’t measure how comfortable you feel chatting with a friend, understanding an unfamiliar accent, or following a fast TV show. If you treat the test result as your main measure of progress, you’ll end up preparing for the test rather than building a broad, flexible ability in Chinese. I have written more about this elsewhere, so I won’t repeat the full discussion here.
• Number of textbooks you have finished. It’s common to talk about level as “I have finished book three in this series”, as if that automatically meant that you’re an intermediate or advanced learner. However, finishing a textbook isn’t the same as mastering everything in it, and different books vary a lot in scope and difficulty. If you focus too much on ticking off books, it’s easy to rush through chapters, skim the exercises and move on without really learning what you’ve just covered. This can create an illusion of advanced learning. 
• How long you have studied Chinese. Saying that you’ve studied Chinese for one, five or ten years sounds like a meaningful way to talk about level, but it only measures how much time has passed, not what you’ve done during that time. You can study very intensively for a few months and learn more than someone who has taken one relaxed class a week for five years. This metric can also be emotionally harmful, since people feel that they should be much better given the number of years, even if their actual study time has been quite limited. I wrote more about why counting years is bunk here.

Instead of relying on these rough labels, you can design benchmarks that stay much closer to real language use.

There are many ways to benchmark, and which one you use depends on what you want to benchmark (see the above article for more specific guidelines). I’d also like to recommend this article about approaches to reading in Chinese, especially the part about benchmarking.

Benchmarking progress in Chinese to stay motivated

Turn a potential enemy into a powerful ally

We use the expression “double-edged sword” to indicate that a situation or method can have both positive and negative consequences. Still, double-edged swords are common all over the world, and if you learn how to use them, you won’t cut yourself just because the blade is sharp on both sides.

The conclusion is that measurable progress can be a powerful motivator, but you need to be aware of what you are measuring and why. If you aren’t, you will end up cutting yourself, but if you are, benchmarking and logging your learning have many benefits!

Editor’s note: This article, originally published in 2012, was rewritten and republished in May 2026.