I recently asked ChatGPT to help me improve Mandarin pronunciation. I told it I would upload a recording and asked it to suggest what to improve, then rate the pronunciation from 1 to 10, where 10 meant native-like. I then uploaded again and asked for follow-up feedback.
I used this sentence: 麻烦你把盐递给我 / 麻煩你把鹽遞給我 (máfan nǐ bǎ yán dì gěi wǒ), “Please pass me the salt.”
Here is a condensed transcript (full version here):
#1: 7/10. Clear and fully understandable. The main issue is tone accuracy and natural tone flow:
- 麻烦: both syllables should have clearly rising second tones, but the rises are not distinct enough. 麻 sounds somewhat like a third tone, and 烦 begins too high and falls slightly.
- 给我: because these are two third tones together, 给 should change to a rising tone, almost like géi wǒ. In the recording, 给 sounds too low and flat, while 我 starts rather high.
- The rhythm is slightly careful and syllable-by-syllable. Linking 递给我 more smoothly would make it sound more native-like.
#2: 7/10. Sounds the same. No improvement.
#3: 8/10. Clear improvement in 麻烦 and fluency, but 给我 still needs work.
#4: 8.5/10. Improvement. 给我 is better, but focus more on natural rhythm and intonation.
#5: 9/10. Another improvement. The overall rhythm and linking are noticeably more native-like.
Sounds quite useful, right?
ChatGPT can give genuinely helpful feedback on word choice and grammar, so it’s not unreasonable to assume that it could also help you improve your Mandarin pronunciation.
Tune in to the Hacking Chinese Podcast to listen to the related episode (#319).
Available on Apple Podcasts, Spotify, YouTube and many other platforms!
No, ChatGPT can’t analyse your Mandarin pronunciation
The problem is that the audio used in the example above isn’t a student recording, but perfectly standard Mandarin produced by a native-speaking teacher from Beijing (you can listen to the clip here).
There are no pronunciation errors, and the only reasonable rating is 10/10.
And this is not the only problem!
The pronunciation didn’t improve between “attempts” because there were no new attempts. I used exactly the same audio every time. Not just the same person saying the same thing, but the same recording.
So, what actually happened then?
ChatGPT invents errors, then makes up a learning arc for you
ChatGPT hallucinated pronunciation errors, then invented an improvement arc as I kept uploading the same audio.
The conversation itself gave it a strong reason to first identify errors and then expect progress, so that is what it did.
For a pronunciation coach, hallucinating feedback is worse than giving no feedback at all.
Since there are several problems here (identifying errors that aren’t there, then making up progress when there isn’t any), it’s worth exploring this further.
I ran independent tests where I presented the same recording in multiple chats (naturally with memory off and outside of a project) to see what would happen. The prompt and the audio were the same as in the transcript shared above.
The ratings ranged from 6.5 to 8.5 out of 10, and the problems highlighted were usually the same as those already listed, but the specific diagnoses also seemed quite random. This makes sense, because given the sentence in question, there are only so many things that typically go wrong (and ChatGPT certainly has discussions about learning tones in its training data).
Is that a Jamaican accent, or are you just making stuff up?
If you want to try this yourself and want to experience some true hallucination, use a language you already know well. For example, record yourself saying “water” once, then ask ChatGPT whether the accent sounds British, American, Australian, Canadian, Jamaican, Indian or South African, and why. Then keep saying (or playing) the word over and over.
When I tried this, ChatGPT went through the possible accents in some arbitrary order, making up plausible explanations of the supposedly different pronunciation features. It clearly has no clue at all what the accent is actually like.
Update: After writing this, I tried this with GPT-Live instead of recording messages. It still gets the accent wrong, but it no longer cycles arbitrarily through the accents.
A hallucinating pronunciation coach is much worse than no coach
If ChatGPT tells you what the capital of Assyria is, you can verify its output with a quick search, but if it tells you that your third tone starts too high or that your second tones don’t rise enough, you can’t easily check whether this is true.
A hallucinating pronunciation coach is nasty because you often can’t tell that it’s hallucinating.
Presumably, you asked for pronunciation feedback because you wanted to identify pronunciation errors in the first place. It’s perfectly possible to make mistakes you can’t hear yourself. This makes it easy to accept a confident diagnosis and waste time fixing the wrong problems.
Sometimes the explanation is wrong, too
Also, bear in mind that the information is sometimes factually incorrect. 麻烦 (máfan) is not typically pronounced with two rising tones, as ChatGPT claims in the supplied transcript. You can read more about the neutral tone here.
The explanation is secondary, though. ChatGPT isn’t detecting a genuine problem and then hallucinating an explanation. In these tests, the pronunciation assessment itself is made up, too.
Is ChatGPT even analysing your pronunciation?
If a chatbot claims that it has analysed pitch contours, F0 patterns or some other acoustic feature (which is visible in the user-facing status messages), that claim doesn’t prove that the subsequent assessment is based on this analysis.
Whatever happens under the hood, identical native audio should:
- Receive full or almost full marks, depending on the standard.
- Receive identical or at least similar assessments each time.
But this is not what ChatGPT does.
I tested this in September 2026, using ChatGPT 5.6 with the highest level of “thinking effort”. General-purpose chatbots are changing quickly, so this may not remain true forever.
I have so far not found a general-purpose chatbot that can analyse your pronunciation, but Claude merits a mention here because it explicitly said it couldn’t do what I asked for (which is of course what ChatGPT should have said).
Many apps that purport to check pronunciation actually check a transcript
My strong suspicion is that ChatGPT transcribes the audio and then makes up the assessment around it. This would explain why it gives you a worse rating if you deliberately change words in the recording and why it says your pronunciation is terrible if you upload complete gibberish.
If it expects you to say something, either because you tell it what the target is or because it’s clear from context, and the transcription then differs from that, the feedback can at least be grounded in something you actually said rather than being completely made up.
Many apps that purport to check your pronunciation work roughly like this:
- Does the transcription match the target? Pass (or, in ChatGPT’s case, start hallucinating).
- Is the transcription different? Give feedback based on the mismatch.
It’s not 100% hallucination, in other words.
ChatGPT’s comprehensibility judgements seem to follow the transcription
I tried probing whether or not transcription differences are the only thing ChatGPT actually analyses. While my tests were far from rigorous enough to show this conclusively, the hypothesis seems promising.
I asked ChatGPT not just to assess pronunciation, but to say how comprehensible my Chinese was. When the utterances were transcribed correctly, ChatGPT tended to say they were understandable.
When I deliberately changed one or several words so that the transcription no longer matched the target, it gave a much lower comprehensibility judgement and correctly pointed out the issue.
For example, if I intended to say 买房子/買房子 (mǎi fángzi), “buy a house”, but this gets transcribed as 卖房子/賣房子 (mài fángzi), “sell a house”, because of a tone error, ChatGPT can say that the tone was wrong.
For more about tone errors that can really mess up communication, see Tone errors in Mandarin that actually can cause misunderstandings.
Tone errors in Mandarin that actually can cause misunderstandings
Speech recognition and pronunciation assessment have different goals
You don’t need a fancy app to use transcription in this way. Just pick up your phone, make sure you have Mandarin selected as input, switch to voice input and start talking. Compare what you intended to say with what is shown on the screen.
It’s tempting to think that computers, including smartphones, listen the way humans do. They don’t, however, and the goals are different, too.
A speech-to-text system uses acoustic input and linguistic context to rank candidates and output the most likely text. More context makes this easier. An isolated syllable gives the system little to work with. A two-syllable word narrows the possibilities further. A full sentence gives so much context that the system can often guess the intended words even when the pronunciation is far from accurate.
For example, I said 我喜欢吃苹果 (wǒ xǐhuan chī píngguǒ), “I like eating apples”, with deliberately flat pitch (mid, flat tone throughout, as if mimicking a robot), essentially removing the lexical tones.
It was still transcribed correctly. That obviously doesn’t mean that my tones were good. It means that, given the acoustic input and all the linguistic context in the sentence, 我喜欢吃苹果 was still the most likely text.
This is also what I found when I tested speech recognition more systematically a few years ago. Context makes a huge difference and means that speech recognition is usually too lenient to use for pronunciation assessment.
You can read the details in the two-part series starting here: Using speech recognition to improve Chinese pronunciation, part 1.
Using speech recognition to improve Chinese pronunciation, part 1
That article is getting old, but it’s an interesting case where the main point becomes more true as it gets older. The better speech recognition becomes, the less useful it becomes for pronunciation assessment.
But don’t humans rely on context too when listening?
Good point! If you’ve read my series about listening comprehension, you might rightfully object and say that humans rely on context, too.
And we do. Humans rely on what has already been said and what we know about the speaker and situation, and we use this information to fill in gaps. We can even go back and correct earlier hypotheses based on subsequent input. Accurate pronunciation (or accurate anything, actually) mostly matters when there isn’t enough context or when the listener can’t predict what you’re going to say.
The importance of tones is inversely proportional to the predictability of what you say
But humans and speech-to-text systems still fail in different ways.
A speech-to-text system has to output something. Even when none of the possibilities is particularly convincing, there is still a least-unlikely candidate it can return. A human listener can instead be unsure, confused or simply fail to understand.
Speech recognition will therefore get some things right that a human would not, and a human will get some things right that speech recognition won’t. They aren’t the same.
For an analogous example with written language, look at the screenshots below from Pleco. I have written the traditional character 龍 twice. Both resulted in the correct character lookup, but only the one on the right is readable for a human.
So, can AI analyse your pronunciation?
Anyway, the takeaway here is that transcription is a very crude way to assess pronunciation, and this means that ChatGPT, and many other apps that claim to analyse your pronunciation, are not very useful for this purpose.
Not all AI systems are chatbots, of course, so I’m not claiming that AI can’t analyse your pronunciation. It’s incredibly hard to know what’s going on under the hood in apps, but some careful probing can at least tell you something.
If you know of an app that actually analyses your pronunciation (they do exist), please let me know. I haven’t had time to check many of these yet, but there’s no theoretical reason AI couldn’t give you helpful pronunciation feedback.
Better ways to improve your Mandarin pronunciation
If you actually want to identify your pronunciation problems and do something about them, here are some things you can try.
- Record yourself and listen back. When you are speaking, your attention is occupied by deciding what to say, finding the words, thinking about how the other person reacts, etc. This is why listening to a recording of yourself can be so useful. It lets you focus entirely on how you sounded, warts and all.
- Mimic native speakers. Listen very carefully to a short recording, mimic the speaker and compare your version with the original. This combines attentive listening, careful production and direct comparison. It’s also entirely free.
- Use minimal-pair bingo. Simply asking someone if your tones are good rarely works. To really know how good your tones are (or some other specific feature, such as aspiration), you need a blind test where the person you ask can’t cheat to make you feel good. Minimal-pair bingo is exactly such a test.
- Make the most out of the feedback you get. Not all feedback is explicit corrections by a professional teacher. In fact, you can learn a lot about your pronunciation by paying attention to how people react when you talk. Naturally, when someone does give you feedback, make sure you encourage them to give you more!
- Work with a pronunciation expert. A good teacher can identify what is wrong, explain why it may be happening and tell you what to do instead. As add-ons to my pronunciation course, Hacking Chinese Pronunciation: Speaking with Confidence, I offer detailed and honest pronunciation assessments as well as individual coaching.
There will probably come a day when ChatGPT can do some or all of this, but it is not today. Chatbots can be useful for many other things in language learning, but pronunciation analysis is not one of them.
