Skip to main content

I expected the translation model to win

Updated 6 min readMarkdown

I came into this test with a lazy assumption: for translation, use the model that was built for translation.

Twenty Japanese and Korean slang terms later, that assumption was gone.

qwen-mt-uni got two right. qwen3.8-max and kimi-k3 got nineteen. deepseek-v4.1-flash got eighteen. glm-5.3 got seventeen.

I am not writing this to dunk on a model. The gap turned out to be the most useful thing I learned about model selection all year, and it applies well beyond translation.

The setup

Five models, one prompt, run through an agent so the wording stayed identical.

Model What it is
qwen-mt-uni Translation model, accepts text, images, documents, audio
qwen3.8-max General-purpose flagship
deepseek-v4.1-flash General-purpose lightweight
glm-5.3 General-purpose mainline, 200k context
kimi-k3 General-purpose flagship, long context

The prompt: translate each term into Chinese internet slang. One per line. Keep it colloquial.

All four general models got every Korean term right. Their misses were Japanese only, and they clustered on the same two words: 草 rendered as surprise rather than laughter, and エモい rendered as 破防了, an emotional direction that runs negative.

Three results that explained everything

"草生える" is Japanese for "that's funny." Built from 草, the character for grass, which Japanese forums use the way English forums use "lol."

qwen-mt-uni returned "草在生长", which means grass is growing.

Technically correct. Semantically dead. A reader of the Chinese output learns that a plant is growing and nothing else.

"어쩔티비" is Korean for something between "so what" and "what are you going to do about it." The second half sounds like "TV."

qwen-mt-uni returned "Eojjeol TV station." It translated the sound of the word, then interpreted that sound as a broadcast channel.

"폼 미쳤다" describes someone in unbelievable form, usually an athlete or performer. 폼 means form, and also foam.

qwen-mt-uni returned "this foam effect is very good."

Three different failure shapes, one shared cause. The model reads the literal surface and commits to it with no room to reconsider.

Three models scored on 20 slang terms

The reason is in the product description

This is not a capability problem. It is a training objective.

The description of qwen-mt-uni says translations correspond strictly to the source, with no additions and no omissions.

Read that again as a product decision rather than a technical limit. For a contract, that sentence is a feature. You do not want a model paraphrasing clause four. For a product manual, it is a feature. For a financial report, it is a feature.

For a subtitle line, it is a bug.

Slang is meaning carried outside the literal text. To translate it, you have to add, drop, and rephrase. Which is exactly what the model was built not to do.

General models were trained on forums, comment sections, and social posts. They learned the register before anyone asked them to translate. When you hand them a slang term, they recognize the situation and answer in it.

Context rescued some of them

Then I put the same terms inside full sentences.

"ワンチャン" alone came out as "one opportunity." Inside "ワンチャン間に合うかも" it became "might still make it." Correct.

"눈물 나" alone was "I am crying." Inside "눈물 나, 진짜 웃겨서" it became "I laughed until I cried." Correct.

Context moved the score from 2 out of 10 to 3 out of 8, with three more that were right in meaning but stiff in tone.

Two never recovered. "草生えるwww" stayed "grass is growing, haha". It learned to add a laugh, but the main clause stayed literal. "폼 미쳤다, 이 무대" went from foam to "the stage design is great." Closer, still not form.

The lesson I took: context helps a constrained model, but it cannot override the constraint. You are asking it to be less literal than it was designed to be.

What the translation model is genuinely better at

The comparison so far is unfair in one respect, so here is the other half.

That model accepts four input types: text, images, documents, and audio. The last three return a finished file, not a string you have to paste somewhere.

I tested all four.

A Chinese product manual came back as a downloadable translated file. The content was right. The section numbering came out as "1. / II. / III." in the same document, mixing Arabic and Roman numerals, so someone still has to fix the layout.

An English pet food poster came back with all three text lines translated. The title and subtitle had merged into one line, "grain free" had broken across two lines, the dollar sign stayed, and the small text printed on the product package was untouched.

A seven-second Chinese voice clip in WAV was rejected as unreadable. Converted to m4a, it returned an English mp3.

This is the trade. General models hand you text. This one hands you files.

If your job is "translate this PDF and send it to a distributor," a general model cannot finish that task at any quality level. It has no file output.

Four billing channels

What I would actually do with this

Split the pipeline by content type rather than by model ranking.

Documents, posters, and voice recordings go to the translation model, because file output is the requirement and layout cleanup is a known, bounded cost. Text that people read as speech goes to a general model, because register matters more than fidelity there.

The pricing supports the split. Four channels, four prices per million tokens: text 65 CNY, document 20, image 32, audio 400. An 8,000-character manual costs roughly 0.52 CNY as text and 0.16 CNY as a PDF. Plain text files bill as characters, not documents, so the container matters.

Audio is the expensive one at six times text, which changes the calculus. One hour of meeting recording is about 36 CNY. Worth it for a spot check, worth a second thought before you translate an archive.

The number that surprised me most was not any of these. It was the editing time. Twenty slang terms took about 25 minutes to rewrite from the translation model's output, and about 5 minutes from the general model's. At any reasonable labor rate, that difference dwarfs the token bill.

So the honest selection rule is not "which model translates better." It is "how much of this output will I have to rewrite, and is the thing I am saving worth more than that time."

Two things to test before you commit: terminology dictionaries did not take effect for me despite returning HTTP 200, and one Korean slang term was blocked by content moderation while translating fine on a general model.

Both are the kind of failure that only shows up when you run your own content through it.


The models are on Alibaba Cloud Bailian, called over the API. Console here, API key here.

No comments yet. Be the first to comment.