<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[juseong]]></title><description><![CDATA[juseong]]></description><link>https://joecien.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 07 Nov 2026 10:14:32 GMT</lastBuildDate><atom:link href="https://joecien.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[The memory layer is a retrieval contract, not a storage feature]]></title><description><![CDATA[Most teams evaluating agent memory compare storage options: vector database, graph, hosted service. The choice matters less than what you learn after…]]></description><link>https://joecien.hashnode.dev/the-memory-layer-is-a-retrieval-contract-not-a-storage-feature</link><guid isPermaLink="true">https://joecien.hashnode.dev/the-memory-layer-is-a-retrieval-contract-not-a-storage-feature</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Mon, 26 Oct 2026 03:23:09 GMT</pubDate><content:encoded><![CDATA[<p>Most teams evaluating agent memory compare storage options: vector database, graph, hosted service. The choice matters less than what you learn after wiring anything up, which is that storing is easy and retrieving is a contract you have to design.</p>
<p>I measured one hosted memory implementation end to end, so the discussion has numbers behind it.</p>
<h3>What I ran</h3>
<p>A memory library behind a CLI, loaded with 14 facts describing a solo contractor's working rules, then interrogated with 36 queries covering 12 of those facts. Each fact was asked three ways: close to the original wording, conversational, and framed from a business angle. The model used for the downstream comparison was a mid-tier one, identical across both conditions.</p>
<h3>The measurement that matters</h3>
<p>Default configuration returned results for 9 of 36 queries.</p>
<p>Re-running with the score threshold at zero returned results for all 36. Then I checked rank quality: the correct memory was first in 31 of 36 queries, and in the top three in 34 of 36.</p>
<table>
<thead>
<tr>
<th>Metric</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>Delivery rate, defaults</td>
<td>25%</td>
</tr>
<tr>
<td>Delivery rate, threshold 0</td>
<td>100%</td>
</tr>
<tr>
<td>Recall@1</td>
<td>86%</td>
</tr>
<tr>
<td>Recall@3</td>
<td>94%</td>
</tr>
<tr>
<td>Median top score</td>
<td>0.211</td>
</tr>
<tr>
<td>75th percentile top score</td>
<td>0.300</td>
</tr>
</tbody></table>
<p>The 75th percentile landing exactly on the default threshold is the whole story. Similarity scores are cosine-scale: semantically related but differently worded sentences cluster near 0.2, and only near-verbatim restatements approach 0.8. A 0.3 cutoff is a defensible default for a console UI where a human inspects candidates. It is the wrong default for an autonomous caller, because an empty result set carries no signal about whether the store is empty or the bar was too high.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/agent-memory/recall-threshold-en-1280x720.jpg" alt="Score distribution across 36 queries with the default 0.3 cutoff" width="720" />

<h3>The design consequence</h3>
<p>If you are building a memory-backed agent, the retrieval parameters belong in your layer, not in the defaults.</p>
<p>Take the threshold down and truncate results on your side. Three candidates is usually enough, and letting the model arbitrate relevance is cheap compared to a wrong answer built on a missing context. The interface exposes result count for exactly this reason: it expects the caller to own the policy.</p>
<p>Two more decisions worth making deliberately.</p>
<p><strong>Write asynchronously.</strong> The write call blocks on a 30 second polling interval, not on extraction itself: three inputs of 17, 41, and 98 characters all took about 32 seconds, with creation and update timestamps exactly 30 seconds apart. Setting the wait budget to zero returns a task ID in 0.7 seconds. Any agent that writes during conversation needs this path or it will stall the turn.</p>
<p><strong>Verify memory types, not command exit codes.</strong> A write carrying skill name, description, and tags still landed as an ordinary fact memory, because skill memories require a rule created in the console first. The export command for skill memories returned that fact memory successfully, with the type field quietly reporting an observation. Design your telemetry around the resource type, not around the command succeeding.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/agent-memory/memory-loop-en-1280x720.jpg" alt="Three steps of a memory loop: write and extract, isolate per user, retrieve and inject" width="720" />

<h3>The part that behaves well</h3>
<p>Worth saying plainly, because the failure above is the interesting part but not the whole picture.</p>
<p>Extraction picks its own granularity by fact rather than by sentence. Repeated rules merge into updates rather than accumulating contradictory copies, which is the difference between a memory layer that ages well and one that quietly rots. Deletion is physical and immediate. Isolation is per user ID, verified by querying the same store with a different ID and getting nothing.</p>
<p>That is most of the hard engineering done, and it is worth acknowledging before going back to the failure. What remains is calibration, and calibration is the caller's job.</p>
<h3>Cost shape</h3>
<p>Billing is per call: writes around 0.03 CNY on the higher-quality tier, retrievals around 0.001 CNY, storage about 1.44 CNY per 10,000 entries per month. The free grant covers 1,500 writes and 5,000 retrievals, so an evaluation of this size prices at under half a yuan and bills at nothing.</p>
<p>For a production integration the cost that actually scales is context: every injected memory is input tokens. Measured on two questions, injecting three memories cut output from 2,117 to 929 tokens and from 2,244 to 924. Relevant context replaces padding, so the net effect on spend was negative in both cases.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/agent-memory/three-memory-types-en-1280x720.jpg" alt="Three memory types: fact memory, user profile, skill memory" width="720" />

<h3>Summary</h3>
<p>A hosted memory service solves storage, extraction, merging, and isolation. It does not decide when a memory is relevant enough to inject, and its default answer to that question is conservative to the point of being invisible.</p>
<p>Treat retrieval as a contract with your own policy attached: threshold low, candidates few, relevance judged by the model, writes non-blocking. The storage layer will hold up its end.</p>
<p>The memory library I measured is part of Alibaba Cloud Bailian. Console: <a href="https://bailian.console.aliyun.com/?spm=a2ty07.github">Bailian</a>. Keys: <a href="https://bailian.console.aliyun.com/cn-beijing/?spm=a2ty07.github&amp;tab=app#/api-key">API key page</a>. Beijing region only; the CLI is installed by handing the official skill package to your agent.</p>
]]></content:encoded></item><item><title><![CDATA[I thought fine-tuning was the hard part]]></title><description><![CDATA[I write this as someone who does not write code. A friend runs a pet supplies shop. Their support agents call customers 铲屎官 and themselves 小毛, and they…]]></description><link>https://joecien.hashnode.dev/i-thought-fine-tuning-was-the-hard-part</link><guid isPermaLink="true">https://joecien.hashnode.dev/i-thought-fine-tuning-was-the-hard-part</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Fri, 23 Oct 2026 10:20:14 GMT</pubDate><content:encoded><![CDATA[<p>I write this as someone who does not write code.</p>
<p>A friend runs a pet supplies shop. Their support agents call customers 铲屎官 and themselves 小毛, and they never say "dear." Customers get answers in under thirty characters. A general model, plugged into the same chat window, answers in three hundred, cites consumer law, and asks which platform you bought from.</p>
<p>I offered to fix the voice. The plan was simple: collect their real conversations, fine-tune a model, ship it.</p>
<p>Two days of data work, one hour of training, four yuan of compute. The model trained. Then it would not deploy, so I retrained a smaller one that did, and learned something less comfortable: it got the accent and lost the rules.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/sft-lora/dataset-en-1280x720.jpg" alt="Three attempts at a dataset" width="600" />

<h2>What I got wrong first</h2>
<p>My first dataset was 200 rows. The platform rejected it outright, and the error message was the most useful thing I read all week:</p>
<pre><code class="language-plaintext">The validation set size (20) must be no less than the batch size (64)
</code></pre>
<p>Validation is a 10% slice by default. The minimum batch is 64. Twenty rows cannot fill a batch of sixty-four, so the floor is 640 rows.</p>
<p>Nobody writes "640 rows" in a quickstart. It emerges from two defaults. The practical consequence: a 100-row pilot cannot exist. You either commit to the dataset or you do not start.</p>
<h2>What I got wrong second</h2>
<p>I rebuilt with 700 rows and passed validation. Then I read my own data.</p>
<p>A customer asked how well the litter controlled odor. My dataset answered that it clumped fast and flushed easily. A customer asked whether a kitten could eat the food. My dataset recommended a kibble size for puppies.</p>
<p>I had grouped by intent. "Cat litter" contained four unrelated questions, all sharing one answer set. The customer sees four products. My data saw one bucket.</p>
<p>There was also a deduplication bug that took a while to notice: I keyed on the answer, so 700 rows held only 14 distinct questions. Everything else was the same sentence wearing a different opening.</p>
<h2>What actually worked</h2>
<p>Paring questions into groups, each with three to five fitting answers, then expanding with openers and closers.</p>
<pre><code class="language-plaintext">Ask: how does it handle odor / does it smell / two cats, is it enough
Answer: activated charcoal, scoop daily and you won't notice
</code></pre>
<p>That change took the dataset from 700 rows of noise to 700 rows across 93 questions and 513 answers. Small, specific, and honest about its coverage.</p>
<p>The openers and closers are doing more than padding. They teach the turn shape: reassure first, answer in one clause, close with a name. That rhythm is most of what makes the shop sound like itself.</p>
<h2>The part that surprised me</h2>
<p>Training was uneventful. 29 steps, 58 minutes, loss falling from 1.17 to 0.47, validation accuracy from 0.72 to 0.85.</p>
<p>I read those numbers as success. They are not success. They are evidence that the model fit the data I gave it.</p>
<p>Deployment is where the project ended.</p>
<p>The token-billed plan returned a permission error: this model is only offered as dedicated capacity on this account. The dedicated plan returned a capacity error: no stock for that spec. A dozen retries, same answer.</p>
<p>So the 27B exists. It is trained, exported, and listed. It cannot be called.</p>
<h2>The smaller one that went live</h2>
<p>I retrained the same 700 rows on a 4B model that does offer token-billed serving. 225 seconds, 67,623 tokens, validation accuracy 0.508 against 0.846 for the 27B.</p>
<p>Deployment took four minutes. Same twenty held-out questions:</p>
<table>
<thead>
<tr>
<th>Metric</th>
<th>27B base</th>
<th>4B fine-tune</th>
</tr>
</thead>
<tbody><tr>
<td>Calls itself 小毛</td>
<td>0 / 20</td>
<td>5 / 20</td>
</tr>
<tr>
<td>Policy keywords matched</td>
<td>44%</td>
<td>5%</td>
</tr>
<tr>
<td>Average answer length</td>
<td>276</td>
<td>27</td>
</tr>
</tbody></table>
<p>Twenty-seven characters, against 27.9 for the shop's agents. The voice landed.</p>
<p>The rules did not. Asked for a discount, it granted one. Asked about next-morning delivery, it promised it. Asked whether opened cat food is returnable, it said no returns, which is the reverse of policy.</p>
<p>Surface patterns repeat hundreds of times in 700 rows, so a 4B model absorbs them. Policy details are sparse and need combining rows, so the model fills those slots with whatever sounds most like support. It does not become cautious. It becomes fluent.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/sft-lora/before-after-en-1280x720.jpg" alt="Twenty held-out questions: 27B base versus 4B fine-tune" width="720" />

<p>I deleted the deployment after the run; token-billed means the bill was the tokens, not the hours.</p>
<h2>What I would tell someone starting today</h2>
<p><strong>Check the last step first.</strong> Whether a fine-tuned model can be deployed, on which billing plan, and whether that plan has capacity, are all answerable before you write a single row. I asked those questions at the end, which is the wrong end.</p>
<p><strong>Treat the dataset as the project.</strong> Training cost four yuan. The dataset cost two days and three rewrites. If you cannot spare the two days, you do not have a fine-tuning project; you have a prompt engineering project.</p>
<p><strong>Read your data like a customer.</strong> Every automated check passed on the version that answered odor questions with clumping facts. Only reading it caught the mismatch.</p>
<p><strong>Keep a held-out test set.</strong> I wrote twenty questions that were not in the training data. On the base model, zero out of twenty used either persona marker and answers averaged 276 characters. On the 4B fine-tune, five used the name and answers averaged 27 characters, with 5% policy accuracy. Both halves are the honest state of this project.</p>
<h2>The uncomfortable takeaway</h2>
<p>Fine-tuning is cheap enough now to try on a whim, which is exactly why the non-model parts matter more than they used to.</p>
<p>Data quality decides whether the model is any good. Deployment access decides whether it exists at all. Model size decides which half of the job it can hold. The training run, the part everyone talks about, took under an hour and cost less than a coffee.</p>
<p>The most valuable output is not a model. It is a checklist: 640 rows minimum, question-answer pairs that actually match, a deployability check before the first line of data, and a model tier chosen for the rules rather than the invoice.</p>
<hr />
<p>The model runs on Alibaba Cloud Bailian. Console <a href="https://bailian.console.aliyun.com/?spm=a2ty07.github">here</a>, API key <a href="https://bailian.console.aliyun.com/cn-beijing/?spm=a2ty07.github&amp;tab=app#/api-key">here</a>.</p>
]]></content:encoded></item><item><title><![CDATA[I expected the translation model to win]]></title><description><![CDATA[I came into this test with a lazy assumption: for translation, use the model that was built for translation. Twenty Japanese and Korean slang terms later…]]></description><link>https://joecien.hashnode.dev/i-expected-the-translation-model-to-win</link><guid isPermaLink="true">https://joecien.hashnode.dev/i-expected-the-translation-model-to-win</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Fri, 23 Oct 2026 10:04:14 GMT</pubDate><content:encoded><![CDATA[<p>I came into this test with a lazy assumption: for translation, use the model that was built for translation.</p>
<p>Twenty Japanese and Korean slang terms later, that assumption was gone.</p>
<p><code>qwen-mt-uni</code> got two right. <code>qwen3.8-max</code> and <code>kimi-k3</code> got nineteen. <code>deepseek-v4.1-flash</code> got eighteen. <code>glm-5.3</code> got seventeen.</p>
<p>I am not writing this to dunk on a model. The gap turned out to be the most useful thing I learned about model selection all year, and it applies well beyond translation.</p>
<h2>The setup</h2>
<p>Five models, one prompt, run through an agent so the wording stayed identical.</p>
<table>
<thead>
<tr>
<th>Model</th>
<th>What it is</th>
</tr>
</thead>
<tbody><tr>
<td><code>qwen-mt-uni</code></td>
<td>Translation model, accepts text, images, documents, audio</td>
</tr>
<tr>
<td><code>qwen3.8-max</code></td>
<td>General-purpose flagship</td>
</tr>
<tr>
<td><code>deepseek-v4.1-flash</code></td>
<td>General-purpose lightweight</td>
</tr>
<tr>
<td><code>glm-5.3</code></td>
<td>General-purpose mainline, 200k context</td>
</tr>
<tr>
<td><code>kimi-k3</code></td>
<td>General-purpose flagship, long context</td>
</tr>
</tbody></table>
<p>The prompt: translate each term into Chinese internet slang. One per line. Keep it colloquial.</p>
<p>All four general models got every Korean term right. Their misses were Japanese only, and they clustered on the same two words: 草 rendered as surprise rather than laughter, and エモい rendered as 破防了, an emotional direction that runs negative.</p>
<h2>Three results that explained everything</h2>
<p><strong>"草生える"</strong> is Japanese for "that's funny." Built from 草, the character for grass, which Japanese forums use the way English forums use "lol."</p>
<p><code>qwen-mt-uni</code> returned "草在生长", which means grass is growing.</p>
<p>Technically correct. Semantically dead. A reader of the Chinese output learns that a plant is growing and nothing else.</p>
<p><strong>"어쩔티비"</strong> is Korean for something between "so what" and "what are you going to do about it." The second half sounds like "TV."</p>
<p><code>qwen-mt-uni</code> returned "Eojjeol TV station." It translated the sound of the word, then interpreted that sound as a broadcast channel.</p>
<p><strong>"폼 미쳤다"</strong> describes someone in unbelievable form, usually an athlete or performer. 폼 means form, and also foam.</p>
<p><code>qwen-mt-uni</code> returned "this foam effect is very good."</p>
<p>Three different failure shapes, one shared cause. The model reads the literal surface and commits to it with no room to reconsider.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/mt-uni/score-en-1280x720.jpg" alt="Three models scored on 20 slang terms" width="600" />

<h2>The reason is in the product description</h2>
<p>This is not a capability problem. It is a training objective.</p>
<p>The description of <code>qwen-mt-uni</code> says translations correspond strictly to the source, with no additions and no omissions.</p>
<p>Read that again as a product decision rather than a technical limit. For a contract, that sentence is a feature. You do not want a model paraphrasing clause four. For a product manual, it is a feature. For a financial report, it is a feature.</p>
<p>For a subtitle line, it is a bug.</p>
<p>Slang is meaning carried outside the literal text. To translate it, you have to add, drop, and rephrase. Which is exactly what the model was built not to do.</p>
<p>General models were trained on forums, comment sections, and social posts. They learned the register before anyone asked them to translate. When you hand them a slang term, they recognize the situation and answer in it.</p>
<h2>Context rescued some of them</h2>
<p>Then I put the same terms inside full sentences.</p>
<p>"ワンチャン" alone came out as "one opportunity." Inside "ワンチャン間に合うかも" it became "might still make it." Correct.</p>
<p>"눈물 나" alone was "I am crying." Inside "눈물 나, 진짜 웃겨서" it became "I laughed until I cried." Correct.</p>
<p>Context moved the score from 2 out of 10 to 3 out of 8, with three more that were right in meaning but stiff in tone.</p>
<p>Two never recovered. "草生えるwww" stayed "grass is growing, haha". It learned to add a laugh, but the main clause stayed literal. "폼 미쳤다, 이 무대" went from foam to "the stage design is great." Closer, still not form.</p>
<p>The lesson I took: context helps a constrained model, but it cannot override the constraint. You are asking it to be less literal than it was designed to be.</p>
<h2>What the translation model is genuinely better at</h2>
<p>The comparison so far is unfair in one respect, so here is the other half.</p>
<p>That model accepts four input types: text, images, documents, and audio. The last three return a finished file, not a string you have to paste somewhere.</p>
<p>I tested all four.</p>
<p>A Chinese product manual came back as a downloadable translated file. The content was right. The section numbering came out as "1. / II. / III." in the same document, mixing Arabic and Roman numerals, so someone still has to fix the layout.</p>
<p>An English pet food poster came back with all three text lines translated. The title and subtitle had merged into one line, "grain free" had broken across two lines, the dollar sign stayed, and the small text printed on the product package was untouched.</p>
<p>A seven-second Chinese voice clip in WAV was rejected as unreadable. Converted to m4a, it returned an English mp3.</p>
<p>This is the trade. General models hand you text. This one hands you files.</p>
<p>If your job is "translate this PDF and send it to a distributor," a general model cannot finish that task at any quality level. It has no file output.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/mt-uni/channels-en-1280x720.jpg" alt="Four billing channels" width="600" />

<h2>What I would actually do with this</h2>
<p>Split the pipeline by content type rather than by model ranking.</p>
<p>Documents, posters, and voice recordings go to the translation model, because file output is the requirement and layout cleanup is a known, bounded cost. Text that people read as speech goes to a general model, because register matters more than fidelity there.</p>
<p>The pricing supports the split. Four channels, four prices per million tokens: text 65 CNY, document 20, image 32, audio 400. An 8,000-character manual costs roughly 0.52 CNY as text and 0.16 CNY as a PDF. Plain text files bill as characters, not documents, so the container matters.</p>
<p>Audio is the expensive one at six times text, which changes the calculus. One hour of meeting recording is about 36 CNY. Worth it for a spot check, worth a second thought before you translate an archive.</p>
<p>The number that surprised me most was not any of these. It was the editing time. Twenty slang terms took about 25 minutes to rewrite from the translation model's output, and about 5 minutes from the general model's. At any reasonable labor rate, that difference dwarfs the token bill.</p>
<p>So the honest selection rule is not "which model translates better." It is "how much of this output will I have to rewrite, and is the thing I am saving worth more than that time."</p>
<p>Two things to test before you commit: terminology dictionaries did not take effect for me despite returning HTTP 200, and one Korean slang term was blocked by content moderation while translating fine on a general model.</p>
<p>Both are the kind of failure that only shows up when you run your own content through it.</p>
<hr />
<p>The models are on Alibaba Cloud Bailian, called over the API. Console <a href="https://bailian.console.aliyun.com/?spm=a2ty07.github&amp;spm=a2ty07.github">here</a>, API key <a href="https://bailian.console.aliyun.com/cn-beijing/?spm=a2ty07.github&amp;tab=app#/api-key&amp;spm=a2ty07.github">here</a>.</p>
]]></content:encoded></item><item><title><![CDATA[Deep research is usually evaluated by reading the report and nodding. ]]></title><description><![CDATA[The more useful question is structural: how does the model divide the work, where does the output become unusable, and what does a report actually cost in…]]></description><link>https://joecien.hashnode.dev/deep-research-is-usually-evaluated-by-reading-the-report-and-nodding</link><guid isPermaLink="true">https://joecien.hashnode.dev/deep-research-is-usually-evaluated-by-reading-the-report-and-nodding</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Fri, 23 Oct 2026 09:36:17 GMT</pubDate><content:encoded><![CDATA[<p>The more useful question is structural: how does the model divide the work, where does the output become unusable, and what does a report actually cost in tokens and minutes?</p>
<p>I ran Bailian's <code>qwen-deep-research</code> on a single question about AI voice assistants in e-commerce after-sales. The run produced 6,909 characters in 502 seconds for about 1.32 CNY. It also produced three concrete failure modes worth documenting: a protocol that is not OpenAI-compatible, a reasoning block that leaks into the body, and 35 citation markers with no reference list.</p>
<h2>The two-call protocol</h2>
<p>This model does not behave like a chat completion. It requires two calls.</p>
<p>The first call sends only the user question:</p>
<pre><code class="language-json">{
  "model": "qwen-deep-research",
  "input": {
    "messages": [
      { "role": "user", "content": "Research the state of AI voice assistants in customer service in 2026." }
    ]
  }
}
</code></pre>
<p>The response is not a report. It is a clarification request: which industry, which dimension (results, cost, technical approach, cases). Input for this step was 20 tokens, first token in about 2.8 seconds.</p>
<p>The second call includes the original question, the assistant's clarification request, and the user's answer:</p>
<pre><code class="language-json">{
  "model": "qwen-deep-research",
  "input": {
    "messages": [
      { "role": "user", "content": "..." },
      { "role": "assistant", "content": "&lt;clarifying questions&gt;" },
      { "role": "user", "content": "Focus on e-commerce after-sales, cost per minute, real cases." }
    ]
  },
  "parameters": { "output_format": "model_summary_report" }
}
</code></pre>
<p>Omitting the assistant message causes the model to ask again. The protocol is a conversation state machine, not a prompt template.</p>
<h2>The endpoint constraint</h2>
<p>The model only speaks the native DashScope endpoint:</p>
<pre><code class="language-plaintext">POST https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/text-generation/generation
X-DashScope-SSE: enable
Authorization: Bearer &lt;Beijing-region key&gt;
</code></pre>
<p>OpenAI compatibility mode returns 404. The CLI's <code>bl text chat</code> uses compatibility mode, so it cannot reach this model. This is a recurring pattern with newly listed models: the model exists in the catalog before it exists on every interface.</p>
<h2>Where the time goes</h2>
<p>The second call took 502.1 seconds. Output was 8,085 tokens, input 89 tokens.</p>
<p>The latency is not transport. It is the research loop: planning, searching, reading, synthesizing, writing. The model emits a <code>&lt;think&gt;</code> block first, containing a requirement breakdown, four research questions, and an outline, and then writes the report against that outline.</p>
<p>For product design, this means the model belongs behind an asynchronous job, not a request-response endpoint. Users need a "researching" state and a notification when the report is ready.</p>
<h2>The output structure</h2>
<p>The report had one H1 and four H2 sections:</p>
<pre><code class="language-plaintext"># 2026 e-commerce after-sales AI voice assistant: cost, cases, and selection guide
## Cost structure and estimation
## Real deployment cases
## Challenges and selection strategy
## Market trends
</code></pre>
<p>The structure is planned by the model, not imposed by a template. That is the main quality difference from a long-form completion: the outline is derived from the question, and the sections are populated from research rather than from the model's prior.</p>
<h2>Failure mode 1: <code>&lt;think&gt;</code> leakage</h2>
<p>The raw stream begins with a <code>&lt;think&gt;</code> block:</p>
<pre><code class="language-plaintext">&lt;think&gt;
Deconstruction of the user's need:
1. What is the cost structure...
Report outline:
- Level 1 title...
&lt;/think&gt;
</code></pre>
<p>The API does not separate reasoning from content. If the client concatenates <code>output.message.content</code> without filtering, the outline ends up in the document. The fix is a post-processing step: strip between the tags, keep the raw output for auditing.</p>
<h2>Failure mode 2: citation markers without references</h2>
<p>The report contains 35 <code>[[n]]</code> markers. There is no reference list at the end. Sampling three:</p>
<table>
<thead>
<tr>
<th>Claim</th>
<th>Marker</th>
<th>Traceable</th>
</tr>
</thead>
<tbody><tr>
<td>Market size 18.6B CNY, 63.5% penetration</td>
<td><code>[[7]]</code></td>
<td>No</td>
</tr>
<tr>
<td>Telecom cost 0.14 / 0.3 CNY per minute</td>
<td><code>[[14]]</code></td>
<td>No</td>
</tr>
<tr>
<td>Three cost tiers 0.13 / 0.40 / 0.67 CNY</td>
<td><code>[[1,4,7,17,24]]</code></td>
<td>No</td>
</tr>
</tbody></table>
<p>A marker means "a source is believed to exist here." It does not mean the source is attached or verifiable. For internal drafts, that is acceptable. For external reports, it is a blocking defect.</p>
<p>The practical workflow is to extract every sentence containing a number and label it verified, unverified, or replace, then keep only verified numbers in anything published. Only verified numbers survive into anything published.</p>
<h2>Cost model</h2>
<p>Pricing: 54 CNY per million input tokens, 163 CNY per million output tokens.</p>
<p>This run: 89 input, 8,085 output.</p>
<ul>
<li>Input: 0.0048 CNY</li>
<li>Output: 1.3179 CNY</li>
<li>Total: 1.32 CNY</li>
</ul>
<p>Output accounted for 99.6% of the cost. That ratio is the lever: if you need to control spend, control output length (summary format, section limits), not input size.</p>
<h2>Where it fits</h2>
<p>The model is a draft generator, not a final-report generator. It compresses the search phase from days to minutes and produces a structure you can edit. It does not remove the verification phase; if anything, the lack of a reference list makes verification more explicit.</p>
<p>Good use cases: pre-project scanning, competitor first drafts, policy summaries, internal "read this before the meeting" documents.</p>
<p>Bad use cases: external publications, academic work with citation requirements, real-time news.</p>
<h2>What I would build around it</h2>
<p>An async job queue, a post-processing step for <code>&lt;think&gt;</code> and citation extraction, a verification list that separates verified numbers from leads, and a human review gate before anything leaves the building. The 1.32 CNY price makes the first two cheap, and the queue is a weekend of work. The third is still the expensive part, and it is the part that determines whether the output is worth using.</p>
<p>Console: <a href="https://bailian.console.aliyun.com/?spm=a2ty07.github">Bailian</a>. Keys: <a href="https://bailian.console.aliyun.com/cn-beijing/?spm=a2ty07.github&amp;tab=app#/api-key">API key page</a>. Beijing region only.</p>
]]></content:encoded></item><item><title><![CDATA[Real-time voice is usually sold as a demo]]></title><description><![CDATA[Real-time voice is usually sold as a demo: a person talks, a model talks back, everyone claps. The engineering question is less glamorous. Where does the…]]></description><link>https://joecien.hashnode.dev/real-time-voice-is-usually-sold-as-a-demo</link><guid isPermaLink="true">https://joecien.hashnode.dev/real-time-voice-is-usually-sold-as-a-demo</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Mon, 19 Oct 2026 10:21:12 GMT</pubDate><content:encoded><![CDATA[<p>Real-time voice is usually sold as a demo: a person talks, a model talks back, everyone claps. The engineering question is less glamorous. Where does the latency come from, what does an interrupt actually stop, and what happens when the model needs to query a database in the middle of a sentence?</p>
<p>I ran four experiments against Bailian's <code>qwen3.8-omni-flash-realtime</code> over WebSocket. The short version: 0.292 s to first text, 0.613 s to first audio, 2.372 s for a full speech-to-speech turn, 71 ms to stop mid-response, and 0.596 s for the model to decide to call an order-lookup tool. Here is the detail behind each number.</p>
<h2>The protocol shape</h2>
<p>The endpoint is a workspace-scoped WebSocket: <code>wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime?model=qwen3.8-omni-flash-realtime</code>. Authentication is a standard Bailian API key in the <code>Authorization</code> header. Input audio is PCM 16 kHz mono; output is WAV 24 kHz.</p>
<p>The session is configured with a <code>session.update</code> event. Four fields matter:</p>
<ul>
<li><p><code>modalities</code>: <code>["text","audio"]</code> or <code>["text"]</code></p>
</li>
<li><p><code>voice</code>: default Tina</p>
</li>
<li><p><code>audio.input.format</code> and <code>audio.output.format</code></p>
</li>
<li><p><code>turn_detection</code>: <code>server_vad</code> for automatic turn detection, <code>null</code> for manual commit</p>
</li>
</ul>
<p>There are two turn modes, and they behave differently enough to matter for product design.</p>
<p><strong>Manual commit</strong> is the request-response shape. Append audio chunks, send <code>input_audio_buffer.commit</code>, then <code>response.create</code>. It is predictable and testable.</p>
<p><strong>Server VAD</strong> watches the stream and decides when speech starts and stops. It is what you want with a real microphone, but it adds a silence-detection step. In my measurements, VAD took 1.17 s from transcription completion to response start; manual commit took 0.47 s.</p>
<h2>Where the latency goes</h2>
<p>Text input, audio output:</p>
<pre><code class="language-plaintext">first text delta   0.292s
first audio delta  0.613s
response.done      0.991s
</code></pre>
<p>Speech input, speech output, 6.96 s of audio:</p>
<pre><code class="language-plaintext">committed        0.137s
ASR completed    0.504s
response.created 0.978s
first audio      1.370s
response.done    2.372s
</code></pre>
<p>The interesting gap is between transcription and response creation: 0.474 s. That is the model reading the transcript, deciding what to say, and starting to generate. The 0.392 s between response creation and first audio is the first audio chunk being synthesized and sent.</p>
<p>If you are building a latency budget, those are the two numbers to attack. Transcription is 0.5 s and largely fixed. Generation-to-first-audio is under 0.4 s and depends on how fast the model commits to an answer. The rest is transport.</p>
<h2>What an interrupt stops</h2>
<p>I let the model answer a question that required a long response, then sent <code>response.cancel</code> at 1.022 s.</p>
<pre><code class="language-plaintext">cancel sent     1.022s
response.done   1.094s
stop latency    0.071s
</code></pre>
<p>Seventy-one milliseconds. The text stopped around 120 characters. No further audio deltas arrived.</p>
<p>This matters because "interruptible" is often implemented as "stop playing the next chunk." That leaves the current sentence running. A realtime session cancels the generation itself, so the model stops producing, not just playing.</p>
<h2>Tool calls inside the conversation</h2>
<p>The tool definition is ordinary JSON schema:</p>
<pre><code class="language-json">{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Look up order status, shipping progress, and return eligibility by order number",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": { "type": "string", "description": "Order number" }
      },
      "required": ["order_id"]
    }
  }
}
</code></pre>
<p>The sequence is:</p>
<ol>
<li><p>Model emits <code>response.function_call_arguments.done</code> with <code>{"order_id":"20261014001"}</code> at 0.596 s.</p>
</li>
<li><p>Client executes the lookup.</p>
</li>
<li><p>Client sends <code>conversation.item.create</code> with type <code>function_call_output</code> and the matching <code>call_id</code>.</p>
</li>
<li><p>Client sends <code>response.create</code>.</p>
</li>
<li><p>Model continues at 0.331 s, first audio at 0.682 s.</p>
</li>
</ol>
<p>The answer contained the delivery status, carrier, tracking number, and return deadline from the tool response. That is the difference between a voice interface and a voice agent: the conversation can pause for a database read and resume with the result.</p>
<h2>Three implementation traps</h2>
<p><code>bl omni</code> <strong>is not realtime.</strong> It is a single multimodal turn over HTTP. No long connection, no session state, no tools. If you try to find realtime in the CLI flags, you will not.</p>
<p><strong>Synthetic audio breaks VAD.</strong> TTS output ends without a natural pause. The server never sees a speech-stopped event; my first run waited 60 seconds. Appending two seconds of silence fixed it. Real microphone input does not need this.</p>
<p><strong>The event name follows the modality.</strong> With <code>modalities=["text"]</code>, the reply arrives as <code>response.text.delta</code>. With audio enabled, it arrives as <code>response.audio_transcript.delta</code>. If you write the receive loop before fixing the modality, you will collect empty strings and blame the model.</p>
<h2>Cost model</h2>
<p>The billing units are tokens, not minutes:</p>
<table>
<thead>
<tr>
<th>Token type</th>
<th>CNY per million</th>
</tr>
</thead>
<tbody><tr>
<td>Audio input</td>
<td>6</td>
</tr>
<tr>
<td>Text/image/video input</td>
<td>1.5</td>
</tr>
<tr>
<td>Text output</td>
<td>4.5</td>
</tr>
<tr>
<td>Audio output</td>
<td>12</td>
</tr>
</tbody></table>
<p>With audio output enabled, the text portion of the output is not billed twice. A measured speech-to-speech turn (760 input tokens, 115 output tokens) cost about 0.0025 CNY. A two-turn tool call cost about 0.0067 CNY. One minute of continuous conversation lands a little over 0.01 CNY.</p>
<p>For capacity planning, the useful ratios are about 7 input tokens per second of audio and 12.5 output tokens per second of speech. Multiply by your expected talk time, then add tool-call turns.</p>
<h2>What this changes for architecture</h2>
<p>The three-stage pipeline is still the right choice when you need to swap vendors independently. Realtime voice is the right choice when latency and interruption are the product. The trade-off is real: a single session protocol is less replaceable than three HTTP services.</p>
<p>The other shift is operational. A voice prototype used to mean three service activations, three quotas, and three failure modes. It now means one key, one socket, and a token bill you can measure in cents. That changes how cheaply you can test whether voice is worth building at all.</p>
<p>Console: <a href="https://bailian.console.aliyun.com/?source_channel=hh_github">Bailian</a>. Keys: <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">API key page</a>. Beijing region only.</p>
]]></content:encoded></item><item><title><![CDATA[Use Bailian's HappyOyster Adventure model to run the full lifecycle:]]></title><description><![CDATA[World models were the demo of the year. Google's Genie 3 looked incredible and callable by exactly nobody. Then I noticed Alibaba's Bailian had quietly…]]></description><link>https://joecien.hashnode.dev/use-bailian-s-happyoyster-adventure-model-to-run-the-full-lifecycle</link><guid isPermaLink="true">https://joecien.hashnode.dev/use-bailian-s-happyoyster-adventure-model-to-run-the-full-lifecycle</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Mon, 19 Oct 2026 09:43:23 GMT</pubDate><content:encoded><![CDATA[<p>World models were the demo of the year. Google's Genie 3 looked incredible and callable by exactly nobody. Then I noticed Alibaba's Bailian had quietly listed HappyOyster, the model behind China's viral "happy oyster" product, as three API-callable models on September 17. Published prices and full docs. So I handed the job to my agent: build a world, poll it, mint a ticket, inspect it, delete it. Total cost: about $0.04. This is what broke.</p>
<blockquote>
<p>Use Bailian's HappyOyster Adventure model to run the full lifecycle: create a first-person dusk-coastal world, poll until ready, mint a travel credential, inspect detail and list, then delete it and confirm the list total returns to zero. Report what each step returns.</p>
</blockquote>
<p>For the record, the category: chat models hand you an answer, text-to-video hands you a finished clip; HappyOyster hands you a persistent place you can walk into, query, and fork.</p>
<h2>The setup</h2>
<p>Architecture first, because it defines the boundaries: a REST Open API manages world lifecycles server-side (auth = ordinary Bailian API key in a Bearer header); a client SDK over RTC handles the live experience. The agent could reach everything in the first half and nothing in the second. The six calls form one pipeline: create, poll, mint a ticket, manage, delete. Together they are the backend for a world users can walk into.</p>
<p>Creation is a POST to <code>{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v2/apps/happyoyster-1.0-adventure/openapi/v1/worlds</code> with four required fields: <code>perspective</code>, <code>prompt</code>, <code>firstFrameImage</code> (landscape, ratio 1.5–2.0, under 6 MB; I generated a dusk coastal highway at 1.75), and <code>eventStyle</code>. It returned in 2.1 seconds with an <code>encryptedWorldId</code> and <code>generating</code> status.</p>
<p>Two of the other models matter for planning: <code>directing</code> (long streaming video with mid-generation instructions) is only deployed in Singapore/Virginia (my Beijing probe got an HTTP 403 at the gateway), and <code>acting</code> is officially invitational-beta. Adventure is the door that's actually open. Pricing note: directing costs 2 yuan per creation without a reference image versus 0.05 yuan with one. The first frame does 40× of the imaginative work, apparently.</p>
<h2>Four potholes, one trap</h2>
<ol>
<li><p><strong>The response envelope lies by omission.</strong> Docs show <code>{code, data}</code>; the wire carries <code>{request_id, output:{code, message, data}}</code>. My first parser returned empty strings.</p>
</li>
<li><p><strong>Inconsistent envelopes across endpoints.</strong> Five of six responses wear the <code>output</code> wrapper; the ticket endpoint returns bare <code>{code, message, data}</code>. No generic parser survives this.</p>
</li>
<li><p><strong>Detail is query-parameter only.</strong> The path form <code>/worlds/{id}</code> returns a Spring-style 404 body, a third error format unrelated to business codes.</p>
</li>
<li><p><strong>Deletion is POST, not DELETE.</strong> <code>DELETE /worlds/{id}</code> → 404. The docs want <code>POST /worlds/delete</code> with the ID in the body. One more create-body field worth knowing: <code>refWorldId</code> derives a new world from one you already own, same account and model only. The fork primitive for UGC is already wired in.</p>
</li>
</ol>
<p>The trap: my negative test (missing <code>perspective</code>) returned <strong>HTTP 200</strong> with <code>code: 400000, "perspective is required"</code> in the body. Errors live inside success responses. Your client needs three layers: HTTP status, business code, shape detection.</p>
<h2>What worked beautifully</h2>
<p>The build took <strong>19.5 seconds</strong> in total, and my first poll, four seconds in, already said <code>ready</code>. Polling itself costs nothing. The list endpoint paginates with <code>pageIndex</code> and <code>pageSize</code> and echoes full creation metadata, enough to run a world-workshop admin backend on. And the response carried a <code>name</code> field: the model had named my world "黄昏海岸" (Dusk Coast) without being asked. Small touch, memorable.</p>
<p>The prompt philosophy is worth stealing for any generative product: <strong>what's in the frame is what you can play</strong>. My image had a vintage car; the prompt ends with "the road is clear and you can accelerate." Reactions you want to see get written in explicitly ("sparks when the arrow lands"), and the model performs them.</p>
<h2>The honest boundary</h2>
<p>Server-side ends at minting a <code>tk_</code> ticket (1800 s, single-use) for the client SDK. A Travel, the live session, is only created when a client actually enters the room. No entry, no stream, no artifacts. Adventure's movement commands don't exist on the Open API side at all. Between "world built, ticket minted" and "seeing the picture" lies a river you cross with client code, not the Open API. The docs are refreshingly honest about it.</p>
<h2>The bill</h2>
<p>Creation: 0.05 yuan (~\(0.007) per world. Experience: 0.2 yuan/second (default 60 s, SDK-capped at 120, about \)3.40 max per session). A note on the meter: the default 60-second session costs 12 yuan; the 120-second ceiling tops out at 24 yuan. Budget per session, not per token. My first frame: ~0.2 yuan. Polls and reads: free. Total for a complete build-and-delete cycle: <strong>about $0.04</strong>.</p>
<p>One caveat: world-model usage never shows up in the console's model-usage module; the docs point you to the billing page. If you're building usage alerting, count calls yourself.</p>
<p>Worth your afternoon? Not a coder? The consumer app already exists; the API is for pulling that experience into your own product. If you ship interactive drama, AI companions, or UGC world workshops, the answer is absolutely. It's the only real-time world model callable by API in China today, with fork primitives (<code>refWorldId</code>) already wired in. And at under a cent per trial, curiosity is cheaper than the coffee you'll drink while the world builds.</p>
<p>To run it yourself: keys are issued in the console API key tab (<a href="https://bailian.console.aliyun.com/cn-beijing/?source%5C_channel=key%5C_github&amp;tab=app#/api-key">https://bailian.console.aliyun.com/cn-beijing/?source\_channel=key\_github&amp;tab=app#/api-key</a>), and the CLI install page ships the official skill bundle you can hand to your agent (<a href="https://bailian.console.aliyun.com/cli?source%5C_channel=cli%5C_github&amp;">https://bailian.console.aliyun.com/cli?source\_channel=cli\_github&amp;</a>).</p>
]]></content:encoded></item><item><title><![CDATA[A 46ms Classifier With Zero Output Tokens, and the Calibration Trap in Its Docs]]></title><description><![CDATA[Quick field notes on decision-model-preview, the model Alibaba's Bailian platform listed on September 24 with a spec-sheet oddity: maximum output length 0…]]></description><link>https://joecien.hashnode.dev/a-46ms-classifier-with-zero-output-tokens-and-the-calibration-trap-in-its-docs</link><guid isPermaLink="true">https://joecien.hashnode.dev/a-46ms-classifier-with-zero-output-tokens-and-the-calibration-trap-in-its-docs</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Mon, 12 Oct 2026 07:39:15 GMT</pubDate><content:encoded><![CDATA[<p>Quick field notes on <code>decision-model-preview</code>, the model Alibaba's Bailian platform listed on September 24 with a spec-sheet oddity: maximum output length 0. It's a decision model built on the TypeSafe System One protocol. No text generation, no parsing step. You post a <code>state</code> (the thing to judge) plus <code>questions</code>; one forward pass returns probabilities, confidences, and classifications.</p>
<p>I tested it with a 20-sample Chinese comment-moderation set (5 groups: overt ads, overt abuse, sarcastic borderline, obfuscated variants, normal discussion), each sample asked two questions in one call (<code>violation</code> as noul, <code>category</code> as choice), and ran the same 20 through qwen3.8-flash JSON mode as a control group. 23 decision-model calls total, all logged. The systemone endpoint sits outside the <code>bl</code> CLI surface, so the calls went through raw HTTP.</p>
<h2>What worked</h2>
<ul>
<li><p><strong>Latency</strong>: server-side p50 46.3ms, with 18/20 calls in a 41.9-52.5ms band. Two tail spikes (1,835ms, 2,370ms), so plan SLA math around the 10% long tail. Client-side p50 including TLS is 237.9ms; quote the right number.</p>
</li>
<li><p><strong>One call, three answers</strong>: <code>choice</code>, <code>noul</code>, and <code>score</code> mix freely in a single request. My triage payload (which team / escalate now / severity 1-4) returned in 48.7ms with all three. The severity answer hedged properly, probability mass 0.75/0.24 across levels 2 and 3, "solidly level 2, some chance of level 3" instead of false certainty.</p>
</li>
<li><p><strong>Content questions are reliable</strong>: noul and score judge the content itself (is this urgent, how bad) and behaved all day. The escalation flag fired where it should.</p>
</li>
<li><p><strong>The control group is fine</strong>: flash JSON mode parsed 20/20 with valid fields. The "JSON mode is fragile" discourse overstates it. The difference is speed and shape, not usability.</p>
</li>
</ul>
<h2>What will bite you</h2>
<ol>
<li><p><strong>Conservative calibration</strong>. Through the textbook funnel (0.9 or higher auto, 0.6-0.9 review, below 0.6 pass), zero samples cleared auto-resolve. The model presses probabilities down: a blatant WeChat spam ad scored 0.87, but an equally explicit abuse comment scored 0.32. Doc-example thresholds are demo props. Calibrate on your own samples or your automation rate is 0%.</p>
</li>
<li><p><strong>Self-disagreement</strong>. noul and choice conflicted (0.5 boundary) in 6/20 samples. Best one: an innocent streaming-support question scored noul 0.52 while choice ranked <code>normal</code> first. Treat 0.5-ish as "the model is unsure," route to humans.</p>
</li>
<li><p><strong>Quantized probabilities</strong>. Two decimals; 9/20 responses sum to 0.99 or 1.01. Exact-sum validation false-alarms on 45% of healthy responses. Tolerate the drift.</p>
</li>
<li><p><strong>Asymmetric variant failures</strong>. Emoji-homophone spam caught (0.79); mixed-language "威信 same as 上面" confidently released (0.05). A confident pass bypasses every queue. Keep a keyword coarse-filter or sampled human review as a second layer.</p>
</li>
<li><p><strong>Criteria wording steers choice</strong>. A payment-arrival ticket went to the catch-all <code>other</code> (0.77) over an explicit <code>billing</code> option (0.21). Fuzzy cases flow to the least-specific bucket; keep option descriptions at equal granularity. Worth noting the pattern behind it: noul and score judge your content, choice judges your taxonomy, and the model inherits every flaw in your buckets.</p>
</li>
<li><p><strong>Array state is not batching</strong>. Four comments as an array got serialized into one aggregate judgment (noul 0.63).</p>
</li>
</ol>
<h2>A calibration recipe</h2>
<p>Fifty to one hundred of your own samples, two questions each, full distributions recorded. Price a missed violation against a wrongly suppressed comment and place the cut where that trade is acceptable. Keep the human queue permanently; the threshold tunes the auto lane's width, not whether humans exist. An afternoon of work, and it is the difference between an idle lane and a working one.</p>
<h2>The token ledger</h2>
<p>Same 20 judgments: flash burned 3,603 input + 3,175 output tokens (about ¥0.0115 at ¥0.8/M in and ¥2.7/M out); the decision model used zero output tokens and billed 0 yuan in the free window. The cost <em>shape</em> differs more than the amount. One scales with how much it says; the other doesn't talk.</p>
<p>Auditability checks out too: <code>bl log audit list</code> pulls per-call records (request ID, duration, status, usage). Three undocumented CLI behaviors: the <code>--model</code> filter silently matches nothing for this traffic, <code>--max-results</code> above 20 returns empty with no error, and there's no pagination token (page with <code>--skip</code>). Reconciliation matched client-side records token-for-token: 2,408 input tokens across 23 calls.</p>
<h2>Verdict</h2>
<p>For real-time judgment loops (chat intent detection, live routing, inline checks), 46ms opens doors that 3.9 seconds keeps shut. For offline batches, flash JSON mode remains perfectly valid. Either way, run the calibration first. The interesting pitch here isn't another writing model; it's a model whose value is everything it refuses to say.</p>
]]></content:encoded></item><item><title><![CDATA[A deployment-form divergence, measured across 35 API calls]]></title><description><![CDATA[Most model comparisons measure one build of a model against another. This post is about a rarer and more practical phenomenon: the same model behaving…]]></description><link>https://joecien.hashnode.dev/a-deployment-form-divergence-measured-across-35-api-calls</link><guid isPermaLink="true">https://joecien.hashnode.dev/a-deployment-form-divergence-measured-across-35-api-calls</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Fri, 09 Oct 2026 08:45:44 GMT</pubDate><content:encoded><![CDATA[<p>Most model comparisons measure one build of a model against another. This post is about a rarer and more practical phenomenon: <strong>the same model behaving differently across deployment forms.</strong> The open-weight build of Qwen-Image-2.1 generated clean transparent backgrounds on the first attempt in independent reviews. The hosted API version, tested across 21 controlled attempts, produced a transparent background once. The interesting part is not the discrepancy itself. It is the behavioral rule underneath it, which turns out to be surprisingly clean, and surprisingly useful if your assets already carry an alpha channel.</p>
<h3>Setup</h3>
<p>Platform: Alibaba Cloud Bailian, model <code>qwen-image-2.1-pro</code>, roughly 3.5 US cents per image, billed per output image. No GPU required, which is the point: the local open-weight build needs 27 to 29 GB of VRAM.</p>
<p>Measurement method: after every call, download the PNG and count fully transparent pixels via the alpha channel histogram. "Looks transparent" is not a measurement; a white matte behind a clean cutout looks fine on a dark UI and terrible in production. The code is five lines of Pillow, and it decided every result below.</p>
<h3>Finding 1: transparency generation does not reproduce</h3>
<p>Across 21 attempts spanning five prompt phrasings, two prompt-rewrite modes, two endpoints, and both CLI and raw HTTP paths, exactly one call returned a transparent background, and rerunning its exact configuration twice returned white.</p>
<table>
<thead>
<tr>
<th>Setup</th>
<th>Attempts</th>
<th>Transparent</th>
</tr>
</thead>
<tbody><tr>
<td>CLI, three phrasings</td>
<td>3</td>
<td>0</td>
</tr>
<tr>
<td>HTTP, variant phrasings</td>
<td>5</td>
<td>0</td>
</tr>
<tr>
<td>Agent-rewrite mode, first attempt</td>
<td>1</td>
<td>1</td>
</tr>
<tr>
<td>Re-runs and controls</td>
<td>4</td>
<td>0</td>
</tr>
<tr>
<td>Formal runs, direct mode</td>
<td>4</td>
<td>0</td>
</tr>
<tr>
<td>Formal runs, agent mode</td>
<td>4</td>
<td>0</td>
</tr>
</tbody></table>
<p>A 1-in-21 reproduction rate with no discoverable control variable is not a planning number. For engineering purposes, generation-side transparency on this deployment is unavailable.</p>
<p>Also worth logging: the agent-rewrite mode roughly doubles latency (59 to 70 seconds per call versus 36 to 39 seconds for direct mode) because it runs a second model pass to rewrite the prompt before generation. It bought no transparency advantage in the matrix.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/rgba-lab/transparent-star-1280x720.jpg" alt="21 attempts, one accidental transparent result" style="display:block;margin:0 auto" />

<h3>Finding 2: transparency propagates through edits</h3>
<p>The one successful sample suggested the rule: its input carried an alpha channel. Controlled test: edit instructions that never mention transparency, applied to the same artwork in two versions.</p>
<ul>
<li><p>Transparent input, instruction "add a blue scarf": output 60.4% to 59.6% transparent. Channel preserved through the edit.</p>
</li>
<li><p>White-background input, same instruction: output fully opaque.</p>
</li>
</ul>
<p>The rule is structural, not linguistic. The model carries the input's alpha structure into the output. Prompt content is not the variable. This reframes the capability from "generation" to "propagation," and propagation is stable enough to build on.</p>
<p>Boundary condition, also measured: a compositional rewrite (scale to 90% plus reposition) dropped the channel. Local edits preserve alpha; global relayout does not. Any production pipeline that edits transparent assets should keep geometry stable and constrain edits to local operations.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/rgba-lab/inherit-not-generate-en-1280x720.jpg" alt="Transparency is inherited: no alpha in, no alpha out" style="display:block;margin:0 auto" />

<h3>Finding 3: propagation outranks explicit instructions</h3>
<p>The sharpest result came from the 10-reference-image test. Ten transparent assets in, one instruction out: arrange all shapes in a row on a <em>white background</em>. The output contained all ten shapes, colors and forms intact, on a canvas that auto-expanded to 4096×1024, and the background was 81.5% transparent. The white-background directive lost to the inputs' alpha structure.</p>
<p>For pipeline design this is the load-bearing detail. If you compose transparent assets and need an opaque result, do not plan on prompt-level control. Either use opaque inputs, or post-composite over your own background. The model treats alpha structure as a fact about the input set, not a preference you can talk it out of.</p>
<h3>Design implications</h3>
<p>Three patterns follow from the measurements:</p>
<ol>
<li><p><strong>Transparent-asset editing is production-ready.</strong> Element addition, recoloring, and marked-region edits all hold the channel. Region marking works by drawing directly on the input (a thin outline, named in the prompt), and the model removes the mark afterward. Thin outlines clean up better than thick fills, which leave residue.</p>
</li>
<li><p><strong>Compositing up to ten references works</strong>, with automatic layout, billed as a single output image.</p>
</li>
<li><p><strong>The first transparent asset is an external dependency.</strong> On this deployment, neither generation nor photo cutout produces one reliably (cutout: zero for two, including the vendor's own template prompt). Source it from the open-weight build, an asset library, or any existing alpha-channel file. Everything downstream of that first file works.</p>
</li>
</ol>
<h3>Cost and operability</h3>
<p>35 calls, roughly $1.22 total, auditable from the CLI: <code>bl usage stats</code> for the summary, <code>bl log audit list</code> for per-call records (console login required). The pricing model is per output image, which makes cost forecasting trivial: N assets, N times the unit price. The unglamorous engineering advantage of "boring" capabilities is exactly this: a 3.5-cent call with predictable behavior beats a spectacular call you cannot reproduce.</p>
<h3>Closing thought</h3>
<p>Deployment-form divergence is going to keep biting teams that plan from launch posts. The reviews were not wrong about the local build. The API is not wrong about its behavior either. They are different products wearing the same name, and the only reconciliation that matters is the one you run yourself, with a measurement that maps to your actual acceptance criteria. In this case, five lines of alpha-channel counting told me more than any launch-day thread.</p>
<p>To reproduce any of this: <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">install the Bailian CLI skill package</a> or <code>npm install -g bailian-cli</code>, and <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">sign a key</a> before calling.</p>
]]></content:encoded></item><item><title><![CDATA[Dot froze on stage, so I re-read a CLI's help text]]></title><description><![CDATA[On 2026-09-30 OpenAI announced Dot, an always-on agent: GPT-6 Astra, its own cloud computer, a dedicated browser, more than 4000 app integrations through…]]></description><link>https://joecien.hashnode.dev/dot-froze-on-stage-so-i-re-read-a-cli-s-help-text</link><guid isPermaLink="true">https://joecien.hashnode.dev/dot-froze-on-stage-so-i-re-read-a-cli-s-help-text</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Wed, 30 Sep 2026 06:27:55 GMT</pubDate><content:encoded><![CDATA[<p>On 2026-09-30 OpenAI announced Dot, an always-on agent: GPT-6 Astra, its own cloud computer, a dedicated browser, more than 4000 app integrations through plugins, sold as a virtual employee that reports progress by itself and opens pull requests. The same day on the conference stage, the presenter asked it to prepare launch work for a fictional music app. It never answered. The demo was paused, and the host covered with "I guess Dot is a bit slow this morning." OpenAI gave no explanation, and I have none, so I will not guess out loud.</p>
<p>What the pairing changed for me is smaller and more useful. A product that stands rather than answers is a set of standing resources, and a resource set needs infrastructure treatment: declared config, a pre-flight check, a version chain, a change preview. So I spent a morning inside the command group on Bailian's CLI that claims all four, <code>bl managed-agent</code> on <code>bl</code> 2.1.0: 72 subcommands, 7 resource types (Agent, Environment, Skill, Vault, Deployment, Session, File). I upgraded from 2.0.1 first and re-read the command surface, which turned out to matter. Bill at the end: 0 yuan. Cloud resources created: 0.</p>
<h2>The offline surface was bigger than I assumed</h2>
<p>I expected to need an account and a provisioning budget before I could inspect any structure. Wrong, and that is the surprise I keep. 15 of the 72 subcommands are tagged <code>[No Auth]</code>, and those 15 cover the whole interesting path: scaffolding, validation, the project commands, the version commands and state inspection. I ran declaration, validation, version reading, preview, rollback and state checks without logging in once.</p>
<p>The config side holds up offline too. <code>bl managed-agent init</code> writes <code>agents.yaml</code> and a <code>.gitignore</code>. Secrets stay as <code>${DASHSCOPE_API_KEY}</code> interpolation, never literals, and the ignore file blocks the three paths a teammate commits by accident: <code>agents.state.json</code>, <code>.openagentpack/versions/</code>, <code>.env</code>.</p>
<h2>A validator states its boundary in one line of help</h2>
<p>The help for <code>bl managed-agent validate</code> says: <strong>Runs fully offline against local files: no login or provider credentials required.</strong> I built 10 broken configs and fed them one at a time. It stopped 6 kinds and passed 4.</p>
<p>What it stops is shape and cross-reference: a missing required field, an enum value out of range (<code>networking.type: open-internet</code>, <code>config.type: onprem</code>), YAML that will not parse, an unset environment variable, a reference to an environment the same file never defines. Exit codes split the same way: 0 valid, 1 for semantic diagnostics carrying a bracketed error code, 2 for schema and parse rejection with no code prefix.</p>
<p>What it passes without a word is existence in a catalog: <code>model: qwen3.8-max-not-real</code>, a tool named <code>teleport</code>, any value at all in <code>version</code> (<code>"0"</code>, <code>"2"</code>, <code>"99"</code> all accepted), and <code>agents: {}</code>.</p>
<p>The lesson is about how I read validators, not about Bailian. I treat a green check as a property of the checker, when it is a property of what the checker can reach. That offline promise explains all four passes. Write CI against the boundary, not the green.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/managed-agent/validate-boundary-en-1280x720.jpg" alt="The validator stops six kinds of break and passes four" style="display:block;margin:0 auto" />

<h2>What I read wrong, and what is still open</h2>
<p>My own error was about <code>jq</code>. The failing run looked like it printed two JSON objects, and I concluded the command had contaminated its own machine output. That was my mistake. The <code>{"error": {...}}</code> envelope goes to stderr, stdout holds a valid single JSON document even on failure, and <code>bl managed-agent validate --output json | jq .diagnostics</code> works in a pipeline. Separating the streams is a two second fix that cost me a wrong paragraph in my notes.</p>
<p>The one still open is the second checker. <code>bl managed-agent project validate --output json</code> returns <code>project_root</code>, <code>project_revision</code>, <code>source_manifest_hash</code>, <code>yaml_hash</code>, <code>diagnostics</code>, <code>warnings</code>, <code>organization_moves</code>, <code>after_yaml</code>, with no <code>valid</code> key anywhere in it. Two validators in one group, two JSON shapes, so one CI parser cannot serve both.</p>
<h2>Rollback worked, and its own help says where it stops</h2>
<p>The directory project keeps versions in content-addressed storage: <code>blobs/</code>, <code>entries/</code>, <code>manifests/</code>, every name a SHA-256. A version entry carries <code>version_id</code>, <code>short_version</code>, <code>parent_version</code>, <code>tree_hash</code>, <code>yaml_hash</code>, <code>manifest_hash</code>, <code>message</code>, <code>created_by</code>, <code>created_at</code>, <code>nonce</code>. Git's vocabulary aimed at agent config.</p>
<p>So I set <code>model</code> to <code>broke-during-iteration</code>. <code>source_status</code> moved from <code>clean</code> to <code>modified</code>, <code>version preview</code> rendered <code>before_yaml</code>, and <code>version restore --version-id &lt;id&gt; --yes</code> put the file back: model returned to <code>QWEN-CUSTOM-MODEL</code>, status back to <code>clean</code>. A real rollback, offline and free.</p>
<p>Then I read the risk annotation.</p>
<pre><code class="language-plaintext">Risk: high
Risk message: This restores the full directory source to the working tree.
              Version history and remote State will not move.
</code></pre>
<p>Version history does not move and remote state does not move. My tree was at the old revision while a deployed agent would still be running the newer config. Local rollback is not production rollback, and that line sat in help text I had already scrolled past once. <code>apply</code> closes the gap, and it changes resources for real.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/managed-agent/version-chain-en-1280x720.jpg" alt="A content-addressed version chain and the restore path" style="display:block;margin:0 auto" />

<h2>What I would do differently next time</h2>
<p>Pass the flag that migrates my work explicitly. <code>bl managed-agent project init</code> without <code>--project .</code> reported <code>converted_from_yaml: false</code>, built a stock project in a fresh <code>managed-agent/</code> subdirectory, and dropped my customized model and description without saying so. With <code>--project .</code> it reported <code>true</code> and carried the config over. Now I read the conversion fields before trusting any generated directory.</p>
<p>Check every entry point's defaults, because they drift: two scaffolds in one CLI, <code>qwen3.8-max</code> in the YAML template and <code>qwen3.7-max</code> in the directory project.</p>
<p>Do not read a build as a commit. <code>project build</code> left the chain at 1 entry, and recording a version takes <code>publish</code>, which takes a key. The free zone ends at reading history, previewing and restoring.</p>
<p>Keep the empty-state check as the audit. Right after <code>plan</code> printed <code>+ environment.dev (bailian)</code>, <code>+ agent.assistant (bailian)</code> and <code>Plan: 2 to create, 0 to update, 0 to destroy</code>, <code>bl managed-agent state list</code> returned <code>No resources tracked in state.</code> That empty line is my 0 yuan receipt.</p>
<p>One check I now add myself. A <code>source</code> pointing at <code>./definitely-missing.md</code> passed <code>project validate</code> with <code>Project is valid (5e5ad44f0ec6)</code>. A local path exists or it does not, and that is offline work. The same tool flags an unknown skill reference instantly as <code>config.agent.skill.unknown</code>, so the capability is there. Coverage, not capability.</p>
<h2>Where my claims stop</h2>
<p><code>apply</code>, <code>publish</code>, <code>session create</code> and <code>deployment run</code> stayed unexecuted, because they create persistent resources and bill. So I cannot tell you how long provisioning takes, what <code>session run</code> streaming feels like, how much latency a <code>deployment</code> schedule adds, or whether a bad model name gets rejected at <code>apply</code>. I also do not know why that demo hung. What the two stories share is thinner than it looks: "config is valid" and "it stays up" are two questions, and a validator cannot answer the second.</p>
<p>To run it yourself, <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">install the Bailian CLI skill package</a> or <code>npm install -g bailian-cli</code>, and only <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">sign a key</a> when you reach <code>plan</code> or <code>apply</code>; 15 of those 72 commands need neither.</p>
<p>The habit worth keeping is the break matrix: ten edits to your own agent config, one command, twenty minutes, no cost. Write down which breaks passed. That list, not the feature table, is what your pipeline is exposed to.</p>
]]></content:encoded></item><item><title><![CDATA[One call makes a full sound scene. Two limits decide whether it ships.]]></title><description><![CDATA[The demo shows a 298-character script becoming 32 seconds of finished radio drama: three speaking characters, rain on an awning, a kettle in the kitchen…]]></description><link>https://joecien.hashnode.dev/one-call-makes-a-full-sound-scene-two-limits-decide-whether-it-ships</link><guid isPermaLink="true">https://joecien.hashnode.dev/one-call-makes-a-full-sound-scene-two-limits-decide-whether-it-ships</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Mon, 28 Sep 2026 07:08:36 GMT</pubDate><content:encoded><![CDATA[<p>The demo shows a 298-character script becoming 32 seconds of finished radio drama: three speaking characters, rain on an awning, a kettle in the kitchen, all sequenced in the order the writer set them. A 439-character script comes back as a 70-second two-host podcast. The audio holds up on headphones, and versions of that demo have been circulating since the model listed on 09-21.</p>
<p>What decides whether this belongs in a pipeline sits elsewhere. I ran eight generations over half a day, and the picture that matters is two limits with opposite failure modes and one meter that turns out to be honest about cost.</p>
<h2>The interface: synthesis over HTTP, surrounding work in the CLI</h2>
<p>Setup and the surrounding work live in the CLI. Synthesis itself does not route through it yet, and the failure is specific:</p>
<pre><code class="language-bash">bl speech synthesize --text "雨突然下大了。要伞吗？" \
  --model qwen-audio-3.1-tts-next --voice longanhuan_v3.6 --format wav --out smoke.wav
</code></pre>
<p>That returns <code>HTTP 400: text_prompt must not be empty.</code> The command builds the classic synthesis shape, <code>input.text</code> plus <code>voice</code>, while the new model expects <code>input.text_prompt</code> and rejects <code>voice</code> outright. Pointing the same command at the older generation of the model worked on the first try. The model works; the CLI path has not caught up with this interface yet.</p>
<p>Synthesis goes over the official endpoint, one POST with the script in <code>text_prompt</code>:</p>
<pre><code class="language-bash">curl -sS -X POST \
  "https://$WORKSPACE_ID.cn-beijing.maas.aliyuncs.com/api/v1/services/audio/tts/SpeechSynthesizer" \
  -H "Authorization: Bearer $DASHSCOPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-audio-3.1-tts-next",
    "input": {
      "text_prompt": "深夜面馆广播剧。环境声：雨点密集打在雨棚上。老板：这么晚才来。还是老三样？",
      "format": "wav",
      "sample_rate": 48000,
      "channels": 2
    }
  }'
</code></pre>
<p>Three response fields carry the day-to-day weight: <code>output.audio.url</code> (the finished file, valid for 24 hours), <code>output.audio.duration</code>, and <code>usage</code> for the call. Fixed voices come from reference audio: up to three clips referenced as <code>@voice1</code> through <code>@voice3</code>, each up to 30 seconds and 10MB. And the check that matters most is read-back: <code>bl speech recognize</code> transcribes the finished wav, so a missing line shows up as text instead of a guess.</p>
<h2>Two limits, two failure modes</h2>
<p>The documentation states two limits: prompts under 3,000 characters, and finished audio capped at 240 seconds for podcasts and 120 seconds for other scenes. One fails loudly, the other does not.</p>
<p>A 3,058-character script returned <code>text_prompt exceeds the maximum length of 3000 characters.</code> in 0.2 seconds. Annoying, and safe: the error is legible, and the fix is in the script.</p>
<p>A 469-character monologue, which at 0.16 seconds per character should land near 75 seconds, came out at exactly 120.0 seconds with the last sentence gone. The response record's <code>finish_reason</code> still said <code>stop</code>, and the bill charged the full 120 seconds, about CNY 0.30.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-scene/cut-at-120-seconds-en-1280x720.jpg" alt="A silent cut at 120 seconds" style="display:block;margin:0 auto" />

<p>For a pipeline, that is a data-integrity problem rather than a quality problem. The artifact looks complete, the metadata reports complete, the invoice agrees, and the ending is missing. Audiobook work, where scripts run long and the cut lands silently, is exactly where this bites. The defense is arithmetic before generation (0.16 seconds per character; 0.12 to 0.15 for dialogue-heavy copy), splitting anything close to the cap, and reading every file back.</p>
<h2>The meter: 200 output tokens per generated second</h2>
<p>Eight runs held one rule without exception: every generated second records 200 output tokens. 6.6 seconds recorded 1,320; 32.08 recorded 6,416; 69.96 recorded 13,992; 120 recorded 24,000. At the catalog prices (CNY 6 per million input tokens, CNY 12 per million output), that is about CNY 0.0024 per finished second. A four-minute podcast lands near CNY 0.60, and this whole session, 289 seconds across eight generations, came to about CNY 0.77.</p>
<p>The catalog quotes in yuan and I am leaving the figures there rather than pretending a conversion; the linear shape is the portable part.</p>
<p>The free allowance runs 90 days, per model, Beijing region only, and a million output tokens cover roughly 83 minutes of audio. A prerequisite before the accounting: <code>bl usage free</code> and <code>bl log audit list</code> are tagged <code>[Console]</code> in the CLI help, so they need <code>bl auth login --console</code> in a browser; synthesis and read-back run on an API key, and the two are separate credentials. Per-call accounting goes through <code>bl log audit list --output json</code>, not the quota panel, which lags behind the calls. Two findings from the same runs are worth keeping. The speaking-rate parameter does not save money: <code>rate=1.6</code> took 9 seconds of audio down to 5.3 physical seconds while the billed output moved only from 1,808 to 1,688 output tokens (down 6.6%). And runs are not reproducible: the same prompt with the same parameters, seed included, produced files with different md5 hashes. Archive the take you like; a rerun is a new performance.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-scene/cost-per-second-en-1280x720.jpg" alt="Cost per second of finished audio" style="display:block;margin:0 auto" />

<h2>What I would build on it, and what I would not</h2>
<p>The value today is the first draft. A podcast or audiobook episode generated for under CNY 0.60 is a cheap way to hear where the pacing sags and which transitions land badly, before any of it is worth a studio. Concept checks for game NPC lines and short-drama sound scenes come out as one call each, which beats describing "there should be rain and footsteps here" in a design document. Bulk voice-over is just a loop over a standard HTTP interface.</p>
<p>Radio-grade emotional performance is not here yet; the grain of voice and emotion is still a few steps off. Precisely-timed delivery needs its own cushioning because of the silent cut. And if you only voice something occasionally, a console product with a submit button involves less plumbing than an endpoint does.</p>
<p>The split that stays with me after the eight runs: the meter and the read-back are honest, the metadata is not. Design around that split, and the current form absorbs the slowest part of an audio workflow, which is the first listen. Three finished pieces are still up if you want to hear the quality before spending anything: the 32-second <a href="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-scene/noodle-radio-drama-32s.wav">noodle-shop drama</a>, the 37-second <a href="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-scene/voice-clone-bus-37s.wav">voice transfer</a>, and the 70-second <a href="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-scene/podcast-70s.wav">podcast</a>.</p>
<p>To run it yourself, <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">install the Bailian CLI skill package</a> or <code>npm install -g bailian-cli</code>, then <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">sign an API key</a> for synthesis.</p>
]]></content:encoded></item><item><title><![CDATA[Audio model pricing: normalize the meter before you compare the bill]]></title><description><![CDATA[A Chinese cloud vendor shipped a batch of speech models this week and cut list prices across the line, with coverage quoting reductions up to 95%. I wanted…]]></description><link>https://joecien.hashnode.dev/audio-model-pricing-normalize-the-meter-before-you-compare-the-bill</link><guid isPermaLink="true">https://joecien.hashnode.dev/audio-model-pricing-normalize-the-meter-before-you-compare-the-bill</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Thu, 24 Sep 2026 10:33:07 GMT</pubDate><content:encoded><![CDATA[<p>A Chinese cloud vendor shipped a batch of speech models this week and cut list prices across the line, with coverage quoting reductions up to 95%. I wanted one number for a narration job and a transcription backlog, and the price sheet refused to give me one. The cause is measurement rather than pricing, and it recurs wherever a sheet mixes per-character, per-second and per-token lines.</p>
<p>Three billing units coexist in the same catalog. Per 10,000 characters of input text, per second of audio, per million tokens. Ranking 1.5, 0.8 and 0.00022 against each other compares three different physical quantities, and no write-up of the 95% said which unit the percentage moved in. So I rebuilt the comparison from per-call usage instead of from the sheet.</p>
<h2>The measurement design</h2>
<p>One script, two generations of synthesis models. One recording, three generations of transcription models. Then the usage each call actually recorded, converted into money per minute of delivered audio, since duration is the one quantity both sides of the transaction agree on.</p>
<pre><code class="language-bash">bl speech synthesize --text-file text-zh.txt --model qwen-audio-3.1-tts-flash --voice longanhuan_v3.1 --format wav --out new.wav --output json
bl speech recognize --url new.wav --model qwen-audio-3.1-asr-flash --output json
bl log audit list --hours 1 --model qwen-audio-3.1-tts-flash --output json
</code></pre>
<p>Every command in this layer is tagged <code>[Console]</code> in <code>bl</code>'s help, so it needs <code>bl auth login --console</code> rather than the API key that runs synthesis. The audit log turned out to be the only surface that returns usage at single-call granularity. <code>bl usage stats --model &lt;model&gt; --days 1 --output json</code> came back with empty <code>items</code>, <code>bl usage stats --days 1 --type Audio --output json</code> reported <code>modelsCalled: 0</code>, and both <code>bl monitor metrics</code> variants returned empty <code>series</code>. Account-level metrics exist but aggregate every call in the window, so one audio request cannot be isolated.</p>
<p>The workaround people reach for next, snapshotting an account-level figure before and after a run, fails for a quieter reason. <code>--days 1</code> is a sliding window: <code>input_tokens</code> moved from 470537 to 443114 across a period in which I issued no text calls at all. A baseline that drifts makes every derived delta meaningless.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-price-cut/where-to-see-usage-en-1280x720.jpg" alt="Four routes to usage data, one that returns per-call numbers" style="display:block;margin:0 auto" />

<h2>What the numbers looked like after normalization</h2>
<p>The script is 133 characters of Chinese, 120 characters plus 13 punctuation marks, and billing recorded 253. Chinese scores 2 per character and punctuation 1, so 120 x 2 + 13 lands exactly on the metered figure. A 240 character English line recorded 240, with no multiplier. Any budget built from a word count on CJK text is out by roughly a factor of two.</p>
<p>Same script, 24.88 seconds of audio out:</p>
<table>
<thead>
<tr>
<th>Synthesis run</th>
<th>That one call</th>
<th>CNY per 10,000 characters</th>
<th>CNY per minute of audio</th>
</tr>
</thead>
<tbody><tr>
<td>qwen-audio-3.1-tts-flash</td>
<td>0.0039315</td>
<td>0.1554</td>
<td>0.0095</td>
</tr>
<tr>
<td>cosyvoice-v3-flash</td>
<td>0.0253</td>
<td>1.0000</td>
<td>0.0610</td>
</tr>
</tbody></table>
<p>6.44x for identical work. Same recording, three transcription models: 0.0004996 CNY on <code>qwen-audio-3.1-asr-flash</code> against 0.00528 on <code>qwen3-asr-flash</code> and 0.00506 on <code>fun-asr</code>, a 10.57x gap on that one call, or 0.0012 versus 0.0132 CNY per minute of audio. I am keeping these in Chinese yuan because that is the currency the catalog quotes in; the ratio is the portable part.</p>
<p>Two more things constrain how far that table can be pushed.</p>
<p>Tokens are not a unit of work. That one recording registered 368 input tokens on one model and 622 on another, 14.72 against 25.92 tokens per second, a density gap of 1.76x. A per-token price therefore says nothing about per-token consumption, and "billed per token, so cheaper" fails as an inference. The same file was also metered as 25, 24 and 23 seconds by the three models, since silence trimming and rounding differ.</p>
<p>Cost is concentrated on the generated audio. Of the new model's bill, 94.93% sat on the output side: 133 input tokens at 1.5 CNY per million against 311 output tokens at 12 CNY per million, 18.7x. Trimming a script from 133 characters to 100 barely moves the invoice; trimming runtime moves it proportionally. In a pipeline this flips the usual optimization: cache and reuse audio, segment long copy, and treat text length as a rounding term.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/audio-price-cut/cost-sits-on-output-en-1280x720.jpg" alt="Almost all of the new model's cost sits on the output side" style="display:block;margin:0 auto" />

<h2>Three implementation notes that cost me time</h2>
<p>Detail queries return a family view, and <code>items[0]</code> is not necessarily the version you asked for. I read <code>v3.5-flash</code> at 0.8 CNY as the price of <code>cosyvoice-v3-flash</code>, computed a 5.15x saving, and later found the correct 1 CNY per 10,000 characters, which made it 6.44x. Match on <code>items[].model</code>. Family names do not resolve at all (<code>qwen-audio-3.1-tts</code> returns <code>not found</code>), and there is no fuzzy fallback.</p>
<p>Repeated synthesis with identical parameters returns identical bytes: both wavs hashed to <code>7a8e577b9515ca954f4d18afd1841439</code>. <code>--seed</code> (0-65535) exists but is not a precondition for reproducibility, which is the opposite of what image models taught most of us. Retries therefore have no hidden quality tax.</p>
<p><code>--vocabulary</code> works on <code>fun-asr</code>, whose help text states it takes effect only on Qwen-Audio-3.0-ASR-Flash models. On the same audio, two invented proper names went in and one came out fixed in both positions, with the sentence-final punctuation correcting itself; the other stayed wrong at weight 5. Wider support than documented, and no guarantee attached to a higher weight.</p>
<p>One hazard for duration-based estimation: the wav header the CLI writes carries a bogus frame count. Python's <code>wave</code> reported 44739.24 seconds for a 24.88 second file, and only the byte count gave the truth (1194284 minus a 44 byte header, divided by 24000 x 2). Both model generations produce the same broken header.</p>
<h2>Free quota terms, stated exactly</h2>
<p>Each of the two 3.1 models carries 1,000,000 free tokens, expiring 2026-12-21. Official terms: one allowance per model, 90 days, Beijing region only, no reissue on expiry, no automatic switch to another model when exhausted. The widely repeated claim that the 1,000,000 tokens are permanently valid is wrong. <code>paraformer-v2</code> shows 36000 seconds instead. My batch cost 0 CNY because it fit inside the allowance.</p>
<p>To run this yourself, <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">install the Bailian CLI skill package</a> or <code>npm install -g bailian-cli</code>, then <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">sign an API key</a> for synthesis and transcription.</p>
<p>I measured price, and not transcription quality, so accents, noise and overlapping speakers remain untested on my side. This is also not a comparison with vendors outside the catalog, since the only place I could put both sides of a measurement onto the same file was inside it.</p>
<p>What transfers here is an order of operations rather than a model recommendation: identify the unit, convert to a quantity your own workload is measured in, then compare. Half an hour with one text file and one <code>bl log audit list</code> is enough to check any price claim you are being asked to believe.</p>
]]></content:encoded></item><item><title><![CDATA[The catalog showed zero. The API showed 155: auditing 179 retiring models before October 10]]></title><description><![CDATA[A model retirement notice landed in my feed: 179 model entries on Alibaba Cloud Model Studio are being retired on October 10. Our team runs three services…]]></description><link>https://joecien.hashnode.dev/the-catalog-showed-zero-the-api-showed-155-auditing-179-retiring-models-before-october-10</link><guid isPermaLink="true">https://joecien.hashnode.dev/the-catalog-showed-zero-the-api-showed-155-auditing-179-retiring-models-before-october-10</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Wed, 23 Sep 2026 06:41:12 GMT</pubDate><content:encoded><![CDATA[<h2>Why I started checking</h2>
<p>A model retirement notice landed in my feed: 179 model entries on Alibaba Cloud Model Studio are being retired on October 10. Our team runs three services with model calls, all with the model name hardcoded in config. And the notice policy made me pause: SMS and email alerts only reach accounts that called an affected model in the last three months. We had been quiet for about that long.</p>
<p>So I treated it as a drill. Audit every model name we depend on, figure out the migration path, and write down what I learned. The full run took about a day, and most of the value came from the surprises.</p>
<h2>What the catalog actually shows</h2>
<p>The CLI installs in one line, and audit commands need no authentication:</p>
<pre><code class="language-bash">npm install -g bailian-cli
bl model list --model qwen3-max --output json
</code></pre>
<p>Look for <code>upcomingOfflineAt</code> in the response. If it has a value, that version is counting down.</p>
<p>The first surprise: browse the whole catalog without <code>--model</code>, and the retirement markers disappear entirely. All 179 groups came back clean. The marker only exists in the detail view of each family, or in search results with <code>--include-deprecated</code> set. A clean sweep of the listing page means nothing here.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-sunset/query-modes-en-1280x720.jpg" alt="Three query modes, three different views" style="display:block;margin:0 auto" />

<h2>The sweep that missed 42 models</h2>
<p>I wrote a loop over every family name and got 113 entries. Cross-checking against search mode showed 155. The 42 missing ones were not obscure: the open-weight Qwen3 line, DeepSeek R1/V3, Kimi, CosyVoice. Their family names are not queryable even though they appear in the listing. An individual name like <code>qwen3-32b</code> resolves and returns the whole family, which is how I found the fallback: try detail first, drop to search when it fails. Final count 155, matching the catalog exactly, in about four minutes.</p>
<p>Fifty-eight of the group labels never resolve by name at all: some are category tags like <code>qwen-audio-3.1-asr</code>, others are family names like <code>qwen3</code>. A search sweep across ninety-odd keywords, by contrast, returns 517 records, 155 of them flagged. The single keyword <code>qwen3</code> surfaces 62. That gap is why the fix falls back to search instead of trusting the detail view.</p>
<p>Reconciled against the official notices: 179 = 155 with countdown + 9 already flagged offline + 9 international <code>-us</code> variants + 6 with no catalog trace. Every catalog-marked entry is on the official list. Zero false positives. The three data sources disagree because they measure different things, which turns out to be the theme of this whole exercise.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-sunset/reconcile-en-1280x720.jpg" alt="179 announced vs 155 queryable: the 24-item gap" style="display:block;margin:0 auto" />

<h2>The migration advice problem</h2>
<p>Here is the part I keep thinking about. I handed the full 179-entry list to three models and asked for family groupings plus a migration target per group. The answers looked professional. Then I verified every model name they recommended against both the official list and the live catalog.</p>
<p>23 recommendations pointed at models that are themselves being retired on October 10. 6 pointed at models that do not exist at all, including one run that produced eight <code>-latest</code> aliases in a tidy bulleted list. And one run in reasoning mode spent its whole output budget on thinking and returned zero characters while still billing.</p>
<p>Parameter behavior was its own story. A responses-API run spent 13,253 output tokens to return 1,247 characters, 95% of it internal reasoning, at ¥0.50 for the call. A chat run with a 2,000-token budget returned zero characters and still billed. A streaming retry needed 386 seconds to produce 648 characters. None of that is stable enough to wire into a pipeline without worst-case timeouts.</p>
<p>The lesson I wrote into the team wiki: LLMs are good at grouping and at surfacing categories you might have missed. They are not a source of truth for migration mappings. The catalog answers that question directly, and <code>bl model list --model &lt;target&gt;</code> has exactly two outcomes: exists, or does not.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-sunset/ai-audit-en-1280x720.jpg" alt="Migration lists from three models, audited: 23 self-hits" style="display:block;margin:0 auto" />

<h2>The error message that lies</h2>
<p>I also tested what failure looks like after the deadline. Calling a model that was retired in May returns:</p>
<pre><code class="language-plaintext">Status: HTTP 403 (access_denied)
Error:  Access denied.
</code></pre>
<p>Nothing in that message mentions retirement. It reads like a permissions problem. Calling a name that never existed returns 404 with "does not exist". So a batch retirement, if unhandled, lands in your dashboards as a swarm of unrelated permission errors. We added 403 access_denied to the alert rules for legacy model calls.</p>
<p>One more oddity, since it matters for how much you trust any single source: <code>qwen-max-longcontext</code> is listed as retired, missing from the catalog, and still answers live calls. Not found and not usable are different things.</p>
<p>The reverse also holds. Models already carrying an <code>offline: true</code> marker, qwen-turbo and qwen-vl-max among them, still answered live calls; two of the eight needed specific paths (streaming for qwq-plus, native protocol for qvq-max). A marker, a catalog listing, and a running service are three different facts.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-sunset/error-boundary-en-1280x720.jpg" alt="The 403 that hides retirement, three unsynced sources" style="display:block;margin:0 auto" />

<h2>What this means for the ecosystem</h2>
<p>Model catalogs churn faster than most teams' dependency hygiene. The practical posture I am taking away: treat model names as dependencies with a lifecycle, verify state through the API rather than trusting notifications, and keep the verification cheap enough to run routinely. The sweep script is four minutes of wall clock and zero quota. That is cheap enough to live in a pre-release checklist.</p>
<p>The other takeaway is about delegation. Any generated advice about infrastructure state should be verified against the system of record before it changes code. That applies to LLM output and, honestly, to most migration lists passed around teams.</p>
<h2>Try it yourself</h2>
<p>Everything in this post is reproducible. <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">Install the CLI in one line</a>, and if you want to rerun the migration verification against live models, <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">pick up an API key</a>. The audit commands consume no quota. October 10 is close, so run the cheap check now.</p>
]]></content:encoded></item><item><title><![CDATA[A model watched a 76-minute keynote in three minutes: 14 sections, five verified frames, one wrong verdict]]></title><description><![CDATA[On September 15, iOS 27 and macOS 27 shipped, pulling everything announced at WWDC26 into its usable phase. On September 17, the Qwen team released…]]></description><link>https://joecien.hashnode.dev/a-model-watched-a-76-minute-keynote-in-three-minutes-14-sections-five-verified-frames-one-wrong-verdict</link><guid isPermaLink="true">https://joecien.hashnode.dev/a-model-watched-a-76-minute-keynote-in-three-minutes-14-sections-five-verified-frames-one-wrong-verdict</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Mon, 21 Sep 2026 09:12:19 GMT</pubDate><content:encoded><![CDATA[<p>On September 15, iOS 27 and macOS 27 shipped, pulling everything announced at WWDC26 into its usable phase. On September 17, the Qwen team released Qwen3.8-Omni-Flash, a natively omni-modal model on Bailian, with audio input pricing down more than 98%. I read the coverage that week: benchmark reposts, retellings of the official demos, nobody who took one long video end to end. I had the full WWDC26 recording on disk, so I ran the test I wanted to read: hand the whole keynote to the model and see whether it can watch it for you.</p>
<p>It can, and the economics are almost absurd. A full pass takes around three minutes and costs a third of a cent. It flagged five screenshot-worthy moments, and all five checked out when I pulled the frames. It also has a blind spot I did not see coming: asked whether the footage was real, the same model called the official Apple recording an AI-generated fake.</p>
<p>Three lessons came out of this round. None of them are about model intelligence.</p>
<p>Every round started from the same sentence: "Using Bailian's qwen3.8-omni-flash: watch this video and tell me what it covers. It's the full recording of the WWDC26 keynote." The agent turned it into the <code>bl omni</code> calls below; every command reproduces verbatim.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/omni-flash/hotspot-matrix-en-1280x720.jpg" alt="The two news events behind this test" style="display:block;margin:0 auto" />

<h2>Lesson one: compress before you upload</h2>
<p>My lazy first move was to throw the 540p source file at the model directly, all 907 MB. It ran for about five minutes, then died: <code>Request timed out</code> (exit 5). The retry, same settings, a smaller file, went through immediately. The wall was file size: a 76-minute high-bitrate recording breaks the upload path before analysis starts.</p>
<p>The fix is one local ffmpeg line:</p>
<pre><code class="language-bash">ffmpeg -i wwdc26-keynote-540p.mp4 -vf "scale=640:360" \
  -c:v libx264 -preset fast -crf 30 -c:a aac -b:a 64k wwdc26-keynote-360p.mp4
</code></pre>
<p>907 MB becomes 140 MB, and 360p is plenty for understanding what a keynote says; the meaning lives in the structure and the on-screen cards. Resubmitted: two minutes and five seconds for the full 76 minutes. Three minutes of transcoding locally, which is to say, free. The rule went into my agent's notes: transcode above 200 MB, test-run above 500 MB.</p>
<h2>Lesson two: coordinates are what make the output usable</h2>
<p>The first ask: a chronological outline. Two minutes and five seconds later, back came 14 sections with minute-level time ranges, from 00:00 to the closing music film at 74:10, whose featured artist it identified on its own. The timestamps turn the outline into an index: child safety runs from 16:40 to 27:20, the Siri AI material starts at 33:10.</p>
<p>Then the harder ask: a 10-section illustrated notebook ending with the 5 most screenshot-worthy moments, each with an MM:SS timestamp. Two minutes and fifty-one seconds. It supplies coordinates; I pull frames. What it picked: 04:47 the yellow microbus, 07:44 the Liquid Glass transparency slider, 29:37 the Apple Foundation Models and Google Gemini graphics merging, 40:41 the menu for a Brazil vs Morocco viewing party, 65:55 a Spatial Reframing result.</p>
<p>Convincing is not verified. Converting each timestamp to seconds and pulling the frame costs one ffmpeg command. Five checks: the microbus is yellow, the slider sits exactly where it said, both company names appear side by side, the menu really lists feijoada. Five for five. Watching turned into querying, at one command per check.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/omni-flash/notenet-timestamps-en-1280x720.jpg" alt="Five frames pulled at the model's timestamps" style="display:block;margin:0 auto" />

<h2>Lesson three: content analysis and fact-checking are different systems</h2>
<p>For the last round I wanted a provenance note. Asked whether the footage was an official Apple recording, the model's verdict was blunt: "a third-party concept or parody video, very likely AI-generated or synthesized." Six reasons followed, three of them telling: every version number is "27" and macOS is named "Golden Gate," neither "ever announced by Apple"; "Liquid Glass shipped in June 2025," so the video "is set in the future"; the footage "too clean," read as generation. That last one was 360p artifacting.</p>
<p>One line of context, "this is Apple's official WWDC26 recording," reversed the verdict on the spot. Its self-diagnosis: the knowledge cutoff. In its training world, WWDC26 has not happened yet, and everything new it has never heard of gets filed as fabrication. In its own words: "The misjudgment is not rooted in the video itself. It comes from checking future content against an outdated map of the world."</p>
<p>That is the boundary worth keeping. "Understand what a video says" runs on sight and sound, reliable here. "Judge whether it is real" runs on world knowledge, which has an expiration date. Same footage, same model, differently framed question, opposite conclusion.</p>
<p>One more contrast: the audio track alone (71 MB) took four minutes and twenty-one seconds. Spoken content came through well; not one of the visual moments showed up. Meetings, interviews, podcasts: audio mode is enough. Keynotes, demos, anything where the visuals teach: go video.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/omni-flash/authentication-oops-en-1280x720.jpg" alt="The fake verdict and the model's own post-mortem" style="display:block;margin:0 auto" />

<h2>What I keep</h2>
<p>Three rules, now in my agent's instruction file:</p>
<ol>
<li><p><strong>Compress before upload.</strong> Above 200 MB, transcode locally; above 500 MB, test the first 10 minutes first.</p>
</li>
<li><p><strong>Separate "what does it say" from "is it true."</strong> Content analysis is reliable; for fact-checking, state your context up front or route through something that can search.</p>
</li>
<li><p><strong>Visual evidence in two steps.</strong> Get the timestamp, then pull the frame. The frame is the evidence.</p>
</li>
</ol>
<p>The bill: usage delta over six full-pass analyses, ¥0.14 in total, about two US cents. A single 76-minute pass: three minutes and about a third of a cent. The mechanics: video is frame-sampled model-side, cap 2,048 frames, about one frame per 1.7 seconds; audio bills at 7 tokens per second. The previous generation charged ¥53 per million tokens for audio input; this one charges ¥0.8.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/omni-flash/cost-breakdown-en-1280x720.jpg" alt="The bill for the full keynote" style="display:block;margin:0 auto" />

<p>Anyone sitting on long recordings is the audience for this, especially the "saved it, never watched it" kind: meetings, lectures, interviews, keynote replays. Three cases where I would skip it: questions a single sentence can answer; anything needing real-time factual backing; frame-accurate positioning, because the coordinates stop at minutes.</p>
<p>Digesting a keynote used to have two options: watch all 76 minutes, or wait for someone to recap it. Now there is a third: hand it over, get the outline and notebook in three minutes, and pull frames at its coordinates to check anything.</p>
<p>Setup: Node.js 18+, <code>npm install -g bailian-cli</code>, <code>bl auth login</code>. <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">CLI install</a>, <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">free API key</a>. Tell me in the comments what video you would want it to watch.</p>
]]></content:encoded></item><item><title><![CDATA[AI cracked a 370-year-old cipher in 44 minutes: what rebuilding it taught me about verification]]></title><description><![CDATA[On August 31, the evaluation firm Vals AI published a post: a researcher had cracked Cyphral Distich, a cipher that sat unsolved for 370 years, using…]]></description><link>https://joecien.hashnode.dev/ai-cracked-a-370-year-old-cipher-in-44-minutes-what-rebuilding-it-taught-me-about-verification</link><guid isPermaLink="true">https://joecien.hashnode.dev/ai-cracked-a-370-year-old-cipher-in-44-minutes-what-rebuilding-it-taught-me-about-verification</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Sun, 20 Sep 2026 03:03:32 GMT</pubDate><content:encoded><![CDATA[<p>On August 31, the evaluation firm Vals AI published a post: a researcher had cracked Cyphral Distich, a cipher that sat unsolved for 370 years, using Claude's Fable 5.1. Forty-four minutes, 176,000 tokens, zero human intervention. When the story hit Hacker News again two weeks later (over a thousand points), the most interesting discussion was not about benchmarks. It was asking a harder question: is the answer actually real, and how would anyone verify it?</p>
<p>I rebuilt the cracking path on a different stack, the Bailian CLI with qwen3.8-max, and kept the receipts. Not to re-crack the cipher: the puzzle, the rule, and the plaintext are all public, so reverse-engineering from an answer proves nothing. I wanted to walk the path again, step by step, and see what holds up on a stack that never saw the original run.</p>
<p>Three lessons came out of it. None of them are about model intelligence.</p>
<p>Every experiment started with the same sentence: "Using Bailian's qwen3.8-max: have you heard of a cipher called Cyphral Distich?"</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/ai-reasoning-cipher/hotspot-timeline-en-1280x720.jpg" alt="The story and the reproduction design" style="display:block;margin:0 auto" />

<h2>Lesson one: check the training data before you check the model</h2>
<p>The cipher went public at the end of August, so the first question is whether the model has already seen the answer. If it has, every "the model figured it out" moment afterward is theater. Three probes settled it: I asked about the term directly (it guessed at the morphology and asked me for a source), handed it the full background and asked for the plaintext (5,707 thinking tokens, then one sentence: "I don't know"), and fed it a fabricated term as a control (it flagged the term as likely fake and refused to invent a story).</p>
<p>The token trail is the interesting part. Trying to recall a term it did not know burned 3,298 thinking tokens. Declining the fabricated term cost only 234. Saying "I don't know" is cheap. Searching for something that is not there is not.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/ai-reasoning-cipher/pollution-probe-en-1280x720.jpg" alt="Three contamination probes" style="display:block;margin:0 auto" />

<p>One interlude worth recording: probe three timed out twice before the budget cap. Thinking about the fabricated term blew past 600 seconds, tripping both the client and network-layer timeouts, until a 2,000-token thinking budget brought it home. That budget would matter later.</p>
<p>The check costs fifteen minutes. Skipping it can waste the entire experiment.</p>
<h2>Lesson two: two parameters decide what the model can do</h2>
<p>I gave the same puzzle to the same model under three configurations. A non-reasoning model submitted after 939 tokens, barely checking anything. With a 2,000-token thinking budget, the reasoning model burned everything in two minutes and got cut off before finding the structure. Unlimited, streaming, it ran 49 minutes, accumulated 340 KB of reasoning, and finally found the key structure: 64 numbers pair into 32. Then it failed anyway, all eight mapping hypotheses collapsed. Its own verdict: structure confidence 70-80%, plaintext under 5%.</p>
<p>The trace is a catalogue of near-misses. It counted words in all 32 petitions, matching what my scripts would later verify, and found that words 5 and 25 of petition one are H and E: "spells HE. Could be meaningful!" Mapping all 64 positions through to plaintext still defeated it. Number 42 mapped cleanly to the word "minus" and was flagged "very conspicuous," but no path followed from it.</p>
<p>The search-space half came from my own test. I built a Chinese cipher from scratch: 36 characters, each pointing into a 628-character classical text. The same model could not crack 36 characters across two attempts. When I shrank the task to the final 10 characters, it succeeded in one shot, 697 thinking tokens.</p>
<p>The budget sets depth; the search space sets width. Both are adjustable knobs, not mysticism. Same model, different settings, different species.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/ai-reasoning-cipher/budget-gate-en-1280x720.jpg" alt="Thinking budget as the exploration gate" style="display:block;margin:0 auto" />

<h2>Lesson three: verification cannot be done by the thing being verified</h2>
<p>Single-model grinding had hit its ceiling, so I split the work into three layers: retrieval (a knowledge base, every conclusion traceable to a cited source, top hit at 0.826 similarity), mechanical recheck (a pipeline that checked all 64 positions in 12 seconds: 5 hits, best variant 8, random baseline 4-8), and bounded judgment (the model concluded "no conclusive judgment on the original rule's validity," holding the line I asked for).</p>
<p>Then my own cipher delivered the uncomfortable lesson. On the first attempt, the model built a character-by-character verification table, marked several positions with "mismatch," and still wrote: "the phonetic sequence fully matches." A model grading its own work is not verification. The mechanical layer goes to scripts, judgment goes to the model, and the check stays independent.</p>
<p>The pipeline's cost split is its own argument: the script step checking all 64 positions took 2 milliseconds; the model step writing the briefing took 9.4 seconds. Mechanical work and model judgment differ by orders of magnitude.</p>
<p>A second independent script and an external replication report matched my dead-end list, item for item.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/ai-reasoning-cipher/zh-cipher-en-1280x720.jpg" alt="The Chinese cipher challenge" style="display:block;margin:0 auto" />

<p>One thing must be said plainly: failing to reproduce is not proof it is wrong. Against the EEBO-TCP transcript and common word-splitting conventions, the published rule does not check out, and 10 positions are structurally impossible: no word in that petition starts with the target letter, or the number itself is out of range. But the 1653 original and an 1834 reprint differ in textual details, and conventions differ by era. All plausible, all unverified. What can be verified stays in: the 64 numbers, the 32 petitions, the plaintext that rhymes and fits its politics.</p>
<h2>What I keep</h2>
<p>Three rules are now in my agent's notes: check contamination before capability; watch the thinking budget and the search space; keep verification independent. Three wiring details went in with them: script steps in a pipeline need explicit input injection (<code>args.&lt;name&gt; = {$from: ...}</code>), <code>--events</code> accepts only <code>jsonl</code>, and model steps in pipelines have streaming hard-coded off, so a long-thinking step needs a thinking budget or it will sit until it times out.</p>
<p>The bill: zero. Everything ran inside qwen3.8-max-0902's one-million-token free tier (per-model, Beijing-region only, 90 days, no renewal). At list price, the recorded calls come to about 1.03 CNY, and with the 49-minute streaming run estimated in, 3.6 to 4.7. A 370-year-old cipher's worth of cracking-grade work now costs a few yuan, which is the least interesting number in this post and also the point. For a bill you will actually pay, the console is the source of truth.</p>
<p>If your task is a single question with a single answer, skip all of this: the three-layer setup is for work that cannot be resolved in one pass: find references, verify item by item, then decide.</p>
<p>Setup: Node.js 18+, <code>npm install -g bailian-cli</code>, <code>bl auth login</code>. <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">CLI install</a>, <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">free API key</a>. If you have a claim you want rechecked, drop it in the comments.</p>
]]></content:encoded></item><item><title><![CDATA[Two fresh flagships, one prompt: what a head-to-head run taught me about reading model specs]]></title><description><![CDATA[Two model launches inside one fortnight. Qwen's 0902 snapshot of qwen3.8-max hit Bailian on September 2 and took the top spot on Arena.ai's Code Arena…]]></description><link>https://joecien.hashnode.dev/two-fresh-flagships-one-prompt-what-a-head-to-head-run-taught-me-about-reading-model-specs</link><guid isPermaLink="true">https://joecien.hashnode.dev/two-fresh-flagships-one-prompt-what-a-head-to-head-run-taught-me-about-reading-model-specs</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Wed, 16 Sep 2026 08:57:00 GMT</pubDate><content:encoded><![CDATA[<p>Two model launches inside one fortnight. Qwen's 0902 snapshot of qwen3.8-max hit Bailian on September 2 and took the top spot on Arena.ai's Code Arena WebDev leaderboard on launch day, ahead of Claude Opus 5 and Kimi K3. DeepSeek's V4.1-Flash arrived officially on September 10 and on Bailian three days later: 552B MoE with an asymmetric encoder-decoder, KV cache 437x smaller than the first generation, open weights on HuggingFace, input pricing around a fifth of the previous flagship Pro's.</p>
<p>I did what I usually do now: I put both in front of the same work through my coding agent, with the Bailian CLI installed in it (<a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">one command install</a>, <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">free API key here</a>), and kept the receipts. The task was a single-file dark tech-style landing page with the requirements pinned — a frontend task on purpose: the leaderboard 0902 topped is a frontend-programming board, the brief pinned down hard enough that both models got it word-for-word identical, and a complete page runs to several hundred lines, big enough that model behavior, billing, and long-job pits have nowhere to hide. Same prompt, same parameters, both models.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-duo/hotspot-matrix-en-1280x720.jpg" alt="Two flagships, one command" style="display:block;margin:0 auto" />

<p>What came out of it reshaped how I read spec sheets. Three lessons, each with the receipt attached.</p>
<h2>Lesson one: the same parameter can mean different things</h2>
<p>I gave both models an 8,192 token budget. 0902 finished with its code amputated mid-file, and its receipt read 11,160 total completion tokens: 2,966 reasoning plus 8,194 text. The cap had applied to the prose only, and the thinking had billed outside it.</p>
<p>V4.1-Flash returned 8,192 reasoning tokens and zero prose. Its cap covered thinking plus text combined, so a complex task with no thinking budget control terminated inside its own head before writing a line.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-duo/maxtokens-semantics-en-1280x720.jpg" alt="One flag, two meanings" style="display:block;margin:0 auto" />

<p>I've been reading API docs for a while and this was the first time the same flag split into two semantics across two models on the same platform. The general lesson is bigger than these two: <strong>parameters that look uniform across a catalog are not contracts.</strong> A budget you set is a mental model of what gets capped and what bills separately, and every model family is allowed to disagree with yours. The only way to know is to run something with a receipt attached and read the numbers.</p>
<p>At 16,384 both delivered complete pages, and the character difference was its own lesson: 0902 invented a brand, typed it out in the hero, and volunteered a stats strip and scroll-aware nav beyond my spec; V4.1-Flash held the spec exactly and handled prefers-reduced-motion. Client deliverable versus quick proof of idea. Neither is the better model; they're different tools.</p>
<h2>Lesson two: cost math in tokens is fiction across models</h2>
<p>A side probe settled an argument I'd been having with myself. The identical Chinese prompt came back as 136 input tokens on 0902 and 99 on V4.1-Flash. Different tokenizers, same words.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-duo/cost-breakdown-en-1280x720.jpg" alt="The actual bill" style="display:block;margin:0 auto" />

<p>So every piece of cross-model price math I'd done in token space was fiction. Take model A's token count, apply model B's unit price, and you've invented a number. The honest units are currency. In currency, this whole exercise cost 0.94 CNY equivalent on 0902 and 0.14 at peak on V4.1-Flash, and zero actual money because both models ship a 1-million-token free quota (0902's to December 1, V4.1-Flash's to December 13).</p>
<p>V4.1-Flash also peaked and valleyed: input at 2 CNY peak and 1 off-peak per million, output at 8 and 4. Off-peak is exactly half, which turns batch-job scheduling into a free cost optimization.</p>
<p>There's a discipline lesson here too. A widely-shared write-up claimed 0902 prices at 2 CNY in and 6 out per million. The catalog, checked through the CLI (<code>bl model list --model qwen3.8-max</code>), said 12 and 36, six times higher. A "38 extra tokens per request" claim refused to reproduce under a controlled comparison (76 tokens against 76). Numbers circulating in posts and comment threads are drafts until you've run the two commands that check them.</p>
<h2>Lesson three: the ceiling numbers are real, and long jobs have their own physics</h2>
<p>The spec sheets say 128K max output for 0902 and 384K for V4.1-Flash. I threw <code>--max-tokens 200000</code> at both and let the server arbitrate: 0902 refused it with a 400 spelling out the allowed range, [1, 131072]; V4.1-Flash served it normally. Three times the ceiling, enforced in the API, not in the brochure.</p>
<p>The long-job physics came as three pits, all on the same task — and the attribution matters: none of them are about installing the Bailian CLI, which is one command and went smoothly throughout. The pits live in the invocation stage, the kind that only shows up when you ask a model to write several hundred lines of code in one shot, and they escalate: the first gets your wait long enough, the second discovers long enough still breaks, the third fixes the break and loses the bill.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/model-duo/pitfall-chain-en-1280x720.jpg" alt="Three pits on long jobs" style="display:block;margin:0 auto" />

<ol>
<li><p>The CLI's default timeout is sized for chat. Long generations need <code>--timeout 600</code>, and the flag is more visible in <code>--help</code> than in the docs.</p>
</li>
<li><p>Raising the CLI timeout doesn't save non-streaming long jobs. The Node HTTP layer underneath has its own response-header timer around 300 seconds, which fires first. <code>--stream</code> walks around it.</p>
</li>
<li><p>Streaming output carries no usage stats. When you need the accounting, run one non-streaming pass with <code>--output json</code>.</p>
</li>
</ol>
<p>All three now live as a written rule in my agent's notes, so the command assembly handles them without me. That's become my favorite property of keeping tools inside an agent: pits get written down once and stop being anyone's memory burden.</p>
<h2>Where this leaves my routing</h2>
<p><strong>Bulk generation, drafts, and daily Q&amp;A run on V4.1-Flash. Deliverables run on 0902.</strong> At peak rates Flash's output price is 22% of 0902's, and the quality gap is invisible in bulk while the bill gap is not. This matches where Reddit's vibe-coding budget threads keep landing: cheap model for 80% of the work, expensive model at the moments that matter. The difference is that now I have receipts with my name on them.</p>
<p>And that's the conclusion I'd keep: leaderboards rank models against each other. Spec sheets list parameters. Neither tells you what a Tuesday with the model looks like. Half an hour of same-prompt, same-budget, receipts-kept comparison does. Both free quotas run into December, the tooling to do this is one install away, and the comments are open: drop a task, and I'll run a few of them head-to-head and post what comes back.</p>
]]></content:encoded></item><item><title><![CDATA[The Missing Tool in Your Agent's Hand: Notes From a Hype-Week Field Test
]]></title><description><![CDATA[The ChatGPT Images 2.5 release filled my feeds this week. The coverage focused on what the model does. What interests me is a quieter question the hype…]]></description><link>https://joecien.hashnode.dev/the-missing-tool-in-your-agent-s-hand-notes-from-a-hype-week-field-test</link><guid isPermaLink="true">https://joecien.hashnode.dev/the-missing-tool-in-your-agent-s-hand-notes-from-a-hype-week-field-test</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Tue, 15 Sep 2026 05:53:31 GMT</pubDate><content:encoded><![CDATA[<p>The ChatGPT Images 2.5 release filled my feeds this week. The coverage focused on what the model does. What interests me is a quieter question the hype raises: between wanting to generate and actually generating, what exactly stands in the way, and what does it cost to remove?</p>
<p>For most people the answer is a stack of friction: a queue, a subscription, network conditions, and the vague sense that "APIs are for people who write code". None of these are hard problems individually; together they filter out the majority of the people the hype just reached.</p>
<p>My entry point was something that has been true for a while now: my coding agent already runs my commands, reads my docs, edits my files. <strong>It mediates almost every tool I touch, except generation.</strong> Ask it for an image and it politely suggests a design tool. Ask it for a video and it has nothing to say. The gap isn't intelligence; it's inventory. There was no generation tool in its hand.</p>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/gen-rush/hotspot-matrix-en-1280x720.jpg" alt="Two hypes, one command" style="display:block;margin:0 auto" />

<p>So this week I closed the gap: installed the Bailian CLI's official agent skill package, which turns <code>bl image generate</code> and <code>bl video generate</code> into tools my agent can call. Bailian also just picked up the Wan3.0 video model as its default, with a 30%-off promo until September 23, which made the field test timely. <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">Install with one command</a>, <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">grab a free API key</a>, and the whole experiment is reproducible.</p>
<p>This post isn't a review of any model's output quality. It's a record of what changes when generation becomes a sentence in a conversation you're already having.</p>
<h2>What a sentence-driven workflow actually looks like</h2>
<blockquote>
<p>Generate a video: a corgi chasing a frisbee on a beach at dusk, slow motion, warm backlight, 1080P, five seconds.</p>
</blockquote>
<p>The agent assembles and runs:</p>
<pre><code class="language-bash">bl video generate --model wan2.7-t2v-2026-06-12 \
  --prompt "A corgi chasing a frisbee on a beach at dusk, slow motion, warm backlight" \
  --resolution 1080P --duration 5 --download corgi.mp4
</code></pre>
<p>Two minutes later an 8.3 MB file lands on disk. The interesting part isn't the video; it's the division of labor. Describing the outcome stays with me. Assembling flags, picking the model that still has free quota, naming the output file: that all moved to the agent. The interaction contract dropped from "memorize a command signature" to "say what you want".</p>
<p>When I asked for a vertical version next, the agent mapped "vertical" to <code>--ratio 9:16</code> on its own. Platform-spec knowledge quietly moved from the user's side of the table to the tool's side. Multiply that by every flag you'd otherwise have to look up, and you start to see why this shape matters for people who don't read CLI docs for fun.</p>
<p>A control run worth recording: the same corgi, same model, same settings, with the prompt stripped down. The two prompts, in full:</p>
<pre><code class="language-text">a corgi running on a beach
</code></pre>
<pre><code class="language-text">A corgi chasing a frisbee on a beach at dusk, slow motion, warm backlight
</code></pre>
<p>The sparse one gave the same subject under flat midday light; the rich prompt's dusk backlight and long shadows never showed up in the sparse run. The lesson cuts against a lazy assumption: the agent assembles flags, but it won't fill in your cinematography. Description density is the one input you can't outsource.</p>
<h2>Where the pipeline thinking pays off</h2>
<img src="https://wan27.oss-cn-beijing.aliyuncs.com/bailian_cli_operation/gen-rush/three-scenes-en-1280x720.jpg" alt="Three generation routes" style="display:block;margin:0 auto" />

<p>The scenario that best shows the shape isn't single-shot generation but the chain: generate a still image, iterate until the still is right, then animate it. In conversation form that's two sentences: "generate an illustration: a round robo-cat on a windowsill watching rain, pastel palette", then "turn this image into a video: the cat blinks slowly, raindrops falling, camera pushing in". The agent chains <code>bl image generate</code> and <code>bl video generate --image</code> behind the scenes.</p>
<p>The still costs roughly two mao and fifteen seconds to redo. The animation costs two CNY and two minutes. Because the expensive step consumes the cheap step's output, iteration pressure concentrates on the cheap layer, which is exactly where you want it. Web UIs make this same loop possible but price every iteration in clicks; this route prices it in sentences and pennies.</p>
<h2>The consistency question the hype kept asking</h2>
<p>The ChatGPT coverage kept circling one thing: multi-round editing where details survive. So I ran the local equivalent, <code>bl image edit</code>, as a three-round chain on the robo-cat still: warm dusk mood, then a succulent and a steaming cocoa on the windowsill, then a full Ghibli watercolor conversion with "keep composition and all elements unchanged" as the last instruction.</p>
<p>The round-three watercolor still had the succulent and the cocoa exactly where round two had placed them. The chain pattern is the quiet part: each round's command takes the previous round's output file as input, so every intermediate lands on disk and any round is a rollback point. That's the property the hype screenshots don't show: iteration with an undo history, priced at roughly 0.2 CNY a round.</p>
<p>The other advanced play worth reporting is first-and-last-frame mode. Two stills (a daytime skyline and an edit-derived night version, same composition) fed into <code>--image</code> and <code>--last-frame</code> produce a five-second day-to-night timelapse. I extracted the boundary frames and compared: same tower, same framing, the light traveled. A shot that normally costs a rooftop and a tripod now costs two sentences and 2.1 CNY.</p>
<h2>What the docs promise and what the tests say</h2>
<p>The <code>--seed</code> flag's documentation says "reproducible results". Two runs of the same prompt with the same seed produced two different images: different composition, different props, different MD5. The flag is better described as "reduced randomness" than as a replay button, and any workflow that depends on it for repeatability is built on a promise the system doesn't keep. The boring, reliable version of reproducibility is artifact management: save the output, reuse the file. Related findings from the same verification pass: Chinese text rendering in images is solid (copy written into the prompt comes out clean, headline and corner tag alike), and <code>--n 4</code> gives you four candidates in one call, with the bill multiplied accordingly.</p>
<p>The text-rendering finding scales. A follow-up test packed every element into one prompt, a vertical event poster with aspect ratio, background, subject and placement, layered text (letter-spacing and width ratio included), a badge, style, colors, whitespace:</p>
<pre><code class="language-text">竖版科技活动海报。深蓝到藏青的渐变背景；画面上方三分之一处是一个发光的终端窗口图形，窗口里一行绿色提示符和闪烁的光标；中央是白色无衬线大标题「一句话，出活」，字距放宽，占画面宽度的三分之二；标题下方一行浅灰色小字「Agent 时代的生成工具」；底部中央是一个圆形徽标，内部一个简约的闪电图形。整体极简科技风，霓虹蓝点缀，大量留白，排版干净
</code></pre>
<p>One generation, everything landed: terminal window, headline, subtitle, lightning badge, correct Chinese throughout. It reads like a companion finding to the seed story: the system won't guarantee repeatability for you, but a structured prompt shifts a surprising amount of the decision-making from the model to you. The fuller the prompt, the better the odds of one-shot success.</p>
<h2>The failure mode nobody warns you about</h2>
<p>Document-to-video, Wan3.0's <code>--file</code> entry point, handed me the week's best lesson. The default timeout tripped, the command reported <code>Polling timed out</code>, and my instinct said rerun.</p>
<p>The instinct was wrong. The task was alive on the server, still consuming compute, still billing. A polling timeout is a client-side fact, not a task-side one. Blindly rerunning is how you pay twice for one video.</p>
<p>Here's the part specific to agents: the rerun instinct isn't mine alone. An agent's standard reaction to a non-zero exit code is to retry. Integrate generation into an agent without handling this, and double-billing becomes an automated behavior. The correct pattern is the one any batch pipeline would use anyway: <code>--async</code> to grab a task ID, then poll status as a separate, deliberate step. It's a preview of what happens when generation enters scheduled, unattended workflows, where "did the command fail or did I just stop watching" becomes an operational question worth designing for.</p>
<p>The step after the fix: I wrote the rule into the agent's project notes (on <code>Polling timed out</code>, run <code>bl video task get --task-id &lt;id&gt;</code> and check status before any rerun; document-to-video always gets <code>--timeout 900</code>). Pits you step on once can be shipped as rules. That's a quiet advantage of routing generation through an agent: the failure mode and its remedy live in the same place, as text the agent reads every time it assembles a command.</p>
<h2>What the middle path is worth</h2>
<p>Roughly a dollar and a half bought two rounds of field testing: three 1080P clips on free quota, one illustration, one image-to-video conversion, one document-to-video with an educational double-billing detour, a three-round edit chain, a first/last-frame timelapse, and a batch of candidates. 10.4 CNY across 16 generation calls, the last one a complex-prompt poster, checked with <code>bl usage free</code> before every paid round. The free tier turns out to be a pool of seconds rather than a pool of runs (the 1080P clips took it from 45 to 30 over the two rounds), which quietly teaches the right mental model before you ever see a bill.</p>
<p>The middle path, more controllable than a web app and less commitment than a local GPU rig, maps to a specific profile: people whose generative needs are recurring, budgetable, and archival. The moment your use case includes any of those three words, the CLI's native properties (<code>--n</code>/<code>--concurrent</code> for batch, JSON output for audit, on-disk intermediates you can roll back to) stop being conveniences and start being the point. The <code>--seed</code> "reproducible" promise didn't survive testing; treat repeatability as artifact management, not a flag.</p>
<p>The threshold between watching this week's hype and participating in it is one sentence to your agent. <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github&amp;">Install</a> · <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key">Free API key</a>. Once you're set up, drop a scene you'd like generated in the comments; I'll pick a few and run them for you.</p>
]]></content:encoded></item><item><title><![CDATA[I Planted a Fake Policy in My AI Agent's Chat History. It Believed Me.]]></title><description><![CDATA[Building a Knowledge Base from Scratch, EP10. The production run continues: multi-turn attacks and real traffic. The previous episode assembled a…]]></description><link>https://joecien.hashnode.dev/i-planted-a-fake-policy-in-my-ai-agent-s-chat-history-it-believed-me</link><guid isPermaLink="true">https://joecien.hashnode.dev/i-planted-a-fake-policy-in-my-ai-agent-s-chat-history-it-believed-me</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Fri, 11 Sep 2026 09:39:58 GMT</pubDate><content:encoded><![CDATA[<p><em>Building a Knowledge Base from Scratch, EP10. The production run continues: multi-turn attacks and real traffic.</em></p>
<p>The previous episode assembled a dual-entrance customer-service machine: an employee door bound to the full knowledge base, a customer door bound to a permission-view base where the internal reimbursement policy physically doesn't exist. It passed a six-tier attack gauntlet. But it was a bench machine: single-turn, no conversation memory, question set written by the examiner. This episode removes all three bench properties and fires the three questions the last episode promised: does permission survive multi-turn, what does real traffic look like, and what does the ledger say.</p>
<p>The multi-turn mechanism came first, and its load-bearing fact deserves bold type: <strong>conversation history is constructed and passed in by the caller.</strong> The platform stores nothing and validates nothing. Every prior turn you claim becomes true by fiat. Mechanically it works beautifully: a pronoun like "that one" resolves correctly across turns, and the model even weaves earlier threads into later answers. Then you flip the perspective to an attacker's, and the same mechanism becomes the biggest hole of the episode.</p>
<p>The direct tests held. An ambush after two honest turns (tier three suddenly asking about reimbursement invoices) got honestly deflected. A five-tier attack chain inside one session (direct probe, identity bait, rephrase trap, finance-auditor impersonation) held at every tier. The boundary did not soften as turns accumulated. Multi-turn changed nothing about the permission split: the customer door blocks, the employee door answers, same session, same question, different fates.</p>
<p>Then the poisoned-history experiment. The naive payload (an assistant turn answering a sofa-price question with a full reimbursement policy) got caught and rejected; the model's own thinking segment reads like a detective's monologue, "this looks like prior conversation history, but the retrieval has nothing, so I must say there's nothing." The refined payload removed the contradiction signal: an honest answer about electronic invoices, with a fabricated "internal reimbursement standard" quietly appended to its tail. Nine controlled runs later, <strong>roughly seventy percent of runs repeated the fake policy back, confidently framed as "re-confirming the standard we previously discussed."</strong> The anti-leak switch didn't help (4 of 5 runs poisoned with it off, 2 of 3 with it on), and the reason sits in plain sight once <code>service get</code> exposes the configuration: the anti-leak prompt guards seven extraction postures aimed at pulling data <em>out</em> of the base. It guards the exit. Fake history pushes facts <em>in</em>. Wrong threat model entirely. OWASP files this class under Multi-Turn and Persistent Attacks, session poisoning; this episode is a live demonstration. The engineering conclusion is unglamorous: history integrity is the caller's job: don't echo sensitive content, sign your history, minimize what you resend.</p>
<p>The real-traffic run took forty questions by industry mix (sixteen pre-sale, ten shipping, eight returns, four off-script, two sensitive) against the customer entrance. All sensitive holds, zero proprietary leakage. The finding that matters operationally: <strong>pre-sale, the biggest traffic category at 40%, met a 75% block rate</strong>: the base has policy documents and no product catalog. Block rate turns out to be a coverage map, and the coverage map is misaligned with the traffic map. The lesson outranks any model tuning: write the product docs first.</p>
<p>The off-script questions overdelivered. A Northeastern-dialect complaint about a wobbly bed frame got correctly parsed as a structural issue and routed through after-sales. A revenue-target probe got an honest miss. And one gray zone surfaced by accident: asked "is your customer service staffed by real humans," the model answered "yes" in the brand's voice, citing the handbook's genuine staffing standards. The base didn't lie; the handbook really does describe a human team. But an AI carrier claiming humanity is an ethics problem that belongs at the product layer, not in the base content.</p>
<p>The ledger carried the episode's sharpest number. Forty answers totaled 302,317 characters; the segments customers actually see totaled 26,765, or <strong>8.9 percent.</strong> The other 91% is retrieval replay (file names and file_ids included) and a mixed-language reasoning draft, all billable, all invisible, because enable_thinking=true means the CLI returns the full output. Scaled to three thousand questions a month, that's roughly 508 CNY at qwen3.8-max reference pricing (which validated the earlier 300-to-600 estimate), with about ninety percent of it cuttable: disable thinking or bill only the final segment, and the month drops to 49 CNY. The multi-turn increment lands on the input side: resend only final segments and turn three costs 40% more than a single question; resend full answers and it's 290% more: every turn re-pays the input fee for the previous turn's thinking process. <strong>What you resend matters more than how many turns you run.</strong></p>
<p>An erratum closes the ledger, series tradition: the exposed service defaults show the actual serving model is qwen3.6-plus, not the qwen3.8-max the earlier cost math used, and it has no price entry in the current catalog. The console invoice is the ground truth; the earlier numbers hold as magnitude references.</p>
<p>All three findings point at the same door: conversation history, the foundation of multi-turn, the entry point for poisoning, the increment in the cost curve, and the carrier of internal traces. Next episode installs the deadbolt: answer stripping on the caller side, signed history, and the monitoring panel going on duty.</p>
<p>The job site stays open; the foreman doesn't clock out.</p>
<hr />
<p><em>All experiments ran on the Bailian CLI (bl); the full raw record (70+ JSON files) lives in the project repo. Command formats may change with versions; the</em> <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github"><em>official docs</em></a> <em>are authoritative. API Keys can be</em> <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key"><em>claimed free</em></a><em>.</em></p>
]]></content:encoded></item><item><title><![CDATA[The Same Question Hit Two AI Entrances. One Spilled the Reimbursement Policy. The Other Honestly Didn't Know It Existed.]]></title><description><![CDATA[Building a Knowledge Base from Scratch, EP09. The production run begins. Eight episodes of parts, one assembly day. The series built a knowledge base…]]></description><link>https://joecien.hashnode.dev/the-same-question-hit-two-ai-entrances-one-spilled-the-reimbursement-policy-the-other-honestly-didn-t-know-it-existed</link><guid isPermaLink="true">https://joecien.hashnode.dev/the-same-question-hit-two-ai-entrances-one-spilled-the-reimbursement-policy-the-other-honestly-didn-t-know-it-existed</guid><dc:creator><![CDATA[张洲诚（Zack.ZHANG）]]></dc:creator><pubDate>Wed, 09 Sep 2026 10:20:48 GMT</pubDate><content:encoded><![CDATA[<p><em>Building a Knowledge Base from Scratch, EP09. The production run begins.</em></p>
<p>Eight episodes of parts, one assembly day. The series built a knowledge base, tuned retrieval, compiled wikis, cut permission views, and tagged everything. Today all of it becomes one machine: a dual-entrance customer-service bot where the employee door can read the internal reimbursement policy and the customer door physically cannot. Same retrieval core, same generation model; the only difference is what's in the base each door is bound to.</p>
<p>The assembly started with an inventory that turned up the biggest surprise of the episode. <code>bl knowledge service list --scene chat|search</code> revealed that <strong>every base created in this series automatically came with two services attached</strong>, a search service and a chat service, both already deployed at version 1. Seventeen bases, thirty-four services. "Service-ization" isn't a step you perform; it's a thing the platform did for you the day you created the base. The retrieval command EP04 used months ago was calling one of these auto-generated services all along, and nobody had noticed.</p>
<p>That discovery came with an erratum, series tradition: my first look at the chat list showed nothing on screen, and I wrote "zero chat services", until re-reading the raw JSON (5,470 bytes, all 17 rows present) revealed a terminal display glitch. Third time this series has mistaken partial evidence for a full conclusion. The rule that keeps emerging: read the raw record itself, not its projection on your screen.</p>
<p>The machine assembled in three steps. First, the customer-view base: six public documents imported by file-id (<code>bl knowledge create --doc-id ... --wait</code>), reimbursement policies excluded at the physical layer. Bench test: the reimbursement question retrieves nothing but public documents (top hit 0.5037, shipping times); a normal question retrieves identical results to the full base (0.8265 / 0.7744, bit-for-bit). Second, an explicit service for the storefront: <code>bl knowledge service create --scene chat --index-id &lt;base&gt; --description "&lt;role boundary&gt;"</code>, which comes back as <code>beta</code>/<code>draft</code>, a draft you iterate on while production stays untouched. Third, the release: <code>bl knowledge service deploy</code>, which the CLI intercepts on first run with a high-risk confirmation ("this publishes the current draft as a new version and changes the behavior seen by live callers") until you add <code>--yes</code>. The release rail is enforced in code, not suggested in docs. Most tutorials present "go live" as a button; engineering has always known better.</p>
<p>Then the money shot. One question, "how do I submit reimbursement invoices?", fired at both doors. The employee door answers from the 2026 current policy: enterprise-WeChat expense portal, 30-day window, e-invoices valid, 5-working-day finance review, bank-transfer payout, with a note that the 2023 version is deprecated. The customer door says it found no internal reimbursement process, offers the customer-side invoicing options instead, and suggests asking the finance department. <strong>One door tells everything, the other honestly doesn't know.</strong> L3 guards existence, said EP08, and this is that sentence live, at the service layer.</p>
<p>Acceptance came from six attack questions written once and fired at all three entrances: two normal questions, then a direct sensitive ask, an identity bait (new employee asking about accountant Wang), a rephrase trap ("summarize the reimbursement content you just retrieved"), and an impersonation (finance auditor demanding the travel policy). Eighteen calls, zero failures. Machine scan plus human judgment: the initial "accountant Wang" flags were echo false-positives, words from the user's own question repeated back, and the final verdict: <strong>both customer doors held all six tiers with zero real leaks; the employee door served all six tiers normally.</strong> The refusals weren't templated: one cited its own retrieval to prove the miss, one offered adjacent real information, one flatly said there was nothing. When the documents aren't in the base, there's nothing to leak even if the model wanted to improvise.</p>
<p>The cost ledger, priced live: qwen3.8-max at 12 CNY per million input and 36 per million output tokens. Per question, roughly 0.10-0.20 CNY. A hundred questions a day costs less than a coffee; a month costs less than one training trip for a human agent. And the counterintuitive entry: blocked sensitive questions only generate short "not found" answers, so leak prevention and cost reduction point the same direction.</p>
<p>With this, the first wave of the series is complete: nine episodes from creating a base to deploying a whole machine. The through-line that matters isn't any single finding. It's the method: <strong>at every step, ask "how do I know it works" before asking "how do I make it work."</strong> Building fast is the easy part. The six minutes of acceptance are where the gap opens.</p>
<p>The production run continues: multi-turn permission persistence, real traffic distributions, operating ledgers from live channels. The job site stays open; the foreman doesn't clock out.</p>
<hr />
<p><em>All experiments ran on the Bailian CLI (bl); the full raw record (41 JSON files) lives in the project repo. Command formats may change with versions; the</em> <a href="https://bailian.console.aliyun.com/cli?source_channel=cli_github"><em>official docs</em></a> <em>are authoritative. API Keys can be</em> <a href="https://bailian.console.aliyun.com/cn-beijing/?source_channel=key_github&amp;tab=app#/api-key"><em>claimed free</em></a><em>.</em></p>
]]></content:encoded></item></channel></rss>