STAGE 01 · THE STARTSTAGE 01 · 起点
Load everything, let the AI figure it out全量导入，让 AI 自己找
Pour the whole knowledge base in and let the model match questions to answers by meaning
把所有知识一股脑塞进去，让大模型自己按"意思相近"去找
We loaded everything and let the model find answers by "closest meaning." Hit rate was poor — and worse, on troubleshooting questions ELSA would confidently make things up, in perfect professional tone.
知识库全量导入，靠"意思相近"去命中内容。结果命中率很低，排查类问题经常一本正经地胡编乱造。
LEARNED学到
A raw AI model on top of a raw knowledge base is a hallucination machine. This is lesson one for anyone who wires an AI up to a knowledge base — no exceptions.
裸大模型 + 裸知识库 = 幻觉制造机。这是所有人做知识库 AI 的第一课，没有例外。
STAGE 02
Search, rebuilt as engineering检索工程化
Treat "finding the right document" as a serious engineering problem in its own right
把"找资料"这件事本身当成一门工程来做
-
Search by meaning AND by exact words, blended. "Video freezing" the concept and "TM1100" the exact model name need different search strategies — ELSA runs both and weighs them together.
"按意思找"和"按关键词找"混合加权。"视频卡顿"这种概念和"TM1100"这种精确型号，需要的是两种不同的找法——ELSA 两条都跑，再加权合并。
-
Cross-language matching — an English question finds the right Chinese checklist, and vice versa.
中英跨语言匹配——英文问题也能命中中文清单，反之亦然。
-
A test set of 239 real questions from actual support work. After tuning, the right document came back first 9 times out of 10 (recall@1 = 0.908) — and every change since gets re-run against this set, so search quality can never silently slip back.
239 条真实提问组成的测试集。调优后，10 次里 9 次第一条就是对的文档（recall@1 = 0.908）——此后任何改动都要在这套题上重新过一遍，检索质量永远不会悄悄倒退。
LEARNED学到
Even at 90%+ search accuracy, satisfaction stayed stuck around ~83% no matter what we tuned. Then we audited the metric itself — and found it polluted: plainly wrong answers were still collecting 👍. A thumbs-up rate cannot be your judge.
检索调到 90%+ 也没用。满意度卡在 ~83% 怎么切都不动，而且扒库发现指标本身被污染——明确答错的回答照样被点赞。点赞率不能当决策裁判。
STAGE 03 · THE WATERSHEDSTAGE 03 · 分水岭
The turn: certainty first形态大转向：确定性优先
The real variable was never "how accurate is the search" — it was "is this answer certain, or composed?"
真正的变量不是"检索准不准"，而是"这答案是确定的，还是拼出来的"
The insight that changed everything: what decides whether people can act on an answer is whether it was read directly out of an authoritative system, or composed by the AI. So the whole system was rebuilt into three tiers:
想明白的那件事：决定"敢不敢照做"的，是答案是从权威系统里直接读出来的，还是 AI 拼出来的。于是整个系统重构成三档：
-
Direct-read lanes — device lookups, warranty, version comparison, manuals, on-call roster… eleven deterministic lanes and counting. On a hit, ELSA copies the record word-for-word with the source linked. No AI writing involved — so nothing to hallucinate.
直读通道——设备查询、保修、版本比对、手册查表、值班名单……先后建了十多个确定性出口。命中就逐字直出、带出处链接。中间没有 AI 写作环节——所以无处可编。
-
The honesty gate — when nothing covers a question, ELSA says so plainly instead of improvising. When it errs, it errs on the side of honesty.
坦白门——覆盖不到的，宁可诚实说不在覆盖范围，绝不即兴发挥——错只错成老实。
-
Tiered expectations — every AI-composed answer is labeled 🟡; direct reads never are. One glance tells you which answers to act on directly. That's the same 🟡 you see in Slack today.
期望分层——AI 综合出的回答强制打 🟡 标；直读回答从不打。一眼就知道哪些能闭眼信。今天你在 Slack 里看到的 🟡，就是这一阶段定下的。
LEARNED学到
When money, compliance, or a frontline engineer about to act on a customer is involved, "usually right" is not good enough. The essence of this stage: take work away from the AI and hand it back to plain, predictable code.
遇到钱、遇到合规、遇到一线要"敢照做"的场景，"大概率对"是不够的。这一阶段的本质，是把大量本来交给 AI 的活收回给纯代码。
STAGE 04 · THE BREAKTHROUGHSTAGE 04 · 突破
Two lanes: Deep analysis arrives双车道：深度分析上线
A fast lane for certainty, a slow lane for hard problems
快车道管确定性，慢车道啃硬问题
A colleague's excellent Q&A bot showed what quality was possible. We broke down why it was good — three root causes — and matched each one: ① a deeper knowledge floor (R&D's own engineering docs were brought in); ② a real time budget per question (the fast lane answers in seconds; the slow lane is allowed minutes); ③ the confidence to actually answer — made legitimate by making it opt-in.
同事的一个答疑机器人展示了质量的可能性。我们拆出它好的三个根源，逐一对齐：①更深的知识底座（接入研发自己的业务逻辑文档）；②单题真实时间预算（快车道秒级作答，慢车道给到分钟级）；③敢答——用"用户主动点按钮"让敢答变得合法。
The architecture is a waterfall, not a fork: every question first runs the full deterministic chain. Only at the honesty gate does ELSA offer the "🔍 Deep analysis (~1–3 min)" button. Click it, and ELSA works the problem the way an engineer would — deciding which source to search, reading, searching again, cross-checking, round after round — every claim cited, hard time cap enforced.
架构关键是瀑布而非分岔：确定性链永远先走全走，走到坦白门才挂出"🔍 深度分析（约 1–3 分钟）"按钮。点了才启动——ELSA 像工程师排查一样干活：自己决定查哪个源、读完再查、交叉印证、一轮接一轮——强制引用出处，硬性时间封顶。
Its first battle avenged ELSA's most embarrassing historical failure: a question it had once answered wrong with full confidence (and still collected a 👍 — see Stage 2). Deep analysis got it right in 6 search rounds and 55 seconds, with evidence from three independent sources. Within two days it was being used more per day than every previous feature combined.
首战就打掉了历史上最丢脸的错答案案例：一个曾自信答错还被点赞的问题（见阶段二），深度分析 6 轮检索、55 秒给出正确答案 + 三个独立来源的交叉证据。上线两天，日均使用量超过此前所有功能的总和。
LEARNED学到
Multi-round search-and-synthesize can answer almost anything the knowledge base actually contains — but only on top of the first three stages. Jump straight from Stage 1 to Stage 4 and it's slow, expensive, and randomly wrong on the easy questions.
只要知识库里有内容，多轮检索再整合就能答好——但前提是前三个阶段的地基。直接从阶段一跳到阶段四不成立：又贵又慢，还会在简单问题上引入随机性。
STAGE 05 · IN PROGRESSSTAGE 05 · 进行中
Repositioning around the two lanes围绕双车道重新站位
Once the quality ceiling broke, every old compromise came up for review
质量瓶颈可破之后，回头清算所有历史妥协
Deep analysis proved the ceiling could be broken — so we went back and re-examined every compromise made before it existed. Features that had quietly stopped being used were retired outright; an auto-learning display that was only right 1 time in 3 went offline; transition-era switches and dead code were swept out. The system is converging on two trunks: fast lane = certainty, slow lane = deep analysis.
深度分析证明瓶颈可破，随即全量重审所有"没有深度分析时代"的妥协功能：用量趋零的旧功能整体退役、准确率仅 1/3 的自动学习展示下线、一批过渡期开关与死代码清扫。系统围绕"快车道确定性 + 慢车道深度分析"两条主干收敛。
LEARNED学到
Every time capability jumps a level, go back and re-audit the old compromises. They were rational when made; now they're debt.
能力每上一个台阶，就该回头重审一遍旧妥协——它们当初是理性的，现在是债务。
The one loop running through it all: errors flow back贯穿全程的一条闭环：错误回流
This is the moat — it belongs to no single stage这是护城河，不属于任何一个阶段
🙁 feedback / ELSA admits "no answer"🙁 差评 / AI 自陈无答案→
auto-logged on the gap board自动进"填坑表"→
human review · quality gate人工审核 · 质量门→
absorbed into the knowledge base吸收进知识库→
follow-up with whoever hit the gap回访差评用户
The quality gate is the point. When we tried absorbing auto-generated answers without human review, accuracy was 1 in 3. Pipelines are easy to build; gates are hard to keep. That lesson paid for itself.
质量门是关键。自动学习不设人审时，准确率只有 1/3——回流管道好建，质量闸门难修，这个教训值回票价。
Alongside it runs a stress-test ledger: 40+ genuinely tricky real questions, re-run in production after every significant change. "It works" has to come with re-runnable proof.
另有信心压测台账：40+ 条真实刁钻问题在生产环境实跑，每次改动全量回归。"说好了"必须附可复跑的证据。
If you take away only three things如果只带走三句话
for anyone building — or judging — a knowledge-base AI给同样在做（或在评估）知识库 AI 的人
1一
Better search has a ceiling. The watershed is separating certain answers from composed ones — not a bigger model, not finer-tuned retrieval.
检索优化有天花板。形态转向——把"确定的"和"拼出来的"分层——才是分水岭，不是换更好的模型或调更准的检索。
2二
Letting the AI work a problem in multiple rounds is the quality breakthrough — but it must stand on a deterministic foundation. Raw, it's slow, costly, and randomly wrong on easy questions.
让 AI 多轮检索、像工程师一样啃问题，是质量突破口——但必须长在确定性地基上，不能裸上：否则又贵又慢，还在简单问题上引入随机性。
3三
An error-feedback loop without a human quality gate pumps pollution straight into your knowledge base. The smoother the pipeline, the faster the poisoning.
错误回流闭环必须有人工确认的质量门，否则是往知识库灌污水。自动化管道越顺畅，没有闸门时污染得越快。