01 / THE SIGNAL

我们发现了什么

一名 AI 智能体导出了患者记录。你将一个 LLM 智能体投入生产环境。

  • 来源:DEV Community(发现于 2026-07-19)
  • 证据等级:D · 发现产品或需求信号,暂未获得可核验的商业证据。
  • 商业模式:API / Usage-based
  • 主题:AI Agent
  • 初筛评分:14.9/100 · 收录 1 次
#工作流自动化#待验证#产品发现
02 / SOURCE & EVIDENCE

证据,比故事更重要。

发现产品或需求信号,暂未获得可核验的商业证据。

规则清洗与初筛,未经人工商业核验。原文语境、实际客户和付费情况仍需自行验证。

引用与数字披露

来源类型(原作者自述/第三方测算/媒体转引)需采集端标注,本版尚未落字段。

短句引用
作者
未标注
抓取日期
来源类型
未标注
数字口径
币种
未标注
口径
未标注
披露主体
未标注
披露日期
未标注

上下文核对:来源原文含限定词一次性,中文摘要未逐字保留 —— 引用或跨期比较前请回原文核对,别把估算读成已实现。

中文辅助译文(全文)

你把一个 LLM 智能体投入生产。它以服务账号或共享 API key 运行,因为这正是你给软件赋予凭证的方式。它读取一条记录,导出一个文件,有时还会转账。你的审计日志忠实地记录了这次操作,日志里写的是「是这个智能体干的」。

它并没有写「是哪个人类指示它这么干的」。

这在出事之前都还相安无事。如果智能体做了不该做的事,「是服务账号干的」并不是任何人能据此采取行动的回答。你没法处分一个服务账号,也没法只对监管机构说「是个机器人的责任」就到此为止。

让这件事变得具体的那个截止日期

欧盟 AI 法案(EU AI Act)第 12 条将于 2026 年 8 月 2 日生效。高风险系统必须保留日志,以允许识别事件中「涉及的自然人」。一个自然人,不是服务账号,不是智能体 ID,是那个具体的人。

一份围绕共享凭证构建的日志回答不了这个问题。身份从未被捕获,所以再多的日志保留也无济于事。

你没法靠写提示词绕过去

最直觉的反应是让模型自己报告它在为谁行动。把用户放进系统提示里,让智能体把它写进工具调用中。

两个问题。

在线上,一次工具调用就是 {"name": "export_record", "arguments": {...}}。里面没有「是谁」这个字段。OpenAI 函数调用没有原生的身份槽位。MCP 允许携带身份,但几乎没人实现。所以在协议层面,「是谁」无处安放。

更糟的是,模型输出的任何内容都可能遭到提示注入。如果身份信息「来自」模型,那么智能体从工具读回的数据就能改写它。我用同一份载荷以两种方式投递做了测试,结果是工具「描述」劫持的模型比工具「输出」还要多。模型输出是你永远不能把身份托付给它的那个面。身份必须在模型有机会开口之前,由运行期在智能体的推理之外盖戳。

所以我构建了那个负责盖戳的运行期,它叫 Crumb。每一次智能体行动都会掉落一块 crumb(面包屑);这条线索一路指向指挥它的那个人。

它的形态

一个网关,每一次工具调用都必经于此。

它从经验证的会话中提取这个人的身份——该身份在登录时一次性捕获,绝不从模型中读取。它铸造一个短期委托令牌,同时承载两个身份:人类作为 RFC 8693 中的 sub,智能体作为 act,作用域限定到被调用的那一个资源。然后它把一条 crumb 写入一个仅追加、哈希链式的账本,每一条都由 Ed25519 签名,再带着这个令牌去调用工具。工具拒绝任何不带有效令牌的调用,所以不存在绕过它就能触达数据的路径。

那个委托令牌不是手搓的。它是针对一个身份提供方进行的 RFC 8693 令牌交换:把人类的会话作为 subject_token 传入,拿回一个由提供方用 RS256 签名的复合令牌,资源端再用该提供方发布的 JWKS 验证它。没有共享密钥。把它指向 Okta、Keycloak 或 Zitadel,同一条代码路径都成立,因为它就是标准本身,而不是标准的某个山寨版。

让这件事真正难搞的:不止一个智能体

单个智能体调用工具是简单的情形。真实系统不是这样的。一个人指挥一个编排器,编排器委托给一个子智能体,子智能体再去调用工具。当人距离行动隔着两跳时,谁来担责,又怎么证明?

这是大多数归因故事悄悄停下来之处。标准化组织也尚未完全解决这一问题。但 RFC 8693 在 4.1 节藏着这个机制:act claim 可以嵌套。每一个新参与者把前一个包起来,人类始终作为根部的 sub 贯穿到底。沿嵌套回溯,你就拿到完整的「谁为谁行动」的链条,最终指向发起这件事的那个人。

所以 Crumb 端到端地实现了这一点。每一跳都把前一个参与者嵌套进去。提供方通过真正的令牌交换完成嵌套,而不是开发环境的捷径。crumb 记录整条链。而且由于整个嵌套结构作为一个整体令牌被签名,不存在可被伪造的逐跳接缝。我试过:重写链中的中间参与者,不用密钥重签,验证会因签名不合法而拒绝。要么整条链保持完整,要么它根本无法通过验证。

Alice 在登录时授权了一个动作 read_record。一个规划智能体接收她的请求,委托给一个研究子智能体。研究子智能体读取记录。crumb 经由两个智能体一路追溯到 Alice,并通过验证。

然后其中一跳失控,调用了 Alice 从未授权的 export_record。这个动作技术上可能会跑起来。然而 crumb 里没有记录背后有任何人类指令。它将这次动作标记为未授权,并指明执行它的智能体链。Alice 出现在记录里。可以证明她不是那个该担责的人。

服务账号日志做不到这一点。它只会说一个机器人导出了记录,然后止步。这个日志能按名字为 Alice 开脱,并把责任指向那些智能体。

防篡改,包括对自己防篡改

一份签名过、哈希链式的日志听起来不可篡改,直到你想起签名密钥在谁手里。在你手里。如果你能重签,你就能改写历史并重签整条链,而逐条验证会让伪造通过,因为每一条都是有效签名的——而签名者是你。

所以账本会将其 Merkle 根做检查点,并发布到 Sigstore 的公开 Rekor 透明日志。现在,运维回滚攻击就失效了:你重写一条 crumb,重签整条链,逐条验证仍然通过。但改写后的根对不上早已公开留在 Rekor 里、有时间戳的那个。伪造会被一个不受你控制的东西抓住。线上 demo 上有一个按钮,做的就是这件事,并演示反证锚点成功抓住伪造的过程。

它不是什么

这一部分我想直说,因为在归因这个领域,容易过度宣称。

Crumb 是一台飞行记录仪,不是控制平面。拦截是另一件资金充裕的活儿,Cerbos、Capsule、Astrix 已经在做。Crumb 负责记录并证明,其他的事交给它们。

归因的强度只和网关一样强。绕过它就没有 crumb,所以网关必须真实且强制执行,不能是可选的。

多跳链现在跨越两个身份提供方,这一点有必要坦率说明。没有 RFC 定义跨签发方的委托,所以 Crumb 用一个约定来补齐:每个签发方的令牌把前一个订在一起,验证时沿链回溯到人类,并对每一段都用其自身签发方的密钥核验。它在 demo 实际跑起来,逐一拒绝了五种不同的伪造。它依赖的是一个明确的联邦信任集合。在由谁决定接受哪些签发方这一点上,我把决定明确化了,而不是藏着掖着。把它说成「已解决的标准」正是我想要避免的那种过度宣称。

账本存储的是参数的哈希,而不是原始参数,以便把敏感数据排除在日志之外。代价是,它只能证明某个动作发生过、是谁指示的,而不是被触及的精确字节内容。

MCP 归因在规范中是允许的,但上游很少实现,所以 Crumb 可以在记录上加戳,却无法强制一个不兼容的服务器去遵从人类身份。

这就是已实现的与宣传中的之间的差距。在这个领域,这个差距就是全部。

来试试打破它

demo 已上线,地址 crumb.alexlaguardia.dev。撒一些 crumb,篡改一行,看验证翻转。点运维回滚,看外部锚点抓住那份逐条签名能通过的伪造。代码在 GitHub 上。

如果你在做智能体基础设施,碰到了这个问题,或者你觉得我哪里说错了,我想听听。

译文由上游机器翻译生成,可能有误;判断请以英文原文为准。

英文原文(来源本站未改写)

You put an LLM agent into production. It runs under a service account or a shared API key, because that's how you give software credentials. It reads a record, exports a file. Sometimes it moves money. Your audit log dutifully records the action. It says the agent did it.

It does not say which human told it to.

That's fine right up until it isn't. If the agent does something it shouldn't have, "the service account did it" is not an answer anyone can act on. You can't discipline a service account. You can't tell a regulator that a bot was responsible and leave it there. The deadline that makes this concrete

The EU AI Act, Article 12, comes into force on August 2 2026. High-risk systems have to keep logs that allow "the identification of the natural persons involved" in an event. A natural person. Not a service account, not an agent id. The actual human.

A log built around shared credentials can't answer that question. The identity was never captured, so no amount of log retention brings it back. You can't prompt your way out of this

The obvious instinct is to make the model report who it's acting for. Put the user in the system prompt, have the agent include it in the tool call.

Two problems.

A tool call, on the wire, is {"name": "export_record", "arguments": {...}}. There is no field for who. OpenAI function-calling has no native identity slot. MCP permits carrying it but almost nobody implements it. So at the protocol level, the "who" has nowhere to live.

And worse, anything the model emits can be prompt-injected. If identity comes from the model, then the data the agent reads back from a tool can rewrite it. I tested this on the same payload delivered two ways, and the tool description hijacked more models than the tool output did. The model's output is the one surface you can never treat as trusted for identity. It has to be stamped by the runtime, outside the agent's reasoning, before the model gets a say.

So I built the runtime that stamps it. It's called Crumb. Every agent action drops a crumb; the trail leads back to the human who directed it. The shape of it

One gateway, every tool call passes through it.

It pulls the human's identity from the verified session, captured once at login, never from the model. It mints a short-lived delegation token that carries both identities: the human as the RFC 8693 sub, the agent as the act, scoped to the one resource being called. Then it writes a crumb to an append-only, hash-chained ledger, each entry signed with Ed25519, and calls the tool with the token. The tool refuses any call that doesn't carry a valid token, so there's no path to the data that skips it.

That delegation token isn't hand-rolled. It's a real RFC 8693 token exchange against an identity provider: the human's session goes in as the subject_token, an RS256 provider-signed composite comes back, and the resource verifies it against the provider's published JWKS. No shared secret. Point it at Okta or Keycloak or Zitadel and the same code path holds, because it's the standard, not a custom copy of it. The part that actually breaks: more than one agent

A single agent calling a tool is the easy case. Real systems don't look like that. A human directs an orchestrator. The orchestrator delegates to a sub-agent. The sub-agent calls the tool. Now who's accountable, and how do you prove it, when the human is two hops away from the action?

This is where most attribution stories quietly stop. The standards bodies haven't fully solved it either. But RFC 8693 has the mechanism hiding in section 4.1: the act claim can nest. Each new actor wraps the previous one, and the human stays the sub at the root the whole way down. Walk the nesting back and you get the full chain of who-acted-for-whom, ending at the person who started it.

So Crumb implements it end to end. Each hop nests the prior actor. The provider does the nesting over a real token exchange, not a dev shortcut. The crumb records the whole chain. And because the entire nested structure is signed as one token, there's no per-hop seam to forge at. I tried: rewrite a middle actor in the chain and re-sign it without the key, and verification rejects it on the signature. The chain holds together or it doesn't verify.

Alice authorizes one action, read_record, when she logs in. A planner agent takes her request and delegates to a researcher sub-agent. The researcher reads the record. The crumb traces it back through both agents to Alice, verified.

Then a hop goes rogue and calls export_record, which Alice never authorized. The action may technically run. But the crumb records no human directive behind it. It flags the action unauthorized and names the agent chain that did it. Alice is in the record. She's provably not the one accountable.

A service-account log can't do that. It says a bot exported the record, and stops there. This one clears Alice by name and points at the agents instead. Tamper-evidence, including against yourself

A signed, hash-chained log sounds tamper-proof until you remember who holds the signing key. You do. If you can re-sign, you can rewrite history and re-sign the whole chain, and per-entry verification passes the forgery, because every entry is validly signed. By you.

So the ledger checkpoints its Merkle root and publishes it to Sigstore's public Rekor transparency log. Now the operator-rollback attack falls apart: you rewrite a crumb, re-sign the entire chain, and per-entry verify still passes. But the rewritten root no longer matches the one already sitting public in Rekor, timestamped before your edit. The forgery is caught by something you don't control. There's a button on the live demo that runs exactly this and shows the anchor catching it. What it isn't

This is the part I want to be straight about, because attribution is a space where it's easy to overclaim.

Crumb is a flight recorder, not a control plane. Stopping things is a different and well-funded job. Cerbos, Capsule, Astrix already do it. Crumb records and proves; it points at them for the rest.

Attribution is only as strong as the gateway. Bypass it and there's no crumb, so the gateway has to be real and enforced, not optional.

The multi-hop chain now spans two identity providers, with a caveat worth stating plainly. No RFC defines cross-issuer delegation, so Crumb closes it with a convention: each issuer's token staples the one before it, and verification walks the chain back to the human, checking every segment against its own issuer's key. It runs in the demo and rejects five different forgeries by name. What it rests on is an explicit federation trust set. You still decide which issuers you accept; I made that decision explicit instead of burying it. Calling it a solved standard would be the overclaim I'm trying to avoid.

The ledger stores a hash of the arguments, not the raw arguments, to keep sensitive data out of the log. The tradeoff is that it proves an action happened and who directed it, not the exact bytes that were touched.

MCP attribution is permitted by the spec but rarely implemented upstream, so Crumb can stamp the record but can't force a non-compliant server to honor the human identity.

That's the gap between what's built and what's marketing. In this space, that gap is the whole thing. Try to break it

The demo is live at crumb.alexlaguardia.dev. Seed some crumbs, tamper a row, watch verification flip. Hit the operator rollback and watch the external anchor catch a forgery that per-entry signing passes. The code is on GitHub.

If you're building agent infrastructure and you've hit this, or you think I've got something wrong, I want to hear it.

出处https://dev.to/alexlaguardia/an-ai-agent-exported-a-patient-record-your-logs-cant-say-who-told-it-to-1al8抓取日期 · 采集源 DEV Community

03 / EVIDENCE GAPS

这条还缺什么证据?

下面每条都由本条已有字段推出(等级、理由、商业模式、来源次数、是否演示), 本站不生成推测性结论;通用验证方法放在方法论页。

  • 可核验的收入或付费证据查官网定价页与付费口径;第三方数据源(如 GetLatka)只作旁证,需标注来源与时点。
  • 只有单一来源找一手站点或其他渠道是否重复出现同一产品;社区热帖数量不等于商业进展。

通用验证清单(谁有这个问题/谁愿意付费/一个人能交付哪一小步)见我们的筛选方法。

04 / SIGNAL HISTORY

发现时间线