你的 AI 智能体 token 账单主要来自输入。以下是削减它的实战指南。

Your AI agent's token bill is mostly input. Here is the field guide to cutting it.

TL;DR - In a toy 10-turn agent session, 16,559 input tokens go out and 3,090 output tokens come back. At Claude Sonnet 4.5 prices input is still 52% of the cost, and all three cuts below shrink it to 3,290 input-equivalent tokens, 80% less. - Truncating tool output and pruning old history alone cut the session's input by 62%, to 6,445 tokens. - Caching a stable prefix bills it at 0.1x after the first call, but a prefix used only once costs 25% more than no caching.

Your agent bill is bigger than the work you remember asking for, and the cause is mechanical. A model API is stateless, so every turn re-sends the system prompt, all tool output so far and the whole conversation. A file listing pasted on turn 2 is billed again on every turn after it. TechCrunch reported that Y Combinator CEO Garry Tan spends up to $200K a year on tokens, and at any scale the repeated input is the part you can shrink. Below: one script to measure it, and three cuts.

Where a 10-turn session's tokens go

Turn 10 carries the results of turns 1 through 9, so a 10-turn session costs the sum of ten growing prompts. In the toy run below that sum is 16,559 input tokens. A real agent reply carries tool calls and code, so we model each reply at about 300 tokens, which makes 3,090 output tokens for the session.

Output costs more per token. Anthropic's pricing page lists Claude Sonnet 4.5 at $3 per million input tokens and $15 per million output tokens, so one output token weighs 5 input tokens. Weighted by price, input is still 52% of the session, a bare majority. The script prints all three numbers.

To count caching in the same units, we use input-equivalent tokens: a cache read counts as 0.1 of a token and a cache write as 1.25.

Measure first: count tokens for each part of a turn

Start with your real baseline: log the usage block of every API response (input and output tokens per call) and sum each per session. That total is your real number. The script then shows where it goes, by counting the system prompt, one tool output and the history separately. The biggest one tells you which cut to try first.

The data is fake, so rerun the script on your own. Every figure is a tiktoken count except the cache lines and the cost-share lines, which are arithmetic from the vendors' documented multipliers and prices, with no API call.

# pip install tiktoken
import tiktoken

enc = tiktoken.get_encoding("o200k_base")

def n(text):
    return len(enc.encode(text))

system = """You are a coding agent. You read files, run commands, and edit code.
Always explain your plan before acting. Follow the repo style guide. When you
write tests, use the project's existing fixtures. Never touch the migrations
folder without asking. Summarize diffs at the end of each task. Be concise but
complete. Handle errors gracefully and retry once on network failures.
[... style guide: 40 more lines of formatting rules ...]"""

# One fake tool output: a 60-line file listing.
tool_output = "\n".join(
    f"src/module_{i:02d}.py: def handler_{i}(request) -> Response: ..."
    for i in range(60)
)

# One fake model reply per turn. A real agent reply carries its tool calls and
# code, so we model it at a few hundred tokens.
reply = (
    "I read the listing. handler_07 builds the response before it validates "
    "the request, which is why test_handler_07 fails. I'll move the check "
    "above the response and add a regression test.\n"
    '{"tool": "edit_file", "path": "src/module_07.py", "patch": "'
    + "\\n".join(
        f"- response = build(request, {i})\\n+ validate(request)\\n+ response = build(request, {i})"
        for i in range(10)
    )
    + '"}'
)

def history_of(turns):
    return "\n".join(
        f"Turn {i}: user asked something, agent read files, ran tests, "
        f"edited two files, tool output followed. {tool_output[:400]}"
        for i in range(turns)
    )

def truncate(text, head=600, tail=200):
    if len(text) <= head + tail:
        return text
    cut = text[head:-tail].count("\n")
    return text[:head] + f"\n[... {cut} lines truncated ...]\n" + text[-tail:]

def turn(history_turns=10, tool=tool_output):
    return "\n".join([system, history_of(history_turns), tool])

full = n(turn())
pruned = n(turn(history_turns=3))
truncated = n(turn(tool=truncate(tool_output)))
both = n(turn(history_turns=3, tool=truncate(tool_output)))

def smaller(x):
    return 100 - x * 100 // full

print(f"system prompt:               {n(system)} tokens")
print(f"one raw tool output:         {n(tool_output)} tokens")
print(f"10 turns of history:         {n(history_of(10))} tokens")
print(f"one full turn as sent:       {full} tokens")
print(f"history pruned (keep 3):     {pruned} tokens ({smaller(pruned)}% smaller)")
print(f"tool output truncated:       {truncated} tokens ({smaller(truncated)}% smaller)")
print(f"both cuts:                   {both} tokens ({smaller(both)}% smaller)")

# A 10-turn session: turn i re-sends i earlier turns of history.
naive = sum(n(turn(history_turns=i)) for i in range(10))
session_cut = sum(n(turn(history_turns=min(i, 3), tool=truncate(tool_output))) for i in range(10))
out = 10 * n(reply)
print(f"10-turn input, no cuts:      {naive} tokens")
print(f"10-turn input, both cuts:    {session_cut} tokens ({100 - session_cut * 100 // naive}% smaller)")

# Cut 3 arithmetic (documented multipliers, no API call): a cache write costs
# 1.25x, a cache read 0.1x. Stable prefix = system prompt + the 3 kept turns.
prefix = n(system + "\n" + history_of(3))
cached = prefix * 1.25 + prefix * 0.1 * 9

# All three cuts stacked, same model. Each turn re-sends the previous turn's
# system prompt + history as a cache read (0.1x), writes whatever the prefix
# grew by (1.25x), and bills the rest of the turn (the tool output) at 1x.
# Assumes pruning happens at fixed points, so the prefix stays valid after
# turn 3, and every call lands inside the 5-minute cache lifetime.
stacked, prev = 0, 0
for i in range(10):
    total = n(turn(history_turns=min(i, 3), tool=truncate(tool_output)))
    pre = n(system + "\n" + history_of(min(i, 3)))
    stacked += prev * 0.1 + (pre - prev) * 1.25 + (total - pre)
    prev = pre
print(f"10-turn input, all 3 cuts:   {stacked:.0f} input-equivalent tokens ({100 - stacked * 100 / naive:.0f}% smaller)")
print(f"10-turn output (replies):    {out} tokens")

# Price-weighted: Sonnet 4.5 lists $3 per million input, $15 per million
# output, so one output token weighs 5 input tokens.
OUT_WEIGHT = 5
print(f"input cost share, no cuts:   {100 * naive / (naive + OUT_WEIGHT * out):.0f}%")
print(f"input cost share, 3 cuts:    {100 * stacked / (stacked + OUT_WEIGHT * out):.0f}%")

print(f"stable prefix:               {prefix} tokens")
print(f"prefix x10, uncached:        {prefix * 10} input-equivalent tokens")
print(f"prefix x10, cached:          {cached:.0f} input-equivalent tokens")
print(f"prefix used once, cached:    {prefix * 1.25:.0f} input-equivalent tokens")

Output:

system prompt:               93 tokens
one raw tool output:         960 tokens
10 turns of history:         1339 tokens
one full turn as sent:       2393 tokens
history pruned (keep 3):     1455 tokens (40% smaller)
tool output truncated:       1663 tokens (31% smaller)
both cuts:                   725 tokens (70% smaller)
10-turn input, no cuts:      16559 tokens
10-turn input, both cuts:    6445 tokens (62% smaller)
10-turn input, all 3 cuts:   3290 input-equivalent tokens (80% smaller)
10-turn output (replies):    3090 tokens
input cost share, no cuts:   52%
input cost share, 3 cuts:    18%
stable prefix:               494 tokens
prefix x10, uncached:        4940 input-equivalent tokens
prefix x10, cached:          1062 input-equivalent tokens
prefix used once, cached:    618 input-equivalent tokens

Try this: replace system, tool_output, history_of and reply with your real system prompt, one real tool result, your logged history and a typical reply.

Cut 1: truncate tool output, keep the head and tail

Tool output is often the biggest block, and most of it is never read again. truncate keeps the first 600 characters and the last 200 and puts a marker in the middle. That takes one turn from 2,393 to 1,663 tokens, 31% smaller.

Keep both ends because errors tend to show up at the top of a log and the summary line at the bottom.

Cap each tool result at a head, a tail and a marker that says how many lines were dropped, so the agent knows it can ask for more.

Try this: log the length of every tool result for one real run. Whatever tops the list gets the cap first.

Cut 2: prune history once a result has been used

After the agent has acted on a tool result, the raw result rarely needs to travel with it. The toy loop keeps only the last 3 turns, which cuts one turn from 2,393 to 1,455 tokens (40%), and both cuts together bring it to 725 (70%). A gentler version keeps a line per old result:

# pseudocode, language-neutral
for each message older than the last 3 turns:
    if it is a tool result:
        replace the body with one line, for example:
        "read src/auth.py (212 lines): login() calls verify_token(); test_login_expired fails"
    otherwise keep it as it is

Over-pruning makes the agent dumber. If an early turn found the name of the failing test, that fact has to survive the prune, as it does in the summary line above. The raw listings and bulky results can go, but any fact a later step will need to quote has to stay.

Try this: after pruning, rerun one task you know the answer to. If it now fails or re-reads files it already read, you cut too much.

Cut 3: keep the prefix identical so the cache can reuse it

Anthropic's prompt caching docs put a cache read at 0.1x the base input price, and OpenAI's prompt caching guide says cached input is discounted up to 95%. The script runs the Anthropic multipliers on the stable prefix, which is the system prompt plus the 3 kept turns, 494 tokens. Sent for 10 turns it bills like 4,940 tokens uncached and like 1,062 with a 1.25x write on turn 1 and 0.1x reads after. Sent once, it bills like 618, 25% more than not caching at all.

That 494-token prefix sits below both vendors' 1,024-token minimums, so the toy cannot show this saving at its own scale. What carries over is the ratio on a real prefix: one write at 1.25x, then reads at 0.1x. The "all 3 cuts" line applies the same multipliers to the whole session, assuming pruning at fixed points and every call inside the 5-minute window.

The 618 is how caching backfires. Three conditions decide which side you land on:

• The prefix must stay identical. The cacheable order is tools, then system prompt, then history. Changing anything early breaks the match for everything after it.
• It must be reused in time. On Anthropic a cache write costs 1.25x, and an entry lives 5 minutes by default. One call that never gets reused costs more than no caching.
• It must be long enough. OpenAI caches prompts of at least 1,024 tokens. Anthropic's minimum is 1,024 for Claude Sonnet 4.5 and ranges from 512 to 4,096 across its other models.

The quiet killer is a timestamp in the system prompt. So is a request ID, or a tool list that comes back in a different order. Any of them makes the prefix different on every call, so on Anthropic you pay the 1.25x write each time and never see a 0.1x read. No error appears; the bill just goes up a little.

Pruning rewrites the history, so the cache match ends where the prune starts: prune at fixed points instead of every turn.

Order the prompt as tools, system prompt, then history, keep timestamps and IDs out of it, and send the same prefix again within 5 minutes.

Try this: on Anthropic, read cache_creation_input_tokens and cache_read_input_tokens from each response's usage block. If reads stay at zero after the second call, something in your prefix changes between turns.

What these cuts will not fix

They shrink input. The output tokens you asked for stay, and after all three cuts the same arithmetic puts input at 18% of the session's cost, so output is now the bigger part of the bill. A loop that wanders for 20 turns instead of 10 re-sends the context ten more times, which no trim recovers.

Where to start:

• Tool output is the biggest block: cut 1.
• History is the biggest block: cut 2.
• The prompt is large and stable: cut 3, and check the usage fields.

Your 15-minute version for tomorrow: run the script with your real system prompt, one real tool result and your logged history. Whichever of the three counts is largest, apply that cut first and run the script again to see the drop. On the toy session the full stack takes 16,559 input tokens to 3,290 input-equivalent ones; your data will land elsewhere, and the rerun tells you where.