Does code mode cut AI agent inference costs by batching tool calls?
Polylane says shifting tool execution to scripts reduces context bloat and parallelizes calls, lowering latency and price per task.
Key points
- Code mode replaces sequential tool calls with a single script that executes multiple functions.
- Tool results are stored in program variables rather than appended to the conversation history.
- This approach allows parallel execution of tool calls within one inference step.
Polylane reported that its agent uses code mode for most tool calls to reduce context size and inference steps.reported The company argues this method is superior to standard tool calling, which appends full results to the conversation history.Polylane says using scripts instead of direct tool calls saves money and time for AI agents.Polylane implements a pattern where tool results populate program variables rather than the context window, enabling parallel execution.
The announcement
According to a report on Lobsters, code mode gives the model a single tool that runs code, exposing other tools as callable functions within that script.reported The model writes a small program, the program executes the necessary tool calls, and only the final return value is sent back to the model.
The context
The outlet noted that code mode has been discussed over the last year primarily as a way to use MCP servers. Polylane stated this framing undersells the concept, arguing it is actually a decision about where tool results go: into the model’s context or into program variables.
Reading between the lines
In standard agents, the model acts as expensive glue between tools, reading growing dumps of JSON at each step. With code mode, the script filters data and fetches jobs in parallel, returning only specific fields. The model sees only the concise return value, not the raw payloads.
The business case
Polylane claims this approach is cheaper and faster because it eliminates redundant inference steps. Standard tool calling requires a full inference step for every call, with the model reasoning about unnecessary data. Code mode consolidates this into one script execution, reducing the volume of data the model processes and the number of paid steps.
Why it matters
- Platform architects can reduce token consumption by keeping large JSON payloads out of the model context.
- Parallel execution within a single script lowers latency compared to sequential tool calling loops.
- The model processes only filtered return values, decreasing the compute cost per complex task.
- This pattern applies to any agent, not just those using specific server protocols like MCP.
Topics: AI agents, inference optimization, tool calling, Polylane, code mode
Updated
Then / now / next
Agents typically made sequential tool calls, appending full results to the conversation context.
Polylane implements code mode to batch calls and filter results before returning them to the model.
Watch for broader adoption of script-based tool execution as a standard pattern for reducing inference costs.
Story timeline
- Then Agents typically made sequential tool calls, appending full results to the conversation context.
- 04 Nov 2026 What even is code mode?
- Now Polylane implements code mode to batch calls and filter results before returning them to the model.
- Next Watch for broader adoption of script-based tool execution as a standard pattern for reducing inference costs.
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Terms in this story
Code mode
A pattern where a model writes a script to execute tools, returning only the final result.
Inference step
A single computation cycle where the model processes input and generates output.
Context
The conversation history and data the model considers when generating a response.
Questions readers ask
How does code mode differ from standard tool calling?
Code mode uses a single script to execute multiple tools in parallel, returning only the final result to the model, whereas standard calling appends each result to the context sequentially.
Does code mode require MCP servers?
No, Polylane states code mode is a decision about result handling and can be used by any agent, independent of MCP.
What is the main cost benefit?
It reduces the number of inference steps and prevents the model from paying for and processing unnecessary data in the context window.
Connected stories
Reading receipt
- Code mode replaces sequential tool calls with a single script that executes multiple functions.
- Tool results are stored in program variables rather than appended to the conversation history.
- This approach allows parallel execution of tool calls within one inference step.
Still open: How does error handling work when multiple tools fail within a single script?
Still open: What is the overhead of interpreting the generated code compared to direct API calls?