Sequence: Inference Economics

Main Ideas And Sequence Order

Blank for collaborative planning.

References

• OpenAI and Amazon announce strategic partnership — OpenAI says AWS will be the exclusive third-party cloud distribution provider for OpenAI Frontier and will help create a Stateful Runtime Environment for production generative AI applications and agents.
• Scaling Managed Agents: Decoupling the brain from the hands — Anthropic describes Managed Agents as a hosted long-horizon agent service built around sessions, harnesses, and sandboxes that can evolve independently.
• Anthropic Introduces Managed Agents to Simplify AI Agent Deployment — Secondary overview of Anthropic Managed Agents as a runtime for secure execution, state handling, tool usage, and operational guarantees.
• Full-Stack Optimizations for Agentic Inference with Dynamo — NVIDIA explains why coding agents create KV-cache pressure through repeated long-context calls, tool definitions, and subagent fan-out.
• NVIDIA Dynamo — NVIDIA’s distributed inference-serving framework for data-center-scale model serving, routing, disaggregated serving, and KV-cache management.
• Multi-Turn Agentic Harnesses — NVIDIA post on streaming reasoning segments, tool-call events, request metadata, parsers, and harness-facing serving behavior for multi-turn agents.
• The More You Buy, the More You Make — NVIDIA blog framing AI factories as infrastructure that produces intelligence measured in tokens, with agentic AI requiring many inference steps per task.
• NVIDIA AI factory glossary — Defines AI factories as infrastructure for the full AI life cycle, with token throughput as a key output measure.
• Agentic AI in the Factory — NVIDIA design guide placing agentic AI as an orchestration layer above inference and data layers.
• LightSeek TokenSpeed thread — Thread/source for TokenSpeed as a practical artifact in the inference-throughput and agent-serving conversation.
• LightSeek TokenSpeed repository — Repository reference for measuring or improving token speed in agentic workloads.
• Sourcegraph MCP server — Sourcegraph reference for colocating code search/navigation capabilities with agent workflows through MCP.
• Coder agent compatibility: Sourcegraph Amp — Notes that Sourcegraph Amp stores conversation threads server-side, relevant to the broader shift from purely local agent loops to managed/server-side agent state.
• HyFunc: Accelerating LLM-based Function Calls for Agentic AI — Paper on reducing redundant context processing and function-call latency through hybrid-model cascades and dynamic templating.
• Optimizing Agentic Language Model Inference via Speculative Tool Calls — Paper on optimizing agent inference when models repeatedly call tools such as search, code execution, and APIs.
• AgentOpt v0.1 Technical Report — Report contrasting server-side efficiency work with client-side optimization for LLM-based agents.