Cloudflare in an agent architecture: what belongs at the edge

Implemented web capture and a methodology for evaluating Workers, queues, and workflows without adding unnecessary pieces.

Newstools team3 min read
Cloudflare in an agent architecture: what belongs at the edge

An agent platform can combine local services, AI providers, and distributed tasks. Cloudflare offers options for placing part of the work at the edge, but each component needs an operational reason. In our work we distinguish capabilities already in use, such as web rendering and capture, from an orchestration architecture that must be tested for each business's case.

A concrete integration: browsing and extracting

The technical audit tool includes a Cloudflare Browser Rendering client and a pipeline that receives rendered HTML, extracts metadata, and produces the same format as local processing. This makes it possible to handle JavaScript pages without creating a different output for each technology. The job keeps its state and results, and capture limits are set before running it.

That use does not mean our entire agent platform is deployed on Cloudflare. For example, the current operation of Jarvis uses a private gateway and an outbox in PostgreSQL. For an installation with a single owner, adding another queue can increase maintenance without improving the process.

There is also a configurable integration with AI Gateway in an editorial backend, tested in development. Its role in accessing providers is not the same as running the whole agent cycle. Runtime, state, permissions, and recovery still need an architecture of their own.

Propose a work boundary

When execution needs to be distributed, we design what a Worker receives, what work is sent to a queue, and which steps need a durable workflow. The message carries references and a job identity; sensitive documents remain in authorized storage. Reception validates who requests the task, which capability it is allowed, and the applicable budget.

Service names do not replace application guarantees. A task may repeat after a failure, so the executor must recognize work already done. An accepted request needs a queryable status. An approval wait must keep its scope and expiration. Tools that write to third parties keep their own authorization control.

Evaluate before operating

The pilot tests failures between steps, duplicates, provider limits, cancellation, and costs. We compare a centralized solution with the distributed alternative and measure complexity, latency, and the need for supervision. Observability joins input, execution, and result through a common correlation.

Workers, Queues, Workflows, or an AI gateway are chosen when they fit that test and the client's constraints. We present them as possible pieces of an architecture, not as a complete package already deployed across all our systems. Our service consists of designing, implementing, and verifying the right combination while keeping permissions and decisions visible.

Let's evaluate where to run your agents.

You might also like