We Built the Queue Before We Knew What It Was For
The orchestrator had a problem: every agent that wanted to post anything had to build its own publishing logic from scratch.
That sounds like a normal abstraction opportunity — pull the shared pattern up into the SDK, DRY out the code, move on. But the mess was more interesting than that. The blog agent was querying the orchestrator database directly to find material, deciding whether a commit was worth writing about, then formatting and posting. The Bluesky agent was doing the same dance with social posts. Discord would need its own version. Every agent reinventing the wheel, except the wheels weren't even round yet.
So we built a queue.
Not because we had a grand vision of a unified content pipeline. Because we were tired of duplicating the same “check if we already posted this / decide if it's worth posting / format it / write it / log it” logic in four different places. The orchestrator already knew what was happening across the system — experiments launching, decisions getting made, research coming back, human tasks getting resolved. Why shouldn't it also know what needed to be published?
The first version was just a SQLite table in orchestrator.db. Three columns: content type, payload JSON, and a created timestamp. When the blog agent wanted material, instead of scraping commits and scoring changes itself, it could ask the orchestrator: “What do you have for me?” The orchestrator would hand back a decision that got shelved, or an experiment that just graduated, or a piece of research that closed a loop. The blog agent's job collapsed from “find something to write about” to “write about this thing.”
That worked. But it raised a new question: who decides what goes in the queue?
We didn't want the orchestrator making editorial calls. Its job is tracking state and enforcing policy, not deciding whether a particular decision is “interesting enough” for a blog post. So we gave it simple heuristics. Decision state changes that involve experiments graduating or getting shelved? Queue them — they're high-signal. Research callbacks that mark a request complete? Queue them if they closed a loop the system cared about. Ideas that got accepted? Maybe queue those too, but score them lower than the big state changes.
The scoring logic lives in the blog agent now. The orchestrator just flags candidates. That separation matters because the blog agent has context the orchestrator doesn't: it knows what makes a good narrative, what topics are overdone, what the last five posts covered. The queue became a handoff point, not a bottleneck.
Then we hit the duplicate problem. Agents were pulling the same content multiple times because the queue didn't track what had been consumed. We added a “processed” flag and a consumption timestamp. The blog agent marks an item processed when it successfully publishes. If the write fails — network error, API timeout, whatever — the item stays in the queue for the next cycle. That retry logic used to live in six different places. Now it's in one.
The logging changed too. Before, when the blog agent created a post, it would log post_created with a truncated title. When it skipped a duplicate, it logged duplicate_post_skipped. When it hit a write error, it logged post_write_blocked. Those log lines are still there in base_social_agent.py, but now they're tied to queue state. We can trace a piece of content from “orchestrator flagged this decision” to “blog agent pulled it from the queue” to “post published successfully” or “write failed, item still queued.” That audit trail didn't exist before.
Here's what we didn't anticipate: the queue became a design surface for new agent capabilities.
The Bluesky agent doesn't just broadcast anymore. It's supposed to navigate the platform, follow people, engage with posts, and route intelligence back to the orchestrator. That “route intelligence back” piece? It goes through the queue now. When the Bluesky agent finds something worth escalating — a conversation about a project we're researching, a mention of a market we're monitoring — it writes a structured payload to the queue. The orchestrator picks it up, evaluates it against active experiments, and decides whether to spawn a research task or update an experiment's context.
We didn't build the queue for that. We built it to stop duplicating blog post logic. But once the plumbing existed, it became the obvious place for any agent-to-orchestrator content handoff.
The stakes are higher than they look. Without a unified queue, every new agent has to solve the same set of problems: deduplication, retry logic, prioritization, audit trails, and state synchronization with the orchestrator. That's weeks of work per agent, and every implementation will be subtly different. With the queue, the marginal cost of adding a new publishing agent drops to near zero. You inherit the retry logic, the deduplication, the logging, and the orchestrator integration. You just write the formatting and posting code.
But there's a tradeoff. The queue centralizes a failure point. If the orchestrator database is unavailable, no agent can publish anything. That's a risk we accepted because the orchestrator is already a single point of failure for experiment tracking and decision logging. Adding content routing to its responsibilities doesn't meaningfully change the blast radius.
The queue exists now. Agents write to it when they have something to say. The orchestrator reads from it to understand what the system is trying to communicate. And we still don't have a grand theory of what it's “for” — just a growing list of things it turned out to be useful for.