Back in January I'd been messing around with my own take on what the industry has started calling a dark factory. My side-project name for it was Black Hole Architecture, and most of the experimenting happened inside a project called Helios: long-running goals, agents defining their own tasks, a Vision doc acting as gravity, time used as a mutex for batch agent work, stateless agents functioning as transition functions over a durable state machine that lived in the repo itself.
That was the fun version, and it was forgiving in the way private experiments are. When the “gravity” got weak, I learned something. When an agent wandered off in a weird direction, the blast radius was my own time and some broken video-engine code. You can do strange things in the garage, because the garage doesn't have stakeholders.
The World Cup project was not the garage.
This was real work here at a climate-focused nonprofit, with real science behind it, real editorial stakes, real partner and audience context. I wasn't about to point a full dark factory at that and hope the thvibes came out scientifically accurate. It needed ordinary human caution and diligence.
But AI still changed the speed, and the prototype got big fast: homepage, match pages, team pages, venues, stadiums, tournament overview, bracket views, shareables, story paths, even a video within just a few days. It started looking like an actual product way earlier than a normal process would ever have allowed.
Mostly that was a win. Speed has side effects, though.
When the team saw the prototype, nobody quite knew where to begin, because there was too much surface area and the artifact looked more settled than it actually was. People were reacting to something product-shaped even though huge chunks of it were still provisional.
And their response was reasonable. I'd handed them something that looked closer to final than it was, and we didn't yet have a good enough system for saying "this part is a proposal, this part is scaffolding, this part needs editorial review, this part is just waiting on an obvious fix."
That's where the story starts.
Review Needed a Better Pen
The team needed a better way to leave notes on the staging prototype.
Reviewing a prototype through Slack threads, meeting comments, screenshots, docs, and side conversations doesn't scale. It barely works when the prototype is small. Once you have that many pages and states and routes and shareable surfaces, feedback loses its context almost immediately. What page was this about? Which branch were they on? Which element? Was the comment about the copy, the layout, the data, or the interaction? Did somebody already fix it, or drop it on purpose? Is that screenshot even from the current version, or from three deploys ago?
Preview Feedback was my answer to that specific problem: a small widget that only existed on staging. A reviewer could select any element on the page, leave a comment, grab a screenshot, and send the whole thing into Notion, where the ticket landed with useful metadata attached: page URL, selected element, coordinates, screenshot, branch context, reporter, status. The UI borrowed heavily from Vercel's comments feature, but from day one it was built around producing the metadata and context an agent would need for a clean hand-off. Notion, because Notion was where we already lived.
It worked, and it seemed helpful. The deeper lesson showed up almost immediately, because we got a ton of tickets. With a prototype surface that large, the team could generate feedback very quickly. To date they've submitted exactly 288 tickets for the World Cup prototype, with a single-day high of 45.
Feedback As a Scheduling System for Attention
The surprising thing wasn't that people had feedback. Of course they had feedback. UI invites feedback the way a couch invites someone to say "actually, can we move this over there?" People are spatially haunted.
You'd expect the result to be a pile of ignored backlog tickets. What surprised me was how manageable everything became once the feedback lived in a durable, queryable system with a decent MCP.
The old process usually protected teams from that pile by routing everything through PMs, EMs, leads, whoever happened to be holding the bag that week. They absorbed the requests, translated them, limited scope, and decided what actually reached implementation. That guardrail still matters. Honestly, it probably matters more with AI, not less. But AI changes the economics around the obvious stuff. When implementation is expensive, nearly every request has to compete for our attention before it can become work. When implementation gets cheap and autonomous, the question shifts. It's no longer only "is this the highest-priority thing?" Sometimes the more practical question is "is this safe and clear enough to just do?"
Which is the opening for a dark corner. A dark factory is impractical but a dark corner could be different. It's a scoped zone where obvious, trusted, low-risk feedback can become work without stealing product-engineering attention from the decisions that actually require thought. The goal is to stop spending time and product judgment on things that have already become obvious.
The Gate
The key primitive is what I ended up calling the gate: the thing that decides whether a piece of feedback is ready to become work right now.
The gate asks whether the work can move safely without another product decision. Whether or not something is safely actionable right now, aka its actionability.
A ticket can be important and low-actionability at the same time. A vague request to change the entire framing of a page might be critical, but if it needs editorial, design, science, and product alignment then it’s obviously not ready to be worked on yet. And a ticket can be small and high-actionability: a confusing label, a broken link, a copied phrase that already has an approved replacement, a layout issue with a clear screenshot and a narrow component target.
Those small tickets aren't worthless either. Some of them are basic needs in the Kano sense, the things users or stakeholders only notice when they fail. The label is clear, the navigation works, the metric makes sense, the page doesn't clip on mobile, etc.
The Score
The gate scores each ticket to decide how ready it is for work based on a few factors. This is different than priority which asks what’s the most important thing to work on right now, it’s more
Vision confidence asks whether the request aligns with the product direction. That direction might live in a formal brief, meeting notes, user research, stakeholder feedback, source docs, or the prototype itself. It doesn't have to be perfect, but it has to be referable, because if nobody can say what the product is trying to become, every piece of feedback becomes equally plausible.
Request clarity asks whether the comment is specific enough to execute. "Change this label to this approved phrase" is clear. "Make this feel more urgent" isn't useless, but it isn't executable yet. That one belongs in a conversation.
Risk and lift ask how much can go wrong. A narrow design tweak on one surface is a different animal from touching shared metric semantics, reusable navigation, data contracts, or a component that shows up on many pages. Risk has to consider the actual codebase and architecture in context of the ticket ask.
Source and trust context asks who's asking and what context they had. A science reviewer pointing at methodology language carries different weight than a casual note about visual polish.
Put those together and you get a practical decision: can this become work now? If yes, delegate it. If not, grill it, split it, park it, and route it to the right people.
What We Actually Did
The World Cup workflow was not autonomous. I drove it by hand.
I read PF tickets, meeting notes, research docs, and source docs. I kicked off agents on the batches of work that looked safe and clear, integrated the results, verified them, updated statuses.
The separation of concerns was mostly operational, meaning I was slicing the work. This agent handles these tickets. That agent stays away from those files. This batch is copy-only, that one is visual polish, these are excluded, these are already done, these are ambiguous, and nobody touches those surfaces.
Architectural separation is the stronger version, where the system itself prevents collisions. Black Hole Architecture depended on that. If two agents can touch the same surface with no rule stopping them, they'll happily dig themselves into the dumb zone, burning time and tokens on merge conflicts.
The World Cup work often ran on operational separation instead. Sometimes agents shared a local workspace, with Codex acting as integrator and verifier. Useful, but not the same thing. It worked because I was paying attention. The future version probably shouldn't require that much of it.
The Grill Loop
Anything that didn't clear the gate went into a grill session: the AI asks one sharp question at a time, with a recommended answer attached when helpful. Inspired by the grill skill by Matt Pocock which I highly recommend trying if you haven’t yet. I had my local coding agent grilling me but in the future you could imagine us building a sandboxed cloud agent version with git access that does this directly in Notion as follow up comments on the original ticket. The grill loop exists to keep things moving without letting a coding agent make assumptions and decisions nobody asked it to make.
A ticket can leave the loop a few ways. It can become highly actionable, if an answer supplies enough clarity to delegate cleanly. It can split, because maybe one piece is ready now while another still needs some thought, so the ready piece goes to an agent and the rest keeps getting grilled. Or it can get parked, because some ideas are real and obviously good and simply not ready yet.
This is the manual fallback loop I'd want in any future system. High-actionability work flows to execution; everything else flows to questions. That's the whole posture.
The Primitives
If I had to name the primitives, these are the ones I'd name.
Vision is the source of gravity, the statement of what the product is trying to become. In the original side-project Black Hole Architecture it was usually just a static VISION.md sitting in the repo. In a normal work environment, vision is smeared across a dozen surfaces and platforms, and you'll want your agent to have access to as many of them as you can manage.
Feedback. A comment in some random Zoom team chat gets lost. A comment with screenshot, route, element, branch, commit SHA, reporter, and status is much closer to being work.
The Gate is the decision point: become work now, get grilled, get deliberated on, or get parked.
The Score is permission logic: vision confidence, request clarity, risk and lift, source and trust context. It is not priority.
Separation of concerns is what makes parallel work safe. Operational separation is a person slicing and coordinating; architectural separation is the system enforcing ownership and boundaries.
Memory is everything the system can use later: PF tickets, Notion status, meeting notes, research docs, source docs, commit history, prototype timeline checkpoints, implementation notes. Memory is what keeps work from turning into folklore.
Closeout is verified completion. You shouldn't trust agents, but they're pretty good at producing receipts. Screenshots and demo videos captured through Playwright cover a lot of frontend verification. Agents have gotten a lot better at computer use and I like to run smoke tests using these directly on staging/prod links as an additional layer to more traditional testing. For quality checks, you can run an adversarial review agent on a loop whenever another one finishes a first pass. Not enough? Run two. Do it in new threads too so they’re less biased. But I definitely believe code quality really ought to be handled through proper systems set up at the start of the work loop rather than waiting for closeout.
The Grill Loop is the human clarification path: one question at a time until the work becomes actionable, splits, or parks.
None of these primitives are fancy, and that's kind of the point.
Dark Corners, Not Dark Factories
The factory won't go completely dark. It will have dark corners.
A dark corner is a bounded autonomous zone inside a human-led product process. The corner has a Vision. It has structured feedback, an Actionability Gate, architectural separation, memory, and closeout. Trusted internal reviewers leave PF tickets; the system scores actionability; high-actionability tickets route into separated execution lanes; agents implement; verification runs; the Notion ticket updates only once evidence exists. Anything under the threshold goes to the Grill Loop, or to product, editorial, design, or science review.
That's the useful version, and it doesn't remove PMs, EMs, designers, editors, scientists, or product engineers from the process. It hands them back time. The point was never to let agents decide what the product should become. The point is to stop asking humans to babysit work that's already clear, bounded, and safe.
External user feedback is a different problem entirely. That's not just more signal — it's untrusted input, and if you wire untrusted input directly into agentic execution, you've changed the security model. Prompt injection, data exposure, tool access, and automated action all start hanging out at the same party, and that party needs adult supervision. External feedback might eventually feed dark corners, but only through quarantine, sanitization, rate limits, review layers, and a much stricter trust boundary. Internal feedback is the safe place to start.
Start with the room where you know who's holding the pen.
What I'd Do Differently Next Time
Before building one giant app-shaped prototype again, I'd build a "reviewable prototype brief" on staging: a page that embeds the live proposed pages or components inline, alongside the rationale, the research context, the open assumptions, and the exact feedback prompts the team should react to. Here's the component. Here's why it exists. Here's what's fake, here's what's real, here's what needs your eyes. Leave feedback right here.
That would have changed the tone of the whole review. A finished-looking app says "approve me" by default. Framed properly, it should say "push on me." Different invitation entirely.
The brief gives reviewers context at the exact moment they're reacting, and it makes the prototype honest about its own state. I still believe in building in the browser when the browser is the right medium but the collaboration surface needs design too.
The Retrospective Lesson
The biggest lesson was not that AI can build fast. That part is no longer surprising.
The lesson was that faster building reveals slow coordination.
It’s easy to ignore when implementation is the slowest part of the system. Once implementation speeds up the handoff surfaces start glowing.
That glow is useful.
So, build dark corners.
Not everywhere. Not the whole factory. Just the places where the work is clear enough, trusted enough, bounded enough, and valuable enough to move without babysitting.
Keep us in the room where judgment happens.
Let the obvious work fall inward.