Back in January I was playing with my version of a dark factory. I called it Black Hole Architecture. The experiment lived inside Helios and used files and a schedule to keep a group of coding agents moving without a manager agent.
It was a good place to try weird ideas. Helios was my project. If a run went wrong, I lost some time and had to repair whatever it did to the video engine. When the gravity got weak, I learned something. Nobody else was waiting on the result.
The World Cup project was not the garage.
This was work for a climate-focused nonprofit. Real science was going into it, and editors, partners, and an eventual audience would all rely on it. I was not going to point an unattended coding system at the project and hope that the science came out right. I stayed in the loop.
Even with me driving, AI changed how fast the prototype grew. Within a few days it had a homepage and routes for matches, teams, venues, stadiums, brackets, and stories. We also had shareable graphics and a video. It looked much closer to a finished product than it actually was.
I knew which parts were placeholders because I had built them. A reviewer did not. They could not tell whether a number had been approved, whether a paragraph was a first draft, or whether an interaction was only there to show an idea. A broken button looked just as intentional as everything around it.
That made the prototype hard to review. People were not confused because they had missed something. The page was giving them the wrong signal.
Review Needed a Better Pen
The team needed a decent way to write directly on the staging prototype.
Slack threads, meeting comments, screenshots, documents, and side conversations can carry feedback for a small page. They fall apart across dozens of routes and states. Context evaporates almost immediately: Which branch was this? Which element? Was the note about copy, data, layout, or behavior? Did somebody fix it already? Is the screenshot from the current version or three deploys ago?
I built Preview Feedback for that problem. It was a small widget available only on staging. A reviewer could select an element, leave a comment, capture a screenshot, and send the package to Notion. The resulting ticket included the page URL, selected element, coordinates, branch context, reporter, screenshot, and status.
The interface borrowed a lot from Vercel's comments feature. The difference was what happened after the click: I designed the ticket around the context a coding agent would need for a clean handoff. We used Notion because that was already where the team worked. No grand platform decision there.
The widget did its job, perhaps too well. Reviewers have submitted exactly 288 tickets on the World Cup prototype so far. The busiest day produced 45.
Feedback Became a Schedule for Attention
Of course people had notes. Interfaces invite opinions the way a couch invites someone to ask whether it might look better against the other wall. People are spatially haunted.
What surprised me was how manageable the pile became once each note lived in a durable, queryable system with usable metadata and a decent MCP connection.
In the old process, product managers, engineering managers, and leads protected the team from that pile. They absorbed requests, translated them, trimmed scope, and decided what deserved implementation time. That guardrail still matters. I suspect it matters more when AI is involved.
What changed was the price of the obvious work. When implementation is expensive, almost every request has to compete for attention before anybody touches it. Once small changes become cheap and can run with less supervision, priority is no longer the only useful filter. Another question appears: Is this request clear and safe enough to do without borrowing human judgment?
That is the opening for a dark corner. Not an autonomous factory, but a bounded part of the process where trusted, low-risk feedback can turn into work. Humans keep the decisions that still contain actual product ambiguity. The already-obvious work stops taking up the same queue.
The Actionability Gate
I started calling this decision point the gate. It answers one question: can this ticket move without asking another person to make a product decision?
That is not the same as asking how important the ticket is. "Reframe this entire page" may be very important and completely unready for an agent. The team still has to agree on what the page should say. A wrong button label may be less important, but if the replacement is already approved, the work is ready.
I look for four things at the gate.
First, can I point to a reason for the change? It may be in the product brief, research, meeting notes, source material, or another ticket. I do not need one perfect `VISION.md` file. I do need something better than the agent deciding what feels right.
Second, is the request literal enough to complete? "Use this approved label" is clear. "Make the page more urgent" is the beginning of a conversation, not an implementation task.
Third, what does the change actually touch? A six-word Notion ticket can still change shared navigation, a data contract, or a component used on twenty pages. The size of the comment tells me almost nothing about the size of the work.
Finally, who left the note? A science reviewer correcting methodology language knows something different from a casual reviewer commenting on spacing. The system needs to keep that context.
The gate is not a priority score. It is permission. The ticket either contains enough information to move, or it goes back for another question.
What We Actually Ran
The World Cup workflow was not autonomous. I drove it by hand.
I read the Preview Feedback tickets along with meeting notes, research, and source documents. Then I pulled together a batch that looked clear and gave it to an agent. I integrated the changes, checked the browser, and updated the ticket when the work was actually finished.
The boundaries mostly lived in my head. I would tell one agent to take a group of copy tickets and another to stay away from the same files. I would remove tickets that were already fixed and call out the pages that were off limits. This worked because I kept watching it.
That is an important difference from Black Hole Architecture. In Helios, the file boundaries belong to the system. Two agents working at the same time should not be able to edit the same surface in the first place.
On this project, agents sometimes shared a local workspace and another coding agent acted as the integrator and verifier. It was useful, and it was much faster than doing every change myself. It was not unattended. If I had walked away, the process would have stopped making sensible choices.
The next version should not require me to remember every lane.
The Grill Loop
What happens when the ticket is not ready? I do not want the agent to guess.
I send it into a grill session instead. The agent asks one specific question at a time and can include a recommended answer when that helps. The pattern was inspired by Matt Pocock's Grill Me skill, which is worth trying.
I ran these conversations through my local coding agent. Eventually I could imagine a cloud version replying to the original Notion ticket, but it would need a sandbox and very limited Git access. The interface is not the important part. The important part is that a vague ticket produces a question instead of an invented decision.
Sometimes one answer is enough. Sometimes a ticket splits in two, with the clear half moving forward and the rest staying in the conversation. Some ideas get parked because they are good ideas at the wrong time.
That is a much more honest failure mode than pretending every comment is ready to become code.
The Pieces I Would Keep
The system eventually resolved into a short set of parts, though they were not all formal software components.
Vision supplies direction. In Helios it was usually a `VISION.md` file. In a normal organization it is smeared across briefs, research, meetings, source documents, and half a dozen tools, so the system needs access to more than one neat file.
Feedback needs enough context to survive the handoff. A stray sentence in a meeting chat is easy to lose. A ticket with a screenshot, route, selected element, branch, commit, reporter, and status is much closer to work.
The gate decides whether the ticket moves, gets grilled, needs deliberation, or parks. The score supports that choice with vision confidence, clarity, risk, lift, source, and trust.
Separation makes parallel execution safe. A person can enforce it operationally, as I did here, or the architecture can enforce file and role ownership.
Memory is everything the next run can inspect: tickets, status, meeting notes, research, source documents, commit history, timeline checkpoints, and implementation notes. Without durable memory, yesterday's lesson becomes today's rumor.
Closeout means verified completion. Agents are good at producing receipts even when I do not trust their claim that a task is done. Playwright screenshots and short demo videos cover a lot of frontend evidence. I also run staging or production smoke tests with computer use, alongside ordinary automated tests. For another angle, an adversarial review agent can inspect a first pass in a fresh thread. Two independent reviews can be useful on risky work.
I still prefer to design quality into the loop at the beginning. Closeout should confirm the system worked, not rescue a careless process at the end.
Dark Corners, Not Dark Factories
I no longer expect the factory to go completely dark. I expect it to develop dark corners.
A dark corner is a bounded autonomous zone inside a process that humans still lead. Trusted internal reviewers leave structured tickets. The gate evaluates actionability. Clear tickets enter separated execution lanes, verification runs, and the Notion status changes only after evidence exists. Everything else goes to the grill loop or back to product, editorial, design, or science review.
This does not remove product managers, engineering managers, designers, editors, scientists, or product engineers. It gives them fewer settled details to babysit. Agents are not deciding what the product should become; they are taking care of work for which the decision has already been made.
I am not ready to connect public feedback to this machinery. A stranger's comment is untrusted input. Give it a path to tools and now I have to worry about prompt injection, permissions, and private data.
Could that input move through quarantine and a human review first? Probably. I have not built it, and I would not pretend the boundary is solved. For now I would begin with colleagues whose role and access I already understand.
What I Would Change Next Time
To put it simply, I built too much before I explained how to review it.
On the next project, the first staging page would be a prototype brief. I would put the actual component on that page and explain why we made it. The research and source links would be right there. So would the questions we need the reviewer to answer.
I would also label the unfinished parts. For example:
- This number comes from the approved dataset. - This interaction is only a placeholder. - This paragraph still needs science and editorial review. - Leave feedback about the chart here.
Reviewers should not need to remember a meeting from three days ago to understand what they are looking at.
This feels obvious after collecting 288 tickets. It was not obvious while I was excited about building the application. I kept adding pages because adding pages was going well.
The result looked finished, so people naturally reviewed it as a finished product. A brief would have told them to pull the idea apart instead. I still want to design in the browser. Next time I will design the review page too.
What the Prototype Taught Me
So what did the prototype actually teach me?
I went into the project thinking that coding speed would be the interesting part. It wasn't. The pages arrived quickly, and then we had to figure out how to review all of them. The 288 tickets tell that story better than any benchmark could.
When development takes several weeks, a poor handoff can hide inside the schedule. There is time to have another meeting or send another document. We did not have that hiding place. The prototype was ready for comments before the review process was ready for the prototype.
Now I look for a very specific kind of work. The decision has already been made. The person asking for it is someone we trust. The agent can stay inside a known group of files. We have a way to check the result when it is done.
That work can move into a dark corner. Everything else still needs the scientist, editor, designer, or product person who owns the decision.
I am not trying to remove those people from the factory. I am trying to give them fewer button labels to babysit.