In April I wrote about opening our codebase to the entire company. 629 merged PRs in 60 days, 40 of them from 11 non-engineers. The question I got most afterwards: what happens once the honeymoon is over?
On 13 October I answered it on stage at the SDLCAI seminar in Espoo. The talk is called "I Let Non-Engineers Vibe-Code Our Production Codebase". Here's the video.
Watch on YouTube · Slides (PDF) · Q&A with Jussi Hacklin and Sini Tistelgrén
The numbers, February to September
- 7831 pull requests opened by agents
- 4582 merged to production
- 807 driven by non-engineers
- 16 non-engineers still shipping code every day
Merged agent PRs per month grew 3,4× from March to September. It never plateaued.
Most of the 807 are not translation fixes, which is what I expected. Designers now send a preview link instead of a Figma file. Product managers fix the annoying thing they've been filing tickets about for years. Customer success ships the change in ten minutes that would have waited six months in a backlog.
What stuck since April
The agent lives in Slack. You tag it like a colleague, in a public channel, and it comes back with a pull request and a preview link. Adoption spreads by watching. PMs watch engineers, engineers watch each other. The vendor matters much less than the pattern. When our Slack integration got repriced, one of our engineers, Suresh, built Leidang: our own runtime for coding agents, agent-agnostic, Slack built in. Nobody noticed the difference. It's going open source under Apache-2.0, and it's already listed on opensource.epilot.cloud with the rest of what we share.
If you can't see the change, you can't own it. Every PR gets a preview link. That's what made it work for people who will never install an IDE.
What broke
A few weeks in, an agent working for a non-engineer found a way to deploy through our CI pipeline. It never reached production. Our security monitoring caught it. We got lucky.
Same day we landed one rule that everybody can remember: agents and non-engineers open as many pull requests as they want. Engineers review and merge. Same bar as any other change.
You'd expect that to slow things down. Non-engineers merging their own PRs went from 36 a month to 1, by design, and the number of merged agent PRs kept growing. Guardrails build trust, and trust drives adoption.
The bigger lesson was about engineers. A default AWS profile can't tell you apart from your agent, so we cut off all laptop access to production AWS. I wrote that up here.
The honest list
- Review is the bottleneck. Writing code is cheap, understanding every change is not. We use AI on that side too.
- Not every PR should ship. About two thirds of non-engineer PRs get merged, the agent average is 59%. A closed PR costs minutes. That's the filter working.
- Agents are confidently wrong. People learn to push back, and non-engineers pick that up fast.
- Engineers over-trust too. My favourite quote of the year, from a PM: "Unfortunately engineers also just trust whatever AI gives them, so we have some weird translations."
If you want to do this
- Hire the coworker. Put the agent in Slack and tag it.
- Open the codebase, with preview links on every PR.
- Guardrails first. Lock merge permissions before you open access, not after.
- Write it down. Your context is the bottleneck, not the model.
The people closest to your customers can change your software now. Are you letting them?
Happy to compare notes with anyone wrestling with the same questions. The original April post has the full setup.