Automated Testing of Electron Apps

I’ve been building my personal photoculler this past week without reading and writing any of the code. Left unattended, coding agents produce slop. I wanted to see if I could prevent them from collapsing under their own slop.

A few weeks ago, I saw a demo of Software Dark Factory techniques from ex-StrongDM folks. The premise is simple: coding agents need rich feedback if they’re going to operate autonomously.

Electron wraps around Chromium, so it can be tested using Chrome CDP. To get started, I reused an agent skill I built to test websites and asked my coding agent to use this skill to test photoculler. The agent can now start the electron app and click through basic user flows. Most of the time, it can verify its own feature, although it sometimes struggles. At the end of a testing session, I ask the agent to reflect on what it struggled with and what could have made its testing session smoother. I use most of its suggestions and ask it to improve the testing skill.

In one of the testing sessions, it struggled to open a folder. That action brings up a native macOS file picker, which Chrome CDP can’t control. After fumbling around, the clanker discovered that it could use osascript instead. When I saw this, I asked the agent to turn that workaround into a helper. Future agents would no longer have to rediscover it.

Another idea I picked up from the StrongDM folks is to give one or more agents a list of user stories to run against your app. The user stories are written in plain English, much like the flows human QAs would manually click through when testing an application end-to-end. The StrongDM folks called this technique Scenario-Based Testing (SBT). I do this periodically with Photoculler. I ask an agent to look at the last few commits and update a SCENARIOS.md file in a separate repo with user stories that define what “done” means for each feature. I instruct my testing agent never to inspect the source code. It treats the software as a black box. At the end of each session, it writes a report with detailed reproduction steps for every problem it finds. I then feed this report back to the builder agent to fix the issues. Keeping the scenarios in a separate repo creates a boundary between builder and tester agent: the builder can’t edit or remove the tests even if it wants to.

Interestingly, my testing agent doesn’t require a frontier model. Given a sufficiently detailed testing skill, even a smaller LLM can do a surprisingly good job as long as it can reliably use tools and follow the workflow.

The interesting constraint may no longer be whether the model can write the software. It may be whether we can give it a good enough feedback loop to know when it’s wrong.

Here is a testing session run by DeepSeek V4 Flash on pi as the testing agent.

Published on June 8, 2026.