A method for catching bugs by having agents draw state diagrams and exhaustively searching with TLA+
lauren shared a method for catching bugs with AI: first have the agent draw the code and user flows into a state diagram, then call the formal verification tool TLA+ to exhaustively search for violating paths. In the open source project pstack, she built a meta-skill that creates skills; users only need to define the rules that must always hold, and the AI will generate a verification suite and flag violating branches. The author believes this approach of first drawing state diagrams and then exhaustively searching according to baseline rules can also be applied to business processes such as cross-department approvals.
Why read this
AI score 65
The original article breaks formal verification into two steps: drawing state diagrams and exhaustive search following ironclad rules, and provides a reusable meta-skill approach that is easy to transfer to other processes.
Topics
View original postthere's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see
View original
there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see
Body · AI translation
The core actions are just two steps: first have the agent draw the code and user flows into a state diagram, then call the formal verification tool TLA+ to exhaustively hunt for bugs.
Normally, when we test software or review code, we rely by default on the human brain to think up test cases, but what humans are worst at is exhaustively covering edge states of multi-device concurrency. Take the chat app notifications demonstrated in her diagram: a user has a channel open on their computer but hasn't clicked into the discussion thread, so the reply gets silently swallowed as read,
or the phone has clearly already seen the message yet it keeps vibrating wildly in the pocket; or even with Do Not Disturb turned on the phone still rings, and the computer wakes from sleep still flashing fake unread badges. These bizarre cross-device interleavings are always missed by manual testing like a sieve, yet real users will inevitably hit them on their very first day online.
The smartest part of her approach is that it democratizes what was once an unattainably high-brow, math-grade formal verification. The reason nobody was willing to use TLA+ before is that hand-writing business requirements into rigorous mathematical specifications is simply too torturous, and the moment the code changes you have to rewrite them.
Now, in the open-source project pstack, she has directly built a meta-skill that creates skills. You only need to define the ironclad rules that must always hold, such as "if Do Not Disturb is on, it must absolutely never vibrate," and the AI will automatically draw the state diagram for you, generate a verification suite, run every possible state branch inside out, and flag and expose all violating paths in red.
This is like repairing an underground maze with hundreds of forks. In the past, finding dead ends relied entirely on sending a few testers in with flashlights to brute-force their way through; even after ten runs they could never touch every remote corner.
In the end, this logic of first drawing a state diagram and then exhaustively enumerating against ironclad rules doesn't belong only to programmers. When we design cross-department approvals, onboarding-transfer-resignation handovers, or bonus calculation processes in a company, what most easily goes wrong is likewise always edge cases colliding.
For example, an employee's leave day happens to trigger a skip-level approval, or during a transfer handover period the old department's permissions are still running. Rather than holding ten post-mortem meetings to argue after an incident, it's better to first have the AI draw the business rules into a state diagram, then hit it one by one with bottom-line rules. Wherever the diagram can't be drawn is exactly the loophole in your rules that you haven't thought through.