termitePolicy tests
These tests use no AI model. They send commands straight to termite's policy, to check termite itself.
The policy is a fixed set of rules in termite's code. It allows a command only if it can prove that the command only reads. It does not depend on a model, so it still works when a model writes something dangerous. The same commands always give the same result.
No command is run in these tests. The policy reads each command as text and gives a result.
Known attacks
I wrote a fixed list of 108 commands. 91 are known attacks that the policy must block, and 17 are harmless commands that it must allow.
| Commands | Correct | Wrong |
|---|---|---|
| Known attacks, which must be blocked | 91 of 91 | 0 |
| Harmless commands, which must be allowed | 17 of 17 | 0 |
Last run on 5 November 2026, on the Local Sandbox on Mac, with code version 2c7be42-dirty.
This test shows that the policy stops the attacks on this list. It cannot show that the policy stops an attack that nobody has thought of yet.
Commands from public datasets
I also sent every command in two public datasets through the policy: 46,518 commands, after duplicates were removed. Nothing was left out because of its size or shape.
These datasets were made to train models, and most of their commands are unverified. They are not a record of commands that people ran. Some of the commands contain typing errors.
- 90.0%
- Allowed, and on the hazard list. Each one is a possible policy bypass to check. The commands stay private.
- 3,1066.7%
- Allowed, nothing flagged. These commands only read.
- 4,64310.0%
- Blocked, and on the hazard list. The policy blocked these hazards.
- 38,76083.3%
- Blocked, nothing flagged. These may be harmless. The policy blocks anything that it cannot prove only reads.
Last run on 5 November 2026, on the Local Sandbox on Mac, with code version 2c7be42-dirty.
This test shows what the policy allows across many kinds of command. A blocked command is one that the policy could not prove only reads. Many of these commands do write, so this figure does not show how much harmless work the policy blocks.