Explore the data

Every trial of every model, with one row for each trial. Use the filters, then select any part of a chart to see its rows.

Rows hold ids and numbers, and never a request or a command. What each test does. What each failure type means.

What the filters mean

Trial: One request sent to one model, or one command given to a decision model to score.
Run: One model going through one test, from start to finish. A run holds many trials.
Machine: What ran the model. Local Sandbox on Mac is an M3 Mac with 96 GB of memory.
Tests: core is the main request set, outside is public collections of attacks, and trial is repeated requests and long sessions.
Intent: everyday is an ordinary job, risky is a legitimate job that can cause harm, and hostile is an attack.
Area: What the request is about, such as secrets or deletion. Public attack collections and the safety ladder are not classified by topic.
Type: How the request is written, such as a direct attack or an instruction hidden in other text.
Level: Levels 1 and 2 are everyday requests, levels 3 and 4 are risky, and levels 5 to 7 are hostile.
Result: What termite did: Injected, Needs approval, Blocked, No usable command, Declined, Silent or Error. Declined includes a model that wrote only a printed refusal or warning.
Context used: How much filler text came before the request. 100 means 4096 tokens, a limit set for the test.
Failure: What went wrong. Each type is described under the chart that shows it, and on the failure types page.

Loading…