assay / replay
HHG-014 · 22 Nov 2016 · $74.96 · online purchase

The bank scored it 0.05. It was one of 28 cards on one phone.

assay is a fraud investigator that runs on a TigerGraph graph. This page replays every query it ran on each case, in order. The graph grows as results come back, and each finding moves a probability that is calibrated against the bank's 5,565 closed cases. Nothing here is staged: the traces come from rerunning the agent and match the committed answer files.

0.964AUC, held-out month
bank's own score: 0.866
10 of 20cards blocked
8 cleared, 2 held
60investigations it
opened unprompted

HHG-014

txn, high p txn, low p card device closed case assay case

Fraud probability—

    What it just learned

    Graph queries

      Nov – Dec 2016 · 80 investigations

      Where it disagrees with the bank

      Every investigation from the exam period. Each dot is assay's final probability, and the brass tick is the bank's risk score for the same transaction. The line between them is the disagreement. Hollow dots are the 60 alerts the agent raised on its own: structuring bursts, device rings, and fraud the bank's score missed. Click a dot to replay that investigation.

      October 2016 · held out · 100,420 transactions

      When it says 0.30, it means three in ten

      The model learns only from the bank's closed cases for July to September. It is then scored on October, which it never saw, and calibrated with cross-fitted isotonic regression.