Behavior drift monitor

What it does: Watches running agents for tool-call patterns that diverge from the pre-deployment baseline; alerts on drift.

1. Vision context

Pre-deployment evaluation establishes what the agent should do. Runtime audit captures what it actually does. The drift monitor closes the loop: when production reality diverges from the evaluation baseline, alert.

This is what Enoki bills as "monitor for drift in production" and what Giskard calls "regressions across agent versions." Together with pre-deployment evaluation (Q8), it gives TapPass parity on the lifecycle story Giskard/Enoki claim.

2. Functional specification

Six signals (architecture §12.4 for the full table). Per-sandbox baselines, configurable thresholds, dashboard alerts with audit replay.

3. Technical design

Lives at tappass/drift/. Reads from audit hash-chain; computes rolling distributions; compares against persisted baseline; fires events into the dashboard alert system.

4. Definition of done

All acceptance_criteria pass.
Demo moment: probe sequence simulating drifted prompt template; alert fires within 30 seconds.
Threshold tuning UI in the dashboard.
False-positive guard: legitimate baseline shifts (operator-acknowledged) reset the comparison.

5. Coordination notes

With pre-deployment-evaluator: evaluator's output is our baseline input. Coordinate on the schema of EvaluationBaseline.

Open questions:

(Q) How long to wait before establishing a baseline if no pre-deployment eval ran? Lean: 7 days of production audit + operator confirmation before baseline is "trusted"; until then, drift alerts are advisory only.

6. Out of scope

Continuous in-production red-teaming (would extend pre-deployment-evaluator into runtime mode).
Auto-remediation (operator decides; we alert).