AI Audit

The AI audit (stage 7) reviews medium+ severity findings with an LLM. The auditor can adjust scores within bounded knobs, flag false positives, surface overlooked details, and recommend a response. Audit results are persisted into each finding’s ai_analysis.

How it works

The audit runs for findings at or above the configured severity threshold (default medium).
It talks to the LLM through an MCP stdio tool server (ai/mcp/audit_mcp_server.py), which exposes tools to the model for reading finding context and writing the verdict back to Rails.
The model can persist its audit via the Rails internal API (PUT /v1/scans/:id/audit), which validates and stores the result.

The verdict

An audit verdict contains:

Field	Constraint
`adjustment`	integer, within `[adjustment_min, adjustment_max]`
`confidence`	0–1
`is_false_positive`	boolean (a reason is required when `true`)
`recommended_action`, `insight`	non-empty, ≤ 300 chars
`overlooked_details`, `relations`, `additional_risks`	optional lists, ≤ 10 items, ≤ 300 chars each

On persistence, Rails recomputes:

adjusted_score — raw score clamped to 0–100, offset by the adjustment.
adjusted_severity — bands: critical ≥ 85, high ≥ 65, medium ≥ 40, low ≥ 20, else info.

Configuration

The audit is configured two ways:

Admin panel (source of truth) — persisted in PlatformSetting:
- audit_config: threshold (default medium), adjustment_min (default -15), adjustment_max (default 15).
- ai_provider: enabled, provider_type, api_key, base_url, default_model, max_tokens, temperature.
Environment fallbacks — AI_AUDIT_ENABLED=1 master switch; AI_ENABLED, AI_PROVIDER, AI_API_KEY, AI_BASE_URL, AI_MODEL fill gaps only when the admin store is unconfigured.

See AI providers for the full provider configuration reference.

Per-finding summaries

After the audit, the LLM’s insight is copied into the finding’s summary, so the dashboard’s AI Summary column shows a genuine per-finding verdict rather than a generic digest.

Deterministic fallback

The platform never blocks a scan or report on a model call. When a provider is unavailable, misconfigured, or fails:

A deterministic AI analysis is produced instead (still finding-specific: score, severity, phishing verdict, threat feeds, risk criteria, registrar, hosting IP).
A warning is logged with details:
- report_ai_provider_unavailable (reason ai_disabled / missing_api_key)
- ai_insights_skipped_no_provider
- report_ai_deterministic_fallback (with scan_id)
- ai_audit_skipped_not_enabled

Keywords

The AI layer is also used for brand keyword expansion (before scans): an LLM infers additional brand keywords from the brand context, persisted through the save_keywords MCP tool and rate-limited to 4 expansions per 30 minutes per user.