System Overview

Brand Protect is a two-application system: a Rails 8.1 web application and a Python 3.14 microservice, orchestrated with Docker Compose and connected by a token-authenticated REST API plus Redis pub/sub.

The two applications

Rails web application (rails_web/)

The user-facing control plane:

• Authentication — custom cookie-based sessions (no Devise), signup, password reset.
• Dashboard — live scan monitoring, findings tables, campaign clusters, reports.
• Brand profile — owned domains, brand names, social handles, keywords, logo, webhook URL.
• Triage — analyst review queue with single and bulk feedback.
• Admin — user registry and platform settings (AI provider, audit configuration).
• Orchestration — starts/stops/resumes scans, dispatches enrichment, generates reports, delivers webhooks.
• Persistence — PostgreSQL tables for users, scans, findings, reports, weights, and settings.
• Real-time — ActionCable streams scan events to the browser; a Redis pub/sub consumer writes Python-produced data into the database.

Python microservice (services/)

The heavy-worker:

• Domain generation — dnstwist-based lookalike permutation.
• Enrichment — RDAP/WHOIS, DNS, HTTP, TLS, screenshots, infrastructure intelligence.
• Social discovery — handle permutation, dorking, Maigret probing, embedding-based identity matching.
• Severity scoring — 29 weighted signals, threat-feed cross-reference, campaign clustering.
• AI audit — LLM review via a model-context-protocol (MCP) stdio server.
• Reporting — WeasyPrint PDF generation and takedown email drafts.

Communication paths

┌──────────────┐   REST (Bearer)   ┌────────────────┐
│    Rails     │ ────────────────► │    Python      │  scan start, enrichment,
│              │ ◄──────────────── │                │  keyword expand, reports,
└──────┬───────┘                   └───────┬────────┘  takedowns, narratives
       │  Redis pub/sub                    │
       │  (progress, findings, profiles)   │
       ▼                                   ▼
   PostgreSQL 16                     Redis 7 ──► GlitchTip (Sentry)

1. REST API (Rails → Python)

PythonApiClient in Rails calls the Python service over HTTP with a shared bearer token:

Call	Purpose
POST /v1/scan/start	Kick off a scan (owned domains, keywords, social handles, enrichment snapshot)
POST /v1/domain/enrich	Enrich a single owned domain
POST /v1/social/enrich	Enrich a single social handle
POST /v1/brand/keyword_expand	LLM keyword inference
POST /v1/report/generate	Generate the full + summary PDFs (base64)
POST /v1/campaign/narratives	LLM campaign narratives
POST /v1/takedown/generate	Takedown email draft

2. REST API (Python → Rails)

V1::InternalApiController lets Python read and write state that Rails owns: scoring weights, analyst feedback samples, campaign stats, the AI provider and audit configuration, derived keywords, brand context, and audit data.

3. Redis pub/sub (Python → Rails)

During a scan, Python publishes progress and results onto Redis channels (brand_protect_cable:*). The Rails scan_db_consumer thread subscribes and:

• merges per-stage progress into scan_runs.stage_states;
• broadcasts progress to the browser via ActionCable;
• upserts findings and social findings;
• upserts domain/social enrichment profiles;
• fires report generation and webhook delivery on scan completion.

4. ActionCable (Rails → Browser)

ScanChannel streams scan_<id> events to the scan owner’s browser. The dashboard also polls GET /dashboard/status every 2 seconds as a fallback/refresh path.

Failure handling

• Auth fails closed — if INTERNAL_API_TOKEN is unset, the internal API returns 401.
• Deterministic fallbacks — PDF reports and takedown emails never depend on a model call; the AI layer degrades to deterministic generation and logs a warning.
• Graceful degradation — optional dependencies (pyjarm, shodan, censys, securitytrails, PIL, ssdeep/tlsh) degrade rather than crash a scan.
• One-shot retry — enrichment retries once if a full run fails.

Continue reading

• The Rails application — controllers, models, jobs, and the pub/sub consumer.
• The Python microservice — packages, pipeline, and messaging.
• Messaging — Redis channels and ActionCable streams.
• Data models — the domain and social finding lifecycles.