Create the folder structure for a competitive intelligence agent.

comp-intel-agent/
├── .env
├── requirements.txt
├── config/
│   ├── rivals.json
│   ├── signals.json
│   └── company_context.json
├── prompts/
│   ├── signal_evaluator.md
│   └── brief_writer.md
├── data/
│   ├── snapshots/
│   ├── signals/
│   ├── briefs/
│   └── logs/
└── src/
    ├── main.py
    ├── logger.py
    ├── http_utils.py
    ├── utils.py
    ├── web_monitor.py
    ├── hiring_monitor.py
    ├── review_monitor.py
    ├── evaluator.py
    ├── brief_generator.py
    └── email_digest.py

Create placeholder files with a short comment in each.
Do not add anything extra.

ANTHROPIC_API_KEY=your_anthropic_key
ANTHROPIC_MODEL_FAST=claude-haiku-3-5-20251001
ANTHROPIC_MODEL_BRIEF=claude-sonnet-4-6
ANTHROPIC_MODEL_CLASSIFIER=claude-haiku-3-5-20251001
DELIVERY_CHANNEL=email

Create requirements.txt with exactly these packages:
requests, beautifulsoup4, python-dotenv, anthropic, jobspy,
feedparser, lxml, twilio

Then run:
pip install -r requirements.txt --break-system-packages

Confirm all packages installed. If any fails, install it
individually and report which one.

Help me identify which competitors are worth monitoring.

Ask me a short sequence of questions, one at a time:
- what my product does and who it's for
- which competitors come up in sales calls
- which tools prospects use instead of us
- which companies we lose deals to, and why
- which adjacent companies seem to be moving into our category

Start with the first question.

Based on my answers, produce a competitor map as a JSON array.

For each competitor include:
- "name"
- "category": "direct" | "indirect" | "comparison_set"
  (direct: same product, same buyer. indirect: different
  product, same underlying problem. comparison_set: appears
  in deals even if not a direct substitute)
- "reason": one sentence on why it belongs in this category
- "monitor_priority": "high" | "medium" | "low"

Do not add companies I didn't mention unless you have strong
reason to infer them. Print the array and wait for my
confirmation before we build the source map.

Using the competitor list we just defined, build a source
map for each one. Create config/rivals.json as valid JSON
(no comments, no trailing commas).

Each entry:
{
  "name": string,
  "tier": "primary" | "secondary",
  "confidence": float 0.0-1.0,
  "notes": string,
  "sources": {
    "homepage": string, "pricing": string or null,
    "changelog": string or null, "blog_rss": string or null,
    "docs": string or null, "integrations": string or null,
    "security": string or null, "careers": string or null,
    "customers": string or null, "g2_slug": string or null,
    "linkedin_company": string or null
  }
}

Infer URLs from company names and standard patterns
(/pricing, /docs, /integrations, /security, /customers).
Do not leave any key missing - use null for unknown values.
Assign "primary" tier to high-priority competitors,
"secondary" to the rest.

Fix the following sources in rivals.json:
[competitor]: docs URL is actually [correct URL]
[competitor]: they don't have a public changelog, set it to null

Read config/rivals.json.
Based on my competitive situation, write config/signals.json
with this structure:

{
  "flag_threshold": 0.6,
  "weights": {
    "pricing_change": float 0.1-1.0,
    "feature_launch": float 0.1-1.0,
    "messaging_change": float 0.1-1.0,
    "documentation_change": float 0.1-1.0,
    "integration_launch": float 0.1-1.0,
    "compliance_signal": float 0.1-1.0,
    "hiring_signal": float 0.1-1.0,
    "customer_win": float 0.1-1.0,
    "competitor_weakness": float 0.1-1.0,
    "activity_trend": float 0.1-1.0,
    "acceleration_trend": float 0.1-1.0
  },
  "strategic_themes": ["pricing_pressure", "enterprise_push",
    "plg_motion", "ai_expansion", "geographic_expansion",
    "vertical_expansion", "compliance_readiness",
    "ecosystem_expansion", "gtm_hiring_push",
    "product_consolidation"]
}

Set weights to my situation: price-sensitive market weighs
pricing_change higher; a feature race weighs feature_launch
and integration_launch higher; visible enterprise hiring
weighs compliance_signal and hiring_signal higher; an early
positioning fight weighs messaging_change higher.

Print a one-line justification for each weight.

Create src/http_utils.py with fetch_with_retries(url, headers,
timeout=15, max_retries=3): default User-Agent, retry on
403/429/5xx and exceptions with exponential backoff
(2s, 4s, 8s), return None after retries exhausted.

Create src/utils.py with:
- clean_html_text(html): strip script/style/nav/footer/header
  and cookie/consent/modal elements, return normalized text
- compute_similarity(a, b): difflib SequenceMatcher ratio
- extract_diff_excerpt(old, new, max_chars=200): first
  meaningful diff block, {removed, added}
- make_signal_id(competitor, signal_type, content, date):
  first 16 hex chars of a SHA-256 hash
- signal_exists(signal_id, signals_dir): filename-as-key check
- load_json_safe(path, default): safe JSON load with fallback

Create src/logger.py: get_logger(module_name) writing to
console at INFO and to data/logs/run_{today}.log at DEBUG.

Create src/web_monitor.py. Every signal saved follows this
schema: signal_id, competitor, signal_type, source_type,
source_url, summary_raw, detected_at, evidence, status "new",
priority_score null, flag null.

Load config/rivals.json and config/signals.json; exit if
either is missing. For each competitor, for each non-null
source URL:

- homepage → messaging_change: diff title/h1/hero/CTA text,
  signal if similarity <= 0.95
- pricing → pricing_change: extract structured pricing where
  possible, signal if similarity <= 0.95
- changelog → feature_launch: one signal per new entry title
- docs → documentation_change: signal on new doc links, plus
  a second pass monitoring up to 5 "important" existing pages
  (API, auth, admin, SSO, webhooks) for content drift below 0.92
- integrations → integration_launch: match a known-brand list
  for high-confidence hits, infer proper nouns for lower-
  confidence hits requiring 2+ co-occurrences
- security → compliance_signal: scan for SOC 2, ISO 27001,
  HIPAA, SSO, SAML, RBAC, MFA, and similar keywords, signal
  on new ones found
- blog_rss → feature_launch: new entry GUIDs not seen before

Store a baseline snapshot per source type on first fetch
(no signal generated). Deduplicate every signal with
signal_exists() before saving. Expose run() and log a summary
of pages checked, signals generated, and failures.

Create src/hiring_monitor.py. For each competitor, scrape
up to 20 recent postings via jobspy (LinkedIn, falling back
to Indeed only if LinkedIn fails). Dedupe against stored jobs
by (title, location).

For each new posting, classify strategic intent with Claude
(model from ANTHROPIC_MODEL_CLASSIFIER) into one of:
product_expansion, market_expansion, go_to_market_push,
customer_operations, infrastructure, leadership_change.
Return {strategic_category, confidence, interpretation}; on
any failure fall back to {"infrastructure", 0.3,
"Classification failed - default assigned."}.

Save each classified posting as a signal using the standard
schema. Expose run() and log postings scanned and new finds.

Create src/review_monitor.py.

G2: for each competitor with a g2_slug, fetch the 10 most
recent reviews. Validate rating is an integer 1-5 and pros/
cons aren't both empty. On first run, save as baseline with
no signals. After that: rating >= 4 → customer_win,
rating <= 2 → competitor_weakness, skip rating 3.

Customer pages: extract candidate names from image alt text
(preferred) and heading/testimonial elements. Filter out
short strings, all-lowercase strings, and common UI phrases
("Learn more", "Case study", etc). Normalize (lowercase,
strip Inc/Ltd/LLC). Baseline on first run; generate
customer_win only for genuinely new normalized names after.

Deduplicate with signal_exists(). Expose run() and log
reviews processed, customer names found, failures.

Create prompts/signal_evaluator.md:

You are a competitive intelligence analyst for a B2B software
startup. Given a signal and context, evaluate:

STRATEGIC RELEVANCE - does this change how prospects compare
us, or require a response?
URGENCY - high (this week), medium (this month), low (track only)
CONFIDENCE - real strategic move, or noise?
STRATEGIC THEME - exactly one from the provided list

Be conservative: 0.8+ strategic_relevance should be rare.
Return valid JSON only:
{strategic_relevance, urgency, confidence,
priority_score = relevance * confidence, flag, summary,
strategic_theme, theme_reasoning, recommended_move}

recommended_move must be specific and actionable - not
"consider updating pricing" but "add a free tier before
quarter-end to neutralize their freemium entry."

Create src/evaluator.py. Load unevaluated signals from
data/signals/. For each: build context (competitor info +
company_context.json if present + valid themes), call Claude
with the evaluator prompt, parse JSON (retry once on failure,
else mark evaluation_failed with score 0).

Score: final_score = priority_score * signal_type weight *
urgency_multiplier (high 1.2, medium 1.0, low 0.8), clamped
0-1. Flag if final_score >= flag_threshold. Merge fields
into the signal file.

TREND DETECTION, after individual scoring, over the last 7
days: group by (competitor, signal_type) and separately by
(competitor, strategic_theme). A group qualifies as an
activity_trend if it has 3+ signals, 2+ with confidence >=
0.5, and no single source_url accounts for more than half of
them. Compare this week's count to the prior week for each
group; if this week is 1.5x+ the prior week and at least 2,
generate an acceleration_trend signal instead.

Expose run() and log signals evaluated, flagged, and trend
signals generated.

Create prompts/brief_writer.md:

You are writing a competitive intelligence brief for a
founder. Lead with the highest-priority signal, no preamble.
For each signal: one sentence on what happened, one on why
it matters, one with the recommended move - three sentences,
no more. Group signals sharing a strategic_theme under one
header. Cap at 7 signals, 400 words total. If fewer than 3
signals were flagged, write a one-paragraph monitoring
summary confirming no significant changes instead.

Format:
[COMPETITOR] [SIGNAL TYPE] — [URGENCY]
What happened: ...
Why it matters: ...
Move: ...

Create src/brief_generator.py. Skip if today's brief already
exists. Load flagged signals not yet included in a brief,
sorted by priority_score, top 10. Call Claude with the
brief_writer prompt and this context. Save to
data/briefs/brief_{today}.txt, mark each included signal, and
print the brief. Expose run().

Create src/email_digest.py. Check data/logs/sent_{today}.json
to avoid double-sending on either channel. Load today's brief
(or "No signals flagged today.").

Email: HTML with a bold header per signal, an urgency colour
chip, three-line content, and a footer noting nothing was
actioned automatically. Send via Gmail SMTP port 587 STARTTLS,
wrapped in try/except so a failure never crashes the run.

WhatsApp: plain text, top 3 signals max, hard 1500-character
limit with truncation. Send via Twilio, same failure handling.

Expose run().

Create src/main.py. Run in order: web_monitor, hiring_monitor,
review_monitor, evaluator, brief_generator, email_digest.
Each step logs its start/end; an unhandled exception in one
step is caught and logged, never blocking the rest. Print a
final summary: signals collected, flagged, trend signals,
brief path, delivery status, log path.

Open the comp-intel-agent project. Run src/main.py.
If the run fails, log the error and stop - do not attempt to
fix code automatically. Report the final summary when complete.