AI Solutions is Signal's before-and-after impact tracker for internal AI tool rollouts at Power Digital. It lives at /solutions (list) and /solutions/:id (detail) under the "Innovation" section of Signal's sidebar, and is visible only to Gandalf-tier users (executive leadership plus the AI/ML team). It answers the question "did this AI deployment actually move the needle for the part of the org that got it?" by freezing a baseline of business metrics and client-health scores when a solution is registered, then continuously comparing current numbers against that baseline. Business metrics (revenue, RPE, seats, retention) originate in Vena (finance contract data); client-health scores (churn risk, sentiment risk, service-expansion readiness) originate in the AI/ML team's churn and service-expansion prediction models, with sentiment ultimately derived from Gong call recordings and Pulse survey responses; the solution registry, snapshots, and team assignments are Signal's own records (entered by users in Signal); the employee picker roster comes from Nova's user directory.
Field | Value |
Route | /solutions (list), /solutions/:id (detail) |
Access | Gandalf only (GandalfRoute on both routes; nav link is gandalfOnly) |
Nav section | Innovation (alongside an external "Garden" link to garden.novapower.io) |
Primary data sources | Vena (business/revenue metrics); churn + service-expansion ML models (client health), with sentiment inputs from Gong calls and Pulse surveys; Signal's own records (solution registry, snapshots, team assignments, access flags); Nova user directory (employee picker roster) |
Writes | Signal's own records: ai_solutions (create/status/notes/delete), ai_solution_snapshots (baseline/manual snapshots, baseline regeneration); system-written weekly periodic snapshots + ai_solution_health_runs log |
Key code refs | src/views/AiSolutionsView.tsx, src/views/AiSolutionDetailView.tsx, src/utils/aiSolutionMetrics.ts, src/components/SolutionHealthTrendChart.tsx, server/aiSolutionTargets.ts, server/aiSolutionHealth.ts, server/aiSolutionHealthSnapshots.ts, server/index.ts (/api/ai-solutions/*), src/services/supabase.ts (getAiSolutions etc.), src/utils/orgStructure.ts |
Last verified | staging @ 98df0c4, verified 2026-07-07 |
Purpose
The AI Solutions module exists because Power Digital's AI/ML team ships AI tools (content generators, automations, agents) into specific parts of the organization, and leadership needs evidence — not anecdotes — that those deployments improve outcomes. Without a measurement discipline, a tool launches, everyone moves on, and months later nobody can say whether revenue-per-employee, retention, or client churn risk changed for the teams that got it.
AI Solutions closes that gap in three steps:
Register the deployment (what, where, when) on the day it goes live, against a division, department, single employee, or a set of employees.
Freeze a baseline — business metrics for the target (revenue, RPE, seats, retention, client counts) plus client-health scores (average churn risk, % of clients at risk, sentiment risk, service-expansion readiness, retention score) as they stood at deployment.
Watch the deltas — the app recomputes "current" values continuously, and a weekly server job snapshots client health every Monday, so improvement or degradation versus baseline is always visible, with the deployment date drawn on a weekly trend chart.
It is a longitudinal before/after ledger for AI investments, feeding leadership decisions about which AI solutions to scale, pause, or kill.
Access & visibility
The AI Solutions module is gated at the Gandalf tier — the top permission tier in Signal, above Admin. Mechanics, all verified in code (staging @ 98df0c4):
Route guard: both /solutions and /solutions/:id are wrapped in GandalfRoute in src/App.tsx; non-Gandalf users are redirected away.
Nav flag: the sidebar link "AI Solutions" sits in the "Innovation" section, which carries gandalfOnly: true — non-Gandalf users never see the section. The Innovation section also contains an external link "Garden" (garden.novapower.io), which is not part of this module.
How the Gandalf flag is granted: the flag is is_gandalf on the user's row in Signal's own user records. A Gandalf user grants or revokes it on the General Access screen (/users, in the bottom-pinned "Access" nav section — itself Gandalf-gated). Two hardcoded overrides exist in src/context/AuthContext.tsx: Sauron super-admin emails always have Gandalf (and cannot have it revoked), and a small GANDALF_ALLOWLIST of emails is treated as Gandalf regardless of the database flag. The resolved flag is cached in the browser's sessionStorage.
Backend: every read endpoint under /api/ai-solutions/* requires a valid Signal session token (requireAuth middleware — any authenticated Signal user token passes; the Gandalf restriction is enforced at the route/UI layer). The one write endpoint (POST /api/ai-solutions/snapshot-health) is ops-only, protected by an X-Snapshot-Secret header, not by user auth.
Data sources & lineage
Never treat the warehouse or caches as a source — they are transport. Origin systems below are where the numbers are actually born.
On-screen element | Origin system | Code path | Refresh | Caveat |
Solutions list, status/type filters, Active/Total badges | Signal's own records (solutions entered by users in Signal — ai_solutions table) | Browser → getAiSolutions() in src/services/supabase.ts | Live read on page load and after every create/delete/status change | — |
Health column on the list (current avg churn risk + delta vs baseline) | Churn ML model scores (current); Signal's own records (frozen baseline health on the earliest baseline snapshot) | GET /api/ai-solutions/health-summary → computeHealthSummary() in server/aiSolutionHealthSnapshots.ts | Server cache ~15 min (code constant) | Fail-soft: shows "—" if the backend is unreachable. Archived solutions are excluded |
"≈N clients will be health-tracked" preview in the create form | Churn ML model client universe + Vena department mapping; for employee targets, Signal's own records (team assignments entered in Manage Leads) | GET /api/ai-solutions/resolve-target → resolveTargetClients() in server/aiSolutionTargets.ts | Debounced live call (400 ms) while filling the form | Excludes fully-churned / contract-ended clients ("actionable book"); employee targets walk back up to 3 months of assignments |
Baseline vs Current business-metric cards (Total Revenue, Avg RPE, Active Clients, Seats, Retention, Employees; or Attributed Revenue / MRR / RPE for employee targets) | Vena (contract revenue, VABO); allocation across employees uses Signal's own records (team assignments + revenue split overrides) | Computed in the browser by captureSolutionMetrics() in src/utils/aiSolutionMetrics.ts from DataContext time-series / employee history (fed by GET /api/assignments) | Baseline is frozen JSON on the baseline snapshot; "Current" is recomputed live from the latest available data month (independent of the global month picker) | Monthly-grained; falls back across scopes (declared → other axis → company-wide) and months, with provenance recorded in capture_meta and shown as badges |
Client Health cards — Avg Churn Risk, % At Risk (≥0.30), Sentiment Risk (0–1), Avg SE Readiness, Avg Retention Score, Clients Tracked | Churn ML model (risk, tiers, MRR context from Vena); sentiment risk derived from Gong call recordings and/or Pulse survey responses; SE readiness + retention score from the service-expansion ML model | GET /api/ai-solutions/:id/health → computeTargetHealth() in server/aiSolutionHealth.ts | Server cache ~15 min per solution (code constant) | Sentiment is only counted for clients with a real sentiment source (gong / gsheets / gong+gsheets); model defaults are excluded. Response carries capturedAt (wall clock) and sentimentAsOf (latest model-run week) — both shown in the UI |
Baseline column of the Client Health cards | Signal's own records (frozen health JSON on the baseline snapshot); if absent, approximated from the weekly model-score history at/just before deployment | snapshotHealth() in AiSolutionDetailView.tsx; backfill via backfillBaselineHealth() in server/aiSolutionHealthSnapshots.ts | Frozen (baseline) or computed on view (approximation) | Provenance is displayed: "frozen at baseline capture", "reconstructed from week of …", or "approximated from weekly history near deployment" |
Weekly health trend chart (avg churn risk + SE readiness lines, deployment-date marker) | Historical weekly runs of the churn and SE ML models | GET /api/ai-solutions/:id/health/history → fetchTargetHealthHistory() in server/aiSolutionHealth.ts (one weekly-aggregated query per score table) | Server cache ~15 min; underlying scores refresh with each weekly model run | Default window is 8 weeks before deployment through today (?weeks=N overrides, capped at 52). Sentiment trend line is absent until the historical score table ships a sentiment column — the code auto-detects it |
Trend Over Time chart (revenue metrics across snapshots) | Signal's own records (stored baseline/manual snapshots) + a live browser-computed "now" point from Vena-derived data | Frontend, AiSolutionDetailView.tsx | New point appears only when someone takes a snapshot or a newer data month lands | Weekly periodic rows are health-only and excluded from this chart |
Per-employee breakdown table (multi-employee targets) | Vena revenue allocated via Signal's own records (team assignments) | Frontend, from live capture + baseline snapshot (getPerEmployeeMetrics) | Live | Client books resolve from current team assignments, not the assignments as of deployment |
Snapshot History table | Signal's own records (ai_solution_snapshots: metrics JSON, health JSON, capture_meta provenance, created_by) | Browser → getAiSolutionSnapshots() in src/services/supabase.ts | Live read | Every row records which scope and data month were actually captured, including any fallback |
Division / Department target pickers | Code constants (canonical Nova org names, hardcoded) | ALL_DIVISIONS (9) / ALL_DEPARTMENTS (31) in src/utils/orgStructure.ts | Only changes with a code deploy | A reorg requires a code update |
Employee target pickers (single + multi-select) | Nova platform user directory (active roster, company-email-filtered) | DataContext.employees ← GET /api/employees in server/index.ts | Server cache 1 hour | Roster excludes terminated users; department label shown per employee comes from Nova |
Weekly automated health snapshots (visible as periodic rows in Snapshot History) | Computed from churn/SE ML model scores; written to Signal's own records | Cron in server/aiSolutionHealthSnapshots.ts (Mondays 09:30 server time + startup catch-up); manual trigger POST /api/ai-solutions/snapshot-health (X-Snapshot-Secret) | Weekly, active solutions only | Idempotent per week per solution; runs logged to ai_solution_health_runs; startup also backfills deployment-week health onto old baselines from historical model scores |
Metrics & definitions
All metrics below appear on the AI Solutions views (/solutions, /solutions/:id). Thresholds are code constants as of Jul 2026 unless noted.
Business metrics (Baseline vs Current cards; origin: Vena, allocated via team assignments recorded in Signal)
Total Revenue (org targets) — summed monthly contract revenue for the target division/department for one data month. Origin: Vena.
Avg RPE (org targets) — revenue per employee: (revenue + VABO) ÷ employee count for the scope/month. Origin: Vena; computed in the browser.
Active Clients — count of clients with revenue activity in the captured month for the scope. Origin: Vena.
Seats — seat count for the scope/month. Origin: Vena contract data.
Retention (org targets) — retention rate from the time-series for the scope/month, shown as a percent. Origin: Vena-derived time series.
Employees (org targets) — derived, not authoritative: round(revenue / rpe); flagged employeeCountDerived in capture provenance.
Attributed Revenue / RPE (employee targets) — the employee's allocated revenue for the month (their share of assigned clients' revenue plus VABO). Note: for a single employee-month, attributedRevenue and rpe are the same underlying number mapped to two cards. Origin: Vena revenue split across Signal team assignments.
MRR (employee targets only) — monthly recurring revenue of the employee's client book. Origin: Vena. Org-target cards deliberately have no MRR row (the org time-series carries none).
VABO (per-employee breakdown) — performance-based revenue for the month; the acronym's expansion is not defined in the app repo. Origin: Vena.
Retention Rate (employee targets) — retained ÷ (retained + churned) clients for the employee-month, ×100. An employee-month with zero retained and zero churned counts as 100% (code behavior).
Client-health metrics (Client Health cards, list Health column, weekly trend; origin: AI/ML prediction models)
Avg Churn Risk — mean of the churn model's 0–1 enhanced risk score across the target's resolved clients. Lower is better. Same per-client scores as Signal's Churn Risk view.
% At Risk (≥0.30) — percent of scored clients whose churn risk is at or above 0.30 (code constant AT_RISK_THRESHOLD, same gate as the churn overview). Lower is better.
Risk tiers (in tooltips/stored data) — Low < 0.3 ≤ Medium < 0.5 ≤ High < 0.6 ≤ Critical (code constants in riskTierOf).
Sentiment Risk (0–1) — mean sentiment-risk norm across clients that have a real sentiment source; lower is better. Sources: Gong call recordings ("gong"), Pulse survey responses (labeled "gsheets"), or both; clients with only a model-default value are excluded. Displayed with the model's as-of week (sentimentAsOf).
Avg SE Readiness — mean service-expansion readiness score (0–1) from the SE ML model; higher is better. Same feed as Signal's Service Expansion view.
Avg Retention Score — mean retention score (0–1) from the SE ML model; higher is better. Distinct from the business "Retention Rate" above.
Clients Tracked — count of clients the target resolved to; context, not a score (rendered neutral).
Health column (list view) — current Avg Churn Risk with an arrow and the delta versus the frozen baseline risk (current − baseline); a down-arrow/green means risk fell (improving). Deltas under 0.01 render flat.
How it's used
Register a new AI deployment (day-of-launch ritual). On /solutions, click New Solution, name it, set the deployed date (defaults to today), pick the target (e.g. Department: Organic Search), sanity-check the "≈N clients will be health-tracked" preview, and hit Create & Capture Baseline. This establishes the measurement contract before memories fade — every later impact claim traces to this frozen baseline.
Monthly impact review of a rollout. Open the solution's detail page (/solutions/:id); read the Baseline vs Current business cards (is RPE/revenue/retention up for the target?) and the Client Health cards (is average churn risk down? % at risk down? SE readiness up?); confirm on the weekly trend chart that movement started around the deployment-date line rather than before it. Decision served: continue investing, iterate, or pause.
Quarterly portfolio scan. On the /solutions list, filter Status to Active and scan the Health column: green down-arrows (risk falling vs baseline) are candidates to scale; red up-arrows warrant opening the detail page. Decision served: rank the AI portfolio for next quarter's resourcing.
Freeze a milestone before a change. Before a major change to a solution (new model version, expanded scope), click Take Snapshot on the detail page to preserve the current state as a manual snapshot; later, Snapshot History and the Trend Over Time chart attribute movement to the right phase.
Track a pilot team. Use the Multiple Employees target to wrap the exact pilot group; the system resolves their client books from team assignments (recorded in Signal's Manage Leads), and the Employee Breakdown table shows per-person RPE deltas vs baseline. Decision served: expand the pilot from N people to the whole department, or not.
Close out a finished experiment. On the detail page, click the status badge and set Completed (keeps history, drops it from the Active count) or Archived (also drops it from the list's Health column and from weekly snapshots); write the conclusion in Notes. Decision served: a durable institutional record of what was tried and what happened.
What users can change here
All user writes from the AI Solutions views land in Signal's own records (entered by users in Signal), directly from the browser:
Create a solution (list view form) → inserts a row into ai_solutions (name, description, target type/id/name, deployed date, status active, notes, creator email), then immediately inserts a baseline row into ai_solution_snapshots — business metrics computed in the creating user's browser for the deployment month, plus a frozen client-health object fetched from the backend (fail-soft: creation never blocks on health). Failures are surfaced in an error banner, not swallowed.
Delete a solution (list view, confirm dialog) → deletes the ai_solutions row and all of its snapshots. Irreversible.
Change status (detail view; click the status badge) → updates ai_solutions.status to Active / Paused / Completed / Archived. Side effects: only active solutions receive the weekly automated health snapshot; archived solutions also drop out of the list view's Health column.
Edit notes (detail view) → updates ai_solutions.notes free text.
Take Snapshot (detail view) → inserts a manual row into ai_solution_snapshots with the latest data month's metrics plus current client health.
Regenerate Baselines (list view) → patches metrics, snapshot_month, and capture_meta on baseline snapshots that captured only empty/zero metrics (e.g. created before the browser's data finished loading). Skips baselines that already have data.
Not editable after creation: name, description, target, and deployed date — fix by delete + recreate. The system itself also writes: the weekly cron inserts periodic (health-only) snapshot rows and logs each run to ai_solution_health_runs, and a startup backfill patches reconstructed health onto baseline snapshots that lack it.
Caveats & data quality
Hardcoded org lists. The Division/Department pickers on the AI Solutions create form read static arrays (9 divisions, 31 departments) in src/utils/orgStructure.ts. A company reorg changes these only via a code deploy.
Cross-axis fallback on seeded rows. Some historical solutions carry mislabeled targets (e.g. "Paid Media" stored as a division; "Marketing Science" renamed to Fusepoint). Matching falls back to the other org axis and, for business metrics, ultimately to company-wide; every fallback is recorded (capture_meta, "fallback" badges, "matched on the other axis" warnings) rather than applied silently.
A "Company-wide" pseudo-target exists in resolution code (target_name === 'Company-wide') but is not offered in the create form — it appears only on legacy/seeded rows and as a business-metrics fallback scope.
Sentiment trend gap. The weekly trend chart cannot plot sentiment history yet — the historical churn-score table has no sentiment column; the code auto-enables the line when one ships. Only the current-vs-baseline sentiment cards work today.
Reconstructed baselines. For solutions created before health tracking existed (health tracking landed Jun 2026), baseline health was rebuilt from historical model-score weeks using today's client roster — historical team membership is not recoverable. The UI labels these "reconstructed from week of …".
Employee targets track current, not historical, client books. The client set always resolves from current team assignments (walking back at most 3 months to find a populated month). A claim like "tracks the same clients since deployment" would be wrong.
Browser-side snapshot capture. Baseline and manual business metrics are computed in the capturing user's browser from their loaded data; early captures could be empty, which is exactly what the "Regenerate Baselines" button repairs.
Fail-soft health. The Health column and Client Health section show "—" or hide entirely when the scoring backend is unreachable (normal in local dev) — not a bug.
Per-client cap. Health snapshots store at most 200 clients per solution (code constant); above that, only the top 50 by risk are stored with a truncation flag — aggregate numbers remain complete.
Thresholds and cadences are code constants as of Jul 2026, not promises: at-risk gate 0.30; risk tiers 0.3/0.5/0.6; server health caches ~15 min; weekly cron Mondays 09:30 server time plus startup catch-up; trend default window 8 weeks pre-deployment, max lookback 52 weeks; employee-picker shows first 50 search results.
Ownership process is not defined in the app repo — who registers solutions, naming conventions, and the intended distinction between "Completed" and "Archived" are team conventions, not code.
Retention-rate quirk: an employee-month with zero retained and zero churned clients counts as 100% retention in the browser-side capture (code behavior; business intent unconfirmed).
Related views
Churn Risk (/predictions/churn) — the AI Solutions health numbers are the same churn-model scores shown there: same feed, same ~15-minute server cache, same 0.30 at-risk threshold and Low/Medium/High/Critical tiers.
Service Expansion (/predictions/se) — SE Readiness and Retention Score in AI Solutions come from the same SE-model feed that view uses.
Manage Leads (/leads) — employee-targeted solutions resolve their client set from the team assignments maintained there; stale assignments mean a stale tracked-client set.
Summary / KPI dashboards — the business-metric cards reuse the same DataContext time-series and employee-history math (Vena revenue, RPE allocation, VABO handling), so numbers reconcile with those views by construction.
General Access (/users, "Access" nav section) — where the Gandalf flag that gates AI Solutions is granted.
Nothing downstream consumes AI Solutions data — it is a terminal reporting module.
FAQ
Q: Why can't I see AI Solutions in my Signal sidebar? A: AI Solutions is gated at the Gandalf tier — Signal's top permission level, above Admin. The "Innovation" nav section and the /solutions routes are hidden unless your user record has the is_gandalf flag (or you are a Sauron super-admin). A Gandalf user can grant the flag on the General Access screen (/users). For a new access grant, ask your manager or the AI/ML team lead.
Q: Where do the churn-risk numbers in AI Solutions come from — are they the same as the Churn Risk page? A: Yes. AI Solutions averages the exact per-client 0–1 risk scores produced by the AI/ML team's churn prediction model — the same scores the Churn Risk view shows, from the same feed with the same ~15-minute server cache. "Avg Churn Risk" for a solution is the mean over the clients its target resolves to.
Q: Why doesn't the revenue on an AI Solution match what I see in Vena? A: The numbers originate in Vena, but AI Solutions shows them for one specific scope and one specific data month. Three things cause apparent mismatches: (1) the baseline is frozen at the deployment month and never updates; (2) the "Current" column uses the latest available data month, ignoring Signal's global month picker; (3) if the target's declared division/department name didn't match revenue data, the capture may have fallen back to the other org axis or to company-wide numbers — the Scope column in Snapshot History and a "fallback" badge tell you exactly which slice was captured. Check the badge before comparing to Vena.
Q: What does "≈N clients will be health-tracked" mean when I create a solution? A: The create form asks the backend to resolve your chosen target to its set of clients using the same logic health tracking will use: division/department targets match against the actionable client universe from the churn model (excluding fully-churned and contract-ended clients); employee targets use the clients assigned to those employees in Signal's team assignments (current month, walking back up to 3 months). If the count is 0 or the name only matched on the other org axis, the form warns you — fix the target type before creating.
Q: What is a "reconstructed" baseline? A: Solutions created before health tracking existed (before Jun 2026) had no frozen health at deployment. A backfill rebuilt their baseline health from the historical weekly model scores nearest the deployment date — but using today's client roster, since historical team membership isn't recoverable. The detail page labels these "reconstructed from week of ". Treat them as an approximation, not a perfect historical record.
Q: What's the difference between baseline, periodic, and manual snapshots in the Snapshot History table? A: baseline — frozen automatically when the solution is created (business metrics for the deployment month + client health at creation); the reference point for all deltas. periodic — written automatically every Monday (09:30 server time, code constant) by a server job for active solutions; health-only, no business metrics. manual — created when someone clicks Take Snapshot on the detail page; freezes the latest month's business metrics plus current health, useful for marking milestones.
Q: Why does the Health column on the solutions list show "—"? A: Three possible reasons: (1) the backend or the scoring data is unreachable — the column fails soft rather than erroring; (2) the solution's target resolved to zero scored clients; (3) the solution is archived, which excludes it from the health summary entirely. If baseline health hasn't been captured yet, you'll see the current risk without a delta arrow instead.
Q: Does changing a solution's status to Archived delete anything? A: No. Archiving keeps the solution row and all snapshots. Side effects: archived solutions stop receiving the weekly automated health snapshot (only Active solutions get those), and they drop out of the list view's Health column. Deleting, by contrast, permanently removes the solution and every snapshot after a confirm dialog.
Q: Can I edit a solution's target or deployed date after creating it? A: No. Name, description, target, and deployed date are immutable after creation — the frozen baseline only makes sense for the original target and date. Only status and notes are editable. To fix a wrong target or date, delete the solution and recreate it (you'll get a fresh baseline as of the new deployment month).
Q: Why is the sentiment line missing from the weekly health trend chart? A: The weekly history comes from the historical churn-model score table, which does not yet carry a sentiment column; the code detects the column automatically and will plot the line once it ships. Until then, sentiment appears only in the current-vs-baseline cards, where "Sentiment Risk (0–1)" averages clients that have a real sentiment source — Gong call recordings and/or Pulse survey responses — and excludes model defaults.
Q: I created a solution and the baseline shows all zeros. Is the data lost? A: No — this happens when the baseline was captured before the browser finished loading revenue data (metrics are computed client-side at creation time). Click Regenerate Baselines on the /solutions list: it re-captures baseline metrics for the deployment month for every solution whose baseline is empty, and leaves healthy baselines untouched.