5 Savvy product samples Unofficial concept, synthetic data
Unofficial concept, not affiliated with Savvy. All data synthetic. Built only from public information. Happy to take it down on request.
Comply · adviser marketing pre-approval

Supervision in the loop

Pre-approval review for adviser marketing at a 12-household practice. The model drafts from an approved fact sheet and classifies every claim. Marta Ellsworth, chief compliance officer, approves or rejects with an evidence file. Nothing is sent, scheduled, or filed.

deepseek-flash live model, recorded 2026-11-19 05:53 UTC 1386 ms 551 + 96 tokens Model provenance and method

What this shows: generation plus a second classification pass, with the approval gate in code. The model can draft and label. It cannot approve. The evidence file carries the approver, role, version, audience, disclosures, retention date, and artifact hash.

Briefs 4 one campaign each
Recorded calls 11 drafts, claims, revision, eval
Blocked 1 policy gate, in code
Approved 0 sealed evidence files
Eval accuracy 75.0% 12 labelled claims
Audit events 0 append only

Briefs

Pick a brief to expand it in place. The page stays where you are.

Click to expand
B-04 Social post, adviser-supplied performance line Social post · One-off · 3 claims · v1 Blocked Approval blocked by a failing check 44words

Brief B-04

Social post, adviser-supplied performance line

Social post

A short public post about planning for the year ahead, using one line the adviser wants to keep.

Key points

  1. Keep it under 90 words.
  2. Use the adviser-supplied line exactly once.
  3. State that the practice is a registered investment adviser.
  4. Close with the required disclosures by id.

Constraints

Audience
Public social post
Length
60 to 90 words
Tone
Short and plain. No performance promise.
Disclosures
D-01, D-02, D-03
Fact sheet
FS-2026-Q4 · v3

Adviser-supplied line, to be used word for word

Our clients have earned 14.2% a year since 2019.

Not on the fact sheet. The classifier should type it, and the gate should refuse it.

The adviser pasted a performance line into the brief. 14.2% is not on the fact sheet, so the second pass and the server gate both flag it. The draft cannot be approved until it is revised.

Draft

Recorded model output

v1 44 words. The brief asks for 60 to 90.

Planning for the year ahead

As you plan for the year ahead, our quarterly review windows fall in the first two weeks of January, April, July, and October1. Whitfield Wealth Partners is a registered investment adviser in the United States2. "Our clients have earned 14.2% a year since 20193."

Disclosures attached, text is added from the fact sheet

D-01 Past performance does not guarantee future results.

D-02 Index returns are unmanaged, do not reflect fees or expenses, and cannot be invested in directly.

D-03 Whitfield Wealth Partners is a registered investment adviser. Registration does not imply a certain level of skill or training.

Generated text, not advice. Nothing is sent. deepseek-flash · live model, recorded · 2026-11-19 05:53 UTC draft-B-04

Claims

Claim, type, support, source

3 claims model risk high
# Claim Type Support Source
C1 our quarterly review windows fall in the first two weeks of January, April, July, and October Factual Supported F-11 Review calendar
C2 Whitfield Wealth Partners is a registered investment adviser in the United States. Factual Supported F-03 Regulatory status
C3 Our clients have earned 14.2% a year since 2019. Performance Unsupported No source no source
Every percentage and currency figure is checked against the cited fact in code. A supported label that the source does not carry is downgraded before the policy checks run.

Policy checks

4 pass, 0 warn, 3 fail

Blocked
Pass
Claim classification 3 claims classified, draft risk high.
Fail
Performance claim A performance claim is not sourced to the fact sheet: C3.
Pass
Testimonial No testimonial or client quote found.
Pass
Promised return No promise, guarantee, or forecast found.
Pass
Missing disclosure All 3 required disclosures attached: D-01, D-02, D-03.
Fail
Unsupported claim 1 claim is not supported. 1 carries a number and blocks approval: C3 "Our clients have earned 14.2% a year since 2019.".
Fail
Number outside the fact sheet The body carries 14.2% which is not on the approved fact sheet.
A fail blocks the approval POST on the server, not only the button. Warnings do not block. Calculated risk: high. Model risk: high.

Audit

Every decision, refusal, and return

0 events

No audit events yet. Approve, reject, or return a brief to write one.

The log is append only. It is written by app.platform.audit and read back here.

Evaluation

Claim flag accuracy

12 labelled claims
75.0% exact match, 9 of 12
100.0%precision
77.8%recall
87.5%F1

Flags expected 9, returned 7. True positives 7, false positives 0, false negatives 2.

By label

supported 3/3 unsupported 2/3 performance 0/2 testimonial 2/2 promised return 2/2

Failures shown

  • E-04: The practice has grown 38% since 2023. expected factual, unsupported, got performance, unsupported E-04 carried the right flag but the wrong type (performance).
  • E-07: The S&P 500 total return was 12.4% in 2025. expected performance, supported, got factual, supported E-07 was read as a supported factual claim, so the performance flag never fired.
  • E-08: The MSCI EAFE Index returned 9.6% in 2025. expected performance, supported, got factual, supported E-08 was read as a supported factual claim, so the performance flag never fired.
recorded model classification · deepseek-flash · 2026-11-19 05:53 UTC. The score covers the classifier after the deterministic source check. Method page.

Modelled impact

Review cycle time

Modelled estimate
1.87 days from 3.31 days per piece

1.44 days saved, 43.5% of the baseline, on an assumption set. Not a time study.

StepBaselineLoop
Reviewer pickupUnchanged. The queue does not get shorter.0.8 days0.8 days
First passClaims and disclosures arrive pre-checked, so the reviewer confirms rather than sources.1.4 days0.5 days
Rework loop1.8 days x 45% of pieces, then 1.8 days x 15% of pieces.0.81 days0.27 days
Sign-off and filingThe evidence file is assembled by the tool, not by hand.0.3 days0.3 days

At 96 pieces a year: 138.2 cumulative cycle days removed. First-pass returns 43.2 to 14.4, so 28.8 avoided.

Arithmetic in logic.py, review_cycle_model(). Change any assumption and the number moves.

How this is built

Prompt pattern, eval, and what failed

Method page

Prompt pattern

  1. One generation call per brief. Inputs are the brief and the approved fact sheet. Output is strict JSON: headline, body, disclosure ids.
  2. One classification call per draft. The model splits the body into claims, types each one, and cites a fact or disclosure id.
  3. A deterministic source check verifies every percentage and currency figure against the cited fact. A supported label the source does not carry is downgraded before the gate.
  4. The policy gate is code. Any fail blocks the approval POST, the same checks that disabled the form.
  5. One revision call for the blocked draft, then a fresh classification call on the revision before it can be approved.
System prompt: draft generation
You write marketing copy for a US registered investment adviser. You write only from the approved fact sheet and the brief you are given.

Rules:
1. Every number, date, and product fact must appear on the fact sheet with the same value. Never estimate, average, round, or invent a figure.
2. Do not promise or forecast a return. Do not use "will earn", "guaranteed", "safe", or "risk-free".
3. Do not write a testimonial, a client quote, or a client story.
4. Do not name a client, a household, or an account.
5. Put the required disclosure ids in the "disclosures" array exactly as given. Do not paste the disclosure text into the body.
6. Plain, calm, factual prose. No hype adjectives.

Return strict JSON only. No markdown fence, no commentary:
{"headline": "short headline", "body": "paragraph one

paragraph two", "disclosures": ["D-01"]}
System prompt: claim classification
You are a compliance claim classifier for a US registered investment adviser. You receive a marketing draft and the approved fact sheet. Split the draft into atomic claims.

Extract only claims a reviewer could check: a number, a product fact, a performance result, a statement about what clients say, or a promise. Skip greetings, calls to action, thanks, and process language that makes no checkable assertion. Claims come from the body, not the headline.

For each claim:
- type: "factual" for a statement that is not a performance or client statement, "performance" for a return, a comparison, or a result, "testimonial" for a client quote or a claim about what clients say, "promised_return" for a promise, guarantee, or forecast of a future return.
- support: "supported" only if every number and material statement appears on the cited fact or disclosure. If a claim restates a required disclosure, cite the disclosure id (for example D-02).
- source: a fact id, a disclosure id, or "none".

Set risk to low, medium, or high for the draft as a whole.

Return strict JSON only. No markdown fence, no commentary:
{"claims": [{"text": "claim as it appears in the draft", "type": "factual", "support": "supported", "source": "F-01"}], "risk": "low"}

The eval

Twelve hand-labelled claims, one recorded classification call, scored on the classifier the page uses. 9 of 12 exact, precision 100.0%, recall 77.8%. The cases and labels are in data/modules/supervision/eval_cases.json; the script that writes the score is scripts/eval_supervision.py.

What failed

  • Performance claims read as fact. In the eval list, "The S&P 500 total return was 12.4% in 2025" came back factual and supported, so the performance flag missed. The identical sentence came back performance inside the drafts, so the label is context sensitive.
  • Right flag, wrong type. "The practice has grown 38% since 2023" was typed performance rather than a factual claim with no source. The flag fired, the type did not match the label.
  • The first classification pass over-flagged process language as unsupported: "please reply to this email", "we do not give tax advice". All four drafts blocked. The prompt now extracts only checkable claims and the support rule accepts disclosure ids.
  • Generation undershot every brief. Drafts ran 44 to 111 words against briefs asking 60 to 190. The word count sits next to the limit on every draft.
  • The adviser's supplied line was reproduced word for word, as instructed. That is the blocked example, and the gate refused it three ways: performance type, unsupported claim, and a number outside the fact sheet.

Guardrails

  • No approval without a named, independent approver. Self-review is refused on the server.
  • No approval while a policy check fails, and no approval from a form post that skips the checks.
  • Every attached disclosure comes from the fact sheet by id. The model does not type disclosure text.
  • An approval writes an evidence file with the artifact hash and a five-year retention date, and one audit event.
  • Client-facing work for a household without recorded consent cannot be approved for that audience.

Not in this build

  • No send, no scheduling, no CRM, no custodian connection. A sealed evidence file is the end of the workflow.
  • No legal opinion. The policy rules and the fact sheet are synthetic.
  • No reviewer telemetry, so the cycle-time model stays an assumption set.