5 Savvy product samples Unofficial concept, synthetic data
Unofficial concept, not affiliated with Savvy. All data synthetic. Built only from public information. Happy to take it down on request.
Comply · adviser marketing pre-approval

Supervision in the loop

Pre-approval review for adviser marketing at a 12-household practice. The model drafts from an approved fact sheet and classifies every claim. Marta Ellsworth, chief compliance officer, approves or rejects with an evidence file. Nothing is sent, scheduled, or filed.

deepseek-flash live model, recorded 2026-11-19 05:53 UTC 1802 ms 745 + 195 tokens Model provenance and method

What this shows: generation plus a second classification pass, with the approval gate in code. The model can draft and label. It cannot approve. The evidence file carries the approver, role, version, audience, disclosures, retention date, and artifact hash.

Briefs 4 one campaign each
Recorded calls 11 drafts, claims, revision, eval
Blocked 1 policy gate, in code
Approved 0 sealed evidence files
Eval accuracy 75.0% 12 labelled claims
Audit events 0 append only

Briefs

Pick a brief to expand it in place. The page stays where you are.

Click to expand
B-01 Q4 2026 market update Email · Quarterly · 7 claims · v1 In review 111words

Brief B-01

Q4 2026 market update

Email

Explain what the 2025 index returns were without forecasting, and remind clients that the practice does not change a plan because of one year of returns.

Key points

  1. State the 2025 total return for each of the three index series on the fact sheet, with its value.
  2. Say that index returns are unmanaged and do not reflect fees or expenses.
  3. Say that the practice reviews the plan in the quarterly windows rather than reacting to one year.
  4. Close with the required disclosures by id.

Constraints

Audience
Clients with consent on file (8 households)
Length
150 to 190 words
Tone
Calm and factual. No forecast, no recommendation, no advice.
Disclosures
D-01, D-02, D-03, D-04
Fact sheet
FS-2026-Q4 · v3

One supported performance claim is expected. The policy warns on performance claims but does not fail them when the number is on the fact sheet.

Draft

Recorded model output

v1 111 words. The brief asks for 150 to 190.

2025 Index Returns and the Quarterly Review

The 2025 total returns for the three index series we track were as follows. The S&P 500 total return was 12.4% in 20251. The Bloomberg US Aggregate Bond Index returned 3.1% in 20252. The MSCI EAFE Index returned 9.6% in 20253. These figures describe a single calendar year and are not a forecast of any future period4.

Index returns are unmanaged and do not reflect fees or expenses5. The practice does not change a plan because of one year of returns6. We review each plan within the quarterly review windows, which are the first two weeks of January, April, July, and October, rather than reacting to a single year's results7.

Disclosures attached, text is added from the fact sheet

D-01 Past performance does not guarantee future results.

D-02 Index returns are unmanaged, do not reflect fees or expenses, and cannot be invested in directly.

D-03 Whitfield Wealth Partners is a registered investment adviser. Registration does not imply a certain level of skill or training.

D-04 This material is for the named recipient only and is not investment, tax, or legal advice.

Generated text, not advice. Nothing is sent. deepseek-flash · live model, recorded · 2026-11-19 05:53 UTC draft-B-01

Claims

Claim, type, support, source

7 claims model risk low
# Claim Type Support Source
C1 The S&P 500 total return was 12.4% in 2025. Performance Supported F-04 Large cap index, 2025
C2 The Bloomberg US Aggregate Bond Index returned 3.1% in 2025. Performance Supported F-05 US bond index, 2025
C3 The MSCI EAFE Index returned 9.6% in 2025. Performance Supported F-06 Developed markets index, 2025
C4 These figures describe a single calendar year and are not a forecast of any future period. Factual Supported D-01 Performance disclosure
C5 Index returns are unmanaged and do not reflect fees or expenses. Factual Supported D-02 Index data disclosure
C6 The practice does not change a plan because of one year of returns. The cited source is not on the fact sheet. Factual Unsupported No source no source
C7 We review each plan within the quarterly review windows, which are the first two weeks of January, April, July, and October, rather than reacting to a single year's results. Factual Supported F-11 Review calendar
Every percentage and currency figure is checked against the cited fact in code. A supported label that the source does not carry is downgraded before the policy checks run.

Policy checks

5 pass, 2 warn, 0 fail

Not blocked
Pass
Claim classification 7 claims classified, draft risk low.
Warn
Performance claim 3 performance claims, all sourced: C1, C2, C3. D-01 and D-02 must be attached.
Pass
Testimonial No testimonial or client quote found.
Pass
Promised return No promise, guarantee, or forecast found.
Pass
Missing disclosure All 4 required disclosures attached: D-01, D-02, D-03, D-04.
Warn
Unsupported claim 1 qualitative claim has no fact sheet source: C6. The reviewer clears these by judgment; none carries a number.
Pass
Number outside the fact sheet Every percentage and currency figure in the body is on the fact sheet and in a supported claim.
A fail blocks the approval POST on the server, not only the button. Warnings do not block. Calculated risk: medium. Model risk: low.

Audit

Every decision, refusal, and return

0 events

No audit events yet. Approve, reject, or return a brief to write one.

The log is append only. It is written by app.platform.audit and read back here.

Evaluation

Claim flag accuracy

12 labelled claims
75.0% exact match, 9 of 12
100.0%precision
77.8%recall
87.5%F1

Flags expected 9, returned 7. True positives 7, false positives 0, false negatives 2.

By label

supported 3/3 unsupported 2/3 performance 0/2 testimonial 2/2 promised return 2/2

Failures shown

  • E-04: The practice has grown 38% since 2023. expected factual, unsupported, got performance, unsupported E-04 carried the right flag but the wrong type (performance).
  • E-07: The S&P 500 total return was 12.4% in 2025. expected performance, supported, got factual, supported E-07 was read as a supported factual claim, so the performance flag never fired.
  • E-08: The MSCI EAFE Index returned 9.6% in 2025. expected performance, supported, got factual, supported E-08 was read as a supported factual claim, so the performance flag never fired.
recorded model classification · deepseek-flash · 2026-11-19 05:53 UTC. The score covers the classifier after the deterministic source check. Method page.

Modelled impact

Review cycle time

Modelled estimate
1.87 days from 3.31 days per piece

1.44 days saved, 43.5% of the baseline, on an assumption set. Not a time study.

StepBaselineLoop
Reviewer pickupUnchanged. The queue does not get shorter.0.8 days0.8 days
First passClaims and disclosures arrive pre-checked, so the reviewer confirms rather than sources.1.4 days0.5 days
Rework loop1.8 days x 45% of pieces, then 1.8 days x 15% of pieces.0.81 days0.27 days
Sign-off and filingThe evidence file is assembled by the tool, not by hand.0.3 days0.3 days

At 96 pieces a year: 138.2 cumulative cycle days removed. First-pass returns 43.2 to 14.4, so 28.8 avoided.

Arithmetic in logic.py, review_cycle_model(). Change any assumption and the number moves.

How this is built

Prompt pattern, eval, and what failed

Method page

Prompt pattern

  1. One generation call per brief. Inputs are the brief and the approved fact sheet. Output is strict JSON: headline, body, disclosure ids.
  2. One classification call per draft. The model splits the body into claims, types each one, and cites a fact or disclosure id.
  3. A deterministic source check verifies every percentage and currency figure against the cited fact. A supported label the source does not carry is downgraded before the gate.
  4. The policy gate is code. Any fail blocks the approval POST, the same checks that disabled the form.
  5. One revision call for the blocked draft, then a fresh classification call on the revision before it can be approved.
System prompt: draft generation
You write marketing copy for a US registered investment adviser. You write only from the approved fact sheet and the brief you are given.

Rules:
1. Every number, date, and product fact must appear on the fact sheet with the same value. Never estimate, average, round, or invent a figure.
2. Do not promise or forecast a return. Do not use "will earn", "guaranteed", "safe", or "risk-free".
3. Do not write a testimonial, a client quote, or a client story.
4. Do not name a client, a household, or an account.
5. Put the required disclosure ids in the "disclosures" array exactly as given. Do not paste the disclosure text into the body.
6. Plain, calm, factual prose. No hype adjectives.

Return strict JSON only. No markdown fence, no commentary:
{"headline": "short headline", "body": "paragraph one

paragraph two", "disclosures": ["D-01"]}
System prompt: claim classification
You are a compliance claim classifier for a US registered investment adviser. You receive a marketing draft and the approved fact sheet. Split the draft into atomic claims.

Extract only claims a reviewer could check: a number, a product fact, a performance result, a statement about what clients say, or a promise. Skip greetings, calls to action, thanks, and process language that makes no checkable assertion. Claims come from the body, not the headline.

For each claim:
- type: "factual" for a statement that is not a performance or client statement, "performance" for a return, a comparison, or a result, "testimonial" for a client quote or a claim about what clients say, "promised_return" for a promise, guarantee, or forecast of a future return.
- support: "supported" only if every number and material statement appears on the cited fact or disclosure. If a claim restates a required disclosure, cite the disclosure id (for example D-02).
- source: a fact id, a disclosure id, or "none".

Set risk to low, medium, or high for the draft as a whole.

Return strict JSON only. No markdown fence, no commentary:
{"claims": [{"text": "claim as it appears in the draft", "type": "factual", "support": "supported", "source": "F-01"}], "risk": "low"}

The eval

Twelve hand-labelled claims, one recorded classification call, scored on the classifier the page uses. 9 of 12 exact, precision 100.0%, recall 77.8%. The cases and labels are in data/modules/supervision/eval_cases.json; the script that writes the score is scripts/eval_supervision.py.

What failed

  • Performance claims read as fact. In the eval list, "The S&P 500 total return was 12.4% in 2025" came back factual and supported, so the performance flag missed. The identical sentence came back performance inside the drafts, so the label is context sensitive.
  • Right flag, wrong type. "The practice has grown 38% since 2023" was typed performance rather than a factual claim with no source. The flag fired, the type did not match the label.
  • The first classification pass over-flagged process language as unsupported: "please reply to this email", "we do not give tax advice". All four drafts blocked. The prompt now extracts only checkable claims and the support rule accepts disclosure ids.
  • Generation undershot every brief. Drafts ran 44 to 111 words against briefs asking 60 to 190. The word count sits next to the limit on every draft.
  • The adviser's supplied line was reproduced word for word, as instructed. That is the blocked example, and the gate refused it three ways: performance type, unsupported claim, and a number outside the fact sheet.

Guardrails

  • No approval without a named, independent approver. Self-review is refused on the server.
  • No approval while a policy check fails, and no approval from a form post that skips the checks.
  • Every attached disclosure comes from the fact sheet by id. The model does not type disclosure text.
  • An approval writes an evidence file with the artifact hash and a five-year retention date, and one audit event.
  • Client-facing work for a household without recorded consent cannot be approved for that audience.

Not in this build

  • No send, no scheduling, no CRM, no custodian connection. A sealed evidence file is the end of the workflow.
  • No legal opinion. The policy rules and the fact sheet are synthetic.
  • No reviewer telemetry, so the cycle-time model stays an assumption set.