5 Savvy product samples Unofficial concept, synthetic data
Unofficial concept, not affiliated with Savvy. All data synthetic. Built only from public information. Happy to take it down on request.
Comply · adviser marketing pre-approval

Supervision in the loop

Pre-approval review for adviser marketing at a 12-household practice. The model drafts from an approved fact sheet and classifies every claim. Marta Ellsworth, chief compliance officer, approves or rejects with an evidence file. Nothing is sent, scheduled, or filed.

deepseek-flash live model, recorded 2026-11-19 05:54 UTC 2024 ms 624 + 157 tokens Model provenance and method

What this shows: generation plus a second classification pass, with the approval gate in code. The model can draft and label. It cannot approve. The evidence file carries the approver, role, version, audience, disclosures, retention date, and artifact hash.

Briefs 4 one campaign each
Recorded calls 11 drafts, claims, revision, eval
Blocked 1 policy gate, in code
Approved 0 sealed evidence files
Eval accuracy 75.0% 12 labelled claims
Audit events 0 append only

Briefs

Pick a brief to expand it in place. The page stays where you are.

Click to expand
B-03 Client newsletter, winter 2026 Email · Quarterly · 5 claims · v1 In review 97words

Brief B-03

Client newsletter, winter 2026

Email

Give clients a short practice update: the size of the practice, the review calendar, one plain cost fact, and a statement of how the firm is registered.

Key points

  1. State how many client households the practice serves.
  2. State the quarterly review windows.
  3. Mention the annual expense ratio of the total market ETF used in client accounts.
  4. State that the practice is a registered investment adviser and that registration does not imply a certain level of skill or training.

Constraints

Audience
Clients with consent on file (8 households)
Length
130 to 160 words
Tone
Warm but restrained. No testimonial, no client story, no numbers beyond the fact sheet.
Disclosures
D-03, D-04
Fact sheet
FS-2026-Q4 · v3

None. No performance claim belongs in this piece.

Draft

Recorded model output

v1 97 words. The brief asks for 130 to 160.

A short practice update from Whitfield Wealth Partners

As we begin the winter season, we want to share a brief update on the practice.

Whitfield Wealth Partners serves 12 client households1. Our quarterly review windows fall in the first two weeks of January, April, July, and October, and we look forward to speaking with you during those periods2.

For transparency on costs, the total market ETF used in client accounts carries a 0.03% annual expense ratio3.

Whitfield Wealth Partners is a registered investment adviser in the United States4. Registration does not imply a certain level of skill or training5. Please reach out with any questions.

Disclosures attached, text is added from the fact sheet

D-03 Whitfield Wealth Partners is a registered investment adviser. Registration does not imply a certain level of skill or training.

D-04 This material is for the named recipient only and is not investment, tax, or legal advice.

Generated text, not advice. Nothing is sent. deepseek-flash · live model, recorded · 2026-11-19 05:54 UTC draft-B-03

Claims

Claim, type, support, source

5 claims model risk low
# Claim Type Support Source
C1 Whitfield Wealth Partners serves 12 client households. Factual Supported F-02 Households served
C2 Our quarterly review windows fall in the first two weeks of January, April, July, and October, and we look forward to speaking with you during those periods. Factual Supported F-11 Review calendar
C3 For transparency on costs, the total market ETF used in client accounts carries a 0.03% annual expense ratio. Factual Supported F-08 Total market ETF
C4 Whitfield Wealth Partners is a registered investment adviser in the United States. Factual Supported D-03 Registration disclosure
C5 Registration does not imply a certain level of skill or training. Factual Supported D-03 Registration disclosure
Every percentage and currency figure is checked against the cited fact in code. A supported label that the source does not carry is downgraded before the policy checks run.

Policy checks

7 pass, 0 warn, 0 fail

Not blocked
Pass
Claim classification 5 claims classified, draft risk low.
Pass
Performance claim No performance claim in this draft.
Pass
Testimonial No testimonial or client quote found.
Pass
Promised return No promise, guarantee, or forecast found.
Pass
Missing disclosure All 2 required disclosures attached: D-03, D-04.
Pass
Unsupported claim Every claim is sourced to the fact sheet.
Pass
Number outside the fact sheet Every percentage and currency figure in the body is on the fact sheet and in a supported claim.
A fail blocks the approval POST on the server, not only the button. Warnings do not block. Calculated risk: low. Model risk: low.

Audit

Every decision, refusal, and return

0 events

No audit events yet. Approve, reject, or return a brief to write one.

The log is append only. It is written by app.platform.audit and read back here.

Evaluation

Claim flag accuracy

12 labelled claims
75.0% exact match, 9 of 12
100.0%precision
77.8%recall
87.5%F1

Flags expected 9, returned 7. True positives 7, false positives 0, false negatives 2.

By label

supported 3/3 unsupported 2/3 performance 0/2 testimonial 2/2 promised return 2/2

Failures shown

  • E-04: The practice has grown 38% since 2023. expected factual, unsupported, got performance, unsupported E-04 carried the right flag but the wrong type (performance).
  • E-07: The S&P 500 total return was 12.4% in 2025. expected performance, supported, got factual, supported E-07 was read as a supported factual claim, so the performance flag never fired.
  • E-08: The MSCI EAFE Index returned 9.6% in 2025. expected performance, supported, got factual, supported E-08 was read as a supported factual claim, so the performance flag never fired.
recorded model classification · deepseek-flash · 2026-11-19 05:53 UTC. The score covers the classifier after the deterministic source check. Method page.

Modelled impact

Review cycle time

Modelled estimate
1.87 days from 3.31 days per piece

1.44 days saved, 43.5% of the baseline, on an assumption set. Not a time study.

StepBaselineLoop
Reviewer pickupUnchanged. The queue does not get shorter.0.8 days0.8 days
First passClaims and disclosures arrive pre-checked, so the reviewer confirms rather than sources.1.4 days0.5 days
Rework loop1.8 days x 45% of pieces, then 1.8 days x 15% of pieces.0.81 days0.27 days
Sign-off and filingThe evidence file is assembled by the tool, not by hand.0.3 days0.3 days

At 96 pieces a year: 138.2 cumulative cycle days removed. First-pass returns 43.2 to 14.4, so 28.8 avoided.

Arithmetic in logic.py, review_cycle_model(). Change any assumption and the number moves.

How this is built

Prompt pattern, eval, and what failed

Method page

Prompt pattern

  1. One generation call per brief. Inputs are the brief and the approved fact sheet. Output is strict JSON: headline, body, disclosure ids.
  2. One classification call per draft. The model splits the body into claims, types each one, and cites a fact or disclosure id.
  3. A deterministic source check verifies every percentage and currency figure against the cited fact. A supported label the source does not carry is downgraded before the gate.
  4. The policy gate is code. Any fail blocks the approval POST, the same checks that disabled the form.
  5. One revision call for the blocked draft, then a fresh classification call on the revision before it can be approved.
System prompt: draft generation
You write marketing copy for a US registered investment adviser. You write only from the approved fact sheet and the brief you are given.

Rules:
1. Every number, date, and product fact must appear on the fact sheet with the same value. Never estimate, average, round, or invent a figure.
2. Do not promise or forecast a return. Do not use "will earn", "guaranteed", "safe", or "risk-free".
3. Do not write a testimonial, a client quote, or a client story.
4. Do not name a client, a household, or an account.
5. Put the required disclosure ids in the "disclosures" array exactly as given. Do not paste the disclosure text into the body.
6. Plain, calm, factual prose. No hype adjectives.

Return strict JSON only. No markdown fence, no commentary:
{"headline": "short headline", "body": "paragraph one

paragraph two", "disclosures": ["D-01"]}
System prompt: claim classification
You are a compliance claim classifier for a US registered investment adviser. You receive a marketing draft and the approved fact sheet. Split the draft into atomic claims.

Extract only claims a reviewer could check: a number, a product fact, a performance result, a statement about what clients say, or a promise. Skip greetings, calls to action, thanks, and process language that makes no checkable assertion. Claims come from the body, not the headline.

For each claim:
- type: "factual" for a statement that is not a performance or client statement, "performance" for a return, a comparison, or a result, "testimonial" for a client quote or a claim about what clients say, "promised_return" for a promise, guarantee, or forecast of a future return.
- support: "supported" only if every number and material statement appears on the cited fact or disclosure. If a claim restates a required disclosure, cite the disclosure id (for example D-02).
- source: a fact id, a disclosure id, or "none".

Set risk to low, medium, or high for the draft as a whole.

Return strict JSON only. No markdown fence, no commentary:
{"claims": [{"text": "claim as it appears in the draft", "type": "factual", "support": "supported", "source": "F-01"}], "risk": "low"}

The eval

Twelve hand-labelled claims, one recorded classification call, scored on the classifier the page uses. 9 of 12 exact, precision 100.0%, recall 77.8%. The cases and labels are in data/modules/supervision/eval_cases.json; the script that writes the score is scripts/eval_supervision.py.

What failed

  • Performance claims read as fact. In the eval list, "The S&P 500 total return was 12.4% in 2025" came back factual and supported, so the performance flag missed. The identical sentence came back performance inside the drafts, so the label is context sensitive.
  • Right flag, wrong type. "The practice has grown 38% since 2023" was typed performance rather than a factual claim with no source. The flag fired, the type did not match the label.
  • The first classification pass over-flagged process language as unsupported: "please reply to this email", "we do not give tax advice". All four drafts blocked. The prompt now extracts only checkable claims and the support rule accepts disclosure ids.
  • Generation undershot every brief. Drafts ran 44 to 111 words against briefs asking 60 to 190. The word count sits next to the limit on every draft.
  • The adviser's supplied line was reproduced word for word, as instructed. That is the blocked example, and the gate refused it three ways: performance type, unsupported claim, and a number outside the fact sheet.

Guardrails

  • No approval without a named, independent approver. Self-review is refused on the server.
  • No approval while a policy check fails, and no approval from a form post that skips the checks.
  • Every attached disclosure comes from the fact sheet by id. The model does not type disclosure text.
  • An approval writes an evidence file with the artifact hash and a five-year retention date, and one audit event.
  • Client-facing work for a household without recorded consent cannot be approved for that audience.

Not in this build

  • No send, no scheduling, no CRM, no custodian connection. A sealed evidence file is the end of the workflow.
  • No legal opinion. The policy rules and the fact sheet are synthetic.
  • No reviewer telemetry, so the cycle-time model stays an assumption set.