Synthetic Personas for Testing AI Digital Twins

An evaluation harness that spins up a fake human personality, runs it through a digital-twin training pipeline, and scores how faithfully the resulting twin reproduces that personality's answers.

@sokkkkaaa

x.com/sokkkkaaavisit ↗

Synthetic Personas for Testing AI Digital Twins

process.log6 fields

toolsClaude Code (Opus)local MLX inference (gemma-3-4b)FastAPI + SQLAlchemy backendFlutter/Riverpod client.

iterationsMany attempts

repeated A/B measurement runs across one session, fidelity climbing 17% → 94%

%_ai97

ai 97

me 3

self-reported · generously rounded

# how_it_was_made

The core problem was that testing the twin normally required a real human to sit through a slow, days-long self-evaluation flow, so I asked for a way to fake the human. We built a synthetic-persona harness: a hand-authored ground-truth personality (a risk-averse schoolteacher, deliberately the opposite of me) that an LLM roleplays through the real onboarding pipeline, after which the trained twin answers a fixed question set and an LLM judge scores each reply against the persona's expected stance. I ran it as a repeatable A/B measurement tool, and each pass surfaced concrete bugs to fix; Claude handled the harness code and diagnosis while I set the metrics that counted (extraction coverage and simulation fidelity). When paid API credits ran dry, we swapped the whole thing onto a local OpenAI-compatible model server so runs cost nothing.

questions0

no questions yet

ask how it was made — the dead-ends are usually the good part

ask about the process…sign in

~/slopfolio·

# prompting_approach

I started from a rough problem statement ("testing is too slow, fake a personality instead") rather than a spec, let it propose the harness design, then drove it as a measurement loop — run, read the metric, fix one thing, re-run. Most of my prompts were single-change follow-ups reacting to the last number.