Fake Data

A fresh dev box has an empty database, so the pages that summarise activity — tag stats, reclassification analytics — render as a wall of empty charts. create_fake_data fills them in.

python3 manage.py create_fake_data <subcommand>
python3 manage.py create_fake_data <subcommand> --delete

This is development data only. Everything it writes is obviously fake, so it can't be mistaken for real curation, and --delete takes it all away again.

Subcommands

Subcommand	Fills in	Module
tags	Variant tags — the tag stats page, and the "Your tagging" card	analysis/fake_variant_tags.py
reclassifications	Germline classifications with a curation history — the reclassification analytics page	classification/fake_reclassifications.py

Each takes --delete, --seed (runs are reproducible), --genome-build and --years, plus its own options — --help on the subcommand lists them.

# ~200k tag events over 5 years, and mix your own user in so their card isn't empty
python3 manage.py create_fake_data tags --also-tag-as jsmith

# 500 classifications, each revisited over 7 years
python3 manage.py create_fake_data reclassifications --classifications 500

Shape matters more than volume

These pages are all about how the data behaves, so a uniform random fill makes them look wrong in a way that's worse than empty — every bar the same height, no tail, no trend. Both modules spend most of their code on shape rather than quantity.

tags gives a handful of tags most of the work, introduces some part way through the period, has users come and go, and lets a few artefact variants soak up hundreds of re-tags.

reclassifications models how curation actually moves: records mostly move one significance bucket at a time, VUS carries nearly all the movement, P and B hardly budge, most reviews confirm the call rather than change it, labs revisit at different rates, and the ACMG criteria that flip are the ones that would justify the direction travelled. --adjacent-percent tunes how often a move goes further than the neighbouring bucket.

Both pick real variants and gene symbols, so the charts group on real symbols and the links go somewhere. That means they need annotated variants to join against — on a database with no annotation they'll report how few they found and create less than you asked for.

Deriving rather than writing

Where a page reads a derived table, the fake data builds the source history and then runs the same derivation the app does, rather than writing the derived rows directly. reclassifications writes a chain of published ClassificationModifications (each with its own clinical significance, curation date and criteria) and hands them to ReclassificationEventBuilder, which is what the analytics page itself calls. So what you see is what a real import would have left behind, and a change to the derivation shows up in the fake data too.

Adding a new one

Write a class in the app the data belongs to with HELP, add_arguments, create and delete, then register it in FAKE_DATA in snpdb/management/commands/create_fake_data.py. analysis/fake_variant_tags.py is the worked example.

Two things to get right:

• Timestamps. created / modified are auto_now_add / auto_now, which would stamp years of history as happening now. Both modules use a _keeping_our_timestamps context manager that turns the fields off around the bulk_create.
• Permissions. bulk_create skips the model's save(), so GuardianPermissionsMixin never assigns anything. Create the GroupObjectPermission rows explicitly — everyone can see (and clean up) fake data, so all_users is all it needs. See Users, permissions and groups.

Make delete find its own records by something structural rather than a list of ids — the fake tag ids, or a lab_record_id prefix — so it still works on a database somebody has since added to.

Not the same as test fixtures

Unit test fixtures are a different thing, and live elsewhere (snpdb/tests/utils/, annotation/tests/test_data_fake_genes.py). Those build the minimum a test needs; this builds a plausible-looking database for a human to look at. See Testing.