Skip to content

Releases: VirtualFlyBrain/VFBquery

v1.22.3

26 Jun 10:05
1a0f655

Choose a tag to compare

What's Changed

Full Changelog: v1.22.2...v1.22.3

v1.22.1 - Fix duplicated term-info cache writes

24 Jun 18:30
72aebdb

Choose a tag to compare

Fix duplicated term-info cache work

get_term_info (and get_template_roi_tree) were double-wrapped with
@with_solr_cache: the *_cached wrappers carried the decorator while also
delegating to an original that is itself decorated with the same bucket. Every
request therefore ran the SOLR cache layer twice — two reads and two writes
for the same key (the duplicated Cached term_info … log lines and doubled Solr
writes after v1.22.0).

These were the only two double-wraps (every other _cached wrapper relies on its
decorated original). Removed the redundant decorators; for template_roi_tree
also forwarded force_refresh, which the outer decorator previously consumed.

No behaviour change beyond halving cache work and Solr writes per term-info
request.

Full changelog: v1.22.0...v1.22.1

v1.22.0 - Term-info parity, coverage & targeting queries

24 Jun 17:46

Choose a tag to compare

Term-info parity, full field coverage, and targeting queries

Moves the v2 term-info side panel onto VFBquery's get_term_info with display
parity against the legacy SOLR-field path, completes field coverage against the
VFB_json_schema_indexer spec, and adds two live query types.

Restored term-info fields (parity with the legacy panel)

  • Class definition references (def_pubs) — inline microref links on the
    description (was dropped).
  • Individual-term synonyms (pub_syn) — synonym block was Class-only.
  • Publication external content (pub_specific_content) — title / PubMed / DOI /
    FlyBase; the block was gated on Publication but the SOLR marker is pub.

Complete field coverage vs the indexer

  • External DB cross-references → Xrefs (e.g. Insect Brain DB, gene FlyBase).
  • related_individuals / term_replaced_byMeta.RelatedIndividuals.
  • DataSet and License homepage link + logo → Meta.Link / Meta.Logo.

New live query types (preview + count badge)

  • SplitsTargeting — splits that target a neuron class.
  • TargetNeurons — neurons targeted by a split class.
  • Cypher mirrors the indexer neuron_split / split_neuron clauses, served from
    pdb and cached; surfaced as queries rather than static fields.

Robustness

  • Term-info no longer 5xx/stalls on cold-miss terms (e.g. a License individual):
    the SOLR cache write defaults to a soft commit so a wedged IndexWriter cannot
    stall the request. Override with VFBQUERY_SOLR_WRITE_COMMIT=true.
  • fill_query_results skips the limit=-1 re-run when a preview is not
    saturated, cutting cold latency on query-heavy SuperTypes.

Tests

  • src/test/test_term_info_parity.py (14) covers every field gap, the License
    smoke test, and the targeting queries.

After release, warm the v3-cached layer:
owlery-cache-reload --only "V3 term info" --force-refresh

Full changelog: v1.21.1...v1.22.0

v1.21.1

16 Jun 17:55
47c6558

Choose a tag to compare

What's changed

  • Gzip-compress Solr result-cache payloads with a 100 MB cap on compressed size, so oversized entries (e.g. AllAlignedImages, ~383 MB raw) can now be cached; adds defensive cache decode and self-healing of corrupt cache entries (#49)
  • Fix graph_builder's single-query-term expectation; increase AllAlignedImages timeout (#48)
  • CI: restrict push trigger to main/dev to avoid duplicate runs on PRs

Full changelog: v1.21.0...v1.21.1

v1.21.0

15 Jun 15:22

Choose a tag to compare

VFBquery v1.21.0

✨ Highlights

Class-level connectivity queries now return a per-subclass breakdown and use
VFB_connect-consistent normalization. Plus a round of caching/versioning
hardening so cache behaviour is correct across code versions and CI never
pollutes the production cache.

🔌 Connectivity by class (get_upstream_class_connectivity /
get_downstream_class_connectivity)

  • Per-subclass results. Results now cover the queried class and each of its
    subclasses that has connectivity instances — the queried term's rows first,
    then one block per subclass. The queried (sub)class for each row fills the
    upstream_class (downstream query) or downstream_class (upstream query) slot,
    so every row carries both upstream_class and downstream_class (queried class +
    partner), matching the v2 table layout. A query_id field is included for
    grouping rows by queried (sub)class.
  • Presynaptic normalization (matches VFB_connect). total_n / connected_n /
    percent_connected now describe the presynaptic (source) side of each
    connection — the queried class for downstream queries, the partner class for
    upstream queries — consistent with VFB_connect.get_connected_neurons_by_type.
    Validated against VFB_connect: identical partner set and matching
    pairwise_connections/total_weight (~99%; residual differences are
    data-snapshot drift).
  • Column names are unchanged (total_n, connected_n, percent_connected,
    pairwise_connections, total_weight, avg_weight), so the v2 frontend renders
    without changes. Subclass enumeration reuses Owlery's cached subclass closure
    (no extra Neo4j round-trip).

⚠️ Behaviour change: these queries now return more rows (subclass blocks),
▎ populate both class columns, and total_n/connected_n are
▎ presynaptic-normalized. Consumers that assumed a single block per query or
▎ queried-class-normalized counts should review.

🗄️ Caching & versioning

  • Cache invalidation is now monotonic. A cached entry is only invalidated when
    it was written by an older version; a newer entry is treated as a miss but
    never deleted. This fixes cross-version cache thrashing and stops an
    older/stale client (or a downgrade) from purging fresher entries.
  • No cache writes from PR checks. VFBQUERY_CACHE_ENABLED=false is now a true
    full bypass (no read/write/purge/cache-server contact), and a new
    VFBQUERY_CACHE_READONLY=true allows reads while suppressing all writes/purges.
    CI uses these so PRs validate live queries and never modify the shared
    production cache.
  • Single source of version truth. The package version now lives only in
    src/vfbquery/_version.py; setup.py, vfbquery.version, and the cache's
    version stamp all derive from it, and releases bump just that file (and commit
    it back to main). Fixes the previous drift between setup.py and init.py.

🐛 Fixes

  • Term-info downloads links now use the absolute
    https://virtualflybrain.org/data/... host (consistent with filemeta); removed
    a dead duplicate get_secure_data_url.
  • Docker image version-verify now reads the version from _version.py (the
    single source), fixing a false "version mismatch" on build.

🔧 CI / internal

  • Performance-test job timeout raised and thresholds adjusted for the live,
    uncached path; cache is writable on push-to-main/scheduled runs (read-only
    only on PRs) so those runs warm the cache.
  • New docs: RELEASING.md, expanded CACHING.md (bypass/read-only flags +
    version invalidation).

v1.20.0 - TermsForPub expression refs + Reference type column

09 Jun 07:20

Choose a tag to compare

1.20.0 — TermsForPub: expression-cited terms + Reference type column

TermsForPub only followed has_reference edges, so a publication cited solely
via the overlaps/part_of relationship pub property (the AnatomyExpressedIn /
TransgeneExpressionHere model) returned nothing — e.g. FBrf0232433 (VT-GAL4
expression data) is cited on ~63k expression edges covering ~5.9k patterns but
has zero has_reference edges.

TermsForPub now lists terms from both mechanisms, deduplicated per term, and
adds a Reference type column: Reference for has_reference sources,
Expression for overlaps/part_of expression patterns (joined if a term is
cited both ways).

The minor bump from 1.19 rotates the major.minor solr_result_cache key.

Validated read-only against prod pdb: FBrf0232433 -> expression patterns (was
empty); FBrf0242477 -> Reference datasets + Expression GAL4 lines.

v1.19.0 - cache all handler-reachable queries

08 Jun 14:27
3a2bfc8

Choose a tag to compare

1.19.0 - cache all handler-reachable queries

Completes the persistent SOLR result-cache coverage: every query-result
function reachable from the HA API handlers is now cached, instead of only a
subset. Adds composite-key persistence for the cross-dataset
query_connectivity, and moves the cache to the dedicated query-cache Solr
(vfbquerycache.virtualflybrain.org:80) with a VFBQUERY_SOLR_URL override.

Coverage sweep: 12 uncached handler entry points -> 0. Cache-hit proofs show
46x-2790x cold->warm speed-ups across the newly-cached query types. The minor
bump rotates the major.minor cache key.

v1.18.0 - clusterExpression Function column

06 Jun 08:39

Choose a tag to compare

1.18.0 — clusterExpression Function column

The migrated "Genes expressed in " (clusterExpression) query did not
surface the gene Function column that v2 prod shows. Gene molecular-function
categories (e.g. Enzyme, Small_molecule_binding) are Neo4j labels on the Gene
node, so Function is now derived from labels(g), excluding the structural and
availability labels (Entity, Class, Individual, Gene, Feature and any has*
facet) and sorted. Added alongside the existing columns; the reverse
expressionCluster query is unchanged.

The minor bump from 1.17 rotates the major.minor solr_result_cache key,
flushing cached clusterExpression results so the Function column appears.

Validated read-only against prod pdb: 128up -> "Enzyme; Small_molecule_binding".

v1.17.0 - carousel up to 5 example images

04 Jun 20:43

Choose a tag to compare

1.17.0 — carousel up to 5 example images for class/dataset rows

AlignedDatasets, AllDatasets, AnatomyExpressedIn and TransgeneExpressionHere
each bound a single representative example image (LIMIT 1) into the Images
cell, so a class/dataset row showed only one thumbnail even when it has many
examples. These four queries aggregate one-to-many example individuals into a
single row, so they now collect up to 5 examples and emit them as a
' | '-joined [![alt](url 'title')](ref) list -- the carousel shape the V2
imageMarkdownToVariableJson splits into a SlideshowImageComponent.

Per-individual queries (instances, templates, NBLAST similarity, connectivity
partners, dataset image listings, terms-for-pub) are one-image-per-row by
design and are unchanged.

The minor bump from 1.16 rotates the major.minor solr_result_cache key,
flushing the single-image cells cached under 1.16.0.

Validated read-only against prod pdb: AlignedDatasets=5, medulla=4, EP=2.

v1.16.0 - rotate cache version to flush stale entries

04 Jun 18:45

Choose a tag to compare

1.16.0 — rotate cache version to flush stale entries

Minor-version bump from 1.15.x. The solr_result_cache key is keyed on the
major.minor package version, so moving 1.15 -> 1.16 invalidates every cached
query result. This forces fresh output that includes the accumulated 1.15.x
query-output fixes:

  • Dataset Reference column linked to publications (FBrf / DOI)
  • Term labels for all link columns (template keeps symbol)
  • Per-class Images carousel (all example images)
  • /reports permalinks, Template anatomy fix, Type-column links, Gene names

No code change beyond the version.