ZCU-PUB: fix slow /browse/subject (~5s→~0.2s) — backport #10057 metadata-browse json.facet/numBuckets#1328
Open
vidiecan wants to merge 2 commits into
Open
ZCU-PUB: fix slow /browse/subject (~5s→~0.2s) — backport #10057 metadata-browse json.facet/numBuckets#1328vidiecan wants to merge 2 commits into
vidiecan wants to merge 2 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Fixes the slow
/browse/subject(and any metadatavalueListbrowse: subject/author/language) on ZCU-PUB. Root cause is upstream DSpace #9244: the pre-7.6.3SolrBrowseDAOrequests the Solr facet withlimit = -1, so Solr returns every distinct value on every page request and the code discards all but the page. Cost ≈ O(distinct values).ZCU-PUB has 254,531 distinct subjects, so subject browse measured ~5.25 s mean (peak 11 s) against
naos-be.zcu.cz. This backports the upstream fix (PR #10057) — ajson.facetterms query with server-sidelimit/offset/numBuckets— which returns only the requested page plus the exact total.Full investigation, measurements and reproduction: dataquest-dev/dspace-customers#668.
Changes
Cherry-pick of the two upstream 7.6.x backport commits (clean for the core fix; one trivial import/method conflict in
SolrServiceImpl.javaresolved to the refactored form):fcc650e1a6— Add limit, offset, and a total-entry-count parameter to the metadata-navigation Solr query (the core fix; touchesSolrBrowseDAO,SolrServiceImpl,DiscoverResult,BrowseEngine).d6ff41d9f5— Refactor browse entries facet query to use JSON facet query.SolrBrowseDAOnow builds ajson.facetterms facet on<field>_filterwithlimit/offset/numBuckets:true;DiscoverResult.getTotalEntries()is populated fromnumBuckets. No change to the date/item (flatBrowse) path.Notes
<field>_filterfields already exist in the 7.6 schema (used by discovery facets).ZCU-PUB/partial-word-match-browsing-using-api); this branch is cut from currentcustomer/zcu-pubHEAD, so that work (if merged) is preserved — please sanity-check partial-word browse after deploy.customer/TUL. ZCU-DATA runs the same code but is unaffected (40 subjects) — no change needed there.Verification
mvn -pl dspace-api -am test-compile checkstyle:check→ BUILD SUCCESS (compiles + checkstyle clean).Post-deploy acceptance (re-run against
https://naos-be.zcu.cz/server/api):GET /discover/browses/subject/entries?page=0&size=20mean < 500 ms (expect ~100–300 ms), andpage=0≈page=5.page.totalElementsunchanged (254,531) — proves thenumBucketstotal wiring is correct.