Add thread-safe TreeSitterPool for concurrent parsing#30
Conversation
Port the pool implementation from ch007m/java-tree-sitter#6. Uses Semaphore + ConcurrentLinkedDeque (no synchronized blocks) so it is safe with virtual threads. Includes tests for borrow/return, reuse, discard, concurrency limiting, and parallel parsing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cmoulliard
left a comment
cmoulliard
left a comment
There was a problem hiding this comment.
Maybe a silly question: What is finally the benefit to have in a pool x instances of treesitter wasm runtime vs having a pool of the parsers that we can use when we would like to parse in parallel string (java, xml, json, etc code) ?
| try (TreeSitterPool pool = TreeSitterPool.create(4)) { | ||
| try (TreeSitterPool.Loan loan = pool.borrow()) { | ||
| try (TreeSitterParser parser = loan.instance().newParser(Language.JAVA); | ||
| TreeSitterTree tree = parser.parseString(source)) { | ||
| System.out.println(tree.rootNode().toSexp()); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Should we add a comment here to tell to the users that if they try to create 5 parsers: json, xml, yaml, properties, java and markdown, then the semaphore will reject ?
There was a problem hiding this comment.
Good question! The pool size limits concurrent borrows, not languages. Each TreeSitter instance can parse any language, so a pool of 4 can serve all 6 grammars — borrow() only blocks when all instances are already in use. I've updated the docs to clarify this.
Just a different level of granularity, also the two options are not exclusive |
Something to be implemented maybe in a separate PR ? |
I think it's a "user"'s concern, but not fussy 🤷 |
|
LGTM. Approved |
2 participants
Port the pool implementation from ch007m/java-tree-sitter#6. Uses Semaphore + ConcurrentLinkedDeque (no synchronized blocks) so it is safe with virtual threads. Includes tests for borrow/return, reuse, discard, concurrency limiting, and parallel parsing.
Supersede #13