Ingest and Search

[datcv-holistics]

Prerequisites

https://github.com/holistics/holistics-core/pull/3045

Summary

Ingest Command

• Ingest sources config is in <repo>/.anfra/context_sources.yml. Sample:

   sources:
     - name: local_ecommerce
       type: postgres
       connection:
         host: localhost
         port: 5432
         database_name: ecommerce
         username: abc
         password: abc
       options:
         query_timeout_sec: 10
     - name: local_aml_repo
       type: aml
       connection:
         path: /path/to/any/aml/repo

  Go passes the file path to anfra-node, instead of source objects (with credentials) via JSON-RPC request bodies. This keeps the Go host thin, with clear boundary. Anfra-node owns ingest semantics and source config interpretation.
• Output to the per-repo ~/.anfra/repos/<repo-id>/catalog/catalog.duckdb path.

Search command

Demo

Screen.Recording.2026-07-23.at.17.09.34.mov

[gemini-code-assist]

Code Review

This pull request introduces the ingest and search commands to build and query a local search catalog using the anfra-node and canal-query sidecars, along with associated tests, client methods, and path helpers. The review feedback points out a potential issue where a failed search command returning a non-OK status could cause a JSON unmarshalling error in renderSearchResults, thereby masking the actual error details. It is recommended to check for a successful status before attempting to render search results.

[gemini-code-assist]

If the search command fails or returns a non-OK status (e.g., StatusInvalid with validation diagnostics), env.Data will contain error details rather than the expected search results structure. Attempting to parse this in renderSearchResults will fail with a JSON unmarshal error, masking the actual error details. Checking env.Status == app.StatusOK ensures that we only attempt to render search results on success, and otherwise fall back to renderTo to display the error/diagnostics as YAML.

Suggested change

	if commandName == "search" {
	if commandName == "search" && env.Status == app.StatusOK {

[khanhhuy]

Search is a specific module, its render function should not be placed at this level

[datcv-holistics]

"rendering" is the CLI Command concern anh, so this function renderSearchResults should not be placed in the internal/search package either.

I think we should have an abstraction "Rendering table" then let the search commands utilize it. Later nhe :frog-hehe: