Skip to content

Fix/quickstart demo flow#75

Merged
shirshanka merged 5 commits into
datahub-project:mainfrom
lakshay-nasa:fix/quickstart-demo-flow
Jun 29, 2026
Merged

Fix/quickstart demo flow#75
shirshanka merged 5 commits into
datahub-project:mainfrom
lakshay-nasa:fix/quickstart-demo-flow

Conversation

@lakshay-nasa

Copy link
Copy Markdown
Collaborator

Summary

  • Auto-generate DataHub token signing keys so fresh installs work with DataHub v1.5.0.6+
  • Add ENGINES_CONFIG to .env.quickstart so the Docker agent reads the mounted config.yaml
  • Use localhost instead of host.docker.internal in quickstart --demo native flow

Test plan

  • Fresh bash quickstart.sh completes without signing key error
  • Agent at http://localhost:8100 shows the demo engine connected after setup
  • analytics-agent quickstart --demo connects to MySQL on all platforms

DataHub v1.5+ requires DATAHUB_TOKEN_SERVICE_SIGNING_KEY and
DATAHUB_TOKEN_SERVICE_SALT to be non-empty. Without them the
system-update container fails on a fresh install. Auto-generate
both with openssl rand if not already set in the environment.
…ted config

The agent defaults to reading config.yaml from ~/.datahub/analytics-agent/
which does not exist inside the container. Without ENGINES_CONFIG pointing
at /app/config.yaml, the bootstrap seeds no engines and the agent starts
with no data source connected.
….internal

The native demo flow runs the agent directly on the host machine, so
MySQL and DataHub GMS are reachable via localhost. Using host.docker.internal
caused DNS resolution failures on some macOS setups and always fails on Linux
where Docker Desktop is not available.
@lakshay-nasa lakshay-nasa requested a review from shirshanka June 5, 2026 05:24
shirshanka and others added 2 commits June 27, 2026 17:39
# Conflicts:
#	backend/src/analytics_agent/quickstart.py
The DataHub v1.5+ signing-key auto-generation relies on `openssl rand`.
Without openssl the keys would be silently empty, reintroducing the
signing-key error this flow fixes. Add it to the prerequisite checks so
the failure is caught up front with a clear install hint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@shirshanka shirshanka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Three small, correct quickstart fixes — verified against current main (rebased; only a one-line docstring conflicted with #74's Fiction Retail rename, resolved to main's wording):

  • Signing-key auto-gen: DataHub v1.5.0.6+ requires DATAHUB_TOKEN_SERVICE_SIGNING_KEY / _SALT to be non-empty; auto-generated via openssl if unset, exported before the compose runs. Correct placement.
  • ENGINES_CONFIG=/app/config.yaml in the .env.quickstart heredoc so the dockerized agent reads the mounted config (no duplicate with #76).
  • localhost for the native --demo flow: correct, since the agent runs natively on the host and reaches the containers via published ports. _HOST_INTERNAL is correctly retained for _ingest_metadata, whose ingestion executor runs inside Docker — the host/internal split is intentional and right.

Verified: import OK, ruff + mypy clean, 10 quickstart unit tests pass.

Pushed one hardening commit (maintainer edit): added check_cmd openssl to the prerequisites, since the signing-key generation depends on it — without it the keys would be silently empty and reintroduce the very error this fixes.

@shirshanka shirshanka merged commit b8e3828 into datahub-project:main Jun 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants