Skip to content

Add Inkling deployment helper for Akash#123

Draft
Tibo2403 wants to merge 4 commits into
mainfrom
agent/add-inkling-akash-deploy
Draft

Add Inkling deployment helper for Akash#123
Tibo2403 wants to merge 4 commits into
mainfrom
agent/add-inkling-akash-deploy

Conversation

@Tibo2403

Copy link
Copy Markdown
Owner

What changed

  • adds scripts/bash/deploy_inkling_akash.sh
  • generates an Akash SDL targeting 1x RTX 3090 by default
  • deploys a LiteLLM gateway to a remote OpenAI-compatible Inkling endpoint
  • explicitly blocks misleading self-host deployment on a single RTX 3090
  • adds docs/inkling-akash.md with setup, security, cost, and test instructions

Why

Inkling cannot be self-hosted on a 24 GiB RTX 3090. The helper preserves the requested Akash GPU option while making the limitation explicit and providing a usable API-gateway deployment path.

Validation

  • script uses strict Bash mode and validates numeric/options input
  • --dry-run prints the generated SDL without writing or deploying
  • deployment requires explicit --deploy and an AKASH_KEY_NAME
  • generated files containing credentials are created with restrictive permissions

Important limitation

The RTX 3090 is not used for inference in API mode; a CPU-only Akash gateway would be cheaper. The documentation calls this out explicitly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant