diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..f22cc06 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,51 @@ +name: Docs + +on: + push: + branches: + - master + # build (but do not deploy) on pull requests, so that a broken link or a + # page missing from the nav fails before it reaches master + pull_request: + branches: + - master + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - name: Install uv + uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 + with: + enable-cache: true + cache-dependency-glob: "pyproject.toml" + - name: Install docs dependencies + run: uv sync --only-group docs + - name: Build docs + run: uv run --no-sync mkdocs build --strict + - name: Upload artifact + uses: actions/upload-pages-artifact@v5 + with: + path: site + + deploy: + if: github.event_name == 'push' + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + runs-on: ubuntu-latest + needs: build + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v5 diff --git a/.gitignore b/.gitignore index 6f3cb1c..e64159a 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ __pycache__ .venv dist uv.lock +site diff --git a/CLAUDE.md b/CLAUDE.md index 908be44..8b1a9d3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,11 +17,14 @@ uv run pytest tests uv run pytest tests/test_conf.py::test_load_conf # a single test uv run pytest -k "run_cmd and bash" # a single shell's parameters + +uv sync --group docs && uv run mkdocs serve # the docs site, with live reload +uv run mkdocs build --strict # what CI builds, warnings fatal ``` ## Testing -The tests that read and write credentials need an OS keyring that unlocks without user interaction. They skip themselves with a message when there is none, so the rest of the suite still runs; `KEYCMD_REQUIRE_OS_KEYRING=1` turns those skips into failures, and CI sets it. Windows needs no setup, macOS needs an unlocked keychain, and Linux needs the tests to run inside a d-bus session with `gnome-keyring` unlocked (see the Testing section of the README for the exact commands). `PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring` with `uv run --with keyrings.alt` avoids the OS keyring entirely. +The tests that read and write credentials need an OS keyring that unlocks without user interaction. They skip themselves with a message when there is none, so the rest of the suite still runs; `KEYCMD_REQUIRE_OS_KEYRING=1` turns those skips into failures, and CI sets it. Windows needs no setup, macOS needs an unlocked keychain, and Linux needs the tests to run inside a d-bus session with `gnome-keyring` unlocked (see `docs/development/testing.md` for the exact commands). `PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring` with `uv run --with keyrings.alt` avoids the OS keyring entirely. `tests/test_wsl.py` covers calling the Windows install of keycmd from a shell inside WSL, and is opt in through `KEYCMD_TEST_WSL=1` on a Windows machine with WSL installed. `tests/test_wsl_interop.py` covers the same boundary as far as it can be reached without one, by faking the Windows process table, and runs everywhere. The rest of the suite stays off that code path entirely: the autouse `outside_wsl` fixture in `tests/conftest.py` clears the flags `wsl.py` detects with, so a run on Windows looks like a run anywhere else. @@ -33,6 +36,12 @@ Things that bite in this suite: - **Do not assume the suite runs unpinned.** `PYTHON_KEYRING_BACKEND` is how the README suggests running the suite without an OS keyring, and it outranks everything `backend.py` does, so a test about remembering has to `delenv` it first or it will be testing the path that deliberately remembers nothing. - Warnings are errors (`filterwarnings` in `pyproject.toml`), so a deprecation in a new Python release fails the suite rather than scrolling past. +## Documentation + +The prose lives in the mkdocs site under `docs/`, built with mkdocs-material and deployed to GitHub Pages by `.github/workflows/docs.yml` on every push to `master` (pull requests build it without deploying, so a broken link fails before it lands). The nav in `mkdocs.yml` is explicit, so a new page has to be added there or the strict build fails on it. Screenshots live in `docs/assets/`. + +The README is a landing page and nothing more: what it says about behaviour it says in a sentence, and links to the page that covers it. New prose belongs on the site โ€” a section that grows in the README is a section that has drifted from its page. + ## Architecture `cli.main` wires the three halves together: `load_conf` produces the configuration, `get_env` turns it into an environment, and `run_cmd`/`run_shell` hand that environment to a shell. `--detect-backend` and `--reset-backend` return before any of it, since neither has a use for a configuration or a command. Errors reach the user through `logs.error`, which exits with status 1 and takes the hint lines that go under the error with it; `logs.vlog` output only appears under `--verbose` and is the first thing to reach for when debugging a configuration. diff --git a/README.md b/README.md index 978035f..e4e08ec 100644 --- a/README.md +++ b/README.md @@ -1,537 +1,68 @@ -# keycmd +# keycmd ๐Ÿ”‘ -[![CI](https://github.com/clinicalgraphics/keycmd/actions/workflows/ci.yml/badge.svg)](https://github.com/clinicalgraphics/keycmd/actions/workflows/ci.yml) -[![PyPI version ](https://badge.fury.io/py/keycmd.svg) -](https://badge.fury.io/py/keycmd) +[![CI](https://github.com/Korijn/keycmd/actions/workflows/ci.yml/badge.svg)](https://github.com/Korijn/keycmd/actions/workflows/ci.yml) +[![PyPI version](https://badge.fury.io/py/keycmd.svg)](https://badge.fury.io/py/keycmd) -Prefix any command with `keycmd` to safely source your secrets and credentials from the OS keyring, instead of risky `.env` files (or worse ๐Ÿ™ˆ). Common applications include npm, pip, poetry, docker, docker compose and kubectl! +**Prefix any command with `keycmd` to source your secrets from the OS keyring**, instead of risky `.env` files (or worse ๐Ÿ™ˆ). Your credentials are exposed as environment variables for exactly one command, and nowhere else. -Supports Windows, macOS and Linux. +๐Ÿ“– **[Documentation](https://korijn.github.io/keycmd)** ยท ๐Ÿ“ฆ **[PyPI](https://pypi.org/project/keycmd/)** ยท ๐Ÿš€ **[Quick Start](https://korijn.github.io/keycmd/getting-started/quick-start/)** -## About +Supports Windows, macOS and Linux. Common applications include npm, pip, uv, poetry, docker, docker compose and kubectl. -The main functionality of `keycmd` is to load secrets from your OS keyring and expose them as environment variables for the duration of a single shell command or alternatively for the lifetime of a subshell. - -This enables you to store sensitive data such as authentication tokens and passwords in your OS keyring, so you no longer need to rely on insecure practises such as `.env` files, or pasting secrets into your terminal. ๐Ÿ˜ฑ - -The most common use case is to load credentials for package managers such as pip and npm when using private package indexes, such as Azure Artifact Feeds. Another common use case is docker build secrets. - -## Installation - -`keycmd` requires Python 3.13 or newer. - -> **Note** -> If you're intending to install `keycmd` in a WSL or pyenv environment, you'll have to skip ahead to the specific installation instructions for those environments. - -### Global installation - -Since `keycmd` is a command line tool, the recommended way to install it is with [uv](https://docs.astral.sh/uv/): +## Quick start ```bash uv tool install keycmd ``` -This installs `keycmd` into its own isolated environment and puts the executable on your `PATH`. Alternatively, install it from pypi using `pip install keycmd`, or whatever alternative python package manager you prefer. - -Note that the executable `keycmd` has to be installed to a folder that is on your `PATH` environment variable, or the command won't be available globally. Assuming you were able to run `pip` just now, the `keycmd` executable should end up in the exact same location and everything should be fine. - -To verify keycmd is installed and available, run `keycmd --version`. - -### pyenv installation - -Now, if you're using pyenv, you're going to have to jump through a few hoops since keycmd needs to be installed globally, which flies directly into the face of what pyenv is trying to accomplish. - -This guide assumes you've also installed [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv), in order to get you the cleanest of setups. โœจ - -> **Note** -> These pyenv instructions are for pyenv on Linux and MacOS. If you are using pyenv-win on Windows, these instructions are most likely not 100% compatible with your setup. - -Run the following commands one by one to install keycmd into its own standalone environment: - -```bash -# run the following commands one by one -pyenv virtualenv 3.13 keycmd -pyenv activate keycmd -pip install keycmd -pathToKeycmd=$(python -c 'import sys; from pathlib import Path; print(Path(sys.executable).parent / "keycmd")') -pyenv deactivate -mkdir -p $HOME/.local/bin -ln -s $pathToKeycmd $HOME/.local/bin/keycmd -``` - -Finally, edit your `~/.bashrc` file (or whatever shell profile you use) to include `~/.local/bin` in your `PATH` variable: - -```bash -export PATH="$HOME/.local/bin:$PATH" -``` - -> **Note** -> This line may already be in place in your `~/.bashrc`, for example, if you installed poetry! It's a common trick used to expose specific binaries on `PATH` when they are in folders that also include binaries that should _not_ be exposed on `PATH`. - -To verify keycmd is installed and available, run `keycmd --version`. - -### WSL installation - -If you're using WSL, you'll run into a wall when you first try to use keycmd. That's because keycmd uses the keyring library to connect to OS keyrings, and keyring will attempt to connect to your linux distro's (probably Ubuntu) keyring background service, which by default isn't actually running in a WSL environment! - -If you did actually set up your linux distro's keyring background service, that's fine, you can continue using it and don't need to perform any additional steps. - -> **Note** -> Just because you installed WSL on your system, does not mean you are actually working in WSL. Think about this for a moment; are you using Python from Windows or from WSL? The instructions here are only necessary if you are actually working in WSL. - -So, if you would like keyring to connect from the WSL environment to your Windows Credential Manager instead, continue with the next steps. - -You have to install keycmd according to the above instructions (globally, or with pyenv) **in Windows**, not in WSL. Then, assuming `keycmd` is on your Windows `PATH`, it should now be available in WSL as well! - -Keep in mind that keycmd is a Windows process in this setup, so left to its own devices it would run your command in a Windows shell, and `keycmd --shell` would open one. It doesn't: when keycmd notices it was called from a distro, it runs your command back inside that distro through `wsl.exe`, and `keycmd --shell` opens a shell there. If you have more than one distro installed, and you are working somewhere on the distro's own file system, keycmd targets the distro you are in rather than the default one. - -Your credentials do not come along by themselves, since neither side of the WSL boundary inherits the other's environment. Only the variables listed in [`WSLENV`](https://devblogs.microsoft.com/commandline/share-environment-vars-between-wsl-and-windows/) make the trip, so keycmd adds the variables from your configuration to it. - -If keycmd gets it wrong, set the `KEYCMD_WSL` environment variable to `0` to keep it on the Windows side, or to `1` to send it through `wsl.exe` regardless. `keycmd --verbose` reports which way it went, and what it based that on. - -### Up- and downgrading - -If at a later point in time, you want to install a different version of keycmd, just use pip again. - -> **Note** -> If you're using pyenv, make sure to activate the virtual environment first with `pyenv activate keycmd`. Don't forget to `pyenv deactivate` afterwards. - -* To upgrade to latest: `pip install -U keycmd` -* To install a specific version: `pip install keycmd==0.6.0` - -## Quickstart - -Now that keycmd is installed, we can perform a quick test to see how it works! - -Let's add a new key to our OS keyring, and then see how we can expose it with keycmd. - -For the purpose of this example, use `my-secret` as the credential name and `my-username` as the... username. I used `foobar` as the password. - -### Windows -On Windows, that means clicking Start and typing "Credential Manager" to find the app. Click the Windows Credentials tab, and click "Add a generic credential". See the screenshots below. - -![Credential Manager](docs/wcm.png) - -![Add Key](docs/wcm-add-key.png) - -### MacOS -On macOS, open Keychain Access (Command-Space bar, type `keychain access`), then press Command-N to add a -new password item. Please note that Account Name holds the username. - -![Add New Password Item](docs/keychain-access-new-password-item.png) - -### Linux -On Linux, the easiest way to add a credential is via python's `keyring` package. Manually adding a password via the GUI does not always allow you to set a username, which is problematic with `keycmd`'s internal `keyring.get_password()` call. Here an example of setting a password: - -```python ->>> import keyring ->>> keyring.set_password("my-secret", "my-username", "foobar") -``` - -after which it appears in the credential manager. This approach should generalize for other distributions (and even other OS's). On Ubuntu `seahorse` is used, which now should show the new password: -![Seahorse](docs/seahorse.png) - -### Configuration -Now, create a `.keycmd` config file in your user home folder. Put the following configuration in the file and save: - -```toml -[keys] -SECRET = { credential = "my-secret", username = "my-username" } -``` - -Finally, open a terminal and run a command to print the secret, so we can see if it worked. That's going to look different depending on what shell you're using, so here's a couple examples: - -* Cmd: `keycmd echo %SECRET%` -* Powershell: `keycmd 'echo $env:SECRET'` -* Bash: `keycmd 'echo $SECRET'` - -You should see the text `foobar` being printed to the terminal. - -You've successfully set up keycmd! ๐Ÿ‘ - -See the [advanced configuration example](#advanced-example) below for a more involved usecase for keycmd, where poetry, npm and docker-compose are all put together. - -## Usage - -The CLI has the following options: - -``` -โฏ keycmd --help -usage: keycmd [-h] [-v] [--version] [--detect-backend] [--reset-backend] - [--shell] - ... - -positional arguments: - command command to run - -options: - -h, --help show this help message and exit - -v, --verbose enable verbose output, useful for configuration debugging - --version print version info - --detect-backend search for the keyring backend now and remember it for - later runs - --reset-backend forget the remembered keyring backend, so the next run - searches again - --shell spawn a subshell instead of running a command -``` - -There are two main ways to use the CLI: - -* `keycmd 'your command'` -* `keycmd --shell` - -The first is the most preferred method, since your secrets will only be exposed as environment variables during a one-off command. The latter is less preferable, but can be convenient if you are debugging some process that depends on the credentials you are exposing. - -Quoting the whole command as one argument is what lets you use your shell's syntax inside it, as in `keycmd 'echo $SECRET | tr a-z A-Z'`: keycmd hands that line to your shell exactly as you typed it, and your shell does the rest. - -You can also write the command out as separate arguments, and then keycmd keeps them separate: - -```bash -# arrives as a single argument, spaces and all -keycmd mytool --message 'hello world' -``` - -Since each argument is passed on as the word it was, your shell's syntax is *not* interpreted a second time in this form. If you want `$SECRET` expanded, either let your own shell expand it, or use the single argument form above. - -## Configuration - -> **Note** -> if you are having trouble configuring keycmd, refer to section [debugging configuration](#debugging-configuration). - -### Locations - -Configuration can be stored in three places (where `~` is the user home folder and `.` is the current working directory when calling `keycmd`): - -- `~/.keycmd` -- all `.keycmd` found while walking file system up from `.` -- first `pyproject.toml` found while walking file system up from `.` - -> **Note:** -> The search for `.keycmd` and `pyproject.toml` will stop at the root of a git repository, and before the user home folder, to ensure your configuration can be applied locally to subtrees of your filesystem. - -Configuration files are loaded and merged in the listed order. - -### Fields - -The options schema is defined as follows: - -* `keys`: dict - * `{key_name}`: dict - an environment variable will be created with this name - * `credential`: str - the name of the credential in your keyring - * `username`: str - the username associated with the credential in your keyring - * `b64`: bool, optional - set to `true` to apply base64 encoding - * `format`: str, optional - apply a format string (applied before base64 encoding) -* `aliases`: dict, optional - * `{alias_name}`: dict - an environment variable will be created with this name - * `key`: str - the key that should be aliased - * `b64`: bool, optional - see `keys.{key_name}.b64` - * `format`: str, optional - see `keys.{key_name}.format` - -### Format strings - -The format string allows you to preprocess the credential before it is exposed as an environment variable. - -The format string is processed using Python's built-in [`str.format`](https://docs.python.org/3/library/stdtypes.html#str.format) so you have access to all formatting functionality supported by that function. - -Three variables are available for use in the format string: - -* `credential` -* `username` -* `password` - -So for example you can put together a basic auth header with a configuration string like this: - -```toml -[keys] -MY_TOKEN = { credential = "MY_TOKEN", username = "azure", format = "{username}:{password}", b64 = true } -``` - -### Aliases - -Aliases can be used to expose the same secret in multiple forms. - -For example, you may have a single Personal Access Token for Azure DevOps, and wish to use the same token for `pip`, `npm` and the REST API. `pip` wants you to provide the token in plain text, `npm` prefers it to be base64-encoded and the REST API is expecting a basic auth header. Aliases make this easy: - -```toml -[keys] -MY_TOKEN = { credential = "azure_secret", username = "azure" } - -[aliases] -MY_TOKEN_B64 = { key = "MY_TOKEN", b64 = true } -MY_TOKEN_BASICAUTH = { key = "MY_TOKEN", format = "{username}:{password}", b64 = true } -``` - -### pyproject.toml example - -You can also store your configuration in `pyproject.toml`, by prefixing the keys with `tool.keycmd`. So if we were to convert the previous example it would look like this: - -```toml -[tool.keycmd.keys] -MY_TOKEN = { credential = "azure_secret", username = "azure" } - -[tool.keycmd.aliases] -MY_TOKEN_B64 = { key = "MY_TOKEN", b64 = true } -MY_TOKEN_BASICAUTH = { key = "MY_TOKEN", format = "{username}:{password}", b64 = true } -``` - -## OpenAI example - -With OpenAI, you're instructed to [use an API key](https://github.com/openai/openai-python#usage) to authenticate with their APIs. When you put that string in a `.env` file, or directly in your code, you risk sharing your API key with the world! ๐Ÿ™…โ€โ™‚๏ธ - -Instead, just put it in your OS keyring, and expose it with keycmd when you run your python scripts or jupyter notebooks. - -For example, if you add it to your OS keyring under the name `my-openai-token` and `your-username`, you would use the following `.keycmd` configuration: +Store a credential in your OS keyring, name it in a `.keycmd` file: ```toml [keys] OPENAI_API_KEY = { credential = "my-openai-token", username = "your-username" } ``` -Now you can run any OpenAI script by just prefixing your commands with `keycmd`. For example: +...and run anything that needs it: ```bash keycmd 'python my_openai_script.py' ``` -Or a jupyter notebook: - -```bash -keycmd 'jupyter notebook' -``` - -That's all! ๐Ÿค˜ Now you can rest easily, knowing your tokens are safe. ๐Ÿ›Œ๐Ÿ’ค - -## Advanced example - -This is an example configuration for Poetry, npm and docker-compose. It should inspire you to see the possibilities keycmd provides thanks to its configuration system. - -In this case, we are authenticating with an Azure DevOps Personal Acces Token to an Azure Artifacts Feed which serves both python and node.js packages. - -Let's begin by creating a Packaging (Read) token in Azure DevOps: - -![Personal access tokens](docs/create-pat.png) - -Make sure to check the Packaging (Read) permission, it's the only permission we need for this example. - -![PAT Permissions](docs/create-pat-2.png) - -In this case, we won't enter it into the OS keyring manually. We'll [let Poetry handle it](https://python-poetry.org/docs/repositories/#configuring-credentials). Let's review our `pyproject.toml` file: - -```toml -[tool.poetry] -name = "my-project" -version = "1.0.0" -description = "" -authors = ["My Name "] - -[[tool.poetry.source]] -name = "main" -url = "https://pkgs.dev.azure.com/my-organization/_packaging/main/pypi/simple/" -priority = "default" - -[tool.poetry.dependencies] -python = "~3.9" - -[build-system] -requires = ["poetry>=1.0.0"] -build-backend = "poetry.core.masonry.api" -``` - -Looks like our poetry source is named `main`, so let's run the appropriate command: - -`poetry config http-basic.main ` - -Poetry will create an entry in the OS keyring, and when you run `poetry install` it will automatically authenticate using that credential. No need for `keycmd` here! - -Next, we're going to piggyback off this credential with keycmd, to reuse it for npm, and for docker-compose. That way, we only have 1 credential to manage (that means updating it when it expires). - -Look up the new credential in your OS keyring, and store the following configuration in a `.keycmd` file. Of course, **review your OS keyring and adjust your configuration to match the credential name and username!** - -```toml -[keys] -PAT = { credential = "credential-name", username = "your-username" } -PAT_B64 = { credential = "credential-name", username = "your-username", b64 = true } -``` - -In this example, we are exposing the same credential twice: - -* As the environment variable `PAT` -* Again but with base64 encoding applied as the environment variable `PAT_B64` - -This is important, because npm requires that we supply the token with base64 encoding, but other tools do not. - -For my npm project, I have a [`.npmrc` file](https://docs.npmjs.com/cli/v7/configuring-npm/npmrc) with the following contents: - -``` -registry=https://pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/ -always-auth=true -//pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/:username=dev -//pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/:_password=${PAT_B64} -//pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/:email=email -//pkgs.dev.azure.com/my_organization/_packaging/main/npm/:username=dev -//pkgs.dev.azure.com/my_organization/_packaging/main/npm/:_password=${PAT_B64} -//pkgs.dev.azure.com/my_organization/_packaging/main/npm/:email=email -``` - -Now, I can set up my `node_modules` just by calling `keycmd 'npm install'`! ๐Ÿš€ - -> **Note** -> npm will complain if you make any calls such as `npm run [...]` without the environment variable set. ๐Ÿ™„ You can set them to the empty string to make npm shut up. I use `export PAT_B64=` (or `setx PAT_B64=` on Windows). - -Additionally, I also have a docker-compose file in this project which is configured as follows: - -```yml -secrets: - token: - environment: PAT - token_b64: - environment: PAT_B64 -``` - -When I call `keycmd 'docker compose build'` these two variables are exposed by keycmd and subsequently they are available as [docker compose build secrets](https://docs.docker.com/compose/use-secrets/). ๐Ÿ‘Œ - -## Debugging configuration - -If you're not getting the results you expected, use the `-v` flag -to debug your configuration. Keycmd will verbosely tell you about all the steps it's taking. - -Here's an example using cmd.exe, otherwise, the command would be `poetry run keycmd -v 'echo $ARTIFACTS_TOKEN_B64'`: - -``` -โฏ poetry run keycmd -v echo %ARTIFACTS_TOKEN_B64% -keycmd: loading config file C:\Users\kvang\.keycmd -keycmd: loading config file C:\Users\kvang\dev\keycmd\pyproject.toml -keycmd: merged config: -{'keys': {'ARTIFACTS_TOKEN': {'credential': 'korijn@poetry-repository-main', - 'username': 'korijn'}, - 'ARTIFACTS_TOKEN_B64': {'b64': True, - 'credential': 'korijn@poetry-repository-main', - 'username': 'korijn'}}} -keycmd: keyring backend: (remembered) -keycmd: exposing credential korijn@poetry-repository-main with user korijn as environment variable ARTIFACTS_TOKEN (b64: False, format: None) -keycmd: exposing credential korijn@poetry-repository-main with user korijn as environment variable ARTIFACTS_TOKEN_B64 (b64: True, format: None) -keycmd: detected shell: C:\Windows\System32\cmd.exe -keycmd: running command: ['C:\\Windows\\System32\\cmd.exe', '/C', 'echo', '%ARTIFACTS_TOKEN_B64%'] -aSdtIG5vdCB0aGF0IHN0dXBpZCA6KQ== -``` - -## Note on keyring backends - -Since keycmd uses keyring as its backend, you're not limited to just working with OS keyrings. ๐Ÿคฏ Any keyring backend will work with keycmd. No special configuration required! - -See the [third party backends](https://github.com/jaraco/keyring/#third-party-backends) list for all options. - -### Startup time - -Left to itself, keyring works out which backend to use by loading every backend registered by every installed package and picking the most suitable one. That search runs on each `keycmd` invocation and, on a machine with a few packages installed, costs more time than the whole of the rest of a `keycmd` run put together. - -The answer, though, is the same every time until the packages on your machine change. So keycmd writes it down the first time it needs a credential, and loads that backend by name on every run after, which on the machine this was measured on takes a run from 0.156s to 0.085s. There is nothing to configure and nothing to read; it just gets faster after the first run. - -You can watch it happen with `--verbose`, which says where the backend came from: - -``` -keycmd: keyring backend: keyring.backends.SecretService.Keyring (found in 0.12s) # the first run -keycmd: keyring backend: keyring.backends.SecretService.Keyring (remembered) # every run after -``` - -The note lives with the rest of your cached files โ€” `%LOCALAPPDATA%\keycmd\backend` on Windows, `~/Library/Caches/keycmd/backend` on macOS, and `$XDG_CACHE_HOME/keycmd/backend` (usually `~/.cache`) on Linux โ€” and deleting it costs you nothing but one slow run. - -keycmd only trusts the note as far as it can check it. If the backend it names has been uninstalled, or is no longer usable because the daemon behind it is not running, the run searches again and writes down what it finds instead. What it cannot notice by itself is a backend that still loads but is no longer the one you want โ€” you installed a better one, or removed a package and want the runner-up. That is what these two are for: - -```bash -keycmd --detect-backend # search now, and remember what turns up -keycmd --reset-backend # forget it, so the next run searches again -``` - -``` -โฏ keycmd --detect-backend -keycmd: remembered keyring backend keyring.backends.SecretService.Keyring, found in 0.12s -``` +The variable exists inside that command, and nowhere else โ€” no `.env` file, no secret pasted into your terminal, nothing left behind afterwards. ๐Ÿ˜ฑ โ†’ ๐Ÿ˜Œ -If you would rather take the whole thing into your own hands, keyring's own `PYTHON_KEYRING_BACKEND` still works and outranks anything keycmd remembers: +Continue with the [Quick Start tutorial](https://korijn.github.io/keycmd/getting-started/quick-start/), which walks through storing the credential on each platform. -```bash -# in your shell profile; use the backend your platform actually uses -export PYTHON_KEYRING_BACKEND=keyring.backends.SecretService.Keyring -``` +## Why keycmd? -`keyring --list-backends` prints the names to choose from. The setting is keyring's own, so it applies to everything else using keyring too, and with it set keycmd has nothing to remember and says so if you ask it to. +* **Your secrets stay in the keyring.** The Windows Credential Manager, the macOS keychain and the Linux secret service already exist to keep credentials safe โ€” keycmd reads from them, so a checked-out repository never has to contain a token. +* **Exposed for one command only**, or for a subshell with `keycmd --shell` when you're debugging. +* **Configuration that follows your project**, merged from your home folder, from `.keycmd` files up the directory tree, and from `pyproject.toml`. +* **One credential, many shapes.** Format strings and aliases expose the same secret as plain text, base64, or a basic auth header โ€” whatever each tool insists on. +* **Any keyring backend**, through [keyring](https://github.com/jaraco/keyring), with no special configuration. -### No backend at all +## Documentation -If keyring finds no backend it can use, there is nowhere for keycmd to read credentials from, and it says so rather than failing on the first lookup: - -``` -โฏ keycmd 'npm install' -keycmd: error: keyring has no backend to read credentials from -keycmd: hint: install one for this platform, or name one you have with PYTHON_KEYRING_BACKEND -keycmd: hint: see https://github.com/jaraco/keyring#third-party-backends -``` +Everything lives at **[korijn.github.io/keycmd](https://korijn.github.io/keycmd)**: -Inside a WSL distribution this usually means the distro's keyring daemon is not running, which is what the [WSL installation](#wsl-installation) instructions above are for; keycmd points you there when it notices it is running in one. Nothing is written down in this case, so there is nothing to reset once you have fixed it. +* [Installation](https://korijn.github.io/keycmd/getting-started/installation/) โ€” globally, under pyenv, or from WSL +* [Running commands](https://korijn.github.io/keycmd/guide/running-commands/) โ€” the two invocation forms, quoting, subshells +* [Configuration](https://korijn.github.io/keycmd/guide/configuration/) โ€” where it lives, keys, format strings, aliases +* [Keyring backends](https://korijn.github.io/keycmd/guide/keyring-backends/) โ€” third party backends, and keycmd's startup time +* [WSL](https://korijn.github.io/keycmd/guide/wsl/) โ€” reaching the Windows Credential Manager from a distribution +* [Troubleshooting](https://korijn.github.io/keycmd/guide/troubleshooting/) โ€” start with `keycmd --verbose` +* [Examples](https://korijn.github.io/keycmd/examples/openai/) โ€” an OpenAI API key, and one Azure DevOps token shared by poetry, npm and docker compose +* [Reference](https://korijn.github.io/keycmd/reference/cli/) โ€” every flag, environment variable and configuration field -## Development +## Contributing -This project uses [uv](https://docs.astral.sh/uv/) for dependency management, [ruff](https://docs.astral.sh/ruff/) for linting and formatting, and [ty](https://docs.astral.sh/ty/) for type checking. - -The `keycmd` package is fully annotated and ships a `py.typed` marker, so the types are available to anything that imports it. Ruff's `ANN` rules keep it that way; the test suite is exempt. +Issues and pull requests are welcome. See [Contributing](https://korijn.github.io/keycmd/development/contributing/) and [Testing](https://korijn.github.io/keycmd/development/testing/) to get set up: ```bash -# create the virtual environment and install all dependencies uv sync - -# install the git hooks that run the checks below on every commit uv run pre-commit install - -# lint, format, typecheck and test -uv run ruff check --fix -uv run ruff format -uv run ty check uv run pytest tests ``` -### Testing - -CI runs the test suite on Windows, macOS and Linux on the latest Python, plus one job on the oldest supported Python to catch anything newer than it allows. The suite adapts to the platform it runs on: it exercises every shell of the platform that is installed (`sh`, `bash` and `zsh` on posix, `cmd` and `powershell` on Windows, and `pwsh` on either, since it installs everywhere and quotes its own way), and it skips the process replacement tests on Windows, which has no `execvpe`. - -The tests that read and write credentials need a real OS keyring that can be unlocked without user interaction. They are skipped with a message if there is no such keyring, so the rest of the suite still runs. Set `KEYCMD_REQUIRE_OS_KEYRING=1` to turn those skips into failures instead; CI sets it so that a broken keyring setup can't quietly reduce the coverage of a run. - -* **Windows**: the credential manager is available to your session out of the box, no setup needed. -* **macOS**: your login keychain works as long as it is unlocked. CI instead creates a throwaway keychain and makes it the default: +## License - ```bash - security create-keychain -p keycmd-test keycmd-test.keychain - security set-keychain-settings keycmd-test.keychain - security unlock-keychain -p keycmd-test keycmd-test.keychain - security list-keychains -d user -s keycmd-test.keychain login.keychain - security default-keychain -s keycmd-test.keychain - ``` - -* **Linux**: the secret service is bound to a d-bus session, so the tests have to run inside one, with an unlocked keyring daemon (install `gnome-keyring` and `dbus-x11` first): - - ```bash - dbus-run-session -- bash -c ' - printf "%s" keycmd-test | gnome-keyring-daemon --unlock --components=secrets - uv run pytest tests - ' - ``` - -### Testing WSL - -The [WSL setup](#wsl-installation) has two halves. Working *inside* WSL, keycmd is a posix process like any other, talking to whichever keyring backend the distro provides; that is the Linux job above, keyring daemon and all. The other half, calling the Windows install of keycmd from a WSL shell to reach the Windows credential manager, crosses the interop boundary, and that is what `tests/test_wsl.py` covers: a credential in the credential manager, a shell inside WSL, and the Windows install of keycmd in between. - -Everything about that boundary that can be decided without a Windows machine is in `tests/test_wsl_interop.py` instead, and runs everywhere: which process tree and working directory mean keycmd was called from a distro, the command lines it builds for `wsl.exe`, and the `WSLENV` that carries the credentials across. - -The end to end tests are opt in, because installing WSL takes a CI job of its own. On a Windows machine that has WSL installed: - -```powershell -$env:KEYCMD_TEST_WSL = 1 -uv run pytest tests/test_wsl.py -``` - -If you would rather not involve your OS keyring at all, point keyring at a file-based backend: - -```bash -uv run --with keyrings.alt pytest tests -# with PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring set in your environment -``` +[MIT](LICENSE) diff --git a/docs/create-pat-2.png b/docs/assets/create-pat-2.png similarity index 100% rename from docs/create-pat-2.png rename to docs/assets/create-pat-2.png diff --git a/docs/create-pat.png b/docs/assets/create-pat.png similarity index 100% rename from docs/create-pat.png rename to docs/assets/create-pat.png diff --git a/docs/keychain-access-new-password-item.png b/docs/assets/keychain-access-new-password-item.png similarity index 100% rename from docs/keychain-access-new-password-item.png rename to docs/assets/keychain-access-new-password-item.png diff --git a/docs/seahorse.png b/docs/assets/seahorse.png similarity index 100% rename from docs/seahorse.png rename to docs/assets/seahorse.png diff --git a/docs/wcm-add-key.png b/docs/assets/wcm-add-key.png similarity index 100% rename from docs/wcm-add-key.png rename to docs/assets/wcm-add-key.png diff --git a/docs/wcm.png b/docs/assets/wcm.png similarity index 100% rename from docs/wcm.png rename to docs/assets/wcm.png diff --git a/docs/development/contributing.md b/docs/development/contributing.md new file mode 100644 index 0000000..2d6dd1f --- /dev/null +++ b/docs/development/contributing.md @@ -0,0 +1,59 @@ +# Contributing + +keycmd is developed on [GitHub](https://github.com/Korijn/keycmd). Issues and pull requests are welcome. + +This project uses [uv](https://docs.astral.sh/uv/) for dependency management, [ruff](https://docs.astral.sh/ruff/) for linting and formatting, and [ty](https://docs.astral.sh/ty/) for type checking. + +## Getting set up + +```bash +# create the virtual environment and install all dependencies +uv sync + +# install the git hooks that run the checks below on every commit +uv run pre-commit install +``` + +## The checks + +```bash +uv run ruff check --fix +uv run ruff format +uv run ty check +uv run pytest tests +``` + +CI runs all four, so the pre-commit hooks are the cheapest place to find out about them. + +## Conventions + +The `keycmd` package is fully annotated and ships a [PEP 561](https://peps.python.org/pep-0561/) `py.typed` marker, so the types are available to anything that imports it. Ruff's `ANN` rules keep it that way; the test suite is exempt. `ty` is configured for the oldest supported Python, so it catches typing features that are newer than `requires-python` allows. + +### Startup time is a feature + +keycmd sits in front of every command a user runs through it, so its own startup is latency the user pays each time. A handful of imports cost more than everything else in the package together, and none of them is needed on every run: + +* `keyring` (and the backend it goes on to discover) โ€” only once a credential is looked up +* `pprint` โ€” only under `--verbose` +* `subprocess` โ€” only on the Windows path, which cannot replace its own process + +Those are imported inside the function that needs them, which keeps `import keycmd.cli` at roughly a third of what it would otherwise cost, and there is a test that fails if one of them wanders back up to module level. Reach for a lazy import when adding a dependency that most runs will not touch, and leave the rest at the top of the file where they belong. + +## The documentation + +This site is built with [MkDocs](https://www.mkdocs.org/) and [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/), from the markdown files under `docs/`. + +```bash +# install the docs dependencies +uv sync --group docs + +# serve the site locally, with live reload, on http://127.0.0.1:8000 +uv run mkdocs serve + +# build it into site/ +uv run mkdocs build +``` + +Every push to `master` builds the site and deploys it to GitHub Pages. + +The nav lives in `mkdocs.yml`; a new page has to be added there to show up. The README is deliberately short and points here, so new prose belongs on this site rather than in the README. diff --git a/docs/development/testing.md b/docs/development/testing.md new file mode 100644 index 0000000..e1d97fe --- /dev/null +++ b/docs/development/testing.md @@ -0,0 +1,75 @@ +# Testing + +```bash +uv run pytest tests + +uv run pytest tests/test_conf.py::test_load_conf # a single test +uv run pytest -k "run_cmd and bash" # a single shell's parameters +``` + +CI runs the test suite on Windows, macOS and Linux on the latest Python, plus one job on the oldest supported Python to catch anything newer than it allows. + +The suite adapts to the platform it runs on. It exercises every shell of the platform that is installed โ€” `sh`, `bash` and `zsh` on posix, `cmd` and `powershell` on Windows, and `pwsh` on either, since it installs everywhere and quotes its own way โ€” and it skips the process replacement tests on Windows, which has no `execvpe`. + +## A keyring to test against + +The tests that read and write credentials need a real OS keyring that can be unlocked without user interaction. They are skipped with a message if there is no such keyring, so the rest of the suite still runs. Set `KEYCMD_REQUIRE_OS_KEYRING=1` to turn those skips into failures instead; CI sets it, so that a broken keyring setup can't quietly reduce the coverage of a run. + +=== "Windows" + + The credential manager is available to your session out of the box, no setup needed. + +=== "macOS" + + Your login keychain works as long as it is unlocked. CI instead creates a throwaway keychain and makes it the default: + + ```bash + security create-keychain -p keycmd-test keycmd-test.keychain + security set-keychain-settings keycmd-test.keychain + security unlock-keychain -p keycmd-test keycmd-test.keychain + security list-keychains -d user -s keycmd-test.keychain login.keychain + security default-keychain -s keycmd-test.keychain + ``` + +=== "Linux" + + The secret service is bound to a d-bus session, so the tests have to run inside one, with an unlocked keyring daemon. Install `gnome-keyring` and `dbus-x11` first: + + ```bash + dbus-run-session -- bash -c ' + printf "%s" keycmd-test | gnome-keyring-daemon --unlock --components=secrets + uv run pytest tests + ' + ``` + +If you would rather not involve your OS keyring at all, point keyring at a file-based backend: + +```bash +uv run --with keyrings.alt pytest tests +# with PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring set in your environment +``` + +## Testing WSL + +The [WSL setup](../guide/wsl.md) has two halves. + +Working *inside* WSL, keycmd is a posix process like any other, talking to whichever keyring backend the distribution provides; that is the Linux job above, keyring daemon and all. + +The other half โ€” calling the Windows install of keycmd from a WSL shell to reach the Windows credential manager โ€” crosses the interop boundary, and that is what `tests/test_wsl.py` covers: a credential in the credential manager, a shell inside WSL, and the Windows install of keycmd in between. + +Everything about that boundary that can be decided without a Windows machine is in `tests/test_wsl_interop.py` instead, and runs everywhere: which process tree and working directory mean keycmd was called from a distribution, the command lines it builds for `wsl.exe`, and the `WSLENV` that carries the credentials across. + +The end to end tests are opt in, because installing WSL takes a CI job of its own. On a Windows machine that has WSL installed: + +```powershell +$env:KEYCMD_TEST_WSL = 1 +uv run pytest tests/test_wsl.py +``` + +## Things that bite in this suite + +* **Never assume a shell.** The `shell` fixture parametrizes over every shell of the platform that is installed, so a test using it runs several times. Ask the `Shell` object for the dialect (`env_var`, `unset_env_var`, `command_not_found_statuses`) instead of branching on the platform. +* **`wsl.exe` mangles its command line**: backslashes disappear and quotes are stripped before the distribution sees them. Pass paths translated to `/mnt/...` by `wsl_path`, unquoted and free of spaces, and keep remote scripts on one line. +* **The remembered backend is redirected, always.** An autouse fixture points `backend.CACHE_HOME` at a folder under `tmp_path`, so a test run neither reads nor writes the note the machine it runs on is using. +* **Do not assume the suite runs unpinned.** `PYTHON_KEYRING_BACKEND` outranks everything `backend.py` does, so a test about remembering has to `delenv` it first, or it will be testing the path that deliberately remembers nothing. +* **Warnings are errors**, so a deprecation in a new Python release fails the suite rather than scrolling past. diff --git a/docs/examples/azure-artifacts.md b/docs/examples/azure-artifacts.md new file mode 100644 index 0000000..0c15754 --- /dev/null +++ b/docs/examples/azure-artifacts.md @@ -0,0 +1,119 @@ +# Azure Artifacts + +This is an example configuration for Poetry, npm and docker compose. It should inspire you to see the possibilities keycmd provides thanks to its configuration system. + +In this case, we are authenticating with an Azure DevOps Personal Access Token to an Azure Artifacts Feed which serves both Python and Node.js packages โ€” with **one** credential in the keyring, shared by three tools that each want it in a different form. + +## Creating the token + +Let's begin by creating a Packaging (Read) token in Azure DevOps: + +![Personal access tokens](../assets/create-pat.png) + +Make sure to check the Packaging (Read) permission, it's the only permission we need for this example. + +![PAT Permissions](../assets/create-pat-2.png) + +## Letting Poetry store it + +In this case, we won't enter the token into the OS keyring manually. We'll [let Poetry handle it](https://python-poetry.org/docs/repositories/#configuring-credentials). Let's review our `pyproject.toml` file: + +```toml +[tool.poetry] +name = "my-project" +version = "1.0.0" +description = "" +authors = ["My Name "] + +[[tool.poetry.source]] +name = "main" +url = "https://pkgs.dev.azure.com/my-organization/_packaging/main/pypi/simple/" +priority = "default" + +[tool.poetry.dependencies] +python = "~3.9" + +[build-system] +requires = ["poetry>=1.0.0"] +build-backend = "poetry.core.masonry.api" +``` + +Looks like our poetry source is named `main`, so let's run the appropriate command: + +```bash +poetry config http-basic.main +``` + +Poetry creates an entry in the OS keyring, and when you run `poetry install` it automatically authenticates using that credential. No need for `keycmd` here! + +## Piggybacking with keycmd + +Next, we're going to piggyback off this credential with keycmd, to reuse it for npm and for docker compose. That way, we only have one credential to manage โ€” which means one credential to update when it expires. + +Look up the new credential in your OS keyring, and store the following configuration in a `.keycmd` file. Of course, **review your OS keyring and adjust your configuration to match the credential name and username!** + +```toml +[keys] +PAT = { credential = "credential-name", username = "your-username" } +PAT_B64 = { credential = "credential-name", username = "your-username", b64 = true } +``` + +We are exposing the same credential twice: + +* as the environment variable `PAT` +* again, with base64 encoding applied, as the environment variable `PAT_B64` + +This is important, because npm requires that we supply the token with base64 encoding, but other tools do not. + +!!! tip "Or use an alias" + + The two entries above look up the same credential twice. An [alias](../guide/configuration.md#aliases) expresses the same thing with a single keyring lookup: + + ```toml + [keys] + PAT = { credential = "credential-name", username = "your-username" } + + [aliases] + PAT_B64 = { key = "PAT", b64 = true } + ``` + +## npm + +For my npm project, I have an [`.npmrc` file](https://docs.npmjs.com/cli/v7/configuring-npm/npmrc) with the following contents: + +```ini +registry=https://pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/ +always-auth=true +//pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/:username=dev +//pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/:_password=${PAT_B64} +//pkgs.dev.azure.com/my_organization/_packaging/main/npm/registry/:email=email +//pkgs.dev.azure.com/my_organization/_packaging/main/npm/:username=dev +//pkgs.dev.azure.com/my_organization/_packaging/main/npm/:_password=${PAT_B64} +//pkgs.dev.azure.com/my_organization/_packaging/main/npm/:email=email +``` + +Now I can set up my `node_modules` just by calling: + +```bash +keycmd 'npm install' +``` + +๐Ÿš€ + +!!! note + + npm will complain if you make any calls such as `npm run [...]` without the environment variable set. ๐Ÿ™„ You can set them to the empty string to make npm shut up. I use `export PAT_B64=` (or `setx PAT_B64=` on Windows). + +## docker compose + +Additionally, I also have a docker compose file in this project which is configured as follows: + +```yaml +secrets: + token: + environment: PAT + token_b64: + environment: PAT_B64 +``` + +When I call `keycmd 'docker compose build'` these two variables are exposed by keycmd and subsequently they are available as [docker compose build secrets](https://docs.docker.com/compose/use-secrets/). ๐Ÿ‘Œ diff --git a/docs/examples/openai.md b/docs/examples/openai.md new file mode 100644 index 0000000..1b5265a --- /dev/null +++ b/docs/examples/openai.md @@ -0,0 +1,36 @@ +# OpenAI API Key + +With OpenAI, you're instructed to [use an API key](https://github.com/openai/openai-python#usage) to authenticate with their APIs. When you put that string in a `.env` file, or directly in your code, you risk sharing your API key with the world! ๐Ÿ™…โ€โ™‚๏ธ + +Instead, just put it in your OS keyring, and expose it with keycmd when you run your Python scripts or Jupyter notebooks. + +## Configuration + +Add the key to your OS keyring โ€” see the [quick start](../getting-started/quick-start.md#1-store-a-credential-in-your-keyring) for how to do that on each platform. Say you store it under the name `my-openai-token` with username `your-username`; then this is the `.keycmd` configuration you need: + +```toml +[keys] +OPENAI_API_KEY = { credential = "my-openai-token", username = "your-username" } +``` + +`OPENAI_API_KEY` is the variable the OpenAI SDK reads by default, so nothing in your code has to change. + +## Usage + +Now you can run any OpenAI script by just prefixing your command with `keycmd`: + +```bash +keycmd 'python my_openai_script.py' +``` + +Or a Jupyter notebook: + +```bash +keycmd 'jupyter notebook' +``` + +That's all! ๐Ÿค˜ Now you can rest easily, knowing your tokens are safe. ๐Ÿ›Œ๐Ÿ’ค + +!!! tip "The same trick works for any SDK" + + Anthropic's `ANTHROPIC_API_KEY`, AWS's `AWS_SECRET_ACCESS_KEY`, a Hugging Face `HF_TOKEN` โ€” anything that reads a credential from the environment is a one-line entry under `[keys]`. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 0000000..1ab1c58 --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,107 @@ +# Installation + +keycmd is a command line tool, so it wants to be installed **globally**, once per machine โ€” not into the virtual environment of each project you use it on. It requires Python 3.13 or newer. + +!!! note "Using WSL or pyenv?" + + Both need a slightly different approach. Skip ahead to [pyenv installation](#pyenv-installation), or to the [WSL guide](../guide/wsl.md), which explains why WSL users install keycmd on the Windows side. + +## Global installation + +The recommended way to install a Python command line tool is with [uv](https://docs.astral.sh/uv/): + +=== "uv" + + ```bash + uv tool install keycmd + ``` + +=== "pipx" + + ```bash + pipx install keycmd + ``` + +=== "pip" + + ```bash + pip install keycmd + ``` + +`uv tool install` and `pipx` both put keycmd in an isolated environment of its own and put the executable on your `PATH`, which is exactly what you want for a tool that is not a dependency of anything. + +Whichever you use, the `keycmd` executable has to end up in a folder that is on your `PATH`, or the command won't be available globally. If you were able to run `uv` or `pip` just now, the executable should land in the same place they did, and everything should be fine. + +To verify keycmd is installed and available: + +```bash +keycmd --version +``` + +Continue with the [Quick Start](quick-start.md). + +## pyenv installation + +If you're using pyenv, you're going to have to jump through a few hoops, since keycmd needs to be installed globally, which flies directly into the face of what pyenv is trying to accomplish. + +This guide assumes you've also installed [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv), in order to get you the cleanest of setups. โœจ + +!!! note + + These instructions are for pyenv on Linux and macOS. If you are using pyenv-win on Windows, they are most likely not 100% compatible with your setup. + +Run the following commands one by one to install keycmd into its own standalone environment: + +```bash +pyenv virtualenv 3.13 keycmd +pyenv activate keycmd +pip install keycmd +pathToKeycmd=$(python -c 'import sys; from pathlib import Path; print(Path(sys.executable).parent / "keycmd")') +pyenv deactivate +mkdir -p $HOME/.local/bin +ln -s $pathToKeycmd $HOME/.local/bin/keycmd +``` + +Finally, edit your `~/.bashrc` file (or whatever shell profile you use) to include `~/.local/bin` in your `PATH`: + +```bash +export PATH="$HOME/.local/bin:$PATH" +``` + +!!! note + + This line may already be in place in your `~/.bashrc` โ€” for example, if you installed poetry. It's a common trick used to expose specific binaries on `PATH` when they live in folders that also contain binaries that should *not* be exposed. + +To verify keycmd is installed and available, run `keycmd --version`. + +## Up- and downgrading + +Install a different version the same way you installed the first one: + +=== "uv" + + ```bash + # upgrade to the latest release + uv tool upgrade keycmd + + # install a specific version + uv tool install keycmd==0.6.0 + ``` + +=== "pipx" + + ```bash + pipx upgrade keycmd + pipx install --force keycmd==0.6.0 + ``` + +=== "pip" + + ```bash + pip install -U keycmd + pip install keycmd==0.6.0 + ``` + +!!! note "pyenv" + + Activate the virtual environment first with `pyenv activate keycmd`, run `pip install -U keycmd`, and don't forget to `pyenv deactivate` afterwards. The symlink in `~/.local/bin` keeps working. diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md new file mode 100644 index 0000000..4324f09 --- /dev/null +++ b/docs/getting-started/quick-start.md @@ -0,0 +1,88 @@ +# Quick Start + +This tutorial takes a few minutes and covers everything keycmd does: you store a credential in your OS keyring, name it in a configuration file, and run a command that can read it. + +For the purpose of this example, use `my-secret` as the credential name and `my-username` as the... username. We'll use `foobar` as the password. + +## 1. Store a credential in your keyring + +How you add a credential depends on your platform. + +=== "Windows" + + Click Start and type "Credential Manager" to find the app. Open the **Windows Credentials** tab, and click **Add a generic credential**. + + ![Credential Manager](../assets/wcm.png) + + ![Add Key](../assets/wcm-add-key.png) + +=== "macOS" + + Open Keychain Access (++cmd+space++, type `keychain access`), then press ++cmd+n++ to add a new password item. Note that **Account Name** holds the username. + + ![Add New Password Item](../assets/keychain-access-new-password-item.png) + +=== "Linux" + + The easiest way to add a credential on Linux is through Python's `keyring` package. Adding a password through the GUI does not always let you set a username, which is a problem for keycmd's internal `keyring.get_password()` call: + + ```python + >>> import keyring + >>> keyring.set_password("my-secret", "my-username", "foobar") + ``` + + After that it shows up in your credential manager. On Ubuntu that's `seahorse`, which should now show the new password: + + ![Seahorse](../assets/seahorse.png) + + This approach generalizes to other distributions โ€” and to other operating systems, for that matter. + +## 2. Write a configuration file + +Create a `.keycmd` file in your user home folder, with the following contents: + +```toml +[keys] +SECRET = { credential = "my-secret", username = "my-username" } +``` + +This says: look up the credential `my-secret` for user `my-username`, and expose the password it holds as the environment variable `SECRET`. + +## 3. Run a command + +Open a terminal and run a command that prints the secret. That looks different depending on the shell you use: + +=== "bash / zsh" + + ```bash + keycmd 'echo $SECRET' + ``` + +=== "PowerShell" + + ```powershell + keycmd 'echo $env:SECRET' + ``` + +=== "cmd" + + ```bat + keycmd echo %SECRET% + ``` + +You should see the text `foobar` printed to your terminal. + +You've successfully set up keycmd! ๐Ÿ‘ + +## What just happened + +keycmd read your configuration, looked `my-secret` up in your OS keyring, put the password in the environment as `SECRET`, and handed that environment to your shell along with your command. When the command finished, the variable went with it: your own shell never had it. + +Note the quotes in the bash and PowerShell examples. Quoting the whole command as one argument is what lets *your command's shell* expand `$SECRET`, rather than your own shell expanding it before keycmd ever sees it. See [Running commands](../guide/running-commands.md) for the details. + +## Where to go next + +* [Running commands](../guide/running-commands.md) โ€” the two ways to invoke keycmd, quoting, and subshells. +* [Configuration](../guide/configuration.md) โ€” where configuration lives, and everything you can put in it. +* [Examples](../examples/openai.md) โ€” an OpenAI API key, and a real world setup where poetry, npm and docker compose share a single Azure DevOps token. +* [Troubleshooting](../guide/troubleshooting.md) โ€” if any of the above did not go as planned, `keycmd --verbose` will tell you why. diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md new file mode 100644 index 0000000..7ab15b0 --- /dev/null +++ b/docs/guide/configuration.md @@ -0,0 +1,97 @@ +# Configuration + +keycmd is configured with TOML: a `[keys]` table naming the credentials to expose, and an optional `[aliases]` table exposing them again in another shape. + +```toml +[keys] +SECRET = { credential = "my-secret", username = "my-username" } +``` + +For the complete schema, see the [configuration reference](../reference/configuration.md). If a configuration is not doing what you expect, [`keycmd --verbose`](troubleshooting.md) prints every file it loaded and the result of merging them. + +## Where configuration lives + +Configuration can be stored in three places, where `~` is your home folder and `.` is the working directory you call `keycmd` from: + +1. `~/.keycmd` +2. all `.keycmd` files found while walking up the file system from `.` +3. the first `pyproject.toml` found while walking up the file system from `.` + +They are loaded and merged in that order, and merged deeply: later sources win per field, so a project can override a single option without restating the whole entry. + +!!! note "The search stays inside your project" + + The search for `.keycmd` and `pyproject.toml` stops at the root of a git repository, and before your home folder, so that configuration applies to a subtree of your file system rather than leaking across projects. + +This is what makes keycmd convenient for teams: a `.keycmd` file committed to a repository names the credentials the project needs, and each developer only has to have those credentials in their own keyring. + +## Keys + +Every entry under `[keys]` becomes one environment variable, named after the entry: + +```toml +[keys] +MY_TOKEN = { credential = "azure_secret", username = "azure" } +``` + +`credential` and `username` together identify the credential in your keyring โ€” they are exactly what is handed to `keyring.get_password()`. The password it returns becomes the value of `MY_TOKEN`. + +Two optional fields change the value before it is exposed: + +* `format` โ€” a format string, applied first +* `b64` โ€” base64 encoding, applied second + +## Format strings + +A format string lets you preprocess the credential before it is exposed as an environment variable. It is processed with Python's built-in [`str.format`](https://docs.python.org/3/library/stdtypes.html#str.format), so everything that function supports is available to you. + +Three variables can be used in the format string: + +* `credential` +* `username` +* `password` + +So a basic auth header, for example, is a format string and a base64 flag: + +```toml +[keys] +MY_TOKEN = { credential = "MY_TOKEN", username = "azure", format = "{username}:{password}", b64 = true } +``` + +`format` is applied before `b64`, which is what makes that combination produce the value a basic auth header wants. + +## Aliases + +Aliases expose the same secret in several forms, without a second lookup in your keyring. + +For example, you may have a single Personal Access Token for Azure DevOps, and want to use the same token for `pip`, `npm` and the REST API. `pip` wants the token in plain text, `npm` prefers it base64-encoded, and the REST API expects a basic auth header. Aliases make this easy: + +```toml +[keys] +MY_TOKEN = { credential = "azure_secret", username = "azure" } + +[aliases] +MY_TOKEN_B64 = { key = "MY_TOKEN", b64 = true } +MY_TOKEN_BASICAUTH = { key = "MY_TOKEN", format = "{username}:{password}", b64 = true } +``` + +An alias names an existing key with `key`, and takes the same `b64` and `format` options as a key does. The keyring is only consulted once, for `MY_TOKEN`; the aliases are derived from what it returned. + +## pyproject.toml + +Configuration can equally well live in a project's `pyproject.toml`, under the `tool.keycmd` table. The previous example becomes: + +```toml +[tool.keycmd.keys] +MY_TOKEN = { credential = "azure_secret", username = "azure" } + +[tool.keycmd.aliases] +MY_TOKEN_B64 = { key = "MY_TOKEN", b64 = true } +MY_TOKEN_BASICAUTH = { key = "MY_TOKEN", format = "{username}:{password}", b64 = true } +``` + +Only the first `pyproject.toml` found on the way up is used, and it is merged last, so it wins over the `.keycmd` files below it. + +!!! warning "Configuration holds names, not secrets" + + Nothing you write in a configuration file is a secret: it names a credential and a user, and the password itself stays in your keyring. That is what makes these files safe to commit โ€” and it is worth keeping it that way. diff --git a/docs/guide/keyring-backends.md b/docs/guide/keyring-backends.md new file mode 100644 index 0000000..3ee3daa --- /dev/null +++ b/docs/guide/keyring-backends.md @@ -0,0 +1,75 @@ +# Keyring Backends + +keycmd reads credentials through [keyring](https://github.com/jaraco/keyring), so you're not limited to just the OS keyrings. ๐Ÿคฏ Any keyring backend works with keycmd, and no special configuration is required. + +See keyring's [third party backends](https://github.com/jaraco/keyring/#third-party-backends) list for all the options. + +## Startup time + +keycmd sits in front of every command you run through it, so its own startup is latency you pay each time. + +Left to itself, keyring works out which backend to use by loading every backend registered by every installed package and picking the most suitable one. That search runs on each `keycmd` invocation and, on a machine with a few packages installed, costs more time than the whole of the rest of a `keycmd` run put together. + +The answer, though, is the same every time until the packages on your machine change. So keycmd writes it down the first time it needs a credential, and loads that backend by name on every run after, which on the machine this was measured on takes a run from 0.156s to 0.085s. There is nothing to configure and nothing to read; it just gets faster after the first run. + +You can watch it happen with `--verbose`, which says where the backend came from: + +``` +keycmd: keyring backend: keyring.backends.SecretService.Keyring (found in 0.12s) # the first run +keycmd: keyring backend: keyring.backends.SecretService.Keyring (remembered) # every run after +``` + +The note lives with the rest of your cached files, and deleting it costs you nothing but one slow run: + +| Platform | Location | +| --- | --- | +| Windows | `%LOCALAPPDATA%\keycmd\backend` | +| macOS | `~/Library/Caches/keycmd/backend` | +| Linux | `$XDG_CACHE_HOME/keycmd/backend` (usually `~/.cache`) | + +## When the note goes stale + +keycmd only trusts the note as far as it can check it. If the backend it names has been uninstalled, or is no longer usable because the daemon behind it is not running, the run searches again and writes down what it finds instead. + +What keycmd cannot notice by itself is a backend that still loads but is no longer the one you want โ€” you installed a better one, or removed a package and want the runner-up. That is what these two flags are for: + +```bash +keycmd --detect-backend # search now, and remember what turns up +keycmd --reset-backend # forget it, so the next run searches again +``` + +``` +โฏ keycmd --detect-backend +keycmd: remembered keyring backend keyring.backends.SecretService.Keyring, found in 0.12s +``` + +Neither flag needs a configuration or a command: they are about the keyring itself, and return before keycmd looks at anything else. + +## Choosing the backend yourself + +If you would rather take the whole thing into your own hands, keyring's own `PYTHON_KEYRING_BACKEND` still works, and outranks anything keycmd remembers: + +```bash +# in your shell profile; use the backend your platform actually uses +export PYTHON_KEYRING_BACKEND=keyring.backends.SecretService.Keyring +``` + +`keyring --list-backends` prints the names to choose from. The setting is keyring's own, so it applies to everything else using keyring too, and with it set keycmd has nothing to remember โ€” and says so if you ask it to: + +``` +โฏ keycmd --detect-backend +keycmd: PYTHON_KEYRING_BACKEND=keyring.backends.SecretService.Keyring already names the backend to use +``` + +## No backend at all + +If keyring finds no backend it can use, there is nowhere for keycmd to read credentials from, and it says so rather than failing on the first lookup: + +``` +โฏ keycmd 'npm install' +keycmd: error: keyring has no backend to read credentials from +keycmd: hint: install one for this platform, or name one you have with PYTHON_KEYRING_BACKEND +keycmd: hint: see https://github.com/jaraco/keyring#third-party-backends +``` + +Inside a WSL distribution this usually means the distro's keyring daemon is not running, which is what the [WSL guide](wsl.md) is for; keycmd points you there when it notices it is running in one. Nothing is written down in this case, so there is nothing to reset once you have fixed it. diff --git a/docs/guide/running-commands.md b/docs/guide/running-commands.md new file mode 100644 index 0000000..58c109e --- /dev/null +++ b/docs/guide/running-commands.md @@ -0,0 +1,66 @@ +# Running Commands + +There are two ways to use keycmd: + +```bash +keycmd 'your command' # run one command with the credentials exposed +keycmd --shell # open a subshell with the credentials exposed +``` + +The first is the preferred one, since your secrets are only exposed as environment variables for the duration of a single command. The second is less preferable, but can be convenient when you are debugging a process that depends on the credentials you are exposing. + +## One command + +In its most common form, keycmd takes the command to run as a single quoted argument: + +```bash +keycmd 'npm install' +``` + +Quoting the whole command as one argument is what lets you use your shell's syntax inside it: + +```bash +keycmd 'echo $SECRET | tr a-z A-Z' +``` + +keycmd hands that line to your shell exactly as you typed it, and your shell does the rest โ€” pipes, redirects, variable expansion and all. This matters especially for the credentials themselves: `$SECRET` has to be expanded by the shell keycmd starts, because that is the only shell the variable exists in. + +## Separate arguments + +You can also write the command out as separate arguments, and then keycmd keeps them separate: + +```bash +# arrives as a single argument, spaces and all +keycmd mytool --message 'hello world' +``` + +Since each argument is passed on as the word it was, your shell's syntax is *not* interpreted a second time in this form. If you want `$SECRET` expanded, either let your own shell expand it, or use the single argument form above. + +!!! tip "Which form should I use?" + + Use the quoted form for anything that needs a shell: pipes, `&&`, redirects, globs, and above all the credentials you came here for. Use separate arguments when you are passing along text that must survive untouched, such as an argument that itself contains `$` or quotes. + +## A subshell + +`keycmd --shell` starts an interactive subshell with the same environment: + +```bash +keycmd --shell +``` + +Every command you run in it has the credentials available, until you exit it. Keep in mind that this means the secrets are in the environment of a long-lived process, and of everything you start from it โ€” which is precisely what the one-off form avoids. + +## Which shell keycmd uses + +keycmd asks [shellingham](https://github.com/sarugaku/shellingham) which shell invoked it, and falls back on `$SHELL` on posix or `%COMSPEC%` on Windows if that fails. In other words, it runs your command in the shell you were already using. + +On posix, keycmd replaces its own process with the shell (`execvpe`), so it does not sit in the process tree waiting around. Windows has no equivalent, so there keycmd runs the shell as a subprocess and passes its exit code along. + +`--verbose` reports what it decided: + +``` +keycmd: detected shell: C:\Windows\System32\cmd.exe +keycmd: running command: ['C:\\Windows\\System32\\cmd.exe', '/C', 'echo', '%ARTIFACTS_TOKEN_B64%'] +``` + +Under WSL the answer is not a shell at all but `wsl.exe`, which is [its own topic](wsl.md). diff --git a/docs/guide/troubleshooting.md b/docs/guide/troubleshooting.md new file mode 100644 index 0000000..810a6c3 --- /dev/null +++ b/docs/guide/troubleshooting.md @@ -0,0 +1,75 @@ +# Troubleshooting + +If you're not getting the results you expected, use the `-v` (`--verbose`) flag. keycmd will verbosely tell you about all the steps it's taking, and that is almost always enough to see what went wrong. + +## Debugging your configuration + +``` +โฏ keycmd -v echo %ARTIFACTS_TOKEN_B64% +keycmd: loading config file C:\Users\kvang\.keycmd +keycmd: loading config file C:\Users\kvang\dev\keycmd\pyproject.toml +keycmd: merged config: +{'keys': {'ARTIFACTS_TOKEN': {'credential': 'korijn@poetry-repository-main', + 'username': 'korijn'}, + 'ARTIFACTS_TOKEN_B64': {'b64': True, + 'credential': 'korijn@poetry-repository-main', + 'username': 'korijn'}}} +keycmd: keyring backend: (remembered) +keycmd: exposing credential korijn@poetry-repository-main with user korijn as environment variable ARTIFACTS_TOKEN (b64: False, format: None) +keycmd: exposing credential korijn@poetry-repository-main with user korijn as environment variable ARTIFACTS_TOKEN_B64 (b64: True, format: None) +keycmd: detected shell: C:\Windows\System32\cmd.exe +keycmd: running command: ['C:\\Windows\\System32\\cmd.exe', '/C', 'echo', '%ARTIFACTS_TOKEN_B64%'] +aSdtIG5vdCB0aGF0IHN0dXBpZCA6KQ== +``` + +That output answers, in order, the four questions a misbehaving run usually comes down to: + +* **Which files were loaded?** If the file you have been editing is not in the list, it is not where keycmd looks. See [where configuration lives](configuration.md#where-configuration-lives) โ€” the search stops at the root of a git repository and before your home folder. +* **What did they merge into?** The merged configuration is the one that counts, and a value you expected can be overridden by a file loaded later. +* **Which backend answered?** See [keyring backends](keyring-backends.md). +* **What was actually run?** Including the shell, and the exact argument vector handed to it. + +The example above uses `cmd.exe`; in bash or PowerShell the command would be quoted as one argument, as in `keycmd -v 'echo $ARTIFACTS_TOKEN_B64'`. + +## Common problems + +### `keycmd: command not found` + +The executable is not on your `PATH`. See [installation](../getting-started/installation.md) โ€” `uv tool install` and `pipx` handle this for you; a plain `pip install` into some environment may not. + +### `keycmd: error: missing command argument` + +keycmd was called with no command to run. Either pass one, or use `keycmd --shell` for a subshell. + +### The variable is empty, or my shell expanded it before keycmd ran + +```bash +keycmd echo $SECRET # your shell expands $SECRET โ€” before keycmd sets it +keycmd 'echo $SECRET' # the shell keycmd starts expands it โ€” correct +``` + +Quote the whole command, so that the shell keycmd starts is the one interpreting it. See [running commands](running-commands.md). + +### `keycmd: error: keyring has no backend to read credentials from` + +keyring found nothing on this machine that it can read credentials from. Install a backend for your platform, or name one with `PYTHON_KEYRING_BACKEND`; see [keyring backends](keyring-backends.md). Inside a WSL distribution this usually means no keyring daemon is running โ€” see the [WSL guide](wsl.md). + +### `keycmd: error: MISSING credential ... as it does not exist` + +The lookup is by credential name *and* username, and both have to match exactly. Check the verbose output for the `exposing credential ... with user ...` line and compare it with what your credential manager shows. On Linux, a password added through a GUI often has no username at all, which is why the [quick start](../getting-started/quick-start.md) sets it through Python's `keyring` package instead. + +### `keycmd: error: MISSING alias key ...` + +An entry under `[aliases]` names a `key` that no entry under `[keys]` defines. Remember that the merged configuration is what counts: an alias in one file can refer to a key that another file was supposed to provide. See [aliases](configuration.md#aliases). + +### npm complains about a missing variable on commands I don't prefix + +npm will complain if you make calls such as `npm run [...]` without the environment variable set. ๐Ÿ™„ You can set them to the empty string to make npm shut up: `export PAT_B64=` (or `setx PAT_B64=` on Windows). + +### A TOML syntax error + +``` +keycmd: error: Expected '=' after a key in a key/value pair (at line 3, column 5) +``` + +One of your configuration files is not valid TOML. The message names the line and column; the file is the last one listed as loading in the verbose output. diff --git a/docs/guide/wsl.md b/docs/guide/wsl.md new file mode 100644 index 0000000..6f76fa2 --- /dev/null +++ b/docs/guide/wsl.md @@ -0,0 +1,52 @@ +# WSL + +If you're using WSL, you'll run into a wall the first time you try to use keycmd. That's because keycmd uses the keyring library to connect to OS keyrings, and keyring will attempt to connect to your Linux distribution's (probably Ubuntu) keyring background service, which by default isn't actually running in a WSL environment. + +There are two ways out of that, and which one you want depends on what you'd rather maintain. + +!!! note "Are you actually working in WSL?" + + Just because you installed WSL on your system does not mean you are working in it. Think about this for a moment: are you using Python from Windows, or from WSL? This page is only relevant if you are actually working inside a distribution. + +## Option 1: run a keyring daemon in the distribution + +If you did set up your Linux distribution's keyring background service, that's fine โ€” you can keep using it and don't need any of the steps below. Inside the distribution keycmd is then an ordinary posix process talking to an ordinary posix keyring, and everything on the rest of this site applies unchanged. + +## Option 2: reach the Windows Credential Manager + +If you would rather have keyring connect from the WSL environment to your Windows Credential Manager, install keycmd according to the [installation instructions](../getting-started/installation.md) **in Windows**, not in WSL. Then, assuming `keycmd` is on your Windows `PATH`, it should now be available in WSL as well. + +That leaves you with one keyring to maintain instead of two, and it is the one your Windows tools already use. + +## How keycmd crosses the boundary + +Keep in mind that keycmd is a Windows process in this setup, so left to its own devices it would run your command in a Windows shell, and `keycmd --shell` would open one. It doesn't: when keycmd notices it was called from a distribution, it runs your command back inside that distribution through `wsl.exe`, and `keycmd --shell` opens a shell there. + +keycmd works out where it was called from by looking at its own process tree and working directory: + +* a `wsl.exe` or `wslhost.exe` ancestor means it was called from a distribution +* a Windows shell (`cmd`, `powershell`, `pwsh`) found first means it was called from Windows after all +* failing both, a UNC working directory (`\\wsl$\...` or `\\wsl.localhost\...`) settles it + +If you have more than one distribution installed, and you are working somewhere on the distribution's own file system, that working directory also names the distribution, and keycmd passes it to `wsl.exe` as `--distribution`, so your command goes to the distribution you are in rather than the default one. + +## Your credentials have to be told to cross + +Your credentials do not come along by themselves, since neither side of the WSL boundary inherits the other's environment. Only the variables listed in [`WSLENV`](https://devblogs.microsoft.com/commandline/share-environment-vars-between-wsl-and-windows/) make the trip, so keycmd adds the variables from your configuration to it. Anything you had already listed in `WSLENV` yourself is kept. + +## When keycmd gets it wrong + +Set the `KEYCMD_WSL` environment variable to override the decision in either direction: + +```bash +KEYCMD_WSL=0 keycmd 'echo $SECRET' # stay on the windows side +KEYCMD_WSL=1 keycmd 'echo $SECRET' # go through wsl.exe regardless +``` + +`keycmd --verbose` reports which way it went, and what it based that on: + +``` +keycmd: windows process tree: keycmd <- wsl <- svchost +keycmd: called from WSL, by way of wsl +keycmd: sharing with WSL as WSLENV=SECRET +``` diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..eeb91f7 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,45 @@ +# keycmd ๐Ÿ”‘ + +Prefix any command with `keycmd` to source your secrets and credentials from the OS keyring, instead of risky `.env` files (or worse ๐Ÿ™ˆ). + +```bash +keycmd 'npm install' +``` + +That's the whole idea. keycmd looks up the credentials named in your configuration, exposes them as environment variables, and runs your command with them โ€” for the duration of that one command, and nowhere else. Nothing is written to disk, nothing is left behind in your shell history, and no `.env` file has to exist. + +It supports Windows, macOS and Linux, and works with npm, pip, uv, poetry, docker, docker compose, kubectl, and anything else that reads a credential from the environment. + +## Why keycmd? + +* **Your secrets stay in the keyring.** The Windows Credential Manager, the macOS keychain and the Linux secret service already exist to keep credentials safe. keycmd reads from them, so a checked-out repository never has to contain a token. +* **Exposed for one command only.** `keycmd 'your command'` sets the variables for that process and nothing else. There is also `keycmd --shell` for a subshell, when you are debugging something that needs them for a while. +* **Configuration that follows your project.** Configuration is merged from your home folder, from `.keycmd` files up the directory tree, and from `pyproject.toml`, so a project can name the credentials it needs without every developer setting them up by hand. +* **One credential, many shapes.** [Format strings and aliases](guide/configuration.md#format-strings) expose the same secret as plain text, base64, or a basic auth header โ€” whatever each tool insists on. +* **Any keyring backend.** keycmd talks to your OS keyring through [keyring](https://github.com/jaraco/keyring), so every [third party backend](https://github.com/jaraco/keyring/#third-party-backends) works too, with no special configuration. + +## A taste of keycmd + +Store a credential in your OS keyring, name it in a `.keycmd` file: + +```toml +[keys] +OPENAI_API_KEY = { credential = "my-openai-token", username = "your-username" } +``` + +...and run anything that needs it: + +```bash +keycmd 'python my_openai_script.py' +keycmd 'jupyter notebook' +``` + +The variable exists inside those commands, and nowhere else. + +## Where to go next + +* [Installation](getting-started/installation.md) โ€” install keycmd globally, or under pyenv, or for use from WSL. +* [Quick Start](getting-started/quick-start.md) โ€” store a credential, write a config file, see it work, in a few minutes. +* [Guide](guide/running-commands.md) โ€” running commands, configuration, keyring backends, WSL and troubleshooting in depth. +* [Examples](examples/openai.md) โ€” an OpenAI API key, and a single Azure DevOps token shared between poetry, npm and docker compose. +* [Reference](reference/cli.md) โ€” every CLI flag, environment variable and configuration field. diff --git a/docs/reference/cli.md b/docs/reference/cli.md new file mode 100644 index 0000000..66e823c --- /dev/null +++ b/docs/reference/cli.md @@ -0,0 +1,102 @@ +# CLI Reference + +``` +โฏ keycmd --help +usage: keycmd [-h] [-v] [--version] [--detect-backend] [--reset-backend] + [--shell] + ... + +positional arguments: + command command to run + +options: + -h, --help show this help message and exit + -v, --verbose enable verbose output, useful for configuration debugging + --version print version info + --detect-backend search for the keyring backend now and remember it for + later runs + --reset-backend forget the remembered keyring backend, so the next run + searches again + --shell spawn a subshell instead of running a command +``` + +## Positional arguments + +### `command` + +The command to run with the credentials exposed as environment variables. + +Given as a single quoted argument, it is handed to your shell as typed, so the shell interprets it โ€” pipes, redirects and variable expansion included: + +```bash +keycmd 'echo $SECRET | tr a-z A-Z' +``` + +Given as several arguments, each is passed on as the word it was, and shell syntax is not interpreted a second time: + +```bash +keycmd mytool --message 'hello world' +``` + +Required, unless `--shell`, `--version`, `--detect-backend` or `--reset-backend` is used. Without one, keycmd exits with `error: missing command argument`. + +See [running commands](../guide/running-commands.md) for the full story. + +## Options + +### `-v`, `--verbose` + +Report every step: the configuration files loaded, the merged configuration, where the keyring backend came from, each credential exposed, the shell detected, and the command line run. See [troubleshooting](../guide/troubleshooting.md). + +Values of credentials are never printed. + +### `--version` + +Print the installed version and exit. + +### `--shell` + +Spawn an interactive subshell with the credentials exposed, instead of running a command. Everything started from that subshell inherits them, until you exit it. + +### `--detect-backend` + +Search for a keyring backend now, and remember it for later runs. Returns without loading a configuration or running a command. + +Use it when your machine changed in a way that leaves the remembered answer valid but wrong โ€” a better backend installed, or a package removed whose backend you no longer want. See [keyring backends](../guide/keyring-backends.md). + +### `--reset-backend` + +Forget the remembered keyring backend, so the next run searches again. Also returns without loading a configuration. + +## Environment variables + +### `PYTHON_KEYRING_BACKEND` + +keyring's own setting, naming the backend class to use, e.g. `keyring.backends.SecretService.Keyring`. It skips the search entirely and outranks anything keycmd has remembered โ€” and with it set, keycmd remembers nothing of its own. Run `keyring --list-backends` for the names to choose from. + +### `KEYCMD_WSL` + +Override keycmd's detection of whether it was called from a WSL distribution. Set it to `0` to keep the run on the Windows side, or to anything else to send it through `wsl.exe` regardless. See the [WSL guide](../guide/wsl.md). + +### `WSLENV` + +Not a setting of keycmd's, but the mechanism it uses: only variables listed in `WSLENV` cross the boundary between Windows and a WSL distribution, so keycmd appends the variables from your configuration to whatever you already had listed there. + +## Files + +| Path | What it is | +| --- | --- | +| `~/.keycmd` | your user-wide configuration | +| `./.keycmd` | per-directory configuration, all of them up to the repository root | +| `./pyproject.toml` | the first one found up the tree, `[tool.keycmd]` table | +| `%LOCALAPPDATA%\keycmd\backend` | the remembered keyring backend, on Windows | +| `~/Library/Caches/keycmd/backend` | the remembered keyring backend, on macOS | +| `$XDG_CACHE_HOME/keycmd/backend` | the remembered keyring backend, on Linux (usually `~/.cache`) | + +The cache file is safe to delete at any moment; it costs one slower run to write again. + +## Exit status + +keycmd exits with `1` and a `keycmd: error: ...` message of its own when a configuration is invalid, a command is missing, or there is no keyring backend to read credentials from. + +Otherwise the exit status is your command's. On posix, keycmd replaces its own process with the shell, so the status is the shell's directly; on Windows it runs the shell as a subprocess and passes the status along. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 0000000..d78c4bf --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,82 @@ +# Configuration Reference + +Configuration is TOML, and lives in `~/.keycmd`, in `.keycmd` files up the directory tree, and in the `[tool.keycmd]` table of a `pyproject.toml`. See [configuration](../guide/configuration.md) for how those are found and merged. + +## Schema + +* `keys`: dict + * `{key_name}`: dict โ€” an environment variable will be created with this name + * `credential`: str โ€” the name of the credential in your keyring + * `username`: str โ€” the username associated with the credential in your keyring + * `b64`: bool, optional โ€” set to `true` to apply base64 encoding + * `format`: str, optional โ€” apply a format string (applied before base64 encoding) +* `aliases`: dict, optional + * `{alias_name}`: dict โ€” an environment variable will be created with this name + * `key`: str โ€” the key that should be aliased + * `b64`: bool, optional โ€” see `keys.{key_name}.b64` + * `format`: str, optional โ€” see `keys.{key_name}.format` + +## Fields + +### `keys.{key_name}.credential` + +**Required.** The name of the credential in your keyring. Together with `username`, this is what is looked up โ€” they are the two arguments of keyring's `get_password()`. A credential that does not exist is a user error, not an empty variable: + +``` +keycmd: error: MISSING credential my-secret with user my-username as it does not exist +``` + +### `keys.{key_name}.username` + +**Required.** The username associated with the credential in your keyring. On macOS this is the keychain item's *Account Name*; on Windows it is the *User name* of the generic credential. + +### `keys.{key_name}.format` + +Optional. A [`str.format`](https://docs.python.org/3/library/stdtypes.html#str.format) format string, applied to the password before it is exposed. Three variables are available: `credential`, `username` and `password`. + +```toml +format = "{username}:{password}" +``` + +### `keys.{key_name}.b64` + +Optional, defaults to `false`. Base64-encode the value, *after* `format` has been applied โ€” which is the order that turns `{username}:{password}` into a basic auth value. + +### `aliases.{alias_name}.key` + +**Required.** The name of an entry under `[keys]` to re-expose. The credential is looked up once, for the key; the alias only applies its own `format` and `b64` to what came back. An alias naming a key that does not exist is an error. + +### `aliases.{alias_name}.format`, `aliases.{alias_name}.b64` + +Optional, and identical in behaviour to the key fields of the same name. They are not inherited from the aliased key: an alias without them exposes the raw password, regardless of what the key does. + +## Examples + +A credential exposed as-is: + +```toml +[keys] +SECRET = { credential = "my-secret", username = "my-username" } +``` + +The same credential in three shapes โ€” plain, base64, and basic auth โ€” with one keyring lookup: + +```toml +[keys] +MY_TOKEN = { credential = "azure_secret", username = "azure" } + +[aliases] +MY_TOKEN_B64 = { key = "MY_TOKEN", b64 = true } +MY_TOKEN_BASICAUTH = { key = "MY_TOKEN", format = "{username}:{password}", b64 = true } +``` + +The same, in a `pyproject.toml`: + +```toml +[tool.keycmd.keys] +MY_TOKEN = { credential = "azure_secret", username = "azure" } + +[tool.keycmd.aliases] +MY_TOKEN_B64 = { key = "MY_TOKEN", b64 = true } +MY_TOKEN_BASICAUTH = { key = "MY_TOKEN", format = "{username}:{password}", b64 = true } +``` diff --git a/keycmd/backend.py b/keycmd/backend.py index 1504d43..b28c987 100644 --- a/keycmd/backend.py +++ b/keycmd/backend.py @@ -47,6 +47,10 @@ # where someone whose machine turned out to have no backend can find one BACKENDS_URL: str = "https://github.com/jaraco/keyring#third-party-backends" +# and where someone inside a distribution can find out why keycmd belongs +# on the windows side of the boundary instead +WSL_DOCS_URL: str = "https://korijn.github.io/keycmd/guide/wsl/" + def cache_home() -> Path: """Where this platform keeps per user files a program can afford to lose @@ -206,13 +210,13 @@ def no_backend() -> NoReturn: f"see {BACKENDS_URL}", ] if in_distro(): - # the README tells WSL users to install keycmd on windows for - # exactly this reason, and this is what not having done so looks - # like from inside the distribution + # the docs tell WSL users to install keycmd on windows for exactly + # this reason, and this is what not having done so looks like from + # inside the distribution hints.append( "inside WSL this usually means no keyring daemon is running;" - " the README explains how to reach the windows credential" - " manager instead" + f" see {WSL_DOCS_URL} to reach the windows credential manager" + " instead" ) error("keyring has no backend to read credentials from", *hints) diff --git a/keycmd/wsl.py b/keycmd/wsl.py index f0c0cc6..845c816 100644 --- a/keycmd/wsl.py +++ b/keycmd/wsl.py @@ -1,6 +1,6 @@ """Reaching a WSL distribution from the windows install of keycmd -The README tells WSL users to install keycmd on windows, so that keyring +The docs tell WSL users to install keycmd on windows, so that keyring talks to the windows credential manager instead of a keyring daemon inside the distribution. That leaves keycmd a windows process with a windows idea of a shell: asked for a subshell it opens cmd, and asked for a command it diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..c2872b2 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,64 @@ +site_name: keycmd +site_description: Run any command with secrets from your OS keyring +site_url: https://korijn.github.io/keycmd +repo_url: https://github.com/Korijn/keycmd +repo_name: Korijn/keycmd + +theme: + name: material + palette: + - media: "(prefers-color-scheme: light)" + scheme: default + primary: indigo + accent: amber + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: indigo + accent: amber + toggle: + icon: material/brightness-4 + name: Switch to light mode + features: + - navigation.sections + - navigation.expand + - content.code.copy + - content.tabs.link + +markdown_extensions: + - admonition + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.superfences + - pymdownx.details + - pymdownx.keys + - pymdownx.tabbed: + alternate_style: true + - attr_list + - md_in_html + +plugins: + - search + +nav: + - Home: index.md + - Getting Started: + - Installation: getting-started/installation.md + - Quick Start: getting-started/quick-start.md + - Guide: + - Running Commands: guide/running-commands.md + - Configuration: guide/configuration.md + - Keyring Backends: guide/keyring-backends.md + - WSL: guide/wsl.md + - Troubleshooting: guide/troubleshooting.md + - Examples: + - OpenAI API Key: examples/openai.md + - Azure Artifacts: examples/azure-artifacts.md + - Reference: + - CLI Reference: reference/cli.md + - Configuration Reference: reference/configuration.md + - Development: + - Contributing: development/contributing.md + - Testing: development/testing.md diff --git a/pyproject.toml b/pyproject.toml index c66e56a..350c0f0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -19,6 +19,7 @@ dependencies = [ keycmd = "keycmd.cli:main" [project.urls] +Documentation = "https://korijn.github.io/keycmd" Repository = "https://github.com/Korijn/keycmd" [dependency-groups] @@ -31,6 +32,10 @@ dev = [ ] ruff = ["ruff"] ty = ["ty"] +docs = [ + "mkdocs>=1,<2", + "mkdocs-material", +] [tool.uv] default-groups = ["dev"]