Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,19 @@ project intends to use Semantic Versioning once its public API reaches stability

## [Unreleased]

### Added

- Native console controls for quiet output, one- or two-level verbosity, skipped/expected-failure
reasons, slow-test duration lists, and explicit automatic/forced/disabled ANSI color handling.

### Changed

- `testenix run` now defaults to a compact per-file report while retaining complete collection and
failure diagnostics plus the final summary. Console rendering remains deterministic and is
emitted after execution rather than presented as live progress.
- Documentation now distinguishes native Testenix rendering from the unchanged pytest output
produced by the transparent `testenix pytest` compatibility bridge.

## [0.2.0] - 2026-07-20

### Added
Expand Down
37 changes: 33 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ If a supported pytest (`>=8.3,<10`) is already installed in the same environment
`testenix` package is sufficient.

Testenix requires Python 3.11 or newer. The project is currently an alpha; pin the version before
using it in CI. Until the first PyPI release is visible, use the GitHub installation below.
using it in CI. Use the GitHub installation below only when you intentionally want unreleased
changes from the current development branch.

To try the current checkout before publication:

Expand All @@ -61,12 +62,14 @@ python -m pip install "testenix[pytest] @ git+https://github.com/polishdataengin
Yes. Testenix provides a transparent compatibility bridge for existing pytest projects:

```bash
testenix pytest -q tests
testenix pytest -q --tb=short tests
```

Everything after `testenix pytest` is forwarded unchanged to the same interpreter as
`python -m pytest`. This preserves pytest collection, `conftest.py`, fixtures, parametrization,
markers, assertion rewriting, plugins, configuration, output, node IDs, and exit codes.
Consequently, output produced by `uv run pytest tests` or `testenix pytest ...` is pytest's UI,
not the native Testenix console reporter.

```bash
testenix pytest tests/test_api.py::test_health -k smoke --maxfail=1
Expand Down Expand Up @@ -164,6 +167,29 @@ Run it with:
testenix run tests
```

The native command uses a compact, file-level report by default and still prints complete failure
details and a final summary. A typical failing run looks like this (the run ID and timings vary):

```console
$ testenix run tests
Testenix | 4 tests | 2 files | 2 workers

PASS tests/test_multiplication.py 2 passed [8ms]
FAIL tests/test_checkout.py 1 passed, 1 failed [12ms]

Problems (1)
FAIL tests/test_checkout.py::test_rejects_expired_card
attempt 1, call: expected status 402, got 200

4 tests, 3 passed, 1 failed in 0.084s
```

Use `-q` to hide the header and file table while retaining collection errors, failure details, and
the final summary. `-v` prints one result row per test; `-vv` also exposes worker, attempt, and
phase metadata. `--show-skips` includes skip and expected-failure reasons, `--durations N` lists
the `N` slowest tests (`--durations 0` lists all), and `--color auto|always|never` controls ANSI
styling.

Plain `test_*` functions are collected without `@test`; the decorator is useful for descriptions,
tags, and per-test timeouts.

Expand All @@ -184,11 +210,14 @@ Command-line options override this table:
```text
testenix run [PATH ...] [--workers auto|N] [--retries N] [--timeout SECONDS]
[--tag TAG ...] [--json FILE] [--junit FILE]
[--history FILE | --no-history]
[--history FILE | --no-history] [-q | -v | -vv]
[--color auto|always|never | --no-color]
[--show-skips] [--durations N]
```

Repeated `--tag` options use AND semantics: a selected test must contain every requested tag.
The console report is always printed. JSON preserves the complete run/test/attempt/phase model,
The console report is always printed after execution; it is deterministic output, not a live
progress display. JSON preserves the complete run/test/attempt/phase model,
JUnit targets CI systems, and SQLite history supplies duration estimates to later runs. History is
enabled at `.testenix/history.sqlite3` by default; use `--no-history` for a side-effect-free run.

Expand Down
14 changes: 12 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ $ python -m pip install "testenix[pytest]"
$ uv add --dev "testenix[pytest]"
```

Until the first PyPI release is visible, install directly from the protected `main` branch:
To evaluate unreleased development changes, install directly from the protected `main` branch:

```console
$ python -m pip install "testenix @ git+https://github.com/polishdataengineer/testenix.git@main"
Expand All @@ -46,7 +46,7 @@ $ python -m pip install "testenix[pytest] @ git+https://github.com/polishdataeng
Run an unchanged pytest suite through its real engine:

```console
$ testenix pytest -q tests
$ testenix pytest -q --tb=short tests
```

Use `testenix run` for native Testenix tests and the built-in scheduler, retries, history, and
Expand Down Expand Up @@ -85,6 +85,16 @@ Run the suite:
$ testenix run tests
```

The default console output is a compact per-file report followed by complete failure details and a
final summary. It is rendered deterministically after the run rather than updated as live progress.
Use `-q` to omit the header and file table, `-v` for one row per test, or `-vv` for worker, attempt,
and phase metadata. Collection errors and failure details remain visible with `-q`.

```console
$ testenix run -q tests
4 tests, 3 passed, 1 skipped in 0.071s
```

The process exits with code `0` when every selected test has a non-gating terminal status.

## Add native metadata
Expand Down
17 changes: 15 additions & 2 deletions docs/guides/pytest-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ configuration:

```console
$ python -m pip install "testenix[pytest]"
$ testenix pytest -q tests
$ testenix pytest -q --tb=short tests
```

For the supported static subset, `testenix migrate pytest tests` can instead create a validated
Expand All @@ -36,6 +36,14 @@ working directory, environment, terminal, standard streams, and pytest's signal
therefore remains responsible for collection, execution, configuration, plugin loading, output,
descendants such as pytest-xdist workers, and exit status.

This also explains the visual style: output from `uv run pytest tests` is rendered by pytest, and
`testenix pytest` deliberately preserves that same renderer. It does not activate Testenix's
compact native console. For shorter pytest output with concise tracebacks, use:

```console
$ testenix pytest -q --tb=short tests
```

```console
$ testenix pytest tests/test_api.py::test_health -k smoke --maxfail=1
$ testenix pytest -m "unit and not slow" tests
Expand Down Expand Up @@ -84,12 +92,17 @@ timeouts, tags, history, event model, JSON reporter, or JUnit reporter. Pass the
pytest or plugin options after the subcommand. For example, use pytest's `--junitxml`, not
Testenix's native `--junit`.

Native presentation options are not interpreted by the bridge either. Arguments such as `-q`,
`-v`, `--color`, `--show-skips`, and `--durations` go straight to pytest and follow pytest's syntax
and semantics. For example, pytest uses `-rs` for skipped reasons, `--durations=N` for its slowest
tests list, and `--color=yes|no|auto`; Testenix does not translate the native spellings.

Pytest and every required plugin must be installed beside the `testenix` executable in the same
interpreter environment. For uv-managed projects, prefer:

```console
$ uv add --dev "testenix[pytest]"
$ uv run testenix pytest -q tests
$ uv run testenix pytest -q --tb=short tests
```

An isolated `uv tool install testenix` environment does not automatically see pytest plugins from
Expand Down
51 changes: 49 additions & 2 deletions docs/guides/reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,60 @@ console and red in JSON because both are derived from one model.

## Console

The console report is always enabled:
The console report is always enabled. Its default mode is compact: Testenix prints an aggregate row
for each source file, then complete failure details and the final summary.

```console
$ testenix run tests
Testenix | 5 tests | 2 files | 2 workers

PASS tests/test_accounts.py 3 passed [9ms]
FAIL tests/test_checkout.py 1 passed, 1 failed [12ms]

Problems (1)
FAIL tests/test_checkout.py::test_rejects_expired_card
attempt 1, call: expected status 402, got 200

5 tests, 4 passed, 1 failed in 0.091s
```

Run IDs and timings naturally vary. The report is assembled in stable source order when execution
finishes; it is not a live-progress display.

Choose the amount of terminal detail without changing execution semantics:

| Mode | Output |
| --- | --- |
| default | Run header, compact per-file table, collection/failure details, final summary. |
| `-q`, `--quiet` | No header or file table; collection/failure details and final summary remain. |
| `-v` | One stable result row per test, including duration. |
| `-vv` | Per-test rows plus worker, attempt, and phase metadata, including captured output. |

Skipped and expected-failure reasons are hidden unless they are requested explicitly:

```console
$ testenix run --show-skips tests
```

List the slowest tests with `--durations N`; use `0` to list every test. The duration section is
printed immediately before the final summary:

```console
$ testenix run --durations 10 tests
$ testenix run --durations 0 tests
```

ANSI styling defaults to `--color auto`, which considers the output terminal plus `NO_COLOR`,
`FORCE_COLOR`, `CI`, and `TERM=dumb`. Force it with `--color always`, or produce plain output with
either `--color never` or `--no-color`:

```console
$ testenix run --no-color tests
```

It prints test outcomes, failure details, and a final summary suitable for local development.
These flags affect the native `testenix run` console only. The transparent `testenix pytest`
bridge preserves pytest's renderer and argument meanings; a concise bridge command is
`testenix pytest -q --tb=short tests`.

## JSON

Expand Down
Loading