From 0d4f43f5dd78f2e7789d2228fd17aa940b37906b Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Thu, 6 Aug 2026 12:05:39 +0900 Subject: [PATCH 1/4] feat(operations): rebuild availability probes on authoritative parent --- .../2026-08-05-availability-probes.md | 144 ++++++++++++++++++ scripts/test_documentation_contracts.py | 36 +++++ .../viewer/controller/HealthController.java | 73 +++++++-- .../controller/HealthControllerTest.java | 95 +++++++++++- 4 files changed, 327 insertions(+), 21 deletions(-) create mode 100644 docs/operations/2026-08-05-availability-probes.md create mode 100644 scripts/test_documentation_contracts.py diff --git a/docs/operations/2026-08-05-availability-probes.md b/docs/operations/2026-08-05-availability-probes.md new file mode 100644 index 00000000..3553bbf9 --- /dev/null +++ b/docs/operations/2026-08-05-availability-probes.md @@ -0,0 +1,144 @@ +# ADR: Separate liveness and readiness probes + +- Status: Accepted +- Date: 2026-08-05 +- Decision owners: Clearfolio maintainers + +## Context + +Clearfolio previously exposed only `GET /healthz`. The implementation returned a +static success payload and described the endpoint as a liveness check, while the +README and architecture documents described the same route as readiness. That +ambiguity lets an orchestrator use a process-alive signal as a traffic-routing +signal. + +Liveness and readiness answer different operational questions. Liveness asks +whether the process is irrecoverably unhealthy and should be restarted. +Readiness asks whether the current instance should receive traffic. A temporary +readiness failure must not cause a restart cascade. + +## Decision + +Clearfolio exposes two unauthenticated, non-cacheable probes on the main +application port: + +| Route | Source of truth | Success | Unavailable | +| --- | --- | --- | --- | +| `GET /healthz` | Spring Boot `LivenessState` | `200 {"status":"ok"}` | `503 {"status":"broken"}` | +| `GET /readyz` | Spring Boot `ReadinessState` | `200 {"status":"ready"}` | `503 {"status":"not_ready"}` | + +Both responses include `Cache-Control: no-store`. The existing successful +`/healthz` payload remains stable for backward compatibility. + +The liveness probe must remain independent of shared external services such as a +database, object store, gateway, or model provider. Restarting an otherwise +recoverable application during an external outage can amplify the outage. +Readiness may later incorporate instance-local conditions that determine +whether this instance can safely accept traffic, such as completed startup +recovery or bounded-queue overload. Such changes must update this ADR and add +executable failure and recovery tests. + +Spring Boot's `ApplicationAvailability` is the in-process source of truth. This +keeps probe semantics available without adding the Actuator dependency or a +second management port and preserves standalone deployment. The implementation +and reference documentation are version-aligned to Spring Boot 3.5.16, the +version managed by this repository. + +## Kubernetes example + +A startup probe protects slow or variable initialization from premature +liveness restarts. Kubernetes suppresses liveness and readiness checks until the +startup probe succeeds. + +```yaml +startupProbe: + httpGet: + path: /healthz + port: 8080 + periodSeconds: 5 + timeoutSeconds: 2 + failureThreshold: 24 # 120-second startup budget +livenessProbe: + httpGet: + path: /healthz + port: 8080 + periodSeconds: 10 + timeoutSeconds: 2 + failureThreshold: 3 +readinessProbe: + httpGet: + path: /readyz + port: 8080 + periodSeconds: 5 + timeoutSeconds: 2 + failureThreshold: 3 + successThreshold: 1 +``` + +Probe timings are deployment inputs rather than application constants. Operators +must tune startup budgets, periods, timeouts, and failure thresholds against +measured startup, overload, and recovery behavior. The example deliberately +uses small dedicated response bodies because Kubernetes determines HTTP probe +success from the status code and recommends minimal health-check payloads. + +## Security and privacy + +- Probe responses disclose only a controlled state label. +- They contain no tenant, document, queue, dependency, credential, build, or + exception details. +- They are intentionally unauthenticated so container orchestrators can call + them, but they do not grant access to protected APIs. +- `Cache-Control: no-store` prevents intermediaries from replaying stale + availability state. +- Each response remains far below Kubernetes' 10 KiB HTTP-probe body read limit. + +## Verification contract + +Automated tests must prove: + +- `CORRECT` liveness returns `200` and `ok`; +- `BROKEN` liveness returns `503` and `broken`; +- `ACCEPTING_TRAFFIC` readiness returns `200` and `ready`; +- `REFUSING_TRAFFIC` readiness returns `503` and `not_ready`; +- every probe response is non-cacheable; +- controller construction fails without the availability provider; +- repository production line and branch coverage remains 100%. + +The release gate also requires exact-head CI, Security Scan, SAST, fuzzing, +automated review, independent approval, and all repository protections. + +## Consequences + +### Positive + +- Kubernetes and other orchestrators can distinguish startup completion, + restart eligibility, and traffic eligibility. +- The `/healthz` success contract remains compatible with existing callers. +- Future readiness signals have an explicit, testable extension point. + +### Trade-offs + +- Operators must configure three probe roles across two routes. +- A static successful liveness response is no longer sufficient when Spring + marks the application `BROKEN`. +- Readiness currently reflects Spring application state, not durable database, + object-store, or queue health. Those dependencies remain separate commercial + hardening slices. + +## Rollback + +A rollback may remove `/readyz` and restore the old `/healthz` implementation, +but deployment manifests must be rolled back at the same time. Do not point both +Kubernetes probes at `/healthz`, because doing so recreates the original semantic +ambiguity. Remove the startup probe only when measured startup behavior and the +replacement deployment policy provide an equivalent startup-failure budget. + +## References + +Broadcom, Inc. (n.d.). *SpringApplication: Application availability (Spring Boot +3.5.16)*. Spring. Retrieved August 5, 2026, from +https://docs.spring.io/spring-boot/3.5/reference/features/spring-application.html#features.spring-application.application-availability + +The Kubernetes Authors. (2026, April 17). *Liveness, readiness, and startup +probes*. Kubernetes. +https://kubernetes.io/docs/concepts/workloads/pods/probes/ diff --git a/scripts/test_documentation_contracts.py b/scripts/test_documentation_contracts.py new file mode 100644 index 00000000..92498265 --- /dev/null +++ b/scripts/test_documentation_contracts.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +"""Regression tests for buyer-facing operational documentation contracts.""" + +from __future__ import annotations + +import unittest +from pathlib import Path + + +REPOSITORY_ROOT = Path(__file__).resolve().parent.parent +THREAT_MODEL_PATH = ( + REPOSITORY_ROOT / "docs" / "security" / "2026-07-02-threat-model-data-handling.md" +) + + +class DocumentationContractsTest(unittest.TestCase): + """Protect terminology that affects deployment and security decisions.""" + + def test_threat_model_distinguishes_liveness_from_readiness(self) -> None: + """Require the threat model to name both probes with their shipped roles.""" + threat_model_lines = THREAT_MODEL_PATH.read_text(encoding="utf-8").splitlines() + health_line = next( + line for line in threat_model_lines if line.startswith("- `GET /healthz`:") + ) + readiness_line = next( + line for line in threat_model_lines if line.startswith("- `GET /readyz`:") + ) + + self.assertRegex(health_line, r": .*liveness probe") + self.assertNotIn("readiness", health_line.lower()) + self.assertRegex(readiness_line, r": .*readiness probe") + self.assertNotIn("liveness", readiness_line.lower()) + + +if __name__ == "__main__": + unittest.main() diff --git a/src/main/java/com/clearfolio/viewer/controller/HealthController.java b/src/main/java/com/clearfolio/viewer/controller/HealthController.java index 9975524c..dcdca47b 100644 --- a/src/main/java/com/clearfolio/viewer/controller/HealthController.java +++ b/src/main/java/com/clearfolio/viewer/controller/HealthController.java @@ -1,36 +1,81 @@ package com.clearfolio.viewer.controller; import java.util.Map; +import java.util.Objects; +import org.springframework.boot.availability.ApplicationAvailability; +import org.springframework.boot.availability.LivenessState; +import org.springframework.boot.availability.ReadinessState; +import org.springframework.http.CacheControl; +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; /** - * Lightweight endpoint used for process-liveness checks. + * Exposes separate liveness and readiness probes on the application port. * - *

This endpoint deliberately reports only whether the application process can - * answer requests. Traffic-readiness semantics are introduced separately so an - * orchestrator never confuses restart eligibility with dependency readiness.

+ *

Liveness answers whether this process can continue operating or needs a + * restart. Readiness answers whether the instance should receive traffic. The + * two signals deliberately remain separate so a temporary readiness failure + * does not trigger a restart cascade.

*/ @RestController -@RequestMapping("/healthz") public class HealthController { + private final ApplicationAvailability applicationAvailability; + /** - * Creates the stateless liveness controller. + * Creates the probe controller from Spring Boot's availability state. + * + * @param applicationAvailability current application availability provider */ - public HealthController() { - // No mutable state or external dependency belongs in the liveness path. + public HealthController(ApplicationAvailability applicationAvailability) { + this.applicationAvailability = Objects.requireNonNull( + applicationAvailability, + "applicationAvailability" + ); } /** - * Returns a static health payload when the service is alive. + * Returns the process liveness state. * - * @return health status payload + * @return {@code 200} with {@code status=ok} when the process can recover, + * otherwise {@code 503} with {@code status=broken} */ - @GetMapping - public Map health() { - return Map.of("status", "ok"); + @GetMapping("/healthz") + public ResponseEntity> liveness() { + return availabilityResponse( + applicationAvailability.getLivenessState() == LivenessState.CORRECT, + "ok", + "broken" + ); + } + + /** + * Returns whether this instance is ready to accept traffic. + * + * @return {@code 200} with {@code status=ready} while accepting traffic, + * otherwise {@code 503} with {@code status=not_ready} + */ + @GetMapping("/readyz") + public ResponseEntity> readiness() { + return availabilityResponse( + applicationAvailability.getReadinessState() == ReadinessState.ACCEPTING_TRAFFIC, + "ready", + "not_ready" + ); + } + + private static ResponseEntity> availabilityResponse( + boolean available, + String availableStatus, + String unavailableStatus + ) { + HttpStatus responseStatus = available ? HttpStatus.OK : HttpStatus.SERVICE_UNAVAILABLE; + String statusValue = available ? availableStatus : unavailableStatus; + return ResponseEntity.status(responseStatus) + .cacheControl(CacheControl.noStore()) + .body(Map.of("status", statusValue)); } } diff --git a/src/test/java/com/clearfolio/viewer/controller/HealthControllerTest.java b/src/test/java/com/clearfolio/viewer/controller/HealthControllerTest.java index 4a8be8f7..f6cc1b7a 100644 --- a/src/test/java/com/clearfolio/viewer/controller/HealthControllerTest.java +++ b/src/test/java/com/clearfolio/viewer/controller/HealthControllerTest.java @@ -1,19 +1,100 @@ package com.clearfolio.viewer.controller; -import static org.assertj.core.api.Assertions.assertThat; - -import java.util.Map; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; import org.junit.jupiter.api.Test; +import org.springframework.boot.availability.ApplicationAvailability; +import org.springframework.boot.availability.LivenessState; +import org.springframework.boot.availability.ReadinessState; +import org.springframework.test.web.reactive.server.WebTestClient; +/** + * Verifies that liveness and readiness expose different operational states. + */ class HealthControllerTest { @Test - void healthControllerReturnsOkPayload() { - final HealthController controller = new HealthController(); + void livenessReturnsOkWhenTheApplicationCanRecover() { + ApplicationAvailability availability = availability( + LivenessState.CORRECT, + ReadinessState.ACCEPTING_TRAFFIC + ); + + client(availability).get() + .uri("/healthz") + .exchange() + .expectStatus().isOk() + .expectHeader().valueEquals("Cache-Control", "no-store") + .expectBody(String.class) + .isEqualTo("{\"status\":\"ok\"}"); + } + + @Test + void livenessReturnsServiceUnavailableForAnUnrecoverableApplication() { + ApplicationAvailability availability = availability( + LivenessState.BROKEN, + ReadinessState.ACCEPTING_TRAFFIC + ); - final Map response = controller.health(); + client(availability).get() + .uri("/healthz") + .exchange() + .expectStatus().isEqualTo(503) + .expectHeader().valueEquals("Cache-Control", "no-store") + .expectBody(String.class) + .isEqualTo("{\"status\":\"broken\"}"); + } + + @Test + void readinessReturnsOkOnlyWhileTrafficCanBeAccepted() { + ApplicationAvailability availability = availability( + LivenessState.CORRECT, + ReadinessState.ACCEPTING_TRAFFIC + ); + + client(availability).get() + .uri("/readyz") + .exchange() + .expectStatus().isOk() + .expectHeader().valueEquals("Cache-Control", "no-store") + .expectBody(String.class) + .isEqualTo("{\"status\":\"ready\"}"); + } + + @Test + void readinessReturnsServiceUnavailableWhileTrafficIsRefused() { + ApplicationAvailability availability = availability( + LivenessState.CORRECT, + ReadinessState.REFUSING_TRAFFIC + ); + + client(availability).get() + .uri("/readyz") + .exchange() + .expectStatus().isEqualTo(503) + .expectHeader().valueEquals("Cache-Control", "no-store") + .expectBody(String.class) + .isEqualTo("{\"status\":\"not_ready\"}"); + } + + @Test + void controllerRejectsMissingAvailabilityStateProvider() { + assertThrows(NullPointerException.class, () -> new HealthController(null)); + } + + private static ApplicationAvailability availability( + LivenessState livenessState, + ReadinessState readinessState + ) { + ApplicationAvailability availability = mock(ApplicationAvailability.class); + when(availability.getLivenessState()).thenReturn(livenessState); + when(availability.getReadinessState()).thenReturn(readinessState); + return availability; + } - assertThat(response).containsEntry("status", "ok"); + private static WebTestClient client(ApplicationAvailability availability) { + return WebTestClient.bindToController(new HealthController(availability)).build(); } } From d89c3780eb7d3d1c7080a3c9f2a7f49dd8cf45d2 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Thu, 6 Aug 2026 12:07:00 +0900 Subject: [PATCH 2/4] docs(operations): align availability architecture and operator guidance --- ARCHITECTURE.md | 23 +- CLAUDE.md | 258 ++++++++++-------- README.md | 11 +- docs/architecture.md | 25 +- .../2026-07-02-threat-model-data-handling.md | 3 +- 5 files changed, 192 insertions(+), 128 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 13dbeda7..57fd1e8e 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,6 +1,6 @@ # Architecture Map -Last updated: 2026-02-23 +Last updated: 2026-08-05 ## System Purpose @@ -22,6 +22,10 @@ Current state: viewer/state API is implemented in this repository; downstream S2 - `GET /viewer/{docId}`: HTML viewer UI entrypoint (loading/failed/ready) that embeds PDF.js. - `ArtifactController` (`src/main/java/com/clearfolio/viewer/controller/ArtifactController.java`) - `GET /artifacts/{docId}.pdf`: serves PDF bytes for SUCCEEDED jobs with basic HTTP Range support. +- `HealthController` (`src/main/java/com/clearfolio/viewer/controller/HealthController.java`) + - `GET /healthz`: process liveness from Spring Boot `LivenessState`. + - `GET /readyz`: traffic readiness from Spring Boot `ReadinessState`. + - Probe payloads disclose only a controlled state label and use `Cache-Control: no-store`. - `DefaultDocumentConversionService` (`src/main/java/com/clearfolio/viewer/service/DefaultDocumentConversionService.java`) - Validation, content hash generation, dedupe lookup, repository persistence, worker enqueue. - PDF passthrough: uploads that declare PDF (extension/content type) and carry the `%PDF-` magic header are seeded into the artifact store as-is, so the original bytes are served instead of a generated placeholder. @@ -46,6 +50,13 @@ Current state: viewer/state API is implemented in this repository; downstream S2 - `ViewerBootstrapResponse` (`src/main/java/com/clearfolio/viewer/api/ViewerBootstrapResponse.java`) - Includes deterministic `sourceExtension` and `rendererAdapter` metadata for viewer adapter bootstrap. +## Availability Model + +- Liveness and readiness are separate operational contracts. +- Liveness determines restart eligibility and must not depend on shared external services. +- Readiness determines whether this instance receives traffic and may later include instance-local startup-recovery or overload signals through Spring availability events. +- The accepted ADR and Kubernetes example are in `docs/operations/2026-08-05-availability-probes.md`. + ## State Model - Status values: `SUBMITTED`, `PROCESSING`, `SUCCEEDED`, `FAILED`. @@ -54,11 +65,10 @@ Current state: viewer/state API is implemented in this repository; downstream S2 ## Operational Gates - Build and test gates are defined in `AGENTS.md` and include: - - `mvn -DskipTests compile` - - `mvn test` - - JaCoCo line/branch 100% for `com.clearfolio.viewer.*` - - JavaDoc gate: `mvn -q -DskipTests javadoc:javadoc` - - Markdown lint for changed docs + - `mvn -B --no-transfer-progress verify` as the single complete merge-evidence command. + - JaCoCo 100% production line and branch coverage for `com.clearfolio.viewer.*` within the `verify` lifecycle. + - Warning-free public Javadoc validation within the same `verify` lifecycle. + - Markdown lint for changed documentation. Mandatory AC list (exact): @@ -85,5 +95,6 @@ Optional tracks: - `docs/diagrams/status-flow.md` - `docs/diagrams/preview-flow.md` - `docs/diagrams/retry-deadletter-flow.md` +- `docs/operations/2026-08-05-availability-probes.md` - `docs/engineering/acceptance-criteria.md` - `docs/workflow/one-day-delivery-plan.md` diff --git a/CLAUDE.md b/CLAUDE.md index c2207fde..2bc1cff0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,144 +1,170 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +This file provides guidance to Claude Code (claude.ai/code) when working with +this repository. ## Read AGENTS.md first -`AGENTS.md` at the repo root is the canonical agent operating guide. It defines the -mandatory quality and security merge gates (compile with zero warnings, tests, -100% JaCoCo coverage, JavaDoc, markdown lint, license/attribution/diligence drift -checks) and the change-management rule that any new gate must be added to -`AGENTS.md` in the same PR. Follow those gates before claiming any change complete. -This file complements `AGENTS.md` with commands and architecture context; when in -doubt, `AGENTS.md` wins. +`AGENTS.md` at the repository root is the canonical agent operating guide. It +defines mandatory quality, security, supply-chain, diligence, and merge gates. +This file complements it with commands and architecture context. When the two +files differ, `AGENTS.md` wins. -Related canonical docs: +Related canonical documents: -- `ARCHITECTURE.md` — root architecture map (components, state model, gate list). -- `docs/architecture.md` — detailed runtime flows and component boundaries. -- `docs/engineering/acceptance-criteria.md` — canonical acceptance policy with exact repro commands and evidence pointers. +- `ARCHITECTURE.md` — root component and state map. +- `docs/architecture.md` — detailed runtime flows and boundaries. +- `docs/engineering/acceptance-criteria.md` — exact-head acceptance policy. +- `docs/operations/2026-08-05-availability-probes.md` — liveness/readiness ADR. - `README.md` — API scope, tenant-header contract, and compatibility notes. ## Common commands -Toolchain: Java 21, Maven (Spring Boot parent 3.5.x). Python 3 for `scripts/`. +Toolchain: Java 21, Maven, Spring Boot 3.5.x, and Python 3 for `scripts/`. ```bash -# Compile (gate: warning/deprecated budget = 0; -Xlint:all -Werror is enforced) -mvn -DskipTests compile +# Canonical local acceptance command. It compiles with -Xlint:all -Werror, +# runs all tests, enforces zero missed JaCoCo lines/branches, and generates +# warning-free public Javadocs. +mvn -B --no-transfer-progress verify -# Full test suite (gate) -mvn test +# Focused test selection during the red-green loop. +mvn -B --no-transfer-progress test -Dtest=ConversionControllerTest +mvn -B --no-transfer-progress test -Dtest=ConversionControllerTest#methodName -# Single test class / single test method (standard Surefire selection) -mvn test -Dtest=ConversionControllerTest -mvn test -Dtest=ConversionControllerTest#methodName +# Buyer-readiness helper tests, matching the repository CI job. +python -m pytest -q scripts -# Coverage gate: JaCoCo line/branch missed must be 0 for com.clearfolio.viewer.* -mvn -q -Djacoco.includes=com.clearfolio.viewer.* \ - org.jacoco:jacoco-maven-plugin:0.8.13:prepare-agent test \ - org.jacoco:jacoco-maven-plugin:0.8.13:report -# Report lands in target/site/jacoco/jacoco.csv - -# JavaDoc gate (must produce no warnings/errors) -mvn -q -DskipTests javadoc:javadoc - -# Run the app locally, then probe readiness +# Run the service and inspect the two distinct availability signals. mvn spring-boot:run -curl -sS http://localhost:8080/healthz - -# Python helper-script unit tests (unittest, no pytest dependency) -python3 -m unittest discover -s scripts +curl -i http://localhost:8080/healthz +curl -i http://localhost:8080/readyz ``` +Do not present `mvn test`, a manually generated coverage report, or an earlier +head as complete merge evidence. The authoritative command is `mvn verify`, and +protected GitHub Checks must be successful for the exact current head. + The license-policy, third-party-attribution, buyer data-room manifest, buyer -readiness scorecard, and Figma deck payload drift checks are Python scripts under -`scripts/`; the exact invocations (with the current evidence/policy file paths) -are listed in `AGENTS.md` and must pass for doc/dependency changes that touch them. +readiness scorecard, and Figma payload drift checks are Python scripts under +`scripts/`. Their exact invocations and current evidence paths are listed in +`AGENTS.md`. + +## GitHub workflow model -There are no repo-local CI workflows in `.github/workflows`; gates are reproduced -locally and evidence is committed under `docs/qa/evidence/`. CodeQL runs through -GitHub default setup (do not add a repo-local advanced CodeQL workflow), and -Dependabot config lives in `.github/dependabot.yml`. +Repository workflows under `.github/workflows/` run CI, Security Scan, SAST +Semgrep, and fuzzing. Organization-central workflows may add review, coverage, +security, and merge-policy evidence. CodeQL uses GitHub default setup; do not add +a duplicate repository-local advanced CodeQL workflow while default setup is +enabled. + +A queued, pending, cancelled, skipped-required, or stale-head run is not passing. +Do not bypass branch protection or counted independent approval. Automated +reviews are advisory unless GitHub recognizes the reviewer identity as having +the permission required by the protected-branch rule. ## What this repository is -Clearfolio Viewer (`com.clearfolio` / `clearfolio-viewer`) is the MVP backend for -an integrated document viewer platform: non-blocking upload submit, async -conversion with retry/dead-letter, job status polling, and a PDF.js viewer served -from the same app. The runtime is Spring WebFlux (Servlet/MVC is explicitly not -the selected stack), logging is Log4j2 (the default Logback starter is excluded), -and the default job/artifact stores are in-memory (a SQL repository profile is -planned but not implemented). +Clearfolio Viewer (`com.clearfolio` / `clearfolio-viewer`) is the backend for an +integrated document-viewing platform. It provides non-blocking upload submission, +asynchronous conversion with retry and dead-letter behavior, status polling, +signed artifact access, and a same-application PDF.js viewer. + +The runtime is Spring WebFlux. Logging is Log4j2; the default Logback starter is +excluded. Conversion jobs currently use an in-memory repository and process-local +lifecycle evidence. Generated artifacts use the filesystem store by default, +with an in-memory implementation available for tests and explicitly configured +local runs. A durable SQL job repository remains a planned production slice. + +Entry point: +`src/main/java/com/clearfolio/viewer/ClearfolioViewerApplication.java`. + +Configuration: -Entry point: `src/main/java/com/clearfolio/viewer/ClearfolioViewerApplication.java`. -Configuration: `src/main/resources/application.yml` (`conversion.*` queue/retry/ -upload limits, `viewer.security.frame-ancestors`) plus the `buyer-demo` Spring -profile (`application-buyer-demo.yml`) for buyer sandbox deployments. +- `src/main/resources/application.yml` — queue, retry, upload, artifact-store, + availability, viewer, tenant-claim, and secret-mount settings. +- `src/main/resources/application-buyer-demo.yml` — buyer sandbox profile. ## High-level architecture -All production code lives in `src/main/java/com/clearfolio/viewer/`: - -- `controller/` — HTTP endpoints and exception mapping. `ConversionController` - (submit `POST /api/v1/convert/jobs`, status polling, operator retry, viewer - bootstrap JSON), `ViewerUiController` (`GET /viewer/{docId}` HTML shell), - `ArtifactController` (`GET /artifacts/{docId}.pdf` with token verification and - single-range support), `AdminController`, `AnalyticsController` (KPI snapshot), - `HealthController` (`/healthz`), `ApiExceptionHandler` (shared error shape: - `errorCode`, optional `code`, `message`, `traceId`, `details`). -- `service/` — `DefaultDocumentValidationService` (extension blocklist — HWP/HWPX - blocked by default — size limits, HMAC-verified policy-override lane), - `DefaultDocumentConversionService` (validation, content-hash dedupe, persist, - enqueue), `DefaultConversionWorker` (bounded executor, retry backoff, - dead-letter fallback, startup recovery sweep for stale leases). -- `repository/` — `ConversionJobRepository` (read/dedupe boundary, in-memory - implementation) and `ConversionJobStateStore` (explicit lifecycle-transition - boundary with a process-local event trail; designed as the seam for the future - SQL implementation). -- `model/` — `ConversionJob` lifecycle (`SUBMITTED`, `PROCESSING`, `SUCCEEDED`, - `FAILED`; retry-exhausted jobs stay `FAILED` with `deadLettered=true`). -- `artifact/` — `ArtifactStore` (in-memory PDF bytes), `PdfBoxArtifactGenerator` - (PDFBox conversion stub), `ArtifactLinkService`/`ArtifactLinkLedger` - (short-lived signed artifact tokens, revocation, read-audit ledger). -- `auth/` — tenant enforcement scaffold: protected JSON APIs require - `X-Clearfolio-Tenant-Id`, `X-Clearfolio-Subject-Id`, `X-Clearfolio-Permissions` - headers (optionally gateway-HMAC-signed); cross-tenant jobs are hidden as 404. - This is not production OIDC/JWT validation. -- `analytics/` — KPI snapshot counters and export ledger. -- `api/` — response/request DTOs; `config/` — `ConversionProperties`, executor, - security-headers WebFilter; `exception/` — domain exceptions. - -Request flow: controller validates and enqueues (returns `202` fast; the request -path must never run conversion inline), the worker converts and stores the PDF, -clients poll status, then fetch viewer bootstrap JSON and the artifact via a -signed link. Static viewer assets (PDF.js shell, demo fixtures) live under -`src/main/resources/static/assets/viewer/`. - -Tests mirror the package tree under `src/test/java/`, including -`config/DependencyPolicyTest` (blocks reintroducing `tika-parsers-standard-package`, -the default Logback starter, or the excluded Jakarta annotation dependency) and -the Jazzer fuzz target `controller/FuzzDownloadFilename` (ClusterFuzzLite marker -in `.clusterfuzzlite/`). - -## Key conventions - -- Coverage is absolute: JaCoCo line/branch missed must stay 0 for - `com.clearfolio.viewer.*`, so every production change ships with tests covering - all new lines and branches. -- Every public type/member needs JavaDoc; the JavaDoc gate fails on any warning. -- The compiler runs with `-Xlint:all -Werror` (`showWarnings`/`showDeprecation` - on), so any warning or deprecated usage breaks the build. -- `checkstyle-suppressions.xml` at the repo root relaxes strict style checks - (Javadoc, magic numbers, line length, etc.) for `src/test/java` only; production - sources get no suppressions. -- Markdown lint applies to changed docs; rule overrides are in - `.markdownlint.yaml`. -- Dependency changes are policy-sensitive: security pins and exclusions in - `pom.xml` carry CVE-annotated comments, `osv-scanner.toml` holds narrowly - scoped, time-boxed ignores, and license/SBOM/attribution evidence under `docs/` - must be updated together (see `AGENTS.md` and `DependencyPolicyTest`). -- Gate evidence is committed under `docs/qa/evidence/`; plans and design docs - under `docs/` use dated filenames (`YYYY-MM-DD-...`). -- `.jules/` holds accumulated lessons (performance, XSS, HMAC canonicalization); - worth scanning before touching validation, viewer JS, or token signing code. +All production code lives under `src/main/java/com/clearfolio/viewer/`. + +- `controller/` + - `ConversionController`: submit, status, retry, delete, viewer bootstrap, and + downloadable PDF endpoints. + - `ViewerUiController`: `GET /viewer/{docId}` HTML viewer shell. + - `ArtifactController`: signed artifact reads with single-range support. + - `AdminController`: tenant-scoped privileged job operations. + - `AnalyticsController`: KPI snapshots and evidence exports. + - `HealthController`: `GET /healthz` liveness and `GET /readyz` readiness, + both sourced from Spring Boot `ApplicationAvailability` and marked + `Cache-Control: no-store`. + - `ApiExceptionHandler`: shared error shape with privacy-safe trace evidence. +- `service/` + - `DefaultDocumentValidationService`: upload constraints, extension policy, + and HMAC-verified exception lane. + - `DefaultDocumentConversionService`: validation, tenant-scoped content-hash + dedupe, persistence, PDF passthrough, and worker enqueue. + - `DefaultConversionWorker`: bounded execution, retry backoff, dead-lettering, + and startup recovery for due jobs and stale leases. +- `repository/` + - `ConversionJobRepository`: read, dedupe, and recoverable-job boundary. + - `ConversionJobStateStore`: lifecycle-transition boundary and event trail. +- `model/` + - `ConversionJob`: `SUBMITTED`, `PROCESSING`, `SUCCEEDED`, and `FAILED`. + Retry-exhausted jobs remain `FAILED` with `deadLettered=true`. +- `artifact/` + - `FileSystemArtifactStore`: default restart-surviving PDF store. + - `InMemoryArtifactStore`: test and explicitly selected local store. + - `PdfBoxArtifactGenerator`: placeholder PDF generation for non-PDF sources; + deterministic real Office conversion remains a separate product slice. + - `ArtifactLinkService` and ledgers: short-lived signed links, revocation, and + read-audit evidence. +- `auth/` + - Tenant and permission enforcement for protected JSON APIs. Gateway-signed + header claims are supported; this is not a complete production OIDC/JWT + implementation. +- `analytics/` + - KPI snapshot counters and export evidence. +- `api/`, `config/`, and `exception/` + - DTOs, configuration, WebFlux filters, and domain exceptions. + +The request path must never perform conversion inline. Controllers validate and +submit bounded work, clients poll status, and the viewer retrieves a signed +artifact only after a successful terminal state. + +## Availability semantics + +- `/healthz` answers whether the process is irrecoverably broken and should be + restarted. Never add shared database, object-store, gateway, or model-provider + dependencies to liveness. +- `/readyz` answers whether this instance should receive traffic. Future + instance-local readiness contributors must publish Spring availability events + and include deterministic failure-and-recovery tests. +- Do not configure both Kubernetes probes against `/healthz`. + +## Tests and gates + +Tests mirror the production package tree under `src/test/java/`. Security-sensitive +parsers and filename paths also have Jazzer targets. + +Key rules: + +- `mvn verify` enforces zero missed production lines and branches using JaCoCo + 0.8.15. +- Maven Javadoc Plugin 3.12.0 runs during `verify` with doclint and fails on any + public API documentation warning or error. +- The compiler uses `-Xlint:all -Werror`; warnings and deprecated production API + usage fail the build. +- Every public production type and member requires useful Javadoc. +- Tests must exercise real behavior, including failure, security, concurrency, + and recovery paths; coverage-only assertions must still represent a valid + contract. +- Markdown lint applies to changed documentation. +- Dependency changes must update security, license, SBOM, attribution, and buyer + diligence evidence together when affected. +- Generated evidence containing local paths, credentials, private runtime + details, or customer data remains local until disclosure review approves it. +- Dated plans and decisions use `YYYY-MM-DD-...` filenames. +- `.jules/` contains accumulated lessons for validation, HMAC canonicalization, + viewer security, and performance; inspect it before modifying those areas. diff --git a/README.md b/README.md index 39584e16..c66cb23b 100644 --- a/README.md +++ b/README.md @@ -12,8 +12,10 @@ asynchronous conversion that produces an in-memory PDF artifact for preview. - `mvn test` 3. Start the app locally: - `mvn spring-boot:run` -4. Check readiness: +4. Check process liveness: - `curl -sS http://localhost:8080/healthz` +5. Check traffic readiness: + - `curl -sS http://localhost:8080/readyz` ## Scope @@ -31,7 +33,8 @@ asynchronous conversion that produces an in-memory PDF artifact for preview. - `GET /api/v1/analytics/kpi-snapshot-exports`: tenant-scoped exported KPI snapshot evidence. - `GET /artifacts/{docId}.pdf`: serves converted PDF bytes (SUCCEEDED jobs only) with single-range support after artifact token verification. - Errors follow shared shape (`errorCode`, optional `code`, `message`, `traceId`, `details`) for 404/409/400/500 paths. -- `GET /healthz`: readiness probe. +- `GET /healthz`: process liveness probe driven by Spring Boot `LivenessState`. +- `GET /readyz`: traffic readiness probe driven by Spring Boot `ReadinessState`. - HWP/HWPX are blocked by configuration. Protected JSON APIs require Clearfolio tenant headers in the current buyer-demo @@ -52,6 +55,9 @@ profile and follow - `GET /viewer/{docId}` remains the canonical entry route, but now serves HTML (PDF.js viewer). - Alias endpoints remain stable, with signed artifact link fields added to viewer bootstrap responses. +- `GET /healthz` preserves the successful `{"status":"ok"}` payload while now + reporting liveness; traffic routing must use the separate `GET /readyz` + readiness probe. Both probe responses use `Cache-Control: no-store`. - Dead-letter terminal cases keep `status=FAILED` in API payloads and set `deadLettered=true` when retries are exhausted. - Dead-lettered jobs can be re-queued by an operator with @@ -128,6 +134,7 @@ Current release claim boundary: - `docs/design/2026-07-02-buyer-demo-kpi-figjam-handoff.md` - `docs/deployment/2026-07-02-buyer-deployment-integration-playbook.md` - `docs/deployment/clearfolio-buyer-connector.openapi.yaml` +- `docs/operations/2026-08-05-availability-probes.md` - `docs/persistence/2026-07-02-durable-conversion-job-repository-plan.md` - `docs/diligence/2026-07-02-buyer-diligence-index.md` - `docs/security/2026-07-02-threat-model-data-handling.md` diff --git a/docs/architecture.md b/docs/architecture.md index e570c63a..0702775e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,6 @@ # Conversion Service Architecture -Last updated: 2026-02-23 +Last updated: 2026-08-05 This repository currently ships an MVP backend for integrated document conversion/viewer entry with a non-blocking web stack. @@ -22,7 +22,23 @@ This repository currently ships an MVP backend for integrated document conversio - Viewer UI flow (`GET /viewer/{docId}`): return HTML shell with mobile-safe loading/failed/ready states; when ready, embed PDF.js. - Bootstrap flow (`GET /api/v1/viewer/{docId}` and `GET /api/v1/convert/viewer/{docId}`): return bootstrap JSON on `SUCCEEDED` with deterministic `sourceExtension`/`rendererAdapter`; return `409` for not-ready/failed states; return `404` when missing. - Artifact flow (`GET /artifacts/{docId}.pdf`): serve converted PDF bytes for `SUCCEEDED` jobs only (single-range support). -- Health flow (`GET /healthz`): readiness probe. +- Liveness flow (`GET /healthz`): report Spring Boot `LivenessState`; return `503` when the process is `BROKEN`. +- Readiness flow (`GET /readyz`): report Spring Boot `ReadinessState`; return `503` while the instance refuses traffic. + +## Availability contract + +Clearfolio separates process restart eligibility from traffic routing. Both +probes run on the application port, expose only controlled state labels, and +return `Cache-Control: no-store`. + +- Liveness must not depend on shared external services. A database, gateway, or + object-store outage must not create a restart cascade. +- Readiness represents whether this instance can accept traffic. Future + instance-local readiness contributors, such as completed startup recovery or + bounded-queue overload, must publish Spring availability events and add + deterministic failure-and-recovery tests. +- The accepted decision and deployment example are documented in + `docs/operations/2026-08-05-availability-probes.md`. ## S2S delivery chain (documented target) @@ -48,7 +64,7 @@ Reference policy: `docs/engineering/acceptance-criteria.md`. ## Component boundaries -- `controller`: HTTP endpoints and exception mapping. +- `controller`: HTTP endpoints, availability probes, and exception mapping. - `service`: validation, policy-override exception lane handling, conversion orchestration, worker execution. - `repository`: job persistence abstraction. - `model`: lifecycle state and retry/dead-letter metadata. @@ -89,6 +105,8 @@ Reference policy: `docs/engineering/acceptance-criteria.md`. | --- | --- | | WebFlux dependency | `pom.xml` | | Submit non-blocking controller path | `src/main/java/com/clearfolio/viewer/controller/ConversionController.java` | +| Liveness/readiness probes | `src/main/java/com/clearfolio/viewer/controller/HealthController.java` | +| Availability probe tests | `src/test/java/com/clearfolio/viewer/controller/HealthControllerTest.java` | | Blocked-format override lane + audit signal | `src/main/java/com/clearfolio/viewer/service/DefaultDocumentValidationService.java` | | Override header contract | `src/main/java/com/clearfolio/viewer/service/PolicyOverrideRequest.java` | | Conversion enqueue orchestration | `src/main/java/com/clearfolio/viewer/service/DefaultDocumentConversionService.java` | @@ -110,4 +128,5 @@ Reference policy: `docs/engineering/acceptance-criteria.md`. - `docs/diagrams/preview-flow.md` - `docs/diagrams/submit-policy-adapter-flow.md` - `docs/diagrams/retry-deadletter-flow.md` +- `docs/operations/2026-08-05-availability-probes.md` - `docs/persistence/2026-07-02-durable-conversion-job-repository-plan.md` diff --git a/docs/security/2026-07-02-threat-model-data-handling.md b/docs/security/2026-07-02-threat-model-data-handling.md index bc951abe..d9905246 100644 --- a/docs/security/2026-07-02-threat-model-data-handling.md +++ b/docs/security/2026-07-02-threat-model-data-handling.md @@ -33,7 +33,8 @@ Primary runtime surfaces: snapshot ledger. - `GET /api/v1/analytics/kpi-snapshot-exports`: tenant-scoped exported KPI snapshot evidence without raw document content. -- `GET /healthz`: readiness probe. +- `GET /healthz`: process liveness probe driven by Spring Boot `LivenessState`. +- `GET /readyz`: traffic readiness probe driven by Spring Boot `ReadinessState`. The current security posture is MVP-grade and evidence-oriented. It has bounded upload size, blocked HWP/HWPX defaults, policy override audit From 4d8b1d43f581e1f84ef6703948cba37b86b26a52 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Thu, 6 Aug 2026 12:08:10 +0900 Subject: [PATCH 3/4] docs(acceptance): require distinct availability probes --- docs/engineering/acceptance-criteria.md | 34 +++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/docs/engineering/acceptance-criteria.md b/docs/engineering/acceptance-criteria.md index dd50c5ab..f1d9ac86 100644 --- a/docs/engineering/acceptance-criteria.md +++ b/docs/engineering/acceptance-criteria.md @@ -31,11 +31,32 @@ a coordinated update to `AGENTS.md`, `CLAUDE.md`, and both architecture maps. and expose status, retry, viewer, and artifact workflows rather than waiting for conversion completion. +## Availability contract + +- `GET /healthz` is process liveness, sourced from Spring Boot + `LivenessState`. It returns `200 {"status":"ok"}` only while the process is + `CORRECT`; `BROKEN` returns `503 {"status":"broken"}`. +- `GET /readyz` is traffic readiness, sourced from Spring Boot + `ReadinessState`. It returns `200 {"status":"ready"}` only while the + instance is `ACCEPTING_TRAFFIC`; otherwise it returns + `503 {"status":"not_ready"}`. +- Both responses use `Cache-Control: no-store` and expose only controlled state + labels. +- Liveness must not depend on a shared database, object store, gateway, model + provider, or another external service. A shared-service outage is not by + itself evidence that this process requires restart. +- Future readiness contributors must be instance-local routing conditions, + publish Spring availability events, and include deterministic failure and + recovery tests. +- The accepted decision, rollback rule, Kubernetes example, and authoritative + references are recorded in + `docs/operations/2026-08-05-availability-probes.md`. + ## Delivery context chain - `Clearfolio Viewer <-> internal WAS -> Azure On-premise Gateway -> Power Platform -> mobile/tablet` - This repository owns the Clearfolio Viewer side of the contract and its state, - authorization, document, artifact, and operational gates. + authorization, document, artifact, availability, and operational gates. ## Required local acceptance commands @@ -69,7 +90,7 @@ code can write report files. | --- | --- | --- | | coverage | JaCoCo 0.8.15 applies bundle-level `LINE` and `BRANCH` `MISSEDCOUNT` limits with a maximum of `0` | `mvn -B --no-transfer-progress verify`; inspect `target/site/jacoco/jacoco.csv` and the exact-head CI job | | docstring | Maven Javadoc Plugin 3.12.0 runs Java 21 doclint for public production APIs and fails on warnings or errors | `mvn -B --no-transfer-progress verify`; inspect `target/reports/apidocs` and the exact-head CI job | -| non-blocking web | Request paths do not execute document conversion inline | `ConversionController`, `DefaultDocumentConversionService`, and their concurrency/integration tests | +| non-blocking web | Request paths do not execute document conversion inline; liveness and readiness remain separate non-blocking probes | `ConversionController`, `DefaultDocumentConversionService`, `HealthController`, and their concurrency/integration tests | | lightweight queue | Capacity, rejection, retry, processing lease, and dead-letter behavior are executable contracts | `ConversionExecutorConfig`, `DefaultConversionWorker`, repository/state-store tests, and exact-head fuzzing | | warning 0 | Java compilation uses `-Xlint:all -Werror`; Maven report acceptance rejects skipped and zero-test evidence | `mvn -B --no-transfer-progress verify`, `python3 scripts/verify_maven_test_reports.py`, and exact-head CI | | deprecated 0 | Deprecated API warnings are build failures | `mvn -B --no-transfer-progress verify` | @@ -109,6 +130,7 @@ code can write report files. - Root architecture map: `ARCHITECTURE.md`. - Detailed architecture: `docs/architecture.md`. +- Availability decision: `docs/operations/2026-08-05-availability-probes.md`. ## References @@ -120,5 +142,13 @@ Apache Software Foundation. (2026). *Surefire reports*. Maven Surefire Plugin. Retrieved August 6, 2026, from https://maven.apache.org/surefire/maven-surefire-plugin/examples/reporting.html +Broadcom, Inc. (n.d.). *SpringApplication: Application availability (Spring Boot +3.5.16)*. Spring. Retrieved August 5, 2026, from +https://docs.spring.io/spring-boot/3.5/reference/features/spring-application.html#features.spring-application.application-availability + JaCoCo. (2026). *JaCoCo Maven plug-in: `jacoco:check`*. Retrieved August 5, 2026, from https://www.jacoco.org/jacoco/trunk/doc/check-mojo.html + +The Kubernetes Authors. (2026, April 17). *Liveness, readiness, and startup +probes*. Kubernetes. +https://kubernetes.io/docs/concepts/workloads/pods/probes/ From 592fa1799fd1d83ab80b6532fb7ba49ba4d61f9d Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Thu, 6 Aug 2026 12:09:30 +0900 Subject: [PATCH 4/4] docs(changelog): record separate availability probes --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b8f8da2b..a2039f3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ ### Added - **UI UX 개선**: 'Details' 버튼 클릭 시, 작업 상세 정보 로드 중에 사용자가 명시적인 로딩 상태를 확인할 수 있도록 'Loading...' 텍스트와 비활성화 상태를 표시하도록 추가했습니다. +- `GET /readyz` traffic-readiness probe를 추가하고 기존 `GET /healthz`를 Spring Boot `LivenessState` 기반 process-liveness probe로 명확히 분리했습니다. 두 경로는 `ApplicationAvailability` 상태를 사용하고 성공·실패 상태 코드와 제한된 응답 payload를 결정적 테스트로 고정합니다. - **관리자용 단건 작업 삭제 및 재시도 API 추가** - 특정 변환 작업을 삭제할 수 있는 `DELETE /api/v1/admin/convert/jobs/{jobId}` 엔드포인트를 추가했습니다. - 실패(dead-lettered) 상태인 작업을 관리자가 재시도 큐에 등록할 수 있는 `POST /api/v1/admin/convert/jobs/{jobId}/retry` 엔드포인트를 추가했습니다. @@ -24,6 +25,7 @@ ### Security +- `/healthz`와 `/readyz`는 tenant, document, queue, dependency, credential, build 또는 exception 세부정보를 노출하지 않고 `Cache-Control: no-store`를 사용합니다. Liveness에는 shared external service 의존성을 추가하지 않아 외부 장애가 restart cascade로 증폭되는 것을 방지합니다. - Maven XML 테스트 보고서 검증기는 각 `testsuite`의 `tests`, `skipped`, `failures`, `errors` 속성을 모두 필수 증거로 요구합니다. 누락된 결과 수를 암묵적으로 0으로 간주하지 않고 fail closed 처리하며, 각 속성 누락 회귀 테스트를 추가했습니다. - Maven XML 테스트 보고서 검증기는 UTF-8만 허용하고 UTF-8 BOM은 수용하며, NUL 바이트·DTD·엔터티 선언을 파싱 전에 거부합니다. UTF-16 같은 대체 인코딩으로 위험 선언을 바이트 검사에서 숨기는 우회와 외부 엔터티 읽기·엔터티 확장형 서비스 거부를 회귀 테스트로 차단했습니다. - Maven XML 테스트 보고서 검증기는 파일당 16 MiB 상한을 적용하고 한 번의 제한된 읽기로 실제 입력 크기를 검증합니다. 테스트 코드가 보고서 파일을 교체하거나 확장해도 크기 사전검사와 파싱 사이의 경쟁 조건을 이용할 수 없습니다.