Skip to content

[CMPT-5391] feat(pipelines): add dr pipeline CRUD, version, and graph commands - #532

Merged
sunny2get merged 15 commits into
datarobot-oss:mainfrom
sunny2get:cmpt-5391/pipelines-crud
Jun 1, 2026
Merged

[CMPT-5391] feat(pipelines): add dr pipeline CRUD, version, and graph commands#532
sunny2get merged 15 commits into
datarobot-oss:mainfrom
sunny2get:cmpt-5391/pipelines-crud

Conversation

@sunny2get

@sunny2get sunny2get commented May 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds dr pipeline CRUD commands: create, get, list, update, del, lock, graph
  • Adds dr pipeline version subcommands: get, list
  • Adds telemetry (telemetry.TrackWith) to all pipeline and version commands
  • All commands hidden behind DATAROBOT_CLI_FEATURE_PIPELINE=true feature gate

Stacked on

Next PRs in stack

  • cmpt-5391/pipelines-runsdr pipeline run subcommands
  • cmpt-5391/pipelines-inputsdr pipeline input subcommands
  • cmpt-5391/pipelines-schedulesdr pipeline schedule subcommands
  • cmpt-5391/pipelines-environmentsdr pipeline environment subcommands

Test plan

  • go build ./... passes
  • task lint passes
  • go test ./cmd/pipeline/... passes
  • DATAROBOT_CLI_FEATURE_PIPELINE=true dr pipeline --help shows all subcommands
  • DATAROBOT_CLI_FEATURE_PIPELINE=true dr pipeline version --help shows get/list

🤖 Generated with Claude Code


Note

Medium Risk
Introduces many authenticated API operations including delete and lock, but behavior is gated, well-tested, and mostly read-only or idempotent-friendly 404 handling.

Overview
This PR turns the previously empty dr pipeline group into a full Pipelines API CLI (still behind DATAROBOT_CLI_FEATURE_PIPELINE), wiring create, list, get, update, delete, lock, graph, and a version subgroup (list / get).

Commands call new internal/pipeline clients for /api/v2/pipelines (multipart upload for create/update, JSON elsewhere), shared --output-format, fileutil path resolution, and scopeflag draft/locked routing for graph. Several verbs treat 404 as a dim message and exit 0; drapi.HTTPError now carries API detail text. Docs add pipeline usage and an endpoint reference.

Telemetry is attached to each new command; coverage is mostly unit tests around flags, rendering, and HTTP helpers.

Reviewed by Cursor Bugbot for commit 6393067. Configure here.

Comment thread internal/pipelines/transport.go Outdated
Comment thread cmd/pipeline/cmd_test.go
@sunny2get
sunny2get force-pushed the cmpt-5391/pipelines-crud branch from 3b4d4c4 to 6327eb4 Compare May 28, 2026 14:46
@sunny2get

Copy link
Copy Markdown
Contributor Author

bugbot run

Comment thread cmd/pipeline/create/cmd.go
Comment thread cmd/pipeline/create/cmd.go Outdated
@sunny2get
sunny2get force-pushed the cmpt-5391/pipelines-crud branch from 5322202 to a4d9113 Compare May 28, 2026 15:32
@sunny2get

Copy link
Copy Markdown
Contributor Author

bugbot run

Comment thread internal/pipeline/pipeline.go
@github-actions

Copy link
Copy Markdown
Contributor

🔐 Fork smoke tests started by maintainer

⏳ Security scans passed. Running smoke tests...

Commit: a4d91132bde0b0279cf7319507c89ec22d2b8fa7
View run

@github-actions

Copy link
Copy Markdown
Contributor

All smoke tests passed! (Fork PR)

✅ Security Scan: success
✅ Linux: success
✅ Windows: success

View run details

@sunny2get

Copy link
Copy Markdown
Contributor Author

bugbot run

@github-actions

Copy link
Copy Markdown
Contributor

🔐 Fork smoke tests started by maintainer

⏳ Security scans passed. Running smoke tests...

Commit: 6393067f3c4b8242f207ae820868ddd4effa39f6
View run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 6393067. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

All smoke tests passed! (Fork PR)

✅ Security Scan: success
✅ Linux: success
✅ Windows: success

View run details

@ajalon1 ajalon1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Everything looks pretty consistent. Thanks for the extensive docs for each command.

Also, telemetry updates look straightforward.

Few minor nits, but nothing blocking.

Comment thread cmd/pipeline/create/cmd.go Outdated
Comment thread cmd/pipeline/graph/cmd_test.go Outdated
Comment thread cmd/pipeline/list/cmd.go Outdated
Comment thread cmd/pipeline/graph/cmd.go Outdated
@sunny2get
sunny2get force-pushed the cmpt-5391/pipelines-crud branch 2 times, most recently from a4f522a to 00e1376 Compare May 29, 2026 18:37
@github-actions

Copy link
Copy Markdown
Contributor

🔐 Fork smoke tests started by maintainer

⏳ Security scans passed. Running smoke tests...

Commit: a4f522ae61c94992fe7e0d389baedf5c1a0ec93a
View run

@github-actions

Copy link
Copy Markdown
Contributor

All smoke tests passed! (Fork PR)

✅ Security Scan: success
✅ Linux: success
✅ Windows: success

View run details

sunny2get and others added 12 commits June 1, 2026 10:11
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces the core pipeline management subcommands:

- create / list / get / update / delete / lock — full pipeline lifecycle
- version list / version get — inspect pipeline versions
- graph — render the pipeline/task DAG (draft or locked scope)
- scopeflag package — shared --pipeline/--scope/--version flag binding

Internal additions:
- internal/pipelines/pipeline.go: CreatePipeline, ListPipelines, GetPipeline,
  UpdatePipeline, DeletePipeline, LockPipeline, multipart upload helpers
- internal/pipelines/version.go: ListVersions, GetVersion, GetGraph

Documentation: pipelines.md CRUD sections, initial pipelines-reference.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
First used by the graph command on this branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Update all cmd/pipeline subcommand files to use internal/pipeline
(singular) package import and pipeline.X symbol references after the
internal/pipelines → internal/pipeline rename.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n, limit defaults, version prefix

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…up commands

Add: alias check, RunE=nil guard, full subcommand list (del/lock/version/graph),
version sub-subcommand registration, and version metadata assertions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace all dr pipelines → dr pipeline and
DATAROBOT_CLI_FEATURE_PIPELINES → DATAROBOT_CLI_FEATURE_PIPELINE
to match the registered command name and feature gate key.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Path

- Add SilenceUsage: true to create, list, and update commands (consistent with del, get, lock, graph, version get/list)
- Extract duplicated resolveFilePath into cmd/pipeline/fileutil package
- Move TestResolveFilePath tests to fileutil package

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
decodeHTTPError was returning fmt.Errorf when the response body had a
"detail" field, so errors.As(err, &httpErr) always failed on FastAPI
404s — users saw raw "HTTP 404 Not Found: Not Found" instead of the
friendly "No pipeline found with id: ..." message.

Fix: always return *drapi.HTTPError (with Detail field carrying the
message). errors.As continues to work for 404 suppression in del/get/graph.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sunny2get and others added 2 commits June 1, 2026 10:11
Covers RenderCreateResponse, RenderPipeline, and RenderPipelines for
both JSON and human output paths, including edge cases (empty list,
no tasks, no description, no latest version, error detail).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eStdout, MarkFlagRequired for graph

- Standardize --mode flag description across create and list to "Pipeline mode: draft or locked"
- Fix list Short description: "List pipeline." -> "List pipelines."
- Replace manual --pipeline guard in graph/cmd.go with cmd.MarkFlagRequired("pipeline")
- Extract captureStdout into cmd/pipeline/internal/testutil.CaptureStdout and remove 6 duplicate definitions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sunny2get
sunny2get force-pushed the cmpt-5391/pipelines-crud branch from 00e1376 to 051edb8 Compare June 1, 2026 14:11
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sunny2get
sunny2get merged commit 5a918a6 into datarobot-oss:main Jun 1, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants