-
Notifications
You must be signed in to change notification settings - Fork 1
COR-1766: report dirty worktree state with corgea scan uploads #150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,6 +18,8 @@ use std::path::Path; | |
|
|
||
| const CHUNK_SIZE: usize = 50 * 1024 * 1024; // 50 MB | ||
| const API_BASE: &str = "/api/v1"; | ||
| const DIRTY_TRUE: &str = "true"; | ||
| const DIRTY_FALSE: &str = "false"; | ||
|
|
||
| fn auth_headers(token: &str) -> HeaderMap { | ||
| let mut headers = HeaderMap::new(); | ||
|
|
@@ -336,6 +338,11 @@ pub fn upload_zip( | |
| if let Some(sha) = &info.sha { | ||
| form = form.part("sha", multipart::Part::text(sha.to_string())); | ||
| } | ||
| // Always send dirty: omitted field = old CLI; "false" = clean tree. | ||
| form = form.part( | ||
| "dirty", | ||
| multipart::Part::text(if info.dirty { DIRTY_TRUE } else { DIRTY_FALSE }), | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This always equates “clean Git status” with “archive represents clean HEAD,” but There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with this finding and think it should be addressed. high: Partial archives are incorrectly marked as clean HEAD The multipart dirty value depends exclusively on Git worktree status. A clean scan using --target or --exclude therefore sends dirty=false even though its archive is not a complete representation of HEAD. The effective upload state must be dirty when packaging options omit tracked content. Proof or reproduction: |
||
| ); | ||
| } | ||
| if let Some(scan_type) = scan_type.clone() { | ||
| let scan_type = if scan_type.contains("blast") { | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,5 +1,5 @@ | ||||||||||||||||
| use crate::utils::terminal::{set_text_color, TerminalColor}; | ||||||||||||||||
| use git2::Repository; | ||||||||||||||||
| use git2::{Repository, StatusOptions}; | ||||||||||||||||
| use globset::{Glob, GlobSetBuilder}; | ||||||||||||||||
| use ignore::WalkBuilder; | ||||||||||||||||
| use std::env; | ||||||||||||||||
|
|
@@ -297,13 +297,33 @@ pub fn get_repo_info(dir: &str) -> Result<Option<RepoInfo>, git2::Error> { | |||||||||||||||
| .map(|commit| commit.id().to_string()) | ||||||||||||||||
| }); | ||||||||||||||||
|
|
||||||||||||||||
| let dirty = is_worktree_dirty(&repo); | ||||||||||||||||
|
|
||||||||||||||||
| Ok(Some(RepoInfo { | ||||||||||||||||
| branch, | ||||||||||||||||
| repo_url: origin_url(&repo), | ||||||||||||||||
| sha, | ||||||||||||||||
| dirty, | ||||||||||||||||
| })) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| /// True when the worktree has modified, staged, or untracked files. | ||||||||||||||||
| /// Gitignored paths alone do not count; submodules are excluded. | ||||||||||||||||
| /// | ||||||||||||||||
| /// Untracked paths that packaging would later drop via `DEFAULT_EXCLUDE_GLOBS` | ||||||||||||||||
| /// still count as dirty (false-positive dirty costs a full scan, not a miss). | ||||||||||||||||
| /// Status errors also treat the tree as dirty so we never claim clean HEAD. | ||||||||||||||||
| fn is_worktree_dirty(repo: &Repository) -> bool { | ||||||||||||||||
| let mut opts = StatusOptions::new(); | ||||||||||||||||
| opts.include_untracked(true) | ||||||||||||||||
| .recurse_untracked_dirs(true) | ||||||||||||||||
| .include_ignored(false) | ||||||||||||||||
| .exclude_submodules(true); | ||||||||||||||||
|
Comment on lines
+318
to
+321
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with this finding and think it should be addressed. high: Modified submodule contents are hidden from dirty detection exclude_submodules(true) prevents submodule changes from making the status nonempty. Because packaging traverses submodule contents, modified bytes can consequently be uploaded with the parent SHA and dirty=false. Include submodule status or exclude submodule content from the archive. Proof or reproduction: |
||||||||||||||||
| repo.statuses(Some(&mut opts)) | ||||||||||||||||
| .map(|s| !s.is_empty()) | ||||||||||||||||
| .unwrap_or(true) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| /// `origin`'s URL, or None when the remote is missing or carries no URL. | ||||||||||||||||
| fn origin_url(repo: &Repository) -> Option<String> { | ||||||||||||||||
| repo.find_remote("origin") | ||||||||||||||||
|
|
@@ -412,6 +432,7 @@ pub struct RepoInfo { | |||||||||||||||
| pub branch: Option<String>, | ||||||||||||||||
| pub repo_url: Option<String>, | ||||||||||||||||
| pub sha: Option<String>, | ||||||||||||||||
| pub dirty: bool, | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[cfg(test)] | ||||||||||||||||
|
|
@@ -440,12 +461,7 @@ mod tests { | |||||||||||||||
| fn get_repo_info_at_root_only_not_nested_cwd() { | ||||||||||||||||
| let dir = tempfile::tempdir().unwrap(); | ||||||||||||||||
| let root = dir.path(); | ||||||||||||||||
| git(root, &["init"]); | ||||||||||||||||
| git(root, &["config", "user.email", "test@example.com"]); | ||||||||||||||||
| git(root, &["config", "user.name", "Test"]); | ||||||||||||||||
| fs::write(root.join("README"), "hi").unwrap(); | ||||||||||||||||
| git(root, &["add", "README"]); | ||||||||||||||||
| git(root, &["commit", "-m", "init"]); | ||||||||||||||||
| init_committed_repo(root); | ||||||||||||||||
|
|
||||||||||||||||
| let root_s = root.to_str().unwrap(); | ||||||||||||||||
| let nested = root.join("pkg").join("inner"); | ||||||||||||||||
|
|
@@ -456,6 +472,7 @@ mod tests { | |||||||||||||||
| .unwrap() | ||||||||||||||||
| .expect("repo root should yield SHA metadata"); | ||||||||||||||||
| assert!(info.sha.is_some()); | ||||||||||||||||
| assert!(!info.dirty, "clean commit should report dirty=false"); | ||||||||||||||||
| assert!(is_at_repo_root(root_s)); | ||||||||||||||||
|
|
||||||||||||||||
| assert!( | ||||||||||||||||
|
|
@@ -465,6 +482,67 @@ mod tests { | |||||||||||||||
| assert!(!is_at_repo_root(nested_s)); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| fn init_committed_repo(root: &std::path::Path) { | ||||||||||||||||
| git(root, &["init"]); | ||||||||||||||||
| git(root, &["config", "user.email", "test@example.com"]); | ||||||||||||||||
| git(root, &["config", "user.name", "Test"]); | ||||||||||||||||
| fs::write(root.join("README"), "hi").unwrap(); | ||||||||||||||||
| git(root, &["add", "README"]); | ||||||||||||||||
| git(root, &["commit", "-m", "init"]); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn get_repo_info_dirty_true_when_tracked_file_modified() { | ||||||||||||||||
| let dir = tempfile::tempdir().unwrap(); | ||||||||||||||||
| let root = dir.path(); | ||||||||||||||||
| init_committed_repo(root); | ||||||||||||||||
| fs::write(root.join("README"), "changed").unwrap(); | ||||||||||||||||
| let info = get_repo_info(root.to_str().unwrap()) | ||||||||||||||||
| .unwrap() | ||||||||||||||||
| .expect("repo info"); | ||||||||||||||||
| assert!(info.dirty); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn get_repo_info_dirty_true_when_change_staged() { | ||||||||||||||||
| let dir = tempfile::tempdir().unwrap(); | ||||||||||||||||
| let root = dir.path(); | ||||||||||||||||
| init_committed_repo(root); | ||||||||||||||||
| fs::write(root.join("README"), "staged").unwrap(); | ||||||||||||||||
| git(root, &["add", "README"]); | ||||||||||||||||
| let info = get_repo_info(root.to_str().unwrap()) | ||||||||||||||||
| .unwrap() | ||||||||||||||||
| .expect("repo info"); | ||||||||||||||||
| assert!(info.dirty); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn get_repo_info_dirty_true_when_untracked_file() { | ||||||||||||||||
| let dir = tempfile::tempdir().unwrap(); | ||||||||||||||||
| let root = dir.path(); | ||||||||||||||||
| init_committed_repo(root); | ||||||||||||||||
| fs::write(root.join("new.py"), "print(1)").unwrap(); | ||||||||||||||||
| let info = get_repo_info(root.to_str().unwrap()) | ||||||||||||||||
| .unwrap() | ||||||||||||||||
| .expect("repo info"); | ||||||||||||||||
| assert!(info.dirty); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn get_repo_info_dirty_false_when_only_gitignored_file() { | ||||||||||||||||
| let dir = tempfile::tempdir().unwrap(); | ||||||||||||||||
| let root = dir.path(); | ||||||||||||||||
| init_committed_repo(root); | ||||||||||||||||
| fs::write(root.join(".gitignore"), "ignored.txt\n").unwrap(); | ||||||||||||||||
| git(root, &["add", ".gitignore"]); | ||||||||||||||||
| git(root, &["commit", "-m", "ignore"]); | ||||||||||||||||
| fs::write(root.join("ignored.txt"), "secret").unwrap(); | ||||||||||||||||
| let info = get_repo_info(root.to_str().unwrap()) | ||||||||||||||||
| .unwrap() | ||||||||||||||||
| .expect("repo info"); | ||||||||||||||||
| assert!(!info.dirty); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn create_zip_from_target_excludes_default_globs() { | ||||||||||||||||
| let dir = tempfile::tempdir().unwrap(); | ||||||||||||||||
|
|
||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
src/scanners/blast.rs:170finishes reading the archive before this metadata sample. If another process completes a commit between those points (a normal possibility in CI/watch workflows), this reads the new clean HEAD andupload_zipsends that new SHA withdirty=false, even though the zip contains the previous/mixed snapshot. That makes the backend's commit-diff incremental decision unsound and can skip analysis of files whose uploaded bytes do not match the advertised commit. Capture repo state both immediately before and after packaging, and only advertise a clean snapshot when both samples are clean and have the same SHA; otherwise fail safe todirty=true. Please cover the reconciliation logic with before/after SHA-change cases.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with this finding and think it should be addressed.
high: Post-packaging Git state may not match archived files
Repository metadata is sampled only after packaging. If HEAD changes while the archive is being created, the upload can advertise the new SHA with dirty=false even though the archive contains files from the previous or a mixed state. Compare state before and after packaging, marking the upload dirty unless both clean samples have the same SHA.
Proof or reproduction: