Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion src/scanners/blast.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ pub fn run(
fs::create_dir_all(&temp_dir).expect("Failed to create temp directory");
let project_name = utils::generic::determine_project_name(project_name.as_deref());
let zip_path = format!("{}/{}.zip", temp_dir.display(), project_name);
let repo_info = utils::generic::get_repo_info("./").unwrap_or_default();
match utils::generic::create_path_if_not_exists(&temp_dir) {
Ok(_) => (),
Err(e) => {
Expand Down Expand Up @@ -207,6 +206,23 @@ pub fn run(
"\r{}Project packaged successfully.\n",
utils::terminal::set_text_color("", utils::terminal::TerminalColor::Green)
);
// Read dirty/sha after packaging so the flag matches the uploaded archive.
let repo_info = utils::generic::get_repo_info("./").unwrap_or_default();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/scanners/blast.rs:170 finishes reading the archive before this metadata sample. If another process completes a commit between those points (a normal possibility in CI/watch workflows), this reads the new clean HEAD and upload_zip sends that new SHA with dirty=false, even though the zip contains the previous/mixed snapshot. That makes the backend's commit-diff incremental decision unsound and can skip analysis of files whose uploaded bytes do not match the advertised commit. Capture repo state both immediately before and after packaging, and only advertise a clean snapshot when both samples are clean and have the same SHA; otherwise fail safe to dirty=true. Please cover the reconciliation logic with before/after SHA-change cases.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

high: Post-packaging Git state may not match archived files

Repository metadata is sampled only after packaging. If HEAD changes while the archive is being created, the upload can advertise the new SHA with dirty=false even though the archive contains files from the previous or a mixed state. Compare state before and after packaging, marking the upload dirty unless both clean samples have the same SHA.

Proof or reproduction:

Start packaging clean commit A, pause after one file is archived, commit changes as B, then resume. The new code observes clean commit B and sends sha=B, dirty=false, although the archive is not commit B.

if let Some(ref info) = repo_info {
if info.dirty {
match info.sha.as_deref() {
Some(sha) => {
let short_sha = &sha[..sha.len().min(7)];
println!(
"Working tree has uncommitted changes - scanning your local files, not commit {short_sha}."
);
}
None => {
println!("Working tree has uncommitted changes - scanning your local files.")
}
}
}
}
println!("\n\nSubmitting scan to Corgea:");
let upload_result = match utils::api::upload_zip(
&zip_path,
Expand Down
7 changes: 7 additions & 0 deletions src/utils/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ use std::path::Path;

const CHUNK_SIZE: usize = 50 * 1024 * 1024; // 50 MB
const API_BASE: &str = "/api/v1";
const DIRTY_TRUE: &str = "true";
const DIRTY_FALSE: &str = "false";

fn auth_headers(token: &str) -> HeaderMap {
let mut headers = HeaderMap::new();
Expand Down Expand Up @@ -336,6 +338,11 @@ pub fn upload_zip(
if let Some(sha) = &info.sha {
form = form.part("sha", multipart::Part::text(sha.to_string()));
}
// Always send dirty: omitted field = old CLI; "false" = clean tree.
form = form.part(
"dirty",
multipart::Part::text(if info.dirty { DIRTY_TRUE } else { DIRTY_FALSE }),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This always equates “clean Git status” with “archive represents clean HEAD,” but blast.rs:87-90 can build a --target archive and blast.rs:170 can omit user---exclude files. Neither selection is sent to the server. Therefore a clean corgea scan blast --target src/a.py (or a clean scan with --exclude) now sends dirty=false for a partial archive and opts it into commit-diff incremental behavior as though it were the full commit. Results can be reused or diffed outside the requested archive scope. Keep the user notice based on actual worktree status, but force the upload's effective dirty/“not exact HEAD” state to true whenever target_str.is_some() or exclude.is_some(); add clean-worktree E2E cases for both options.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

high: Partial archives are incorrectly marked as clean HEAD

The multipart dirty value depends exclusively on Git worktree status. A clean scan using --target or --exclude therefore sends dirty=false even though its archive is not a complete representation of HEAD. The effective upload state must be dirty when packaging options omit tracked content.

Proof or reproduction:

In a clean repository containing src/a.py and src/b.py, run `corgea scan blast --target src/a.py`. `info.dirty` is false, so the changed code sends `dirty=false`, while the archive omits the tracked src/b.py.

);
}
if let Some(scan_type) = scan_type.clone() {
let scan_type = if scan_type.contains("blast") {
Expand Down
92 changes: 85 additions & 7 deletions src/utils/generic.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::utils::terminal::{set_text_color, TerminalColor};
use git2::Repository;
use git2::{Repository, StatusOptions};
use globset::{Glob, GlobSetBuilder};
use ignore::WalkBuilder;
use std::env;
Expand Down Expand Up @@ -297,13 +297,33 @@ pub fn get_repo_info(dir: &str) -> Result<Option<RepoInfo>, git2::Error> {
.map(|commit| commit.id().to_string())
});

let dirty = is_worktree_dirty(&repo);

Ok(Some(RepoInfo {
branch,
repo_url: origin_url(&repo),
sha,
dirty,
}))
}

/// True when the worktree has modified, staged, or untracked files.
/// Gitignored paths alone do not count; submodules are excluded.
///
/// Untracked paths that packaging would later drop via `DEFAULT_EXCLUDE_GLOBS`
/// still count as dirty (false-positive dirty costs a full scan, not a miss).
/// Status errors also treat the tree as dirty so we never claim clean HEAD.
fn is_worktree_dirty(repo: &Repository) -> bool {
let mut opts = StatusOptions::new();
opts.include_untracked(true)
.recurse_untracked_dirs(true)
.include_ignored(false)
.exclude_submodules(true);
Comment on lines +318 to +321

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exclude_submodules(true) hides a dirty checked-out submodule from this status, while the archive walker at generic.rs:77 still traverses that directory and packages its source files. Thus modified submodule bytes can be uploaded with the parent SHA and dirty=false, allowing an incorrect incremental scan. Include submodule status (or, less usefully, exclude submodule contents from packaging); the minimal safe fix is:

Suggested change
opts.include_untracked(true)
.recurse_untracked_dirs(true)
.include_ignored(false)
.exclude_submodules(true);
opts.include_untracked(true)
.recurse_untracked_dirs(true)
.include_ignored(false);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this finding and think it should be addressed.

high: Modified submodule contents are hidden from dirty detection

exclude_submodules(true) prevents submodule changes from making the status nonempty. Because packaging traverses submodule contents, modified bytes can consequently be uploaded with the parent SHA and dirty=false. Include submodule status or exclude submodule content from the archive.

Proof or reproduction:

Create and commit a submodule, modify a tracked file inside its checkout without updating the parent index, then scan. The status query excludes that submodule and returns empty, causing dirty=false despite the modified file being packaged.

repo.statuses(Some(&mut opts))
.map(|s| !s.is_empty())
.unwrap_or(true)
}

/// `origin`'s URL, or None when the remote is missing or carries no URL.
fn origin_url(repo: &Repository) -> Option<String> {
repo.find_remote("origin")
Expand Down Expand Up @@ -412,6 +432,7 @@ pub struct RepoInfo {
pub branch: Option<String>,
pub repo_url: Option<String>,
pub sha: Option<String>,
pub dirty: bool,
}

#[cfg(test)]
Expand Down Expand Up @@ -440,12 +461,7 @@ mod tests {
fn get_repo_info_at_root_only_not_nested_cwd() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
git(root, &["init"]);
git(root, &["config", "user.email", "test@example.com"]);
git(root, &["config", "user.name", "Test"]);
fs::write(root.join("README"), "hi").unwrap();
git(root, &["add", "README"]);
git(root, &["commit", "-m", "init"]);
init_committed_repo(root);

let root_s = root.to_str().unwrap();
let nested = root.join("pkg").join("inner");
Expand All @@ -456,6 +472,7 @@ mod tests {
.unwrap()
.expect("repo root should yield SHA metadata");
assert!(info.sha.is_some());
assert!(!info.dirty, "clean commit should report dirty=false");
assert!(is_at_repo_root(root_s));

assert!(
Expand All @@ -465,6 +482,67 @@ mod tests {
assert!(!is_at_repo_root(nested_s));
}

fn init_committed_repo(root: &std::path::Path) {
git(root, &["init"]);
git(root, &["config", "user.email", "test@example.com"]);
git(root, &["config", "user.name", "Test"]);
fs::write(root.join("README"), "hi").unwrap();
git(root, &["add", "README"]);
git(root, &["commit", "-m", "init"]);
}

#[test]
fn get_repo_info_dirty_true_when_tracked_file_modified() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
fs::write(root.join("README"), "changed").unwrap();
let info = get_repo_info(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(info.dirty);
}

#[test]
fn get_repo_info_dirty_true_when_change_staged() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
fs::write(root.join("README"), "staged").unwrap();
git(root, &["add", "README"]);
let info = get_repo_info(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(info.dirty);
}

#[test]
fn get_repo_info_dirty_true_when_untracked_file() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
fs::write(root.join("new.py"), "print(1)").unwrap();
let info = get_repo_info(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(info.dirty);
}

#[test]
fn get_repo_info_dirty_false_when_only_gitignored_file() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
fs::write(root.join(".gitignore"), "ignored.txt\n").unwrap();
git(root, &["add", ".gitignore"]);
git(root, &["commit", "-m", "ignore"]);
fs::write(root.join("ignored.txt"), "secret").unwrap();
let info = get_repo_info(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(!info.dirty);
}

#[test]
fn create_zip_from_target_excludes_default_globs() {
let dir = tempfile::tempdir().unwrap();
Expand Down
20 changes: 15 additions & 5 deletions tests/cloud_commands_e2e/common/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -767,12 +767,17 @@ pub(crate) fn assert_issue_summary(stdout: &str, context: &str) {
}

pub(crate) fn blast_plan(sha: &str) -> Vec<ExpectedRequest> {
blast_upload_plan(sha, false, true)
}

/// BLAST upload contract. `include_sca` covers `--fail-on malicious` (SCA fetch).
pub(crate) fn blast_upload_plan(sha: &str, dirty: bool, include_sca: bool) -> Vec<ExpectedRequest> {
let patch_sha = sha.to_string();
let dirty_value = if dirty { "true" } else { "false" }.to_string();
let patch_path = "/api/v1/start-scan/transfer-123/".to_string();
let detail_path = "/api/v1/scan/blast-scan-123".to_string();
let issue_path = "/api/v1/scan/blast-scan-123/issues".to_string();
let sca_path = "/api/v1/scan/blast-scan-123/issues/sca".to_string();
vec![
let mut plan = vec![
verify_request(),
expected_request(
"start BLAST upload",
Expand Down Expand Up @@ -808,6 +813,7 @@ pub(crate) fn blast_plan(sha: &str) -> Vec<ExpectedRequest> {
"https://github.com/corgea/cloud-e2e.git",
)?;
assert_multipart_text_field(request, "sha", &patch_sha)?;
assert_multipart_text_field(request, "dirty", &dirty_value)?;
assert_body_contains(request, b"name=\"chunk_data\"")
},
json_response(json!({
Expand All @@ -829,14 +835,18 @@ pub(crate) fn blast_plan(sha: &str) -> Vec<ExpectedRequest> {
},
json_response(empty_issue_page()),
),
expected_request(
];
if include_sca {
let sca_path = "/api/v1/scan/blast-scan-123/issues/sca".to_string();
plan.push(expected_request(
"read malicious SCA issues",
move |request| {
assert_authenticated_request(request, Method::GET, &sca_path)?;
assert_query(request, "page", "1")?;
assert_query(request, "page_size", "30")
},
json_response(malicious_sca_issue_page()),
),
]
));
}
plan
}
27 changes: 27 additions & 0 deletions tests/cloud_commands_e2e/scan_list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ fn scan_fail_on_malicious_sends_sha_and_list_renders_it() {
scan_stdout.contains("matched --fail-on malicious"),
"{scan_context}"
);
assert!(
!scan_stdout.contains("Working tree has uncommitted changes"),
"clean tree must not print dirty notice\n{scan_context}"
);

let list_response_sha = project.sha.clone();
let list_api = ApiStub::start(vec![
Expand Down Expand Up @@ -72,6 +76,29 @@ fn scan_fail_on_malicious_sends_sha_and_list_renders_it() {
assert!(list_stdout.contains(&project.sha[..8]), "{list_context}");
}

#[test]
fn scan_dirty_worktree_sends_dirty_true_and_prints_notice() {
let project = git_project();
std::fs::write(project.path().join("main.py"), "print('dirty')\n")
.expect("modify tracked file");
let short_sha = &project.sha[..7];
let scan_api = ApiStub::start(blast_upload_plan(&project.sha, true, false));
let (mut scan_command, _scan_home) = cloud_command(&scan_api, project.path());
scan_command.args(["scan", "blast", "--project-name", "cloud-e2e"]);

let scan_output = run_with_timeout(scan_command, &scan_api);
let scan_transcript = scan_api.assert_finished();
let scan_context = output_context(&scan_output, &scan_transcript);
assert_eq!(scan_output.status.code(), Some(0), "{scan_context}");
let scan_stdout = String::from_utf8_lossy(&scan_output.stdout);
assert!(
scan_stdout.contains(&format!(
"Working tree has uncommitted changes - scanning your local files, not commit {short_sha}."
)),
"{scan_context}"
);
}

#[test]
fn list_json_returns_filtered_scan_contract() {
let project = TempDir::new().expect("create list project");
Expand Down
Loading