commit and its corresponding functions

[wubozhi]

I found that the commit id does not match the functions and changes you provide in each data sample.

For example, I search the commit accoriding to the commit id, and see what functions are changed during the commit. Then I find the functions changed in the commit are not the functions that shown in your data.

I wonder why? Am I correct? or I make a mistake?

[zyh1121]

Hi @wubozhi, thanks for the message.

Just wanted to double check if I got your question correctly. Are you saying

• (1) the commit info (the commit field) saved (e.g.) doesn't match the actual version where the issue is from, or
• (2) the commit info is correct but the functions changed by the commit as captured by touched_by_commit (e.g.) are not correct?

If it's (1), I was wondering which commit hash you were checking. Basically, for each issue, two consecutive version hashes (e.g.) and the commit hash in between are relevant. For example, regarding positive samples, the before was the version where the issue is from, and the after was the version where the issue is fixed by the commit. So the commit hash is the after version hash, which is specified in sample["commit"]["url"] (e.g.).

If it's (2), would it be possible to have an example such that I can understand the problem better?

[wubozhi]

Hi @zyh1121 , thanks so much for your reply.

nginx_labeler_1.json.txt
nginx_after_fix_extractor_0.json.txt

These two json files are extracted from nginx.tar.gz you provided.

I have the following questions:

1. Through beyondcompared, you can find that for some samples, the after_fix_extractor_0 version and labeler_1 version have lable 0 and 1 respectively, but their functions are the same. This is weird.

For example, sample id: nginx_a8f0ddde6aef8e5868bf4ac5b5b8c80be83dc601, you can see

2. Some different samples have the same commit id, i.e., version_pair or version . For exmple, sample id: nginx_a8f0ddde6aef8e5868bf4ac5b5b8c80be83dc601_1 and nginx_f269d57b5c669ef3d2cb304e7e10f365098872fb_1, they have the same commit id. I am a little confused why should not different samples come from different commit ids? Otherwise, how can you get the corresponding after-fix version? That's also the reason I ask the question yesterday, because I found the commit id does not match the functions and changes you provide in each data sample.
   

Thanks again for your reply, hope you can help me clear my confusion.

[zyh1121]

Thanks for the details, @wubozhi. I see the confusions

For (1), they are produced by two different labellers, please refer to this section in README for details. In particular, for the examples you gave, nginx_after_fix_extractor_0 essentially refers to the code after fix, where the bug is supposedly fixed by the commit. That's why the git commit hash is cf3134... (the after commit version). On the other hand, nginx_labler_1 refers to the bug before the fix. So it's git commit hash is b0b7... (the before commit version). Does it make sense?

I am a little confused why should not different samples come from different commit ids?

The reason is the static analyzer can find multiple issues in a same version. some are real and some are FPs. In the example you gave, the bug type of the upper one is BUFFER_OVERRUN_L4 and the lower one is INTERGER_OVERFLOW_L5. And they may sit at different parts in the code base. I didn't see why different samples should come from different commits.

Otherwise, how can you get the corresponding after-fix version?

The heuristic for after_fix_labeller is to look at the places in the after-fix version, that are corresponding to the trace locations identified in the before-fix version. Since we are talking about different bugs, it's not relevant if they come from the same or different version.

because I found the commit id does not match the functions and changes you provide in each data sample.

I am sorry. still not following. Let me try and see if I can address your question. sample["functions"] and sample["url"]["changes"] are different things:

• sample["functions"] are functions in the trace steps identified by the static analyzer, which are relevant to the issue. They have nothing to do with the commit changes.
• sample["commit"]["changes"] are the places touched by the commit, whose hash is the after version hash, which is also captured in sample["commit"]["url"].

Please refer to the field description in README for more info

[wubozhi]

Thanks so much for your patience. I'm still confused.

For (1), "nginx_after_fix_extractor_0 essentially refers to the code after fix, where the bug is supposedly fixed by the commit ", you mean the functions in "nginx_after_fix_extractor_0" are the code after fix? But can you provide the corresponding commit id? Accoring to the data in "nginx_after_fix_extractor_0", itscommit id may be one of the following? But I found both of them are not related to the functions in "nginx_after_fix_extractor_0". Do I make some mistakes?

"version_pair": {
"before": "b0b7b5a356c7dedf4a1867e94d8a36cbb6dc3da1",
"after": "cf31347ee84fdaa02f768e641d1a2f1352b6a56a"
}

Actually I know what you mean, but my question is why the samples with same functions are labeled as 0 and 1 respectively in nginx_after_fix_extractor_0.json.txt and nginx_labeler_1? You mean the labels in nginx_after_fix_extractor_0.json.txt are not necessarily correct?

Maybe I'll tell you my intentions so you can understand what I'm thinking: I am going to train a vulnerability identification model with your dataset. Here I use the samples in nginx_labeler_1 as positive sample, and nginx_after_fix_extractor_0.json.txt as negative sample. Could I do that?

Thanks again. Hope I do not disturb you too much.