running error

[laybebe]

Thanks for your wonderful jobs! I followed the “GET_STARTED.md” to test the TAO model on the Open-vocabulary MOT benchmark, however, the code threw an error.

[siyuanliii]

Have you run the conversion scripts for the TAO annotations file?

[laybebe]

Thanks for your reply. I ensure that I have not omitted any steps. I will double-check and run it again.

[AndyBarcia]

This problem is apparently caused because the trackers doesn't actually detect anything with the default configuration (at least for me), because this line sets the score threshold too high:

rcnn_test_cfg.score_thr = 1 / len(text_features) + 0.01

https://github.com/SysCV/ovtrack/blob/e4f20e0974753539a47cf96fae46efad8adfd3b6/ovtrack/models/roi_heads/ovtrack_roi_head.py#L526C9-L526C64

Changing it to the following gives me the best results, although I'm not sure if this is the correct way of doing it.

rcnn_test_cfg.score_thr = (1 / len(text_features)) * 1.001

[siyuanliii]

Thanks for pointing out this problem, and sorry for the inconvenience. I accidentally added this line. It was intended for dynamic adjusting the inference threshold on any video with any class prompt. If you check in detail, you will find the confidence score varies depending on the number of classes to track. This is due to the nature of the softmax function. I added that line to dynamically adjust the proper threshold for any number of class prompts. However, this is not needed for inference on standard benchmarks. I update the code accordingly.

I also noticed that adding the line you suggested to dynamically adjust the rcnn confidence threshold works very well! It can give better overall but only slightly worse classification performance on TAO validation benchmark.
I will add this line as default with a hyperparameter "dynamic_rcnn_thre" to control it.