error when use seglst cal wer

[Wang-Zeyu]

Hi，

When I use the above seglst format to calculate wer, I get the following error：
ValueError: (Session ID, speaker) pairs are not unique. The SISO-WER needs unique session IDs or (session ID, speaker) pairs.

Do I need to manually merge words with the same Session ID based on chronological order and then calculate the wer, or is there a problem with my usage

Thanks！

[boeddeker]

Hi, do you mind sharing, want you want to do? The error message shows SISO-WER, so you were calculating the classical WER, that is defined on an utterance level. This means there is a unique mapping between reference sentence and estimate.

I guess you have a meeting or conversation as recording. In that case, the cpwer or the tcpwer may be the right one. There is also the orc wer, but it looks like you estimate the speaker, so you don't need this.

[Wang-Zeyu]

Thank you very much for your response.
Yes, I have some meeting transcriptions, and I would like to know how many of the errors are caused by ASR and how many by diarization. Therefore, I used files in the seglst format to separately calculate TCPWER and classical WER. However, there was an error when calculating WER, so I think I should concatenate the texts belonging to the same audio ID in chronological order before calculating WER.
Thank you again.

[boeddeker]

Often tcpWER is considered to count diarization, since many diarization systems don't identify the actual speaker, only that multiple utterance belong to the same speaker.

Just in case, this package has the ORC-WER, which is an WER, that doesn't count diarization errors on utterance level.
Additionally, die DIcpWER can give an indicator of assignment errors.

Probably, we should add an option to disable the calculation of the permutation in the cpWERand tcpWER, to support your case.