Does CLAP able to train on ASR dataset?

[MonolithFoundation]

Does CLAP able to train on ASR dataset?

[mrowebot]

@MonolithFoundation Yes, you should be able to train on such a dataset as the READMe mentions the following in the Pretrained Models section:

For music and speech: music_speech_epoch_15_esc_89.25.pt
For speech, music and general audio: music_speech_audioset_epoch_15_esc_89.98.pt

You would need to convert whatever ASR dataset you have to the Webdataset format for training to use this data. Have a look at the processing steps and the following repo: https://github.com/LAION-AI/audio-dataset

[lukewys]

The speech dataset is not very "ASR"-heavy, as far as I remember they also contains the description of the speaker and language. I don't how CLAP performs in ASR-like transcription task.