The padding value '_' is used to fit sequences into fixed-length batches, but it’s unnecessary for the model to "learn" that this should always be zero. Instead, we should explicitly handle it.
For example, we could mask padding in the model workflow to ignore it during training/inference.
Otherwise, we could also set padding to zero during length regulation so it "vanishes."
The padding value '_' is used to fit sequences into fixed-length batches, but it’s unnecessary for the model to "learn" that this should always be zero. Instead, we should explicitly handle it.
For example, we could mask padding in the model workflow to ignore it during training/inference.
Otherwise, we could also set padding to zero during length regulation so it "vanishes."