Duration pitch loss not used.

[jaykim9870]

Hello, I was looking into your code and it seems like the code does not consider the duration_pitch_loss.

naturalspeech2-pytorch/naturalspeech2_pytorch/naturalspeech2_pytorch.py

Line 1522 in 659bec7

duration_pitch_loss = 0.

Maybe, it might be related to the aux_loss you have made.

naturalspeech2-pytorch/naturalspeech2_pytorch/naturalspeech2_pytorch.py

Line 1600 in 659bec7

aux_loss = (duration_loss * self.duration_loss_weight) \

Thanks for the great work!

[wonwooo]

@jaykim9870
I have the same question.
You're thinking that code should be changed like below. Right?

before : return loss + (self.rvq_cross_entropy_loss_weight * ce_loss) + duration_pitch_loss

fixed : return loss + (self.rvq_cross_entropy_loss_weight * ce_loss) + aux_loss

[jaykim9870]

@wonwooo
Yes, that would do.

FYI, There are some other issues like wavenet based diffusion model as the model size is very different from the original paper. As far as I have investigated, the model architecture is too different so it may affect the model performance. If you are working based on this project, you may also need to check those out!