Reproduce for BERT reweight using SAMA

[liaodisen]

Hello,

I am trying to reproduce the result for table 1 for SAMA. However, using hypergradient will not make much difference on the validation results. I am able to get the baseline results, so is there anything I need to pay attention to when I reproduce the results? After the meta net is learned, do I need to retrain the BERT?

Thanks