training efficiency

[adogwangwang]

hello，thanks for your efforts, I have problems with training efficiency, how long time it takes to train your Tokenizer with your data? I have been training a tokenizer for a long time and it haven't finished.

[taishan1994]

I haven't tested a lot of the text, you can look it up on its official github.