Number of Pre-training Steps

Our final model trains for 1 Mio steps. We show finetuning results for priors after fewer training steps. At 105K steps, the model already performs nearly on par with the final model trained for 1 Mio. steps. Please see the paper for metrics and a discussion.