This is the homepage for paper Metadata Conditioning Accelerates Language Model Pre-training. We propose a new pre-training method named metadata conditioning then cooldown (MeCo): it conditions ...