In this work, we investigate how optimization, data distribution, loss function, and model architecture in LM pre-training influences the emergence of attention sink ...

Abstract: Applications of Large Language Models (LLM) for source code analysis and related tasks arising during the development of an industrial static analyzer are becoming increasingly relevant due ...

Abstract: As quadruped controllers approach greater maturity for locomotion on level ground, a next challenge relates to enabling these systems to carefully choose contacts in cluttered or ...