Abstract: The inner product between two activation vectors is crucial for implementing neural networks with the attention mechanism. In this work, we propose and experimentally validate a novel inner ...