[P] I replaced Dot-Product Attention with distance-based RBF-Attention (so you don't have to...)

I recently asked myself what would happen if we replaced the standard dot-product in self-attention with a different distance metric, e.g. an rbf-kernel? Standard dot-product attention has this quirk where a key vector can "bully" the softmax simply by having a massive magnitude. A random key that