Transformer Concept

Positional Encoding

Slide the position — each one gets a unique fingerprint of sine waves.

PE vector · 16 dimspos 6
−1+1
posenc.py