Neural Networks & LLM Components · Medium

Sinusoidal Positional Encoding

Build deterministic position vectors from paired sine and cosine frequencies.

Task

Implement sinusoidal_positional_encoding(sequence_length: int, d_model: int) -> list[list[float]]. Construct the standard sinusoidal positional encoding matrix with one row per sequence position and one column per model dimension.

Requirements

sequence_length and d_model are non-negative integers.
Use sine for even dimensions and cosine for odd dimensions.
Each even/odd dimension pair must use the same frequency scale.
Support odd d_model values, zero sequence length, and zero model width.

Example

sinusoidal_positional_encoding(2, 4)
# [
#   [0.0,        1.0,       0.0,        1.0],
#   [0.84147098, 0.54030231, 0.00999983, 0.99995000],
# ]