An ordered list of numbers that represents a point or direction in continuous space—embeddings and activations are vectors at different stages of the model.
Papers and code often say "vector" when they mean a 1-D tensor: one row from an embedding table, a query vector, or a hidden state. Knowing that vectors are the building blocks inside tensors helps you follow shape notation from embeddings through attention.
Simple Example
A 768-dimensional embedding for one token is a vector of length 768. Stacking many positions produces a matrix or higher-rank tensor, but each row is still a vector in hidden space.
Common Confusions
A vector is not the same as a sparse one-hot encoding: learned embeddings are dense vectors with values in every dimension. "Dense vector" usually contrasts with sparse bag-of-words features, not with tensors in general.