LLM embeddings

LLM embeddings are dense vector representations: they convert text into high-dimensional float arrays, enabling machines to process semantic meaning for tasks like RAG.

LLM embeddings are the core mechanism for semantic understanding, transforming raw text—words, phrases, or documents—into dense numerical vectors. These vectors, often with 1536 or 3072 dimensions (e.g., OpenAI’s `text-embedding-3-small`), capture the conceptual relatedness of data. By calculating vector distance (cosine similarity), systems identify semantic similarity, not just keyword matches. This capability is critical for modern AI applications: powering highly accurate semantic search, enabling efficient data clustering, and serving as the foundation for Retrieval-Augmented Generation (RAG) workflows.

https://platform.openai.com/docs/guides/embeddings