Embedding

An embedding is a learned mapping that converts discrete, categorical data — a word, a product, an image, a user — into a continuous vector of real numbers. The key property is that items with similar meanings or characteristics are mapped to nearby points in the vector space. For example, in a well-trained word embedding, the vectors for 'king' and 'queen' are close together, as are 'Paris' and 'France.' This allows mathematical operations on meanings: the famous equation vector('king') − vector('man') + vector('woman') ≈ vector('queen') illustrates how embeddings capture semantic relationships.

Embeddings are fundamental to modern deep learning because neural networks operate on continuous numbers, not discrete symbols. Before processing a word, an NLP model converts it to an embedding vector. Before comparing images, a vision model encodes them as embedding vectors. Word2Vec (2013) and GloVe popularized word embeddings, but modern systems use contextual embeddings — where the same word gets different vectors depending on its surrounding context — as computed by Transformer models like BERT and GPT. Sentence and document embeddings extend this to longer text spans.

Beyond NLP, embeddings are now used across virtually every AI domain. Recommendation systems embed users and items into a shared space to compute personalized similarity. Image search engines embed photographs for visual similarity matching. Graph Neural Networks compute embeddings for nodes in social networks or molecular structures. The rise of vector databases (Pinecone, Weaviate, Qdrant) reflects the centrality of embeddings in modern AI infrastructure — they provide the backbone for semantic search, retrieval-augmented generation (RAG), and multimodal AI.