LLM-fused personalization at scale
Spotify fuses user embeddings into LLM token space for steerable recommendations AI EngineerTL;DW
- Spotify embeds user vectors into LLM token space as 'soft tokens' to inject personalization into generative recommendations without retraining on 750M+ users.
- Semantic IDs compress content embeddings (tracks, episodes) into 4-6 tokens hierarchically, enabling LLMs to auto-regressively predict next items like words.
- User taste profile feature exposes what Spotify knows about you and accepts text edits to dynamically update the generative model's understanding in real time.
- Moving from siloed multi-stage ranking pipelines (candidate generation → rankers) to unified transformer backbone supporting steerable, natural-language-driven recommendations.
- Spotify jointly embeds users, tracks, and episodes in same vector space, visualizable as a hypersphere where proximity reveals taste neighborhoods and unexplored regions.
- Combining Spotify's knowledge (user/content vectors) with open-weight LLM world knowledge via fine-tuning and semantic ID domain adaptation improves steerability and explainability.
- Transformer-based sequential user models replace older autoencoder approach, treating listening history as context like prompts in language models for better personalization.
TL;DW
- Spotify embeds user vectors into LLM token space as 'soft tokens' to inject personalization into generative recommendations without retraining on 750M+ users.
- Semantic IDs compress content embeddings (tracks, episodes) into 4-6 tokens hierarchically, enabling LLMs to auto-regressively predict next items like words.
- User taste profile feature exposes what Spotify knows about you and accepts text edits to dynamically update the generative model's understanding in real time.
- Moving from siloed multi-stage ranking pipelines (candidate generation → rankers) to unified transformer backbone supporting steerable, natural-language-driven recommendations.
- Spotify jointly embeds users, tracks, and episodes in same vector space, visualizable as a hypersphere where proximity reveals taste neighborhoods and unexplored regions.
- Combining Spotify's knowledge (user/content vectors) with open-weight LLM world knowledge via fine-tuning and semantic ID domain adaptation improves steerability and explainability.
- Transformer-based sequential user models replace older autoencoder approach, treating listening history as context like prompts in language models for better personalization.
Spotify's AI Foundation team replaces multi-stage collaborative filtering with a generative model built on three components: transformer-based user embeddings, semantic IDs that compress content into hierarchical tokens, and a soft tokenization layer that projects user state into LLM embedding space. Deployed for podcast recommendations; rolling out via the Taste Profile feature.