BERT-based Text Embeddings vs Word2Vec
viaGlassdoor
Question: What is the difference between BERT-based text embeddings and Word2Vec? Key points: Word2Vec produces static, context-independent embeddings - each word has exactly one vector regardless of usage. BERT produces contextual embeddings - the same word gets a different vector depending on surrounding context, because BERT is a deep bidirectional Transformer trained with masked-language-modeling. BERT embeddings capture polysemy and richer syntactic/semantic context at the cost of higher compute; Word2Vec is lightweight and fast but cannot disambiguate word senses.
asked …