Word Count / Log Aggregation

Problem Given a stream of text lines, count word frequencies and return the top k most frequent words.

Input / Output

• Input: a stream (or large list) of lines, and an integer k.
• Output: the k words with the highest frequency, most frequent first.

Constraints

• The stream may be very large — discuss bounding memory when the vocabulary or input does not fit in RAM.
• Define tie-breaking between words of equal frequency (e.g. lexicographic).

Example

• lines ["the cat", "the dog", "the cat"], k=2 -> ["the", "cat"] (the=3, cat=2, dog=1).