Senior DE: PySpark/pandas live coding + payroll data-pipeline star-schema design; expected syntax from memory
The interview experience was subpar. The HR didn't verbally inform that I'd need to have PySpark set up locally. It was mentioned in the interview email but hidden in the blocks of unnecessary text. So that led to a bit of back & forth. Later, when the interview happened, the interviewer expected me to remember the entire PySpark & pandas syntax without taking the help of AI to write the same piece of code that you will use AI to write for daily work. While the interview problem itself is not difficult but expecting to memorise the syntax for the whole thing seemed a bit unfair. The task: build a pipeline to transform raw payroll payout data (inconsistent date formats, nested JSON amounts, mixed currencies) into a clean query-able format — design a star schema, decide an ingestion strategy for daily files, and implement the transformation in Python/DataFrame or SQL to answer business questions (total payouts per currency, average salary per currency by department, flatten the amounts JSON).
The loop · 1 round
PySpark/pandas live coding + data pipeline design (star schema modeling, ingestion strategy)