13 Real Machine Learning System Design Interview Questions and Answers

-- For Applied Scientists and Machine Learning Engineers --

This is a curated, evolving list of real machine learning system design interview questions and answers, designed by a Staff ML Scientist who is still actively interviewing candidates. Practicing these questions will help you prepare for ML Scientist, ML Engineer, Applied Scientist, and Data Scientist roles at FAANG and similar-tier companies.

Problem	Topics	Difficulty
( Subscription required ) API patterns	APIs, GraphQL, REST	Easy
( Subscription required ) Build an ML system to predict Ad clicks	ML system design, Feature engineering, Data exploration, ML modeling, Monitoring, Deployment, Business metrics	Hard
( Subscription required ) Cloud vs. on-device deployment	Deployment, Cloud, Edge	Medium
( Subscription required ) Complex vs. simple deployment	Deployment	Easy
( Subscription required ) Crons, schedulers, orchestrattors	ML infra	Medium
( Subscription required ) Data, model, and pipeline parallelism	Parallelism	Medium
( Subscription required ) Debug an ML model	Best practices	Medium
( Subscription required ) How to speed up inference	Inference	Easy
( Subscription required ) ML system design tools and use cases	ML infra, CDN, Kafka, Reddis, Dynamo, Cassandra, Chubby, PGVector, DBT, Feast, MLFlow, Statsig, Airflow, Fiddler	Hard
( Subscription required ) Online prediction, vs. batch prediction	Inference	Medium
( Subscription required ) Simple model deployment process	Deployment, Docker	Easy
( Subscription required ) Training tracking	Best practices	Medium
( Subscription required ) Types of data distribution shifts	Train-serving skew, Covariate shift, Label shift, Concept shift	Medium