An open-source distributed processing engine for large-scale data, built around in-memory computation.
One engine, several APIs: Spark SQL (DataFrames), Structured Streaming, MLlib (machine learning), GraphX.
Why it displaced MapReduce
Keeps intermediate results in memory instead of writing every stage to disk, so iterative jobs run orders of magnitude faster than Apache Hadoop MapReduce.
Where it runs on AWS
Amazon EMR — full control of the Spark cluster and version (EMR on EC2, EMR Serverless, EMR on EKS).
AWS Glue — serverless Spark for ETL, no cluster to size.
Exam signal
“Spark”, “in-memory big data processing” → EMR (or Glue when the question emphasizes serverless ETL).