Skip to main content

Apache Hive

  • What it is
    • A data warehouse layer over Apache Hadoop that lets you query files with SQL-like HiveQL instead of writing MapReduce or Apache Spark code.
  • The metastore
    • Hive keeps a metastore of table definitions — schema, partitions, file locations — that other engines read.
    • On AWS, the AWS Glue Data Catalog is the managed, shared replacement for the Hive metastore, and is Hive-metastore-compatible.
  • Where it runs on AWS
    • Amazon EMR, alongside Spark and Presto on the same cluster.
  • Not to be confused with
    • Amazon Athena — serverless SQL over S3 with no cluster at all. Hive needs a cluster; it suits long-running batch SQL rather than ad hoc queries.

Linked from