Skip to content

Norman's Deep Dives

Lim Xing Kang Norman's adventures in data ingestion, processing and warehousing.

  • Home
  • About
  • Linkedin
  • Github

Tag: kubernetes

Cloud-Agnostic Big Data Processing with Kubernetes, Spark and Minio

In this article I will discuss how to build a cloud agnostic Big Data processing and storage solution running entirely in Kubernetes. This design avoids vendor lock-in by using only open-source technologies and avoiding cloud-managed products such as S3 and Amazon ElasticMapReduce in favour of MinIO and Apache Spark

frenoid AWS, kubernetes, minio, spark 1 Comment May 4, 2022 16 Minutes

Build a Data Lake with Trino, Kubernetes, Helm, and Glue

How to create a Data Lake in AWS using S3 as the storage layer, Glue as the metastore, and Trino on Kubernetes as the query engine.

frenoid Uncategorized Leave a comment February 19, 2022June 12, 2022 13 Minutes

Run Trino/Presto on Minikube on AWS

This is the beginning of a new series centering on the use of Kubernetes to host Big Data infrastructure. In this article I will run a single-node Trino cluster in local Kubernetes cluster called minikube

frenoid Uncategorized 2 Comments December 7, 2021February 19, 2022 14 Minutes
Blog at WordPress.com.
  • Follow Following
    • Norman's Deep Dives
    • Already have a WordPress.com account? Log in now.
    • Norman's Deep Dives
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...