Back to portfolio

Big Data, Distributed Systems

Spark/Kafka Real-Time Streaming Pipeline

Dockerized real-time data pipeline built on Hadoop, Hive, and Kafka.

Key results

99%

uptime

KafkaSparkHDFSHive

Big Data, Distributed Systems

Spark/Kafka Real-Time Streaming Pipeline

99% uptime

Overview

A containerized streaming pipeline demonstrating distributed systems fundamentals.

Topics

SparkKafkaHadoopDocker

Stack

  • Apache Spark
  • Apache Kafka
  • Hadoop HDFS
  • Hive
  • Docker
  • Scala
  • Zookeeper
GitHub

More projects

View all

Pitch, GITEX Africa 2024

SafeLens AI

ClarityBot feature

Applied AI

ClarityBot

PDF

Fintech, Cloud

CryptoVault