Skip to content
Projects · Aug 2022 – Dec 2022 · PES University

Mini‑Kafka Implementation (Python)

A compact, educational Kafka‑like broker in Python: topics, producers/consumers, and message flow — built to understand distributed log systems from the inside.

Distributed Systems Pub/Sub Replication Concurrency

Overview

A from‑scratch mini implementation inspired by Kafka to learn how log‑based messaging systems work. It models core concepts like topics, producers, and consumers.

Problem

It’s easy to use Kafka and hard to truly understand it. The goal was to internalize the design by building a simplified broker — and documenting the trade‑offs.

Approach

Designed a simple broker protocol and implemented producer/consumer interactions. Modeled topic storage and message retrieval patterns to mirror log semantics. Kept the project intentionally compact so each component is readable and teachable.

Impact

Shows systems thinking and curiosity: building infrastructure primitives, reasoning about messaging, and explaining design choices — a standout for backend/distributed roles.