To implement this project, I used Go for the backend services, gRPC for efficient inter-service communication, ffmpeg for video segmentation, SQLite for metadata storage, and HTML/CSS for the frontend interface.
The architecture consists of three main components: the frontend provides a straightforward interface where users can upload videos and watch them; the storage nodes simulate a cluster of servers exposed via gRPC that hold the video chunks, with each node responsible for a portion of the overall data; and the coordinator manages node membership, handles consistent hashing, and keeps everything in sync. When an user uploads a video, ffmpeg segments it into smaller chunks, which are then distributed across the cluster based on the consistent hash. When someone wants to watch the video, the system fetches only the chunks it needs. This method reduces bandwidth usage and allows playback to begin as soon as possible.
In the real world, servers go offline due to failures or scheduled maintenance, which is why fault tolerance was a key feature. I tested this by deliberately taking nodes offline during playback. The video continued streaming without interruption because the data is replicated across multiple nodes. When a server goes down, the system reroutes to grab chunks from another node.
This project helped me reinforce concepts covered in class about distributed systems, and it was truly interesting how this system was pieced together. I developed a deeper appreciation for the engineering success that companies like YouTube, Netflix, and Spotify went through to deliver seamless user experiences.