A Netflix Backend Implementation

Here we implemented a sample of Netflix’s backend. We had designed a Domain Driven Architecture using microservices and Kafka. In order to simulate a real scenario, we picked up a real dataset with 1,000 of the most popular movies on IMDB in the last 10 years.

Keywords: microservice, python, kafka, flask, restful-api

Motivation

As a home work for my Master of Business Administration class in Software Engineer at FIAP, we decided to propose a backend implementation for Netflix.

With this implementation we intend to provide APIs to:
- List movies by gender
- Visualize movie details
- Provide a vote option to like movies
- Provide a reminder list of movies
- Search movies by keyword
- List ranked movies by gender
- Open a technical support for problems
- List watched movies

Dataset

In order to simulate a real scenario, we picked up an IMDB dataset from Kaggle with 1,000 of the most popular movies on IMDB in the last 10 years. The data set contains a lot of information on multiple movies, however, in order to simplify our implementation, we selected only some fields.

IMDB dataset

We also designed a script to filter this dataset and build a populated database schema for our microservices.

Architecture

Architecture design

Here we propose a domain driven architecture with microservices. In order to solve a non-functional requirement we implemented a Kafka communication between some services.

All microservices are coded with Python 3.7+, Flask web framework and Clean Architecture principles.

Movies

The movies microservice take account of the movie domain. Their APIs answers questions about movie details, movie genres and searches for movies. Its database is built by the movie scraper which reads our IMDB dataset and populates the movie’s database.

Ranking

The ranking microservice takes into account the votes and movies ranking. Their APIs collects users likes and computes the ranking.

Reminder

The reminder microservice collects the reminder and watch later movies list. Their APIs collect users desire for reminder a movie and to set movies as watched.

Support

The support microservice posts the user message to a Kafka topic in order to notify the technical support team.

Technical Support

The technical support reads from the support-topic and logs a message simulating a service order creation.

Movie Scraper

The movie scraper processes our dataset to build a populated database schema with all movies.

Here we use the pandas library, which reads our dataset CSV file and, given the MAX_MOVIES variable, adds the movies to movies service.

Running

We provide a bootstrap.sh script to simplify the stack boot. It runs a docker-compose.yml which starts all services, databases and Kafka. After that, the scripts sleeps for 30 seconds waiting for all container to be set up. So it runs the docker-compose-scraper.yml which starts to populate our movies database.

Bootstrap script output

It’s possible to customize the amount of movies changing the MAX_MOVIES variable at docker-compose-scraper.yml. As our database is limited to a maximum of a thousand movies, we defined 10 as the default amount.

Testing

As a simple way to test our stack, we provide a postman collection that has sample requests for all proposed APIs.

Postman collection

Conclusion

Here we designed a Domain Driven Architecture using microservices and Kafka. Our intention was to build a simple solution for Netflix Backend. All code can be found at https://github.com/victoramsantos/netflix-backend.

Poeta, amante de tecnologia e apaixonado por conhecimento. Trabalho aplicando a Engenharia de Software à cultura DevOps e de SRE.