A Netflix Backend Implementation
Here we implemented a sample of Netflix’s backend. We had designed a Domain Driven Architecture using microservices and Kafka. In order to simulate a real scenario, we picked up a real dataset with 1,000 of the most popular movies on IMDB in the last 10 years.
Keywords: microservice, python, kafka, flask, restful-api
Motivation
As a home work for my Master of Business Administration class in Software Engineer at FIAP, we decided to propose a backend implementation for Netflix.
With this implementation we intend to provide APIs to:
- List movies by gender
- Visualize movie details
- Provide a vote option to like movies
- Provide a reminder list of movies
- Search movies by keyword
- List ranked movies by gender
- Open a technical support for problems
- List watched movies
Dataset
In order to simulate a real scenario, we picked up an IMDB dataset from Kaggle with 1,000 of the most popular movies on IMDB in the last 10 years. The data set contains a lot of information on multiple movies, however, in order to simplify our implementation, we selected only some fields.
We also designed a script to filter this dataset and build a populated database schema for our microservices.
Architecture
Here we propose a domain driven architecture with microservices. In order to solve a non-functional requirement we implemented a Kafka communication between some services.
All microservices are coded with Python 3.7+, Flask web framework and Clean Architecture principles.
Movies
The movies microservice take account of the movie domain. Their APIs answers questions about movie details, movie genres and searches for movies. Its database is built by the movie scraper which reads our IMDB dataset and populates the movie’s database.
Ranking
The ranking microservice takes into account the votes and movies ranking. Their APIs collects users likes and computes the ranking.
Reminder
The reminder microservice collects the reminder and watch later movies list. Their APIs collect users desire for reminder a movie and to set movies as watched.
Support
The support microservice posts the user message to a Kafka topic in order to notify the technical support team.
Technical Support
The technical support reads from the support-topic and logs a message simulating a service order creation.
Movie Scraper
The movie scraper processes our dataset to build a populated database schema with all movies.
Here we use the pandas library, which reads our dataset CSV file and, given the MAX_MOVIES variable, adds the movies to movies service.
Running
We provide a bootstrap.sh script to simplify the stack boot. It runs a docker-compose.yml which starts all services, databases and Kafka. After that, the scripts sleeps for 30 seconds waiting for all container to be set up. So it runs the docker-compose-scraper.yml which starts to populate our movies database.
It’s possible to customize the amount of movies changing the MAX_MOVIES variable at docker-compose-scraper.yml. As our database is limited to a maximum of a thousand movies, we defined 10 as the default amount.
Testing
As a simple way to test our stack, we provide a postman collection that has sample requests for all proposed APIs.
Conclusion
Here we designed a Domain Driven Architecture using microservices and Kafka. Our intention was to build a simple solution for Netflix Backend. All code can be found at https://github.com/victoramsantos/netflix-backend.
