System Design: Products API

The goal of the project was to build a scalable RESTful API for an existing e-commerce web application and optimize it to withstand the web scale traffic loads. Working in a team of three engineers, we inherited a legacy codebase and each member took ownership of a micro service that will maintain the existing application data set. I was responsible for redesigning and building a backend server and database for the products API service.

Tech Stack

API Development & Optimization

• Design and evaluate RDBMS and DBMS and consider tradeoffs: selected PostgreSQL

• Performe an ETL Process to transfer the full application data set (20M+) into PostgreSQL database

• Optimize queries using B-tree indexes, connecting pooling and building aggregate tables

Deployment

• Containerize the database and server using Docker
• Set up NGINX load balancer with ip_hash method for horizontal scaling and reduce latency by 800%
• Scale microservice to handle 3000 RPS by deploying 3 Node/Express servers and database on AWS EC2

Load & Stress Testing

• Used AutoCannon to simulate load on the server in development environment
• Conducted cloud-based performance and stress testing on loader.io with randomized product IDs
• Achieved 3000 RPS with latency 10ms with 0% error rate

Endpoints	/:product_id	/:product_id/styles	/:product_id/related
Avg res time	4ms	5ms	4ms
Min/Max	3/125ms	3/88ms	3/67ms
Err rate	0%	0%	0%

Getting Started

Prerequisites

• npm

npm install npm@latest -g

Installation

1. Clone the repo

   git clone https://github.com/sdc-cinnamon/sdc-products.git

2. Install NPM packages

   npm install

3. Set up .env file in the server's root directory and add the following info

    PORT="paste_port_number"
    PG_PORT="paste_pg_port_number"
    PG_USER="paste_username"
    PG_PASSWORD="paste_password"
    PG_HOST='paste_IP_address'
    PG_DBNAME="page_pg_database_name"

(back to top)