This repository has been archived by the owner on Dec 10, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 22
docs: add API design overview #16
Open
sf-clouder
wants to merge
14
commits into
Superformula:master
Choose a base branch
from
sf-clouder:sf-code-test
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
d82182c
docs: add API design overview
sf-clouder d3f03d0
chore: project init
sf-clouder c0f870e
style: install prettier and eslint
sf-clouder a8e7a87
chore(cdk): install aws cdk core
sf-clouder 74cff4f
chore(cdk): install cdk packages for apig, dynamo, lambda, sqs
sf-clouder d3ed2eb
feat(stack): basic cdk stack with dynamo table
sf-clouder fa3035a
feat(dynamo): add gsi for sk-state-city-zip
sf-clouder 3e24cc3
feat(stack): add api gateway with resources for /users and /users/{id}
sf-clouder 8fb7ee0
chore(lambda): install clone, clean, merge and logging packages
sf-clouder 8051c09
feat(stack): add lambda layer to stack
sf-clouder 72a48bf
feat(lambda): add create, read, update, delete functions
sf-clouder ef79d9b
feat(api): add user model to api for payload validation in front of c…
sf-clouder 1d59d3b
docs: remove outdated api design overview
sf-clouder e3154f8
docs(readme): modify readme.md with info about architecture, deployme…
sf-clouder File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
{ | ||
"extends": ["plugin:prettier/recommended"], | ||
"plugins": ["prettier"], | ||
"rules": { | ||
"prettier/prettier": "error" | ||
}, | ||
"parserOptions": { | ||
"ecmaVersion": 2018 | ||
}, | ||
"env": { | ||
"es6": true | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
node_modules | ||
.vscode | ||
.env | ||
|
||
# CDK asset staging directory | ||
.cdk.staging | ||
cdk.out |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
{ | ||
"printWidth": 140, | ||
"singleQuote": true, | ||
"arrowParens": "avoid", | ||
"trailingComma": "all", | ||
"tabWidth": 4, | ||
"useTabs": false, | ||
"semi": true | ||
} | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,106 +1,184 @@ | ||
# Superformula Back-end Developer Test | ||
# SF Users API | ||
|
||
Be sure to read **all** of this document carefully, and follow the guidelines within. | ||
## About | ||
|
||
## Context | ||
The "SF Users" API is built and deployed using the AWS Cloud Development Kit (CDK). The stack consists of API Gateway endpoints integrated with Lambda running Node.js. Data storage is in DynamoDB. | ||
|
||
Build a RESTful API that can `create/read/update/delete` user data from a persistence store. | ||
Payload validation is handled within API Gateway, prior to hitting Lambda. This avoids triggering Lambda executions when a payload is invalid. | ||
|
||
### User Model | ||
The DynamoDB data store intentionally uses generically named partion and sort keys, 'pk' and 'sk', to allow for future expansion, using access patterns not yet determined. Dynamo tables are commonly stacked with multiple kinds of data. In our case, using 'pk' and 'sk' would allow for storage of other non-user specific data down the road. There is a Global Secondary Index which makes it possible to do partial lookups against a sort key of state#city#zip, in order to return users in a particular state, or city, or zipcode. | ||
|
||
``` | ||
There is one Lambda Layer containing the NPM packages leveraged by the four CRUD lambdas. Individual Lambdas handle Create, Read, Update and Delete actions. Resuse of HTTP connection is enabled. There is an SQS dead letter queue attached to each Lambda. | ||
|
||
Logging from lambda is done to CloudWatch with recommended AWS attributes, like this: | ||
|
||
```json | ||
{ | ||
"id": "xxx", // user ID (must be unique) | ||
"name": "backend test", // user name | ||
"dob": "", // date of birth | ||
"address": "", // user address | ||
"description": "", // user description | ||
"createdAt": "" // user created date | ||
"updatedAt": "" // user updated date | ||
"timestamp": "2019-08-22 18:17:33,774", | ||
"level": "INFO", | ||
"location": "handler:1", | ||
"service": "payment", | ||
"lambda_function_name": "test", | ||
"lambda_function_memory_size": "128", | ||
"lambda_function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test", | ||
"lambda_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72", | ||
"cold_start": "true", | ||
"message": "User created" | ||
} | ||
``` | ||
|
||
### Functionality | ||
Future plans and thoughts: | ||
|
||
- Enhanced user model schema checking. Currently required values, and min/max length are enforced. Pattern matching could be added for more granular validation of dates, for instance. | ||
- Break out the CDK stack and Lambda code into modules | ||
- Better sanitize some errors generated by AWS resources (like Dynamo), so ARNs and other AWS specific information is not revealed to the client using the API | ||
- Tests for API endpoints and the AWS services created with the CDK | ||
- Process any data deposited in the Lamda DLQs | ||
- Use direct API Gateway service integrations to other AWS resources, bypassing Lambda, if it make sense. | ||
|
||
## Install & Deploy | ||
|
||
- The API should follow typical RESTful API design pattern. | ||
- The data should be saved in the DB. | ||
- Provide proper API documentation. | ||
- Proper error handling should be used. | ||
Ensure you have the AWS CLI installed and configured with your AWS credentials | ||
|
||
- Install: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html | ||
- Configure: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html | ||
|
||
Install the AWS CDK | ||
|
||
``` | ||
npm install -g aws-cdk | ||
``` | ||
|
||
## What We Care About | ||
Clone the repository | ||
|
||
Use any libraries that you would normally use if this were a real production App. Please note: we're interested in your code & the way you solve the problem, not how well you can use a particular library or feature. | ||
``` | ||
git clone <this repository url> | ||
``` | ||
|
||
Go to the repository's directory and install. This installs packages for the CDK stack and for the Lambda Layer. | ||
|
||
``` | ||
cd <local repo directory> | ||
npm install | ||
``` | ||
|
||
NOTE: If this is your first time running the AWS CDK in this AWS account and/or region, you need to boostrap the CDK | ||
|
||
``` | ||
cdk boostrap | ||
``` | ||
|
||
_We're interested in your method and how you approach the problem just as much as we're interested in the end result._ | ||
Optional: Synthesize the stack to see the CloudFormation template the CDK generates | ||
|
||
Here's what you should strive for: | ||
``` | ||
cdk synth | ||
``` | ||
|
||
- Good use of current Node.js & API design best practices. | ||
- Solid testing approach. | ||
- Extensible code. | ||
Deploy the stack (note the various options) | ||
|
||
If you have not been specifically asked, you may pick either `Implementation Path: Docker Containers` or `Implementation Path: Cloud-native` requirements below. | ||
``` | ||
// Example #1: standard 'DEV' stack | ||
cdk deploy | ||
|
||
## Implementation Path: Docker Containers | ||
// Example #2: deploys to a stack named 'TEST' | ||
cdk deploy -c stage=test | ||
|
||
### Basic Requirements | ||
// Example #3: deploys to a stack named 'PROD' with using a specific AWS profile. | ||
cdk deploy -c stage=prod --profile MY_PROFILE_NAME | ||
``` | ||
|
||
- Use Node.js `LTS` and any framework of your choice. | ||
- Use any persistence store. NoSQL DB is preferred. | ||
- Write concise and clear commit messages. | ||
- Write clear **documentation** on how it has been designed and how to run the code. | ||
After a few minutes the stack should respond with the API endpoint | ||
|
||
### Bonus | ||
``` | ||
// Example output (your endpoint URL will be different) | ||
|
||
- Provide proper unit tests. | ||
- Add a read only endpoint to fetch location information based off the user's address (use [NASA](https://api.nasa.gov/api.html) or [Mapbox](https://www.mapbox.com/api-documentation/) APIs) | ||
- Use Docker containers. | ||
- Utilize Docker Compose. | ||
- Setup a CircleCI config to build/test/deploy the service. | ||
- Leverage Terraform or other infrastructure management tooling to spin up needed resources. | ||
- Providing an online demo is welcomed, but not required. | ||
// NOTE: The endpoint will not have the trailing '/users' which you will need to add when interacting the api. | ||
|
||
### Advanced Requirements | ||
Outputs: | ||
SF-UserStack-DEV.SFUserStackDEVUsersAPIEndpointC5532769 = https://epvz8ugbl7.execute-api.us-east-1.amazonaws.com/prod/ | ||
``` | ||
|
||
These may be used for further challenges. You can freely skip these if you are not asked to do them; feel free to try out if you feel up to it. | ||
## Create a User | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Have you ever used API Gateway export to dynamically generate API documentation? https://docs.aws.amazon.com/cli/latest/reference/apigateway/get-export.html There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you for your review, Afaq. Yes, the ability to export was a primary reason behind defining the methods and validation within API Gateway instead of proxying and handling it within Lambda. |
||
|
||
- Use [hapi](https://hapijs.com/) to build the core feature and use a different framework (such as Express or Loopback) to handle HTTP requests. | ||
- Provide a complete user auth (authentication/authorization/etc) strategy, such as OAuth. | ||
- Provide a complete error handling and logging (when/how/etc) strategy. | ||
- Use a NoSQL DB and build a filter feature that can filter records with some of the attributes such as username. Do not use query languages such as MongoDB Query or Couchbase N1QL. | ||
> Method: POST | ||
> | ||
> Endpoint: https://<API_ID>.execute-api.<AWS_REGION>.amazonaws.com/prod/users/ | ||
> | ||
> Returns: 200 and the system generated user id | ||
|
||
```json | ||
// Expected payload: all fields required, except 'description' | ||
|
||
## Implementation Path: Cloud-native | ||
{ | ||
"name": "Jane Smith", | ||
"dob": "YYYY-MM-DD", | ||
"address": { | ||
"street": "123 Main Street", | ||
"city": "Austin", | ||
"state": "TX", | ||
"zip": "78729-1234" | ||
}, | ||
"description": "this is the description" | ||
} | ||
``` | ||
|
||
### Basic Requirements | ||
```json | ||
// Return payload | ||
|
||
- Create each endpoint as an individual AWS Lambda in Node.js | ||
- Use any AWS Database-as-a-Service persistence store. DynamoDB is preferred. | ||
- Write concise and clear commit messages. | ||
- Write clear **documentation** on how it has been designed and how to run the code. | ||
{ | ||
"id": "118bb552-cded-43be-bc90-74973320a780" | ||
} | ||
``` | ||
|
||
### Bonus | ||
## Read a User | ||
|
||
- Use Infrastructure-as-code tooling that can be used to deploy all resources to an AWS account. Examples: CloudFormation / SAM, Terraform, Serverless Framework, etc. | ||
- Provide proper unit tests. | ||
- Use API Gateway to expose AWS Lambdas | ||
- Providing an online demo is welcomed, but not required. | ||
- Bundle npm modules into your Lambdas | ||
> Method: GET | ||
> | ||
> Endpoint: https://<API_ID>.execute-api.<AWS_REGION>.amazonaws.com/prod/users/<USER_ID> | ||
> | ||
> Returns: 200 and a payload | ||
|
||
### Advanced Requirements | ||
```json | ||
// Return payload | ||
|
||
These may be used for further challenges. You can freely skip these; feel free to try out if you feel up to it. | ||
{ | ||
"id": "5da9791d-dcfa-4a46-ba64-ad5a933426eb", | ||
"address": { | ||
"zip": "55466", | ||
"state": "MT", | ||
"city": "Billings", | ||
"street": "123 Main Street" | ||
}, | ||
"description": "this is the description", | ||
"name": "Tim Williams", | ||
"dob": "1984-04-01" | ||
} | ||
``` | ||
|
||
- Describe your strategy for Lambda error handling, retries, and DLQs | ||
- Describe your cloud-native logging, monitoring, and alarming strategy across all endpoints | ||
- Build a filter feature that can filter records with some of the attributes such as username. | ||
## Update a User | ||
|
||
## Q&A | ||
> Method: PATCH | ||
> | ||
> Endpoint: https://<API_ID>.execute-api.<AWS_REGION>.amazonaws.com/prod/users/<USER_ID> | ||
> | ||
> Returns: 204 | ||
|
||
> Where should I send back the result when I'm done? | ||
```json | ||
// Expected payload: This is an exammple updating 3 fields | ||
|
||
Fork this repo and send us a pull request when you think you are done. There is no deadline for this task unless otherwise noted to you directly. | ||
{ | ||
"name": "NEW NAME", | ||
"dob": "YYYY-MM-DD", | ||
"address": { | ||
"city": "NEW CITY" | ||
} | ||
} | ||
``` | ||
|
||
> What if I have a question? | ||
## Delete a User | ||
|
||
Create a new issue in this repo and we will respond and get back to you quickly. | ||
> Method: DELETE | ||
> | ||
> Endpoint: https://<API_ID>.execute-api.<AWS_REGION>.amazonaws.com/prod/users/<USER_ID> | ||
> | ||
> Returns: 204 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
#!/usr/bin/env node | ||
const { App, Tag } = (cdk = require('@aws-cdk/core')); | ||
const { UsersStack } = require('../lib/UsersStack'); | ||
|
||
const app = new App(); | ||
const STACK_NAME = process.env.STACK_NAME || 'SF-UserStack'; | ||
const STAGE = (app.node.tryGetContext('stage') || 'dev').toUpperCase(); | ||
|
||
const stack = new UsersStack(app, STAGE, { | ||
stackName: `${STACK_NAME}-${STAGE}`, | ||
}); | ||
|
||
Tag.add(stack, 'stack', STACK_NAME); | ||
Tag.add(stack, 'stage', STAGE); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"app": "node bin/create-stack.js" | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed some of your files lack newlines at the end of the file.
have you considered using editorconfig or similar tool to auto-add line endings?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a great suggestion. I read up on why an EOF newline can be helpful. Prior, I knew about it as an option, but didn't understand the reason behind it until you pointed it out. Thank you.