-
-
Notifications
You must be signed in to change notification settings - Fork 176
feat(infra): enhance server infrastructure with Redis connection pool and observability stack #1259
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
abretonc7s
wants to merge
25
commits into
main
Choose a base branch
from
redis-consolidation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5,864
−789
Open
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
779c38d
feat(redis): implement connection pooling and key migration for Redis
abretonc7s 9a3fe15
feat(analytics): add analytics server and client packages
abretonc7s 86abc4c
feat(protocol): introduce architectural changes for communication layer
abretonc7s d9b9417
refactor(analytics): update server URL references and introduce analy…
abretonc7s 451f6c5
refactor(analytics): streamline analytics handling and server URL con…
abretonc7s 4291b68
Merge remote-tracking branch 'origin/main' into redis-consolidation
abretonc7s c2effb4
feat(analytics-server): add Dockerfile for analytics server setup
abretonc7s d5bd5cb
feat(redis): implement connection pool and enhance Redis integration
abretonc7s 5034c56
feat(monitoring): integrate Prometheus and Grafana into Docker Compose
abretonc7s 32035a6
feat(docker): enhance Docker Compose setup for Redis and monitoring
abretonc7s 3be6ef3
feat(monitoring): integrate Loki and Promtail for enhanced logging
abretonc7s 9eb0887
feat: wip
abretonc7s c078cb5
feat(docs): update README and Grafana dashboards for improved clarity…
abretonc7s 3349f7f
fix(docs): update README to include Loki in background services setup
abretonc7s fa956fc
chore: remove migration status endpoint from socket server
abretonc7s 38b32ad
refactor(redis): improve health monitoring and logging logic
abretonc7s 8611177
feat: cleanup
abretonc7s 01c69b9
feat: cleanup
abretonc7s ab872e0
fix: pr comments
abretonc7s 38486eb
fix: pr comments
abretonc7s 7137851
fix: pipeline
abretonc7s 16477fd
feat: pr comments
abretonc7s b0ace18
feat: lockfile
abretonc7s 8c9e3e8
fix: pipeline
abretonc7s 768e716
fix: pipeline
abretonc7s File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# analytics-client | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
{ | ||
"name": "@metamask/sdk-analytics-client", | ||
"packageManager": "[email protected]", | ||
"scripts": { | ||
"build": "echo 'Na'", | ||
"build:dev": "echo 'Na'", | ||
"dev": "echo 'Na'", | ||
"build:post-tsc": "echo 'Na'", | ||
"build:pre-tsc": "echo 'Na'", | ||
"size": "echo 'Na'", | ||
"clean": "echo 'Na'", | ||
"lint": "echo 'Na'", | ||
"lint:changelog": "echo 'Na'", | ||
"lint:eslint": "echo 'Na'", | ||
"lint:fix": "echo 'Na'", | ||
"lint:misc": "echo 'Na'", | ||
"publish:preview": "echo 'Na'", | ||
"prepack": "echo 'Na'", | ||
"reset": "echo 'Na'", | ||
"test": "echo 'Na'", | ||
"test:e2e": "echo 'Na'", | ||
"test:coverage": "echo 'Na'", | ||
"test:ci": "echo 'Na'", | ||
"test:dev": "echo 'Na'", | ||
"watch": "echo 'Na'" | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
dist/ | ||
node_modules/ | ||
*.js | ||
*.d.ts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
module.exports = { | ||
root: true, | ||
parser: '@typescript-eslint/parser', | ||
plugins: ['@typescript-eslint'], | ||
extends: [ | ||
'eslint:recommended', | ||
'plugin:@typescript-eslint/recommended', | ||
'prettier', | ||
], | ||
env: { | ||
node: true, | ||
es6: true, | ||
}, | ||
rules: { | ||
'@typescript-eslint/explicit-module-boundary-types': 'off', | ||
'@typescript-eslint/no-explicit-any': 'warn', | ||
'@typescript-eslint/no-unused-vars': ['error', { argsIgnorePattern: '^_' }], | ||
}, | ||
}; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Dependencies | ||
node_modules/ | ||
yarn.lock | ||
package-lock.json | ||
|
||
# Build output | ||
dist/ | ||
|
||
# Environment variables | ||
.env | ||
.env.local | ||
.env.*.local | ||
|
||
# Logs | ||
logs/ | ||
*.log | ||
npm-debug.log* | ||
yarn-debug.log* | ||
yarn-error.log* | ||
|
||
# IDE | ||
.idea/ | ||
.vscode/ | ||
*.swp | ||
*.swo | ||
|
||
# OS | ||
.DS_Store | ||
Thumbs.db |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
dist/ | ||
node_modules/ | ||
*.js | ||
*.d.ts | ||
package.json | ||
package-lock.json | ||
yarn.lock |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
{ | ||
"semi": true, | ||
"trailingComma": "es5", | ||
"singleQuote": true, | ||
"printWidth": 100, | ||
"tabWidth": 2 | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# Build stage | ||
FROM node:18-alpine AS builder | ||
|
||
# Install build dependencies and build the project | ||
WORKDIR /app | ||
COPY package.json ./ | ||
RUN yarn install | ||
COPY . . | ||
RUN yarn build | ||
|
||
# Runtime stage | ||
FROM node:18-alpine | ||
|
||
# Install runtime dependencies | ||
WORKDIR /app | ||
COPY --from=builder /app/package.json ./ | ||
RUN yarn install --production | ||
|
||
# Copy built project and .env file from the build stage | ||
COPY --from=builder /app/dist ./dist | ||
# Do not copy .env file, it should be mounted separately | ||
# COPY .env ./ | ||
|
||
# Expose the server port | ||
EXPOSE 2002 | ||
|
||
# Start the server | ||
CMD ["node", "dist/src/index.js"] | ||
# CMD ["sh", "-c", "DEBUG= node dist/index.js"] | ||
|
||
# Start the server with DEBUG mode enabled | ||
# CMD ["sh", "-c", "DEBUG=socket.io-redis-streams-adapter node dist/index.js"] | ||
# CMD ["sh", "-c", "DEBUG=socket.io-redis node dist/index.js"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# @metamask/analytics-server | ||
|
||
Analytics server for MetaMask SDK. | ||
|
||
## Prerequisites | ||
|
||
- Node.js | ||
- Yarn | ||
- Docker (Optional, for containerized deployment) | ||
|
||
## Local Development | ||
|
||
1. **Install Dependencies:** | ||
```bash | ||
yarn install | ||
``` | ||
2. **Configure Environment:** | ||
Copy `.env.sample` (if it exists, otherwise create `.env`) and fill in the necessary environment variables. | ||
3. **Build the Code:** | ||
```bash | ||
yarn build | ||
``` | ||
4. **Run the Server:** | ||
* For production mode (uses compiled code): | ||
```bash | ||
yarn start | ||
``` | ||
* For development mode (uses ts-node): | ||
```bash | ||
yarn dev | ||
``` | ||
|
||
The server will typically run on the port specified in your `.env` file (defaulting to 2002 if not set). | ||
|
||
## Running with Docker | ||
|
||
1. **Build the Docker Image:** | ||
```bash | ||
docker build -t metamask/analytics-server . | ||
``` | ||
2. **Run the Docker Container:** | ||
Make sure to provide the necessary environment variables, for example by using an `.env` file and the `--env-file` flag. | ||
```bash | ||
docker run -p 2002:2002 --env-file .env metamask/analytics-server | ||
``` | ||
* Replace `2002:2002` if the server uses a different port. | ||
* The container exposes port 2002 by default. | ||
|
||
## Configuration | ||
|
||
The server is configured using environment variables. These can be placed in a `.env` file in the root directory for local development. See `.env.sample` (if available) for required variables. | ||
|
||
When running with Docker, environment variables should be passed to the container (e.g., using `--env-file` or `-e` flags). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
{ | ||
"name": "@metamask/analytics-server", | ||
"version": "1.0.0", | ||
"private": true, | ||
"description": "Analytics server for MetaMask SDK", | ||
"main": "dist/src/index.js", | ||
"scripts": { | ||
"build": "tsc", | ||
"start": "node dist/src/index.js", | ||
"dev": "ts-node src/index.ts", | ||
"lint": "eslint . --ext .ts", | ||
"lint:fix": "eslint . --ext .ts --fix", | ||
"format": "prettier --write \"src/**/*.ts\"", | ||
"typecheck": "tsc --noEmit", | ||
"allow-scripts": "allow-scripts" | ||
}, | ||
"dependencies": { | ||
"analytics-node": "^6.2.0", | ||
"body-parser": "^1.20.2", | ||
"cors": "^2.8.5", | ||
"dotenv": "^16.3.1", | ||
"express": "^4.18.2", | ||
"express-rate-limit": "^7.1.5", | ||
"helmet": "^5.1.1", | ||
"ioredis": "^5.6.0", | ||
abretonc7s marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"winston": "^3.11.0" | ||
}, | ||
"devDependencies": { | ||
"@lavamoat/allow-scripts": "^2.3.1", | ||
"@types/analytics-node": "^3.1.13", | ||
"@types/body-parser": "^1.19.4", | ||
"@types/cors": "^2.8.15", | ||
"@types/express": "^4.17.20", | ||
"@types/node": "^20.4.1", | ||
"@typescript-eslint/eslint-plugin": "^4.20.0", | ||
"@typescript-eslint/parser": "^4.20.0", | ||
"eslint": "^7.30.0", | ||
"eslint-config-prettier": "^8.3.0", | ||
"eslint-plugin-prettier": "^3.4.0", | ||
"prettier": "^2.8.8", | ||
"ts-node": "^10.9.1", | ||
"typescript": "^4.3.2" | ||
} | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.