Skip to content

Feat: AWS S3 support #4646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Feat: AWS S3 support #4646

wants to merge 3 commits into from

Conversation

I-Info
Copy link
Contributor

@I-Info I-Info commented Apr 23, 2025 •
edited by sweep-ai bot
Loading

Sweep Summary Sweep

Adds AWS S3 object storage support to FastGPT as an alternative file storage option for chat files.

  • Implemented S3 client integration in packages/service/common/file/s3/index.ts using Minio library for bucket operations and presigned URL generation.
  • Created a new API endpoint in projects/app/src/pages/api/common/file/s3PresignedUpload.ts to handle S3 presigned URL generation with proper authentication.
  • Modified the file upload controller in projects/app/src/web/common/file/controller.ts to use S3 for chat files while keeping MongoDB GridFS for dataset files.
  • Added S3 configuration parameters to .env.template and updated system initialization to verify S3 bucket existence on startup.

Ask Sweep AI questions about this PR

I-Info added 2 commits April 23, 2025 10:27
@I-Info
init: s3
b1a721d
@I-Info
Refactor S3 presigned URL functions and integrate with API
ebf254a 
- Added support for custom expiration and metadata in S3 presigned URL functions.
- Replaced GridFS upload logic with S3 presigned URL-based uploads.
- Updated API endpoints to handle S3 presigned uploads and return preview URLs.
- Enhanced file upload controller to support S3 uploads with progress tracking.
```
@github-actions GitHub Actions
Copy link

Preview mcp_server Image: ghcr.io/labring/fastgpt-pr:fatsgpt_mcp_server_ebf254a5e4daca0b9d8386f448485418d3a4ab25

@github-actions GitHub Actions
Copy link

Preview sandbox Image: ghcr.io/labring/fastgpt-pr:fatsgpt_sandbox_ebf254a5e4daca0b9d8386f448485418d3a4ab25

@github-actions GitHub Actions
Copy link

Preview fastgpt Image: ghcr.io/labring/fastgpt-pr:fatsgpt_ebf254a5e4daca0b9d8386f448485418d3a4ab25

@github-actions GitHub Actions
Copy link

Preview mcp_server Image: ghcr.io/labring/fastgpt-pr:fatsgpt_mcp_server_5ba33941bd8378f86e07e2381f942eafae7d8a74

@github-actions GitHub Actions
Copy link

Preview sandbox Image: ghcr.io/labring/fastgpt-pr:fatsgpt_sandbox_5ba33941bd8378f86e07e2381f942eafae7d8a74

@github-actions GitHub Actions
Copy link

Preview fastgpt Image: ghcr.io/labring/fastgpt-pr:fatsgpt_5ba33941bd8378f86e07e2381f942eafae7d8a74

@I-Info
Refactor S3 client initialization and error handling
7f21b30 
- Add `isS3ClientInitialized` to check S3 client readiness
- Update functions to handle uninitialized S3 client gracefully
- Introduce `removeObjectsByPrefix` for bulk deletion by prefix
- Modify `deleteChatFiles` to remove S3 files by chat ID
- Adjust S3 upload logic to include chat ID in file paths
- Add fallback to legacy upload for unsupported cases
@github-actions GitHub Actions
Copy link

Preview sandbox Image: ghcr.io/labring/fastgpt-pr:fatsgpt_sandbox_7f21b307ceb8238a7eaa93d256e8717c958deff2

@github-actions GitHub Actions
Copy link

Preview mcp_server Image: ghcr.io/labring/fastgpt-pr:fatsgpt_mcp_server_7f21b307ceb8238a7eaa93d256e8717c958deff2

@github-actions GitHub Actions
Copy link

Preview fastgpt Image: ghcr.io/labring/fastgpt-pr:fatsgpt_7f21b307ceb8238a7eaa93d256e8717c958deff2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@I-Info