Skip to content

Commit 2447985

Browse files
melihciraymelihciray
committed
feat(filePath):s3 file upload into specified path
1 parent 15b9c60 commit 2447985

File tree

5 files changed

+545
-12
lines changed

5 files changed

+545
-12
lines changed

‎packages/s3-store/IMPLEMENTATION_SUMMARY.md‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# S3 Object Prefix Implementation - Summary
2+
3+
## Changes Made
4+
5+
I've successfully implemented the S3 object prefix feature that allows you to organize uploaded files in folder structures within your S3 bucket.
6+
7+
### Modified Files
8+
9+
**`/packages/s3-store/src/index.ts`**
10+
11+
1. **Added `objectPrefix` option to the `Options` type** (line ~49):
12+
```typescript
13+
objectPrefix?: string
14+
```
15+
- Optional parameter
16+
- Used to create pseudo-directory structures in S3
17+
- Example: `"uploads/"` or `"my-app/files/2024/"`
18+
19+
2. **Added `objectPrefix` class property** (line ~109):
20+
```typescript
21+
protected objectPrefix: string
22+
```
23+
24+
3. **Initialize `objectPrefix` in constructor** (line ~116):
25+
```typescript
26+
this.objectPrefix = objectPrefix ?? ''
27+
```
28+
- Defaults to empty string if not provided (maintains backward compatibility)
29+
30+
4. **Updated `infoKey()` method** (line ~224):
31+
```typescript
32+
protected infoKey(id: string) {
33+
return `${this.objectPrefix}${id}.info`
34+
}
35+
```
36+
37+
5. **Updated `partKey()` method** (line ~228):
38+
```typescript
39+
protected partKey(id: string, isIncomplete = false) {
40+
if (isIncomplete) {
41+
id += '.part'
42+
}
43+
return `${this.objectPrefix}${id}`
44+
}
45+
```
46+
47+
6. **Updated all S3 operations to use the prefix**:
48+
- `uploadPart()` - Uses `this.partKey(metadata.file.id)`
49+
- `create()` - Uses `this.partKey(upload.id)` for Key
50+
- `read()` - Uses `this.partKey(id)`
51+
- `finishMultipartUpload()` - Uses `this.partKey(metadata.file.id)`
52+
- `retrieveParts()` - Uses `this.partKey(id)`
53+
- `remove()` - Uses `this.partKey(id)` and `this.infoKey(id)`
54+
55+
### Created Files
56+
57+
**`/packages/s3-store/OBJECT_PREFIX_EXAMPLE.md`**
58+
- Complete usage documentation
59+
- Before/after examples showing folder structures
60+
- Multiple use cases (single folder, nested folders)
61+
62+
## How It Works
63+
64+
### Without objectPrefix (default behavior):
65+
```
66+
my-bucket/
67+
├── file-abc123
68+
├── file-abc123.info
69+
├── file-xyz789
70+
└── file-xyz789.info
71+
```
72+
73+
### With objectPrefix: 'uploads/':
74+
```
75+
my-bucket/
76+
└── uploads/
77+
├── file-abc123
78+
├── file-abc123.info
79+
├── file-xyz789
80+
└── file-xyz789.info
81+
```
82+
83+
### With objectPrefix: 'my-app/uploads/2024/':
84+
```
85+
my-bucket/
86+
└── my-app/
87+
└── uploads/
88+
└── 2024/
89+
├── file-abc123
90+
├── file-abc123.info
91+
├── file-xyz789
92+
└── file-xyz789.info
93+
```
94+
95+
## Usage Example
96+
97+
```typescript
98+
import {S3Store} from '@tus/s3-store';
99+
100+
const store = new S3Store({
101+
objectPrefix: 'uploads/', // Add this line!
102+
103+
s3ClientConfig: {
104+
bucket: 'my-bucket',
105+
region: 'us-east-1',
106+
credentials: {
107+
accessKeyId: 'YOUR_ACCESS_KEY',
108+
secretAccessKey: 'YOUR_SECRET_KEY',
109+
},
110+
},
111+
});
112+
113+
// Now all uploads will be stored in the 'uploads/' folder
114+
```
115+
116+
## Backward Compatibility
117+
118+
✅ **Fully backward compatible** - If `objectPrefix` is not provided, it defaults to an empty string, maintaining the current behavior of storing files at the bucket root.
119+
120+
## Testing Recommendations
121+
122+
1. Test with no prefix (default behavior)
123+
2. Test with single folder: `'uploads/'`
124+
3. Test with nested folders: `'my-app/uploads/2024/'`
125+
4. Test with prefix without trailing slash (should still work)
126+
5. Verify all operations work:
127+
- Upload files
128+
- Resume uploads
129+
- Read files
130+
- Delete files
131+
- List expired files
132+
133+
## Benefits
134+
135+
- **Organization**: Keep uploads organized in logical folder structures
136+
- **Multi-tenant**: Separate uploads by user/tenant: `'users/user-123/uploads/'`
137+
- **Time-based**: Organize by date: `'uploads/2024/10/'`
138+
- **Environment separation**: Different folders for dev/staging/prod
139+
- **Easier lifecycle policies**: Apply S3 lifecycle rules to specific prefixes
140+
- **Better billing analysis**: Track storage costs by prefix
141+
142+
## Notes
143+
144+
- The prefix applies to all S3 objects: main files, `.info` metadata files, and `.part` incomplete files
145+
- Make sure to include trailing slash for folder-like structures
146+
- S3 doesn't have real folders - prefixes create a "pseudo-directory" structure
147+
- All existing code continues to work without any changes

‎packages/s3-store/OBJECT_PREFIX_EXAMPLE.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# S3 Object Prefix Example
2+
3+
The S3Store now supports an `objectPrefix` option that allows you to organize your uploads in a folder structure within your S3 bucket.
4+
5+
## Usage
6+
7+
```typescript
8+
import {S3Store} from '@tus/s3-store';
9+
10+
const store = new S3Store({
11+
// Specify a folder path for your uploads
12+
objectPrefix: 'uploads/', // Files will be stored as: uploads/<file-id>
13+
14+
// Or create nested folders
15+
// objectPrefix: 'my-app/uploads/2024/', // Files: my-app/uploads/2024/<file-id>
16+
17+
s3ClientConfig: {
18+
bucket: 'my-bucket',
19+
region: 'us-east-1',
20+
credentials: {
21+
accessKeyId: 'your-access-key',
22+
secretAccessKey: 'your-secret-key',
23+
},
24+
},
25+
});
26+
```
27+
28+
## Before and After
29+
30+
### Without `objectPrefix` (default):
31+
```
32+
my-bucket/
33+
├── file-id-1
34+
├── file-id-1.info
35+
├── file-id-2
36+
└── file-id-2.info
37+
```
38+
39+
### With `objectPrefix: 'uploads/'`:
40+
```
41+
my-bucket/
42+
└── uploads/
43+
├── file-id-1
44+
├── file-id-1.info
45+
├── file-id-2
46+
└── file-id-2.info
47+
```
48+
49+
### With `objectPrefix: 'my-app/uploads/2024/'`:
50+
```
51+
my-bucket/
52+
└── my-app/
53+
└── uploads/
54+
└── 2024/
55+
├── file-id-1
56+
├── file-id-1.info
57+
├── file-id-2
58+
└── file-id-2.info
59+
```
60+
61+
## Notes
62+
63+
- The prefix is optional - if not provided, files will be stored at the bucket root (current behavior)
64+
- Make sure to include a trailing slash if you want a folder-like structure
65+
- The prefix applies to both the upload file and its `.info` metadata file
66+
- The prefix is also applied to incomplete parts (`.part` files)

‎packages/s3-store/QUICK_REFERENCE.md‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# Quick Reference: S3 Object Prefix
2+
3+
## TL;DR
4+
5+
Add `objectPrefix` to your S3Store configuration to organize files in folders:
6+
7+
```typescript
8+
const store = new S3Store({
9+
objectPrefix: 'uploads/', // ← Add this!
10+
s3ClientConfig: { /* ... */ }
11+
});
12+
```
13+
14+
## Common Use Cases
15+
16+
### 1. Simple folder organization
17+
```typescript
18+
objectPrefix: 'uploads/'
19+
// Result: bucket/uploads/file-id
20+
```
21+
22+
### 2. Multi-tenant application
23+
```typescript
24+
objectPrefix: `tenants/${tenantId}/uploads/`
25+
// Result: bucket/tenants/abc-123/uploads/file-id
26+
```
27+
28+
### 3. Date-based organization
29+
```typescript
30+
const date = new Date();
31+
objectPrefix: `uploads/${date.getFullYear()}/${date.getMonth() + 1}/`
32+
// Result: bucket/uploads/2024/10/file-id
33+
```
34+
35+
### 4. Environment separation
36+
```typescript
37+
const env = process.env.NODE_ENV;
38+
objectPrefix: `${env}/uploads/`
39+
// Result: bucket/production/uploads/file-id or bucket/dev/uploads/file-id
40+
```
41+
42+
### 5. File type segregation
43+
```typescript
44+
objectPrefix: 'media/images/'
45+
// Result: bucket/media/images/file-id
46+
```
47+
48+
### 6. User-specific uploads
49+
```typescript
50+
objectPrefix: `users/${userId}/files/`
51+
// Result: bucket/users/user-456/files/file-id
52+
```
53+
54+
## What Gets Prefixed?
55+
56+
✅ Main upload file: `bucket/prefix/file-id`
57+
✅ Info metadata file: `bucket/prefix/file-id.info`
58+
✅ Incomplete parts: `bucket/prefix/file-id.part`
59+
60+
## Tips
61+
62+
- Always use trailing slash for folder-like structure: `'uploads/'` ✅ not `'uploads'`
63+
- Prefix is optional - without it, files go to bucket root (backward compatible)
64+
- S3 doesn't have real folders - prefixes create visual organization
65+
- Use prefixes to apply different lifecycle policies to different file types
66+
- Combine with S3 bucket policies for fine-grained access control
67+
68+
## S3 Console View
69+
70+
With `objectPrefix: 'uploads/'`, your S3 console will show:
71+
72+
```
73+
📁 my-bucket
74+
└── 📁 uploads
75+
├── 📄 file-abc123
76+
├── 📄 file-abc123.info
77+
├── 📄 file-xyz789
78+
└── 📄 file-xyz789.info
79+
```
80+
81+
Without prefix:
82+
83+
```
84+
📁 my-bucket
85+
├── 📄 file-abc123
86+
├── 📄 file-abc123.info
87+
├── 📄 file-xyz789
88+
└── 📄 file-xyz789.info
89+
```
90+
91+
## Complete Example
92+
93+
```typescript
94+
import {Server} from '@tus/server';
95+
import {S3Store} from '@tus/s3-store';
96+
97+
const tusServer = new Server({
98+
path: '/files',
99+
datastore: new S3Store({
100+
objectPrefix: 'uploads/',
101+
s3ClientConfig: {
102+
bucket: 'my-bucket',
103+
region: 'us-east-1',
104+
credentials: {
105+
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
106+
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
107+
},
108+
},
109+
}),
110+
});
111+
```
112+
113+
## Migration from No Prefix
114+
115+
If you're adding a prefix to an existing setup:
116+
117+
1. **Option A**: Keep old files where they are, new files use prefix (recommended)
118+
2. **Option B**: Use AWS CLI to move existing files:
119+
```bash
120+
aws s3 mv s3://my-bucket/ s3://my-bucket/uploads/ --recursive
121+
```
122+
123+
## Benefits
124+
125+
🎯 **Organization**: Logical folder structure
126+
🔐 **Security**: Apply IAM policies per prefix
127+
💰 **Cost**: Track storage costs by prefix
128+
⏰ **Lifecycle**: Different retention policies per folder
129+
🧹 **Cleanup**: Easy to delete all files in a prefix
130+
📊 **Analytics**: Better insights with S3 analytics

0 commit comments

Comments
 (0)