Issue with TUSKit uploads pausing in the background and high memory usage when pausing and resuming uploads.

[muthuvelWorkflowlabs]

I'm encountering an issue with the TusKit iOS library where uploads pause when the app goes into the background and automatically resume when it returns to the foreground. To address this, I tried setting the chunk size to 0, which allowed the uploads to continue in the background. However, this approach leads to the app being terminated by the system due to excessive memory usage when pausing and resuming uploads of files larger than 2GB
Note: This crash does not occur if the upload runs continuously without being paused and resumed.

Steps to Reproduce:
1 Implement resumable uploads using TusKit in an iOS app.
2 Set the chunk size to 0 in the TUSClient initialization.
3 Attempt to upload a file larger than 2GB.
4 Observe that the app crashes due to high memory usage when uploads are paused and resumed.

Expected Behavior:
• The uploads should continue in the background without causing a memory-related crash.

Actual Behavior:
• The app crashes due to high memory consumption when attempting to pause and resume uploads of files larger than 2GB with the chunk size set to 0.
• The app is not running in the background when I set the chunk size other than 0, eg: If I set the chunk size as 20MB, the app will upload in the background until 20MB. Then pauses and waits for the app to come in foreground to upload the next chunk.

Code Snippet:
func initializeTusClient(){
do {
let identifier = "com.workflowlabs.newsflowkmp.tus"
let tusClient = try TUSClient(
server: URL(string: Urls().uploadFile)!,
sessionIdentifier: "TUS DEMO",
sessionConfiguration: .background(withIdentifier: identifier),
storageDirectory: URL(string: "/TUS")!,
chunkSize: 0 // This causes high memory usage for large files when uploads are paused and resumed.
)
Self.uploadCommon = UploadCommonIosImpl(tusClient: tusClient)
logDebug(message: "Upload initiated", tag: "UploadTusIos")
} catch let error {
logError(error: "Failed to create tusClient instance")
logError(error: error)
}
}

Environment:
• iOS version: 18.3.2
• TusKit version: 3.4.3
• Device: iPhone 11

Additional Context:
• Setting the chunk size to a non-zero value avoids the crash but results in the upload pausing when the app goes into the background.
• Any insights on how to balance background upload continuation with memory usage would be appreciated.

For old code snippets please refer to Pr #195

Error logs
#3 0x000000010143ddbc in UploadDataTask.prepareUploadFile() at /Users/muthuvelekambaranathan/Downloads/NewsFlow2ChatFeatureFinished/NewsFlowKMP/iosApp/Pods/TUSKit/Sources/TUSKit/Tasks/UploadDataTask.swift:186
#4 0x000000010143cd68 in UploadDataTask.run(completed:) at /Users/muthuvelekambaranathan/Downloads/NewsFlow2ChatFeatureFinished/NewsFlowKMP/iosApp/Pods/TUSKit/Sources/TUSKit/Tasks/UploadDataTask.swift:85
#6 0x000000010141bcc4 in closure #1 in Scheduler.checkProcessNextTask() at /Users/muthuvelekambaranathan/Downloads/NewsFlow2ChatFeatureFinished/NewsFlowKMP/iosApp/Pods/TUSKit/Sources/TUSKit/Scheduler.swift:104
Thread 48: EXC_RESOURCE (RESOURCE_TYPE_MEMORY: high watermark memory limit exceeded) (limit=2098 MB)

[donnywals]

HI @muthuvelWorkflowlabs thanks for reporting this issue.

Background uploads not resuming for chunked uploads is expected; iOS penalizes apps that start new network transfers while an app is in the background hence why we're not doing this.

You should be able to upload large files without memory issues; this is something that we've solved in the past but I'm not sure we tested >=2GB (my largest test file at the moment is 1GB). I'll try a 2GB file on my end and see what I can do to fix this as soon as possible. I will be out on vacation after next week so hopefully it's something that can be addressed before then but otherwise I'll have to ask you to be patient.

edit
Ah I'm now seeing that this was resolved initially but remains an issue when uploads get paused and resumed.

[donnywals]

@muthuvelWorkflowlabs #209 should fix this issue. If you have a chance I would be forever grateful if you could verify this before I merge it in a few days.

[muthuvelWorkflowlabs]

@donnywals Thank you for your continued support and for sharing the fix in the dw/memory-usage branch.

In version 3.5.0, when we pause an upload and try to resume it, the upload starts from zero. However, in version 3.4.3, the upload correctly resumes from where it left off after being paused. In the dw/memory-usage branch, pausing and resuming also causes the upload to restart from the beginning, instead of continuing from the last uploaded chunk.

If it's possible to retain the resumable behavior seen in 3.4.3 while addressing the memory issue, it would be incredibly helpful.

[donnywals]

That would definitely be desirable 😅 I will see what changed and fix the underlying issue

[donnywals]

@muthuvelWorkflowlabs The PR is updated with some fixes that re-instate the desired resume behavior. If you have a chance to test from that branch I'm happy to merge it and cut a release if it works as expected for you

[muthuvelWorkflowlabs]

Hi @donnywals ,

Thanks a ton for the quick turnaround! I pulled the latest changes from the dw/memory-usage branch, and everything’s working perfectly now—uploads resume exactly where they left off with no memory issues. Really appreciate your help on this!

[donnywals]

Perfect! I've just merged the PR and will be releasing TUSKit 3.6.0 today.