Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REQUEST] Make Filestream Client Requests With No Logging Asynchronous #398

Open
ryanohoro opened this issue Sep 26, 2023 · 0 comments
Open
Assignees
Labels
enhancement New feature or request

Comments

@ryanohoro
Copy link
Collaborator

Is your feature request related to a problem? Please describe.

The filestream client waits for a request to fully complete before terminating the thread. This counts towards the concurrency limit. When streaming a large volume of files, and additionally not using/collecting Strelka events on the client side e.g. "Discard" mode, the client should not wait for events. This should vastly increase throughput for busy systems like network sensors.

Describe the solution you'd like

If the filestream client has no response entries in the config file (response: dict is emtpy), the client should immediately terminate the request after sending the file.

Describe alternatives you've considered

It may be possible to set the Filestream client timeout to a low threshold to quickly release waiting requests. However, that may negatively impact requests with large file sizes.

Additional context

Relevant code sections

sem := make(chan int, conf.Throughput.Concurrency)
defer close(sem)
// Create buffered channel to collect responses
responses := make(chan *strelka.ScanResponse, 100)
defer close(responses)
// Set callback for completed requests, returning events
wgResponse.Add(1)
if conf.Response.Log != "" {
go func() {
rpc.LogResponses(responses, conf.Response.Log)
wgResponse.Done()
}()
log.Printf("responses will be logged to %v", conf.Response.Log)
} else if conf.Response.Report != 0 {
go func() {
rpc.ReportResponses(responses, conf.Response.Report)
wgResponse.Done()
}()
log.Printf("responses will be reported every %v", conf.Response.Report)
} else {
go func() {
rpc.DiscardResponses(responses)
wgResponse.Done()
}()
log.Println("responses will be discarded")
}

// Wait for the response from the frontend containing the event
for {
resp, err := scanFile.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Println(errToMsg(err))
break
}
// Add the event to the responses channel
responses <- resp
}

@ryanohoro ryanohoro added the enhancement New feature or request label Sep 26, 2023
@ryanohoro ryanohoro changed the title [REQUEST] Make Filestream Client Requests Optionally Asyncronous [REQUEST] Make Filestream Client Requests With No Logging Asynchronous Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants