Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement AccessLog support for Ballerina #1969

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

AzeemMuzammil
Copy link
Member

@AzeemMuzammil AzeemMuzammil commented Apr 25, 2024

Description

This pull request addresses the proposal detailed in ballerina-platform/ballerina-library#6111. The implementation has been completed, and comprehensive load tests were conducted to evaluate the performance with and without the access log feature enabled.

The load tests were executed for 30 minutes to ensure stability and performance consistency. Different backends and testing tools were utilized for HTTP/1.1 and HTTP/2 passthrough scenarios:

HTTP/1.1 Passthrough Scenarios:

  • Backend: Netty
  • Testing Tool: JMeter

HTTP/2 Passthrough Scenarios:

  • Backend: Netty for HTTP/2 Client and Custom Ballerina BE for HTTP/1.1 Client (Netty backend was not used as it crashes under extensive load when multiple streams are created)
  • Testing Tool: h2load

These configurations were chosen to handle each protocol's specific requirements and challenges effectively. Additionally, we have added the load test results and memory usage metrics for the HTTP/2 passthrough scenarios, providing detailed insights into the performance and resource consumption under heavy load conditions.

Load Test Results:

h1-h1-pass-through scenario (Apache JMeter)

  1. AccessLog Disabled:
load_test_h1_h1_log_disabled
  1. AccessLog Enabled:
load_test_h1_h1_log_enabled

h1-h2-pass-through scenario (Apache JMeter)

  1. AccessLog Disabled:
load_test_h1_h2_log_disabled
  1. AccessLog Enabled:
load_test_h1_h2_log_enabled

h2-h2-pass-through scenario (h2load)

  1. AccessLog Disabled:

Result
h2_h2_log_disabled_result

Heap
h2_h2_log_disabled_heap

CPU
h2_h2_log_disabled_cpu

  1. AccessLog Enabled:

Result
h2_h2_log_enabled_result

Heap
h2_h2_log_enabled_heap

CPU
h2_h2_log_enabled_cpu

h2-h1-pass-through scenario (h2load)

  1. AccessLog Disabled:

Result
h2_h1_log_disabled_result

Heap
h2_h1_log_disabled_heap

CPU
h2_h1_log_disabled_cpu

  1. AccessLog Enabled:

Result
h2_h1_log_enabled_result

Heap
h2_h1_log_enabled_heap

CPU
h2_h1_log_enabled_cpu

Examples

Checklist

  • Linked to an issue
  • Updated the changelog
  • Added tests
  • Updated the spec
  • Checked native-image compatibility
  • Checked the impact on OpenAPI generation

@@ -144,6 +145,9 @@ public synchronized void addHttpContent(HttpContent httpContent) {
public HttpContent getHttpContent() {
HttpContent httpContent = this.blockingEntityCollector.getHttpContent();
this.contentObservable.notifyGetListener(httpContent);
if (httpContent != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blockingEntityCollector can contain multiple http contents right? In this case we only consider 1 content and set it to the carbon msg. Is that correct?

@@ -53,6 +59,7 @@ public class ReceivingEntityBody implements SenderState {
private final Http2TargetHandler http2TargetHandler;
private final Http2ClientChannel http2ClientChannel;
private final Http2TargetHandler.Http2RequestWriter http2RequestWriter;
private Long contentLength = 0L;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not sure about storing the content length in the state itself. while receiving the entity body, there is a possibility for the state to change right? if it changes, when receiving the response body again, a new ReceivingEntityBody state will be created that doesn't have the previous contentLength value. This is a possibility right? @TharmiganK

Copy link

This PR has been open for more than 15 days with no activity. This will be closed in 3 days unless the stale label is removed or commented.

@github-actions github-actions bot added the Stale label May 18, 2024
@TharmiganK TharmiganK removed the Stale label May 19, 2024
@AzeemMuzammil AzeemMuzammil marked this pull request as ready for review May 28, 2024 05:49
@AzeemMuzammil AzeemMuzammil force-pushed the fb-access-log-improvement branch 3 times, most recently from 252ce36 to a8d2f46 Compare June 5, 2024 05:32
Copy link

sonarcloud bot commented Jun 12, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@AzeemMuzammil AzeemMuzammil changed the base branch from 2201.8.x to master June 18, 2024 04:18
private List<HttpAccessLogMessage> getHttpAccessLogMessages(HttpCarbonMessage request) {
Object outboundAccessLogMessagesObject = request.getProperty(OUTBOUND_ACCESS_LOG_MESSAGES);
if (outboundAccessLogMessagesObject instanceof List<?> rawList) {
for (Object item : rawList) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a possibility for the rawList to contain other LogMessage types than HttpAccessLogMessage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ATM, no only HttpAccessLogMessage

changelog.md Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants